One of my major goals in this project is to streamline the process of creating a visualization by freeing myself from ImageJ. I’m currently working on a Python script which nominally accomplishes this goal, and I figure this is as good a time as any to give you all an update.
My shotorg script makes use of several free software tools. Perhaps the most essential is shotdetect, a program written by Google software engineer Johan Mathe that computes the time at which each shot begins and exports the data to an xml file. From there it’s just a matter of using the Beautiful Soup Python library to read the data (specifically the millisecond at which each shot begins and how many milliseconds it lasts) and feed arguments to ffmpeg, which will rip the specified frames from the film and place them in their own directory.
organize function on 2001: A Space Odyssey according to my current settings creates 596 folders and 27570 png files totaling 133.9 MB. This is the point in the process when I would normally turn to ImageJ; however, I’ve found that ImageMagick is able to do almost everything I want faster, simpler, and more reliably than ImageJ. I can also run ImageMagick from the command line, which makes it much easier to integrate into my Python script. The two visualization functions I’ve written so far,
loop, transform each shot into a 1-row “film strip” montage and an animated gif, respectively. I wanted to make the
visualize function go even further, composing each film strip into a larger montage of each shot in the film arranged chronologically from top to bottom, but doing so causes ImageMagick to run out of memory long before completing its task. Luckily I was able to run the function on several chunks of the film and finish the composition in GIMP.
This visualization’s original version was 112.1 MB with a resolution of 33000×21204, and each individual frame was only 80×36! I think the key element that sets a visualization like this apart from a bar chart (which it certainly resembles) is its interactivity. You can zoom in on any point in the graph until an individual frame fills your entire screen. However, this feature is for the most part neutralized when the image is resized for the web, and my computer can barely handle 100 MB images when I view them locally!
I’ll write more about how useful I think this process is in my next post. Meanwhile, if you’re interested in duplicating my results, feel free to download my horrifically crude and inelegant Python script and customize it for your computer. Creating visualizations for single scenes is super easy, just follow these steps:
ffmpeg -ss (start time) -t (duration of clip) -i (yourvideo) -r (frame rate) -s (size of ripped frames) -f image2 out%03d.png
This will start ripping frames from your video file at a specified point, naming them out000.png, out002.png, etc. (change %03d for more initial zeros). For example, entering
ffmpeg -ss 01:23:45 -t 00:00:05 -i /home/user/videos/my_video.mp4 -r 12 -s 1920x1080 -f image2 out%02d.png
will result start ripping 1920×1080 resolution frames from
my_video.mp4 starting at 1:23:45 for 5 seconds, saving them as out00.png to out60.png. Now we can make a montage:
montage *.png -tile (colxrow) -geometry +0+0 -background black out.png
or a gif:
convert -delay 15 -loop 0 *.png out.gif