Archive

Tag Archives: programming

I’m going a bit off-piste with this post, and looking at the hidden-in-plain-view potential of some of the libraries, very familiar to Linux geeks, but pretty much unknown to the outside world.

Although I talk about the past a lot, I’m not nostalgic. I don’t miss the frustrations of the past. I just like to remind myself how good things are now. By the way, I am by no means a computer historian, and my opinions and experience are subjective. If I post anything that is factually incorrect, please let me know.

I realise some of this might be bleeding obvious to the Linux world, but those of us cloistered for years in the walled-garden of the Mac OS X GUI, it’s quite a revelation.

As I’ve said before, I’m a lapsed programmer and used to write commercial software using framework applications, primarily FileMaker Pro 3.0 amongst others. FileMaker has developed into a very sophisticated rapid application development (RAD) platform now that FileMaker Inc have got around to including the features that us developers were clamouring for in the 1990s.

Given a choice, I am very happy tinkering with nice GUI-based visual programming / scripting tools, but the compelling reason for me to get my hands dirty at the command line is the fact that nothing gets in the way.

One of my most enduring love / hate relationships has been with HyperCard / SuperCard. I remember being excited, almost to the point of peeing myself, when I first got wind of HyperCard. My programming days started with BBC BASIC, then Spectrum BASIC, then the woefully under-implemented Commodore 64 BASIC. After that things just got worse and worse and Atari ST BASIC was absolutely useless. I can’t even remember if there was an implementation for the Amiga, but I did try CanDo which was promising but way ahead of its time. As the years passed it got more and more hard work to get to a command line and I kind of missed the direct simplicity of BBC or Spectrum BASIC, which were at least broadly complete implementations that you could actually do stuff with.

Update: I completely forgot about STOS on the ST and AMOS on the Amiga, which were both brilliant.
http://en.wikipedia.org/wiki/STOS_BASIC
http://en.wikipedia.org/wiki/AMOS_(programming_language)

HyperCard was a revelation to me as it was on the desktop, and you could easily create GUI-based applications with it, and it had a pretty decent plain-English programming language. You could actually do stuff with it.
http://en.wikipedia.org/wiki/HyperCard

As desktop computers “improved”, programming became more and more remote from the user, and there was a very frustrating period in the late 80s and early 90s where application software was not mature enough, but programming not accessible enough, to plug all the gaps in our productivity.

Fortunately those days are over.

There were many good things about HyperCard, but it was lacking in some fundamental functions, and for me, they were colour and structured graphics. HyperCard was strictly monochrome and bitmap only, and although Apple eventually did include plugins that supported colour graphics, it was an afterthought and not adequately implemented. Apple neglected HyperCard for years and it is now a minority interest tool. I still use it for programming on old compact Macs as it is one of the few programming tools that will work on machines with no more than 4 Mb RAM.

Silicon Beach SuperCard seemed to be the obvious successor, the so-called “HyperCard on steroids”. Its own language, SuperTalk, was an extended and mostly compatible development of HyperCard’s HyperTalk, and it supported 8 bit colour and script-controllable vector graphics.
http://en.wikipedia.org/wiki/Silicon_Beach_Software

SuperCard, whilst not neglected in the same way, passed from one owner to another for years, developing only gradually. I cannot tell you just how closely related it is to the original, but SuperCard appears to have evolved into something called LiveCode by Runtime Revolution.
http://www.runrev.com/

LiveCode deploys on a number of platforms including Windows, Linux, OS X, iOS and Android, and looks very promising. However, it seems to be lacking one of the fundamental features that I need, and that is to append single images to a movie file. Please correct me if I’m wrong, but I can’t find any mention of it, although I’m still looking into it.

And this is my big frustration when using very-high-level development tools. I tried using Apple Automator and QuickTime Pro player to assemble digital stills into movie files. Again, maybe it’s there but I couldn’t find it without having to operate the menu items in the player, and I really don’t want to go back to the days of using marionette software like Keyquencer (remember that?) or similar to operate programs via their menus. In my experience, automated clicking of buttons and selecting menu items is just not reliable or fast enough.

Adobe Photoshop’s Actions is a very powerful tool but it’s quite slow (although things may well have changed since CS3).

And this is when we get back to Linux, GIMP and ImageMagick. Whilst I have been banging my head against a brick wall for years, looking for the ideal development platform, Linux tools and libraries have been quietly maturing under my nose. I recently looked into using ImageMagick, which can do, well, everything. For the uninitiated, ImageMagick is a “software suite to create, edit, compose, or convert bitmap images” and can also procedurally create images by manipulating graphics or drawing shapes.

There are some very coherent and complete help files, written by Anthony Thyssen, and they seem to go on forever, documenting more and more features, available via command line. And the command line bit is the real killer because it means I can write a program and just call the single function I want, and add it to a Bash script.
http://www.imagemagick.org/Usage/

My own interest is in batch-converting, montaging of multiple images and format conversion of data sets, often comprising tens of thousands of images. I have managed to use gPhoto2 and FFmpeg to assemble the images into movies, and ImageMagick / GIMP will help me to manipulate the images if need be.

More soon…

Advertisements

THE CODE PUBLISHED HERE COMES WITH ABSOLUTELY NO WARRANTY WHATSOEVER, SO PLEASE USE AT YOUR OWN RISK.

I am basing my code on existing bits of open source stuff scavenged from various places and cobbled together. At some point in the future I hope to make some sort of point-and-click front end for it, but in the meantime it’s just command-line code, and at the moment, pretty crude.

Here is a slightly updated version of my previous script, which now clears the work folders and downloads from the camera using gPhoto2.

# !/bin/bash
clear
echo “Hello Monkey world”
#gphoto2 —

cd /home/richard/Desktop/RBTest/newtest
rm *.JPG
rm *.jpg
cd /home/richard/Desktop/RBTest/testtemp
rm *.JPG
rm *.jpg

cd /home/richard/Desktop/RBTest/newtest
gphoto2 –get-all-files

x=1; for i in $(ls -t -r *JPG); do counter=$(printf %04d $x); ln “$i” /home/richard/Desktop/RBTest/testtemp/img_”$counter”.jpg; x=$(($x+1)); done

cd /home/richard/Desktop/RBTest/testtemp
ffmpeg -r 25 -i img_%04d.jpg -s 640×480 -qscale 1 -vcodec mjpeg movie.avi

This is pretty clunky but at least it’s getting somewhere. At some point I’ll find out how to reference files case-insensitive.

On some of the older cameras, such as my A620, the camera saves images in discrete folders of only 200 images each, which means a lot of effort in post-production if it has to be done by hand.

I think the next logical steps are to compile multiple folders of images and also to name the destination movie file discretely. The name could be created from some format information, a date/time stamp, and the model of the camera which can be derived from gPhoto2.

Googling something about recursive folders I found this:

http://stackoverflow.com/questions/245698/list-files-recursively-in-linux-with-path-relative-to-the-current-directory

However, this does not return them in creation time order and, as far as I could google, there doesn’t seem to be an argument to produce that.

After googling something about Linux pipeline commands I found a reference to xargs, and Bob’s your uncle. I can hardly believe I found it so quickly, but I remember Martyn at Access Space telling me about pipelining so I used a bit of intuition and worked out how to take the results from the find command and sort them afterwards.

http://www.cyberciti.biz/faq/linux-unix-bsd-xargs-construct-argument-lists-utility/

Here is my Bride of Frankenstein, which takes a folder full of folders (full of images), lists them in creation order, renames them and compiles them into a movie file. Simples!

# !/bin/bash
clear
echo “Hello Monkey Planet”

cd /home/richard/Desktop/RBTest/testtemp
rm *.JPG
rm *.jpg

cd /home/richard/Desktop/RBTest/newtest

x=1; for i in $(find $(pwd) -name \*.JPG | xargs ls -t -r); do counter=$(printf %04d $x); ln “$i” /home/richard/Desktop/RBTest/testtemp/img_”$counter”.jpg; x=$(($x+1)); done

cd /home/richard/Desktop/RBTest/testtemp
ffmpeg -r 25 -i img_%04d.jpg -s 640×480 -qscale 1 -vcodec mjpeg movie.avi

There are some technologies that none of us will miss, such as CRTs, the SCSI interface, VHS and (for me) optical storage media (CDs and DVDs). But one technological era I am particularly glad to leave behind is the 1980s, characterised by cassette-loading or actual typing in of huge BASIC programs, distributed in magazines.

These days we have an embarrassment of riches in the form of the internet. For a programmer, it's a goldmine of free resources. However, something that is often missing is an exhaustive deconstruction of the code, particularly useful to noobs. I'm a lapsed programmer, so I don't find it too hard, but Linux is new to me and has some strange “conventions”.

Anyway, as promised here is some actual code that I found (well, Google found it and I copied it) with some explanation about what the bits and pieces do. I will probably update this a bit and please let me know if you spot any errors.

FFMpeg requires an unbroken sequence of image files starting at 1 and will fail if the numbers are not continuous. This is not optional and so it is best to rename them as a matter of course.

The following two code snippets are from stackoverflow.com and I've adapted them slightly.

http://stackoverflow.com/questions/2829113/ffmpeg-create-a-video-from-images

This is the script for renaming the files.

x=1; for i in *jpg; do counter=$(printf %03d $x); ln "$i" /tmp/img"$counter".jpg; x=$(($x+1)); done

And here is the original FFMpeg command from the same source.

ffmpeg -f image2 -i /tmp/img%03d.jpg /tmp/a.mpg

Note:

24 seconds to ininitalise the Powershot A640 and another 9 minutes to download 2,595 images totalling 2.7 Gb (over USB 2 on 1st generation MacBook (1.83 GHz Intel dual core) running Ubuntu Linux 11.04.

Sort by creation date/time

x=1; for i in $(ls -t -r *JPG); do counter=$(printf %04d $x); ln “$i” /tmp/img_”$counter”.jpg; x=$(($x+1)); done

The $( some stuff ) contruct eveluates everything within the brackets before using the result. In this case, $(ls -t -r *JPG) returns a list of files ending in “JPG”, sorted by ascending creation time. The for i in $(ls -t -r *JPG) is a for..next loop where i is each file ending in “JPG”, sorted into creation time order.

The snippet counter=$(printf %04d $x) resolves the integer variable x into a 4-digit decimal number with leading zeros and puts it into the variable counter. The leading zeros are necessary in order to maintain the 8-character file names so that they sort alpha-numerically. The snippet x=$(($x+1)) increments the variable x by integer 1 after each renaming operation.

It’s important to get the capitalisation correct and the destination pathname, or it will just fail.

The -t modifier sorts by creation time in reverse order, most recent first. The -r modifier reverses that order to, in this case, change the order back to the correct chronological order.

Canon Powershot cameras automatically number the images files in the format “IMG_0001.JPG” and restarts the numbering after 9,999 images. It will automatically create a new folder every 2,000 images and will create a new folder and restart from 0001 once it gets to IMG_9999.JPG.

Consequently, it is possible that for a timelapse shoot, the file names will not be in continous alpha-numeric order. Sometimes they will be, but it’s as well to assume that they won’t. Also, gPhoto2 does not preserve the folder structure if you download using the gphoto2 –get-all-files command, and will dump the whole lot into a single folder.

In this case, it is very likely that the alpha-numeric order will not be strictly chronological. As a result, it was necessary to include an amendment to the code that will order them into chronological order before renaming them.

More soon.