I'm creating an open source sync program called ClearSkies, with the help of some others. While unit testing can be done like usual, integration testing involves some additional trickery. This post contains what has worked well for me so far.
As a preamble, the first thing that I feel I need to be able to do with any test is to be able to iterate quicky. I have a tool to do this, called auto-go, which looks in the current directory for a file called .auto-go, and then searches the parent directory and all ancestors until it finds it. The magical part is how it is set up in vim. I make it so that if I press ",r", it saves my file and then runs auto-go, even in insert mode.
There are other methods for running tests automatically, such as with Guard, but I prefer this one because it lets me adjust my area of focus quickly (by changing the contents of .auto-go).
Integration Test with bash
To do an integration test, we need to run two copies of the program simultaneously. When I wrote a very similar program during 2007 I'd launch each copy by hand and then connect them together, which was tedious and slowed my progress. Basically, two copies of the program need to be launched and then the progress of the sync can be checked by looking directly at the directory contents:
rm -rf /tmp/1 /tmp/2 ./clearskies start --data-dir=/tmp/1 ./clearskies start --data-dir=/tmp/2 mkdir /tmp/1/stuff echo some file contents > /tmp/1/stuff/test-file code=`./clearskies --data-dir=/tmp/1 share /tmp/1/stuff read-write` ./clearskies --data-dir=/tmp/2 add /tmp/2/stuff $code echo "Waiting for test-file to sync" while ! test -e /tmp/2/stuff/test-file do sleep 0.1 done ./clearskies stop --data-dir=/tmp/1 ./clearskies stop --data-dir=/tmp/2
Of course, it's nice to see the logging output of both programs, so they should be launched in the Foreground:
./clearskies start --no-fork --data-dir=/tmp/1 & ./clearskies start --no-fork --data-dir=/tmp/2 &
It's then hard to distinguish between the output of both programs, so I tried launching each in its own gnome-terminal:
gnome-terminal -e './clearskies start --no-fork --data-dir=/tmp/1' gnome-terminal -e './clearskies start --no-fork --data-dir=/tmp/2'
I used the gnome-terminal version for quite a while, but I found it was annoying that the new windows would steal my focus, stopping me from interrupting them.
So, I started them in tmux instead:
if [ -z "$TMUX" ] then exec tmux new-session $0 fi tmux split-window -v './clearskies start --no-fork --data-dir=/tmp/1' tmux split-window -v './clearskies start --no-fork --data-dir=/tmp/2' tmux select-layout even-vertical tmux select-pane -t 0
This works fantastically well.
One problem with this approach is that if there are bugs involving the interaction of both programs, it's hard to figure out the timeline of events. I solved this by having each program write to a log file with a timestamp at the beginning of the line, including the test's output, and then combined them into a single log file, sorted by the time.
I had to capture
ctrl-c in my main script so that I could shut down the other
programs, and I also had the extra windows sleep after launching so that I
could see any exceptions.
See the actual integration script for more details.