Kyle

When GNU released Automake 1.13, they made some interesting (and unfortunate) decisions about their test harness.

Automake has (and has had since version 1.11, back in May 2009) two different test harnesses: a serial one and a parallel one. The serial one is simpler, but the parallel one has a lot of helpful features. (In this case, parallel means running multiple tests at the same time, not that the harness is good for tests that happen to themselves be parallel.) Importantly, these two harnesses are mutually incompatible (e.g. the use of TEST_ENVIRONMENT is often necessary in the serial harness and will seriously break the parallel harness). The default test harness, of course, has always been the serial one for lots of very good reasons, including backwards compatibility. However, starting in automake 1.13 (which is now the standard on up-to-date Ubuntu, and will become common), whether it’s a good idea or a bad idea, the default test harness behavior is now the parallel one. There is a way to specify that you want the original behavior (the “serial-tests” option to automake), however, that option was only added to automake 1.12 (released in April 2012), and since automake aborts when it sees an unsupported option, that path doesn’t really provide much backward compatibility.

Now, it’s clear that the automake authors think the parallel test harness is the way of the future, and is probably what people should be using for new projects. However, this has impact on backwards compatibility. For instance, the version of automake that comes with RHEL 5.8 (which still supported and relatively common) is 1.9.6 - which was released waaay back in July 2005!

One option for dealing with the latest wrinkle in automake is to adopt the parallel test harness, make 1.11 the required minimum version, and simply abandon support for developing on older systems. That may or may not be a viable option. Another option is to support both harnesses somehow, e.g. by providing two different Makefile.am files (or more, depending on how complex your test folder tree is) and swapping between them via some sort of script (e.g. autogen.sh, which many people use). This, however, is a maintainence nightmare. And the third option is to stick with the serial test harness, attempt detection of the autoconf version in the configure script and conditionally define the “serial-tests” option only when necessary. This will maintain compatibility with old versions, but is somewhat fragile.

An example of this latter is, assuming your AM_INIT_AUTOMAKE line looks like this:

AM_INIT_AUTOMAKE([1.9 no-define foreign])

…to make it look like this:

AM_INIT_AUTOMAKE([1.9 no-define foreign]
  m4_ifdef([AM_EXTRA_RECURSIVE_TARGETS], [serial-tests]))