Handling Variant Configuration Files

This post originally appeared on the Software Carpentry website.

One of our learners came to us with a problem last week. The program she uses depends on some complex configuration files, which she'd like to store in version control. However, a couple of parameters change depending on the machine the program is running on. She doesn't want to check those changes into version control over and over again; what should she do?

To make this more concrete, imagine that her configuration file is a Makefile containing instructions to rebuild a set of files. Initially, it looks like this:

summary.dat : left.dat right.dat
        summarize left.dat right.dat > summary.dat

That works fine on one machine, but on another, the program summarize has been installed as sum7. She could do this:

SUMMARIZE=summarize # on Linux
# SUMMARIZE=sum7 # on Mac OS X

summary.dat : left.dat right.dat
        ${SUMMARIZE} left.dat right.dat > summary.dat

but then she'd have to edit the file to uncomment one line, and recomment the other, whenever she switched from her Mac laptop to her desktop Linux machine and vice versa. Here's what she can do instead:

ifeq ($(shell uname),Linux)
        SUMMARIZE=summarize
else
        SUMMARIZE=sum7
endif

summary.dat : left.dat right.dat
        ${SUMMARIZE} left.dat right.dat > summary.dat

The trick is that Makefiles (and most other grown-up configuration files) allow conditionals and functions, just like programs. The function $(shell whatever) runs a shell command; ifeq then checks if that command's output is the string Linux, and the Makefile's variable SUMMARIZE is set accordingly.

Another way to approach the problem is to put the machine-dependent parameters in separate files, and include one of those in the main file. For example, the Makefile could be written like this:

include settings.mk

summary.dat : left.dat right.dat
        ${SUMMARIZE} left.dat right.dat > summary.dat

We would then put two files in version control—one for Linux:

# settings-linux.mk
SUMMARIZE=summarize

and one for Mac OS X:

# settings-macosx.mk
SUMMARIZE-sum7

Notice that neither of these is called settings.mk. The first time we check out on a machine, we manually copy either settings-linux.mk or settings-macosx.mk to create settings.mk. The include in the main Makefile then finds what it's looking for, and everything runs. If someone changes the settings for a particular platform, our next version control update will get the new platform-specific file, and we'll have to re-copy it to install it.

That manual copying step is why I'm not a fan of this second approach. There are ways to have the copying done automatically, but they all basically come down to including a conditional in the main Makefile, and if we're going to do that, we might as well use that for setting the parameter values anyway. However, some configuration file formats don't support conditionals, so the "include if available and fail if not" trick is still worth knowing.