Ikke's Blog

Writing Makefiles the manual way, and using autotools

As promised, the article on writing Makefiles. As an extra, I also include how to build a basic/simple autotools project.

Writing Makefiles, the manual way

Let's get started.
First of all, you need all files used in my previous article, i.e. test-signal.gob and test-signal-test.c

There are some rules of thumb you can use when writing a Makefile manally:

1. Find out what programs you need to process your source files


2. Find out what packages you need: which headers, and which libraries


3. Make a list of all source files, and find out which ones are dependent on others

Let's get through these step by step:

1. What programs do we need? We need gob2 of course, to process our gob file. Next to this, we need the stuff you need most of the time when creating a Makefile: a compiler and a linker. And guess what, GCC can do both things.


2. What packages do we need? Remember the command line thing you had to use when compiling the test-signal executable?

   gcc -Wall `pkg-config --libs --cflags gobject-2.0 glib-2.0` -o testsignal test-signal-test.c test-signal.c

   We don't even need this line to figure out what we need: we're building a gobject, so we need all libs and headers provided by the gobject package, version 2.x in our case, same thing for glib.
   How can we find out where these things are located? Well, the smart people at freedesktop.org made a nifty tool called pkg-config. When a library is installed, it can install a pkg-config resource file, which lists the directories where it's stuff is installed. For some samples of these files, check /usr/lib/pkg-config. The pkg-config command line utility can parse these files and give you the information you need.


3. What source files have we got, and which one needs which? We got 2 files, test-signal.gob and test-signal-test.c. In the end we want to generate an executable called test-signal, which needs test-signal.c (the GObject implementation file) and test-signal-test.c.
   We will work in 2 stages here: first we'll compile all necessary .c files to an object file (.o), then in the end link all object files together in a nice executable.
   Here's what should happen: test-signal.gob should be parsed by gob to create our test-signal.c and test-signal.h file, test-signal.c must be compiled, test-signal-test.c must be compiled, and finaly test-signal.o and test-signal-test.o should be linked into test-signal.

First a little intermezzo. You might be asking "If I need to make all these lists, what's the use? Can't I just write some Bash script which executes the gcc command all at once?". Well, no. Make does more than just executing some commands. It also checks whether it *should* do something. Imagine you got 100 source files. You got all of them compiled, linked into some executable, run that one, and find some bug. You fix it by editing some lines in one file, and re-execute your huge gcc command. Gcc would recompile all 100 files, which will take some time.
If you use a Makefile make will find out only one file has been altered since the last build, it'll let gcc recompile only that file, then relink all object files together, which will take less time. Next to this: once using autotools, everything becomes much simpeler ;-)

Ok, now we got all prerequisites. Let's get started writing our Makefile.
I always tend to use the same format when writing one (although I dont write many of them, I use autotools ;-)). I start with defining the executables:

GOB2=gob2
CC=gcc
LD=gcc

This is not necessary, but can be usefull sometimes. Now we can use these variables later on. If we want to change the linker, we only have to edit the Makefile in one place.

Next comes the definition of the CFLAGS, the flags given to the compiler (CC) when compiling a sourcefile into an object file:

CFLAGS=`pkg-config --cflags glib-2.0`
CFLAGS+=`pkg-config --cflags gobject-2.0`
CFLAGS+=-Wall -g

As you can see, we request the CFLAGS necessary for glib-2.0 and gobject-2.0 (2.x, actually) by querying pkg-config. In the end we add 2 compiler flags: -Wall, which tells gcc to show all possible warnings, and -g, which tells gcc to include debugging information. This can result in a somewhat bigger executable, but it is very usefull if we want to debug the program using GDB.

Now we define the LDFLAGS, the flags given to the linker:

LDFLAGS=`pkg-config --libs glib-2.0`
LDFLAGS+=`pkg-config --libs gobject-2.0`

This should be fairly self-explaining.

This project only consists of one executable, so I define an "OBJS" variable including all the object files needed to build our executable:

OBJS=test-signal.o test-signal-test.o

Now comes the name of the executable we want to create:

PROG=test-signal

Now the magic starts. Until now we only defined some variables for later use. We were not forced to do so, it's just more convenient later on. Actually, when building small things, we can just use re-use this Makefile, and only have to change the variable definitions (OBJS, PROG, maybe some CFLAGS and LDFLAGS).

Now some rules follow, which tell make how to handle files:

%.c %.h %-private.h: %.gob
        $(GOB2) $<

This rule tells make: "When a foo.c, foo.h or foo-private.h is needed, and not existant, it should be made from foo.gob, by executing "$(GOB2) $<" which gets expanded as "gob2 foo.gob"". Indeed, % represent "a string", $(GOB2) is the variable we defined at the beginning and gets expanded as "gob2", and $< gets expanded as the first item on the right of the semicolon (":").
One big thing to watch out for: Makefiles are indenting-sensitive but you may not use spaces. So in the last Makefile fragment there's a [tab] before $(GOB2).
The format of a rule is very simple:

filenametobuild: dependenciestobuilditfrom
[tab]what to do first[enter]
[tab]what to do next if necessary[enter]

etc.

Next we define how to link our $(PROG):

$(PROG): $(OBJS)
        $(CC) $(LDFLAGS) $(OBJS) -o $(PROG)

$(PROG) is built out of $(OBJS) by issuing "$(CC) $(LDFLAGS) $(OBJS) -o $(PROG)", expanded to "gcc `pkg-config --libs gobject-2.0` `pkg-config --libs glib-2.0` test-signal.o test-signal-test.o -o test-signal"

We still have to tell make how to create object files out of source files:

%.o: %.c
        $(CC) $(CFLAGS) -c $<

You should be able to figure out what this does by yourself.

Now we add some convenience targets:

all: $(PROG)

default: $(PROG)

clean:
        rm -f $(OBJS) $(PROG)
        rm -f test-signal.[ch]
        rm -f test-signal-private.h

This enables us to just type "make", which will start building the "default" target, or "make all", which will build the "all" target, or "make clean" to "build" the "clean" target.
Notice a target

1. is not forced to "do" something ("all" and "default"). You can just say "it depends on "foo" and/or "bar", which will be built then, and


2. the dependencies of a target may be empty ("clean")

Let's end with the complete Makefile:

GOB2=gob2
CC=gcc
LD=gcc

CFLAGS=`pkg-config --cflags glib-2.0`
CFLAGS+=`pkg-config --cflags gobject-2.0`
CFLAGS+=-Wall -g

LDFLAGS=`pkg-config --libs glib-2.0`
LDFLAGS+=`pkg-config --libs gobject-2.0`

OBJS=test-signal.o test-signal-test.o

PROG=test-signal

%.c %.h %-private.h: %.gob
        $(GOB2) $<

$(PROG): $(OBJS)
        $(CC) $(LDFLAGS) $(OBJS) -o $(PROG)

%.o: %.c
        $(CC) $(CFLAGS) -c $<

all: $(PROG)

default: $(PROG)

clean:
        rm -f $(OBJS) $(PROG)
        rm -f test-signal.[ch]
        rm -f test-signal-private.h

Save this file as a file called "Makefile" in your source directory, and type "make". This should be the result:

gob2 test-signal.gob
gcc `pkg-config --cflags glib-2.0` `pkg-config --cflags gobject-2.0` -Wall -g -c test-signal.c
test-signal.c: In function `test_signal_testsignal':
test-signal.c:141: warning: implicit declaration of function `memset'
gcc `pkg-config --cflags glib-2.0` `pkg-config --cflags gobject-2.0` -Wall -g -c test-signal-test.c
gcc `pkg-config --libs glib-2.0` `pkg-config --libs gobject-2.0` test-signal.o test-signal-test.o -o test-signal
rm test-signal.c

Notice the "rm test-signal.c": make removes the files it generated itself, so when you update test-signal.gob, it will tell gob to reconstruct the c file, otherwise the c file wouldnt get updated.

Now you should be able to execute ./test-signal.

Ok, we got a nice Makefile now, but it took some time to write it, isn't it? And we don't have a "normal" FOSS install method like ./configure, make, make install...

Well, that's what we are going to do now. The following stuff is much easier than writing Makefiles by hand (you know, FOSS devs are lazy people ;-)), but it is useful to know how Makefiles are formatted, and how they work, though.

Ok, time for some really 1337 (couldn't stop it) stuff: introducing GNU Autotools.

Using the GNU Autotools to build your project

Didn't you ever want to be able to write such a neat ./configure script yourself? Here's how to do it with our sample project :-)

Autotools consist of a bunch of utilities, most of them starting with "auto" (duh). There's autoconf to generate a "configure" script from a file you provide, there's the automake script that creates Makefile's for you (actually, it does not create Makefiles. Read on), and many more.

This is how it works: the configure script will look up a bunch of stuff for you (or you provide it using --with-foo=... etc), it will do some tests to figure out whether the project should compile and run cleanly on your system, and in the end it will generate some files.
A boilerplate for these generated files should be provided by you, called "thefile.in". The generated file will be "thefile" then. Inside "thefile.in", you can use variables like these: "@FOO@", which will be substituted by the configure script.
That's the system automake uses: you write a simple Makefile.am file (read on on how to do this), automake generates a long and difficult Makefile.in file, which gets processed by configure to create the final Makefile.

GNU Autotools have some strict rules (although it is possible, but not advisable, to get around them). Source files should be in the src/ subdirectory, and some files are required in the root directory of the project. We'll find out which ones these are later on.

Let's get started by creating our initial project directory layout:

#Go into an empty directory
mkdir src
cd src
cp /foo/bar/test-signal.gob ./
cp /foo/bar/test-signal-test.c ./
cd ..

Initial task: once more, figure out what the required dependencies are. As mentioned in the first part, we need gobject-2.0 and glib-2.0. Next to this, we need a working C compiler.

We can start writing a configure.in file now, in the root dir of our project, which will be processed by autoconf to generate out ./configure script. Here's what it could look like, comments (starting with "dnl") inline:

dnl Register ourselves to autoconf, giving the main source file
AC_INIT(src/test-signal-test.c)

dnl Init Automake, giving the program name and version. More parameters (author and author's email) are optional
AM_INIT_AUTOMAKE(TestSignal, 0.1)
dnl Enable maintainer mode (debugging flags etc)
AM_MAINTAINER_MODE

dnl Check whether we got a good C compiler. Variable "CC" will be defined and expanded in the .in files
AC_PROG_CC

dnl GOB2 macro, to check whether gob version >=x.y.z (here >=2.0.0) is found. Variable "GOB2" will be substituted/expanded
GOB2_CHECK([2.0.0])

dnl Use built-in macro's to query pkg-config. First parameter is a variable name we'll use later on, second is the package to check for (with optional minimal version), third is the thing to do if the package is found, fourth if not
PKG_CHECK_MODULES(GLIB, glib-2.0, have_glib=true, have_glib=false)
if test "x${have_glib}" = "xfalse" ; then
        AC_MSG_ERROR([No Glib package information found])
fi
dnl So glib-2.0 is found. Remember the first parameter in the previous command, GLIB? Well, GLIB_CFLAGS now contains the output of `pkg-config --cflags glib-2.0`, same thing for GLIB_LIBS with --libs instead of --cflags
dnl AC_SUBST tells configure to substitute the given value in the provided .in files
AC_SUBST(GLIB_CFLAGS)
AC_SUBST(GLIB_LIBS)

dnl Same thing for gobject-2.0
PKG_CHECK_MODULES(GOBJECT, gobject-2.0, have_gobject=true, have_gobject=false)
if test "x${have_gobject}" = "xfalse" ; then
        AC_MSG_ERROR([No GObject package information found])
fi
AC_SUBST(GOBJECT_CFLAGS)
AC_SUBST(GOBJECT_LIBS)

dnl Here we tell the configure script which files to *create*, so we leave out the .in part
AC_OUTPUT([
        Makefile        \
        src/Makefile
])

Currently Makefile.in and src/Makefile.in don't exist yet, they will be created by automake later on.

This file is a very simple one, it can be a tedious job to create complex configure.in files :-/

Next thing: Makefile.am files, which will be processed by automake to create the Makefile.in's.
In the project root dir, this file can be very simple:

SUBDIRS = src

We just define the subdir(s) of the current dir.
src/Makefile.am is a little more complex:

INCLUDES = $(GLIB_CFLAGS) $(GOBJECT_CFLAGS)

bin_PROGRAMS = test-signal

test_signal_SOURCES = test-signal.c test-signal-test.c

test_signal_LDADD = $(GLIB_LIBS) $(GOBJECT_LIBS)

%.c %.h %-private.h: %.gob
        @GOB2@ $<

Some explanation:

• "INCLUDES" is a variable that will be given to every compile call to "$(CC)", here we only need the glib and gobject includes. Remember these values will be setted by the configure script.
• "bin_PROGRAMS" is a variable defining the names of all targets we want to be installed as an executable in the bin directory (/usr/local/bin if no prefix is given to ./configure)
• "test_signal_SOURCES" is a variable defining which files are needed to make the target "test-signal". Notice "-" being replaced with "_" here, which is a common thing in automake files.
• "test_signal_LDADD" defines which parameters to offer to the linker when linking the input object files to the "test-signal" executable. We could have used a global "LDADD" variable here, like "INCLUDES", or have made "INCLUDES" non-global by using "test_signal_INCLUDES". In large projects this can make a big difference.
• The last part is the one we also used in our hand-written Makefile. It will be put like this in the resulting Makefile.in by automake, so make will know how to build .c and .h files from a .gob file.
  Automake puts all non-automake-specific stuff that's in Makefile.am in the resulting Makefile.in.

Now everything is done. At least, almost ;-) We still need to call out autotool scripts.
It's easy to do this from a shell script. Most projects call this script "autogen.sh", so will we. This script can be fairly complex, ours will be braindead easy:

#!/bin/bash
aclocal
autoconf
automake -a

As you can see, we first call aclocal (part of the automake package), then autoconf, then automake with the -a flag.
Why "aclocal"? Autoconf, which creates a configure script out of a configure.in file, uses M4, a complex macro system, to do this. aclocal copies the necessary macro definitions for your system to the right place.

Run this script, or enter the commands by hand. aclocal can give a lot of warnings, don't bother about these.
If this is the first time you run the script, you will see some automake errors in the end, and the script execution will fail:

Makefile.am: required file `./NEWS' not found
Makefile.am: required file `./README' not found
Makefile.am: required file `./AUTHORS' not found
Makefile.am: required file `./ChangeLog' not found

These are the required files I mentioned earlier. We'll just touch them for now so they "exist", although contain no usefull information:

touch NEWS README AUTHORS ChangeLog

Now restart the autogen.sh script, everything should pass now.

If you take a look now, you'll see a couple of Makefile.in files are created now (who will be converted into Makefiles by configure, remember?), and the configure script.

Let's take our work to the test, and run ./configure. You'll see the usual output, and everything should pass fine (we don't have a lot of prerequisites :-))

Now let's try if our project builds fine:

make

Jay, lot of compiler commands, no errors. Fun :-)

To finish, test whether the executable works:

cd src
./test-signal

The same output as before appears, we're member of the Autotools User Group now ;-)

That's it for now, more stuff should follow: "The Glib mainloop", "Debugging using GDB", and a from-scratch "program" we'll create. I only lack the time to write everything ;-)

I'd like to convert these articles to some format so they can be used outside this blog (converted to PDF, HTML,...). Docbook seems to be a good format to do this, I need to get used to it first, though. If someone knows a good Docbook editor (next to Conglomerate), please let me know.