The Makefile utility
Motivation
Small programs single file
“Not so small” programs :
– Many lines of code– Multiple components– More than one programmer
Mozilla
Mozilla Directory Structure
One very large directory tree26,000 files2,500 subdirectories240 file types
One very large directory tree26,000 files2,500 subdirectories240 file types
Mozilla Directory References
1,521 directory referencesAvg. fan-in/fan-out 12Max fan-in=100Max fan-out=42Median fan-in=5Median fan-out=12
1,521 directory referencesAvg. fan-in/fan-out 12Max fan-in=100Max fan-out=42Median fan-in=5Median fan-out=12
Build-levelFile (atomic entity)
–Source–Documentation
Directory tree (container) = source tree
Build process–To build/install software–Driven by make/ANT/…
Configuration process–To control build process–Driven by configure/configuration files/…
Build-level interfaces–Build interface–Configuration interface
Mozilla Build Level
1,350 Makefiles40,000 LOC build instructions16,000 LOC configurationCyclic dependenciesTwo-phase build processCentralized build/configuration knowledgeCode duplicationComponent implementations scattered
1,350 Makefiles40,000 LOC build instructions16,000 LOC configurationCyclic dependenciesTwo-phase build processCentralized build/configuration knowledgeCode duplicationComponent implementations scattered
Motivation – continued
Problems:
– Long files are harder to manage
(for both programmers and machines)– Every change requires long compilation– Many programmers can not modify the
same file simultaneously– Division to components is desired
Motivation – continued
Solution : divide project to multiple files Targets:
– Good division to components– Minimum compilation when something is
changed– Easy maintenance of project structure,
dependencies and creation
Project maintenance
Done in Unix by the Makefile mechanism A makefile is a file (script) containing :
– Project structure (files, dependencies)– Instructions for files creation
The make command reads a makefile, understands the project structure and makes up the executable
Note that the Makefile mechanism is not limited to C programs
Project structure
Project structure and dependencies can be represented as a DAG (= Directed Acyclic Graph)
Example :– Program contains 3 files– main.c., sum.c, sum.h– sum.h included in both .c files– Executable should be the file sum
sum (exe)
sum.omain.o
sum.csum.h sum.hmain.c
Make: Header Dependencies What if …
– Sensor.cc changes?
– Sensor.h changes?
– Robot.h changes?
Pattern rule ignores header dependencies!
Requires unnecessary “make clean; make”
Let gcc figure out the header dependencies for us!
The –MM option produces make rules in a .d file which we can then include in the Makefile
Sensor.h Sensor.cc
Robot.h Robot.cc
robotest.cc
makefile
sum: main.o sum.o
gcc –o sum main.o sum.o
main.o: main.c sum.h
gcc –c main.c
sum.o: sum.c sum.h
gcc –c sum.c
Rule syntax
main.o: main.c sum.h
gcc –c main.c
tab
dependency action
Rule
Equivalent makefiles
.o depends (by default) on corresponding .c file. Therefore, equivalent makefile is:
sum: main.o sum.ogcc –o sum main.o sum.o
main.o: sum.hgcc –c main.c
sum.o: sum.hgcc –c sum.c
Equivalent makefiles - continued
We can compress identical dependencies and use built-in macros to get another (shorter) equivalent makefile :
sum: main.o sum.o
gcc –o $@ main.o sum.o
main.o sum.o: sum.h
gcc –c $*.c
make operation
Project dependencies tree is constructed Target of first rule should be created We go down the tree to see if there is a target
that should be recreated. This is the case when the target file is older than one of its dependencies
In this case we recreate the target file according to the action specified, on our way up the tree. Consequently, more files may need to be recreated
If something is changed, linking is usually necessary
make operation - continued
make operation ensures minimum compilation, when the project structure is written properly
Do not write something like:
prog: main.c sum1.c sum2.c
gcc –o prog main.c sum1.c sum2.c
which requires compilation of all project when something is changed
Make operation - example
File Last Modified
sum 10:03
main.o 09:56
sum.o 09:35
main.c 10:45
sum.c 09:14
sum.h 08:39
Make operation - example
Operations performed:
gcc –c main.c
gcc –o sum main.o sum.o
main.o should be recompiled (main.c is newer).
Consequently, main.o is newer than sum and therefore sum should be recreated (by re-linking).
Another makefile example# Makefile to compare sorting routines BASE = /home/blufox/baseCC = gccCFLAGS = -O –WallEFILE = $(BASE)/bin/compare_sortsINCLS = -I$(LOC)/includeLIBS = $(LOC)/lib/g_lib.a \ $(LOC)/lib/h_lib.aLOC = /usr/local
OBJS = main.o another_qsort.o chk_order.o \ compare.o quicksort.o
$(EFILE): $(OBJS)@echo “linking …”@$(CC) $(CFLAGS) –o $@ $(OBJS) $(LIBS)
$(OBJS): compare_sorts.h$(CC) $(CFLAGS) $(INCLS) –c $*.c
# Clean intermediate filesclean:
rm *~ $(OBJS)
Example - continued
We can define multiple targets in a makefile Target clean – has an empty set of
dependencies. Used to clean intermediate files.
make– Will create the compare_sorts executable
make clean– Will remove intermediate files
Passing parameters to makefile
We can pass parameters to a makefile by specifying them along with their values in the command line.
For example: make PAR1=1 PAR2=soft1
will call the makefile with 2 parameters: PAR1 is assigned the value “1” and PAR2 is assigned the value “soft1”. The same names should be used within the makefile to access these variables (using the usual “$(VAR_NAME)” syntax)
Passing parameters - continued
Note that assigning a value to a variable within the makefile overrides any value passed from the command line.
For example: command line : make PAR=1
in the makefile:PAR = 2
PAR value within the makefile will be 2, overriding the value sent from the command line
Conditional statements
Simple conditional statements can be included in a makefile.
Usual syntax is:
ifeq (value1, value2)
body of if
else
body of else
endif
Conditional statements - example
sum: main.o sum.ogcc –o sum main.o sum.o
main.o: main.c sum.hgcc –c main.c
#deciding which file to compile to create sum.oifeq ($(USE_SUM), 1)
sum.o: sum1.c sum.hgcc –c sum1.c –o $@
elsesum.o: sum2.c sum.h
gcc –c sum2.c –o $@
endif
Make: Advanced Options Text manipulation functions
– $(patsubst pattern,replacement,text)– $(patsubst %.o,%.cc,<list of objfiles>)
Pattern rules– Uses a pattern in the target with % as wildcard– Matched % can be used in dependencies as well– Simple Example:
%.o : %.cc <tab>command …
Pattern rules with automatic variables– $@ full target name– $< first dependency– $* string which matched % wildcard– Advance Example:
%.o : %.cc <tab>$(CC) $(CCFLAGS) –c $< $(INCPATHS)
Make: A Simple ExampleCC=g++ # Compiler to useFLAGS=-g # Compile flagsMASLAB_ROOT=maslab-software # Maslab software root directoryLIB_DIR=$(MASLAB_ROOT)/liborc # orc-related library directoryINC_DIR=$(MASLAB_ROOT)/liborc # orc-related include directoryLIBS=-lm –lpthread -lorc # Library files
all : helloworld
helloworld.o : helloworld.cc$(CC) $(FLAGS) –c $*.cc –o $@
helloworld: helloworld.o $(CC) -o helloworld helloworld.o $(LIBS)
clean: rm -f *.o helloworld
Make: Example MakefileMASLABROOT = /mnt/maslab/software/maslab-software-current
#This is the list of places to look for include filesINCPATHS = -I$(MASLABROOT)/libs/liborc \
-I$(MASLABROOT)/libs/libim \ -I$(MASLABROOT)/libs/libbotserver \ -I/opt/intel/ipp/include
#This is the list of places to look for librariesLIBPATHS = -L$(MASLABROOT)/libs/liborc \
-L$(MASLABROOT)/libs/libim \ -L$(MASLABROOT)/libs/libbotserver \ -L/opt/intel/ipp/sharedlib
#This is the names of the librariesLIBS = -lippipx -lippcvpx -lim -lorc -lbotserver -lm -lpthread -ljpeg -lpng
#This is the compiler to useCC = g++
# This is your c++ file extension (usually cc or cpp)CCEXT = cc
Make: Example Makefile (cont)CCFLAGS = -g -Wall
# This rule builds everything. You should put the names of all your
# programs in this list.
all : robotest
# This rule says how to turn any .cpp file into a .o file.
%.o : %.$(CCEXT)
$(CC) $(CCFLAGS) -c $< $(INCPATHS)
#----------------------------------------------------------------------
# robotest
PROGRAM_NAME = robotest
PROGRAM_OBJS = robotest.o BumpSensor.o Robot.o
GLOBAL_OBJS += $(PROGRAM_OBJS)
JUNK += $(PROGRAM_NAME)
$(PROGRAM_NAME): $(PROGRAM_OBJS)
$(CC) $(PROGRAM_OBJS) -o $(PROGRAM_NAME) $(LIBS) $(LIBPATHS)
chmod a+x $(PROGRAM_NAME)
Extending Example Makefile
#----------------------------------------------------------------------
# Sensor Incremental Test
PROGRAM_NAME = sensor-test
PROGRAM_OBJS = sensor.t.o BumpSensor.o
GLOBAL_OBJS += $(PROGRAM_OBJS)
JUNK += $(PROGRAM_NAME)
$(PROGRAM_NAME): $(PROGRAM_OBJS)
$(CC) $(PROGRAM_OBJS) -o $(PROGRAM_NAME) $(LIBS) $(LIBPATHS)
chmod a+x $(PROGRAM_NAME)
So to add a new executable program, simply cut and paste the robotest section and enter your own PROGRAM_NAME and PROGRAM_OBJS
Extending example MakefileDEPENDS += $(patsubst %.o, %.d, $(GLOBAL_OBJS))
JUNK += $(DEPENDS)
include $(DEPENDS)
depend : $(DEPENDS)
depend.$(CCEXT) = set -e; $(CC) $(CCFLAGS) -MM $(DEFS) $(INCPATHS)
depend.filt = sed 's/\($*\)\.o[ :]*/\1.o $@ : /g' > $@; [ -s $@ ] || rm -f $@
%.d: %.$(CCEXT)
$(depend.cc) $< | $(depend.filt)
#======================================================================
# Miscellaneous Rules
#----------------------------------------------------------------------
# This rule cleans your directory. You could also add commands here
# to remove program files and any other files that should be cleaned
clean:
rm -f $(JUNK) *~ *.o core.[0-9]* core
ANT:Another Nice Tool
What is Ant?
Java-based Build tool from Apache De facto standard for building, packaging, and
installing Java applications Accomplishes same objectives that make does on
Unix based systems Files are written in XML
Why Ant?
Many claim.. That – Unlike makefiles, Ant files work cross platform
- No need for multiple, complex makefiles depending on the operating system.- Tasks declared in platform independent way; Ant engine translates to OS specific commands.
But still need to know (potentially) complex path/install data. So not as general as they claim
Easy to create own Ant “tasks”, in addition to core tasks
Installing Ant
Download Ant binary distribution from:
http://ant.apache.org/bindownload.cgi Set ANT_HOME to where you installed Ant Include $ANT_HOME/bin in PATH Make sure JAVA_HOME is set to point to JDK
Running Ant
Type “ant” at the command line Automatically looks for build.xml file in current
directory to run Type “ant –buildfile buildfile.xml” to specify
another build file to run.
Ant Output
Sample build.xml<project name="MyProject" default="dist" basedir="."> <property name="src" location="src"/> <property name="build" location="build"/> <property name="dist" location="dist"/>
<target name="init"> <tstamp/> <mkdir dir="${build}"/> </target>
<target name="compile" depends="init" > <javac srcdir="${src}" destdir="${build}"/> </target>
<target name="dist" depends="compile" > <mkdir dir="${dist}/lib"/> <jar jarfile="${dist}/lib/MyProject-${DSTAMP}.jar" basedir="${build}"/> </target>
<target name="clean" > <delete dir="${build}"/> <delete dir="${dist}"/> </target></project>
Ant Overview: Project
Each build file contains exactly one project and at least one target
Project tags specify the basic project attributes and have 3 properties:
- name
- default target
- basedir Example: <project name=“MyProject” default=“build” basedir=“.”>
Ant Overview: Targets
Target is a build module in Ant Each target contains task(s) for Ant to do One must be a project default Overall structure of targets:
<target name="A"/>
<target name="B" depends="A"/>
<target name="C" depends="B"/>
<target name="D" depends="C,B,A"/>
Ant Overview: Tasks
Each target comprises one or more tasks Task is a piece of executable Java code (e.g.
javac, jar, etc) Tasks do the actual “build” work in Ant Ant has core (built in) tasks and the ability to
create own tasks
Ant Overview: Tasks
Example: <target name="prepare" depends="init“ >
<mkdir dir="${build}" /> </target>
<target name="build" depends="copy" >
<javac srcdir="src" destdir="${build}">
<include name="**/*.java" />
</javac> </target>
Ant Overview: Core Tasks
javac – Runs the Java Compiler java – Runs the Java Virtual Machine jar (and war) – Create JAR files mkdir – Makes a directory copy – Copies files to specified location delete – Deletes specified files cvs – Invokes CVS commands from Ant
Ant Overview: Writing Own Task
Create a Java class that extends org.apache.tools.ant.Task For each attribute, write a setter method that is
public void and takes a single argument Write a public void execute() method, with no
arguments, that throws a BuildException -- this method implements the task itself
Ant Overview: Properties
Special task for setting up build file properties: Example:
<property name=“src” value=“/home/src”/> Can use ${src} anywhere in build file to denote
/home/src Ant provides access to all system properties as if
defined by the <property> task
Ant Overview: Path Structures
Ant provides means to set various environment variables like PATH and CLASSPATH.
Example of setting CLASSPATH:
<classpath>
<pathelement path="${classpath}"/>
<pathelement location="lib/helper.jar"/>
</classpath>
Command Line Arguments
-buildfile buildfile – specify build file to use targetname – specify target to run (instead of
running default) -verbose, -quiet, -debug – Allows control over
the logging information Ant outputs -logger classname – Allows user to specify their
own classes for logging Ant events
IDE Integration
Eclipse, NetBeans, JBuilder, VisualAge, and almost any other Java IDE has Ant integration built-in to the system
Refer to each IDE’s documentation for how to use Ant with that IDE
Web Development with Ant
Tomcat comes with special Ant tasks to ease Web application development and deployment
Copy $TOMCAT_HOME/server/lib/catalina-ant.jar to $ANT_HOME/lib
Ant tasks for Tomcat:
- install
- reload
- deploy
- remove
Documentation/References
Good tutorial for makefiles http://www.gnu.org/manual/make-3.80/make.html
Good tutorial for cmakef
http://www.cmake.org/HTML/Documentation.html ANT User Manual:
http://ant.apache.org/manual/index.html
Sun’s Web development tutorial (Ant and JSPs):http://java.sun.com/webservices/docs/1.2/tutorial/doc/GettingStarted3.html
Standardized Build Interface
Different build systems exist, e.g., –make, ANT, shell scripts, IDE
Different software systems require different build actions, e.g., –make
–make bootstrap, make, make install
make cleanmake allmake checkmake (un)installmake dist
make cleanmake allmake checkmake (un)installmake dist
The real Problem
How do we handle platform specific issues?– Providing a different Makefile for each architecture– Using Autoconf, Automake and Libtool
The installer needs only– Bourne shell– C compilers– Make program
Standardized Configuration InterfaceDifferent mechanisms exist to control software
construction, e.g.,–configuration files, configuration tools, Makefile
editing
Standardized configuration interface to enable uniform compile-time configuration:
configure --helpconfigure --prefix=/usrconfigure --with-aterm=/usr/libconfigure --with-optimization=trueconfigure --with-debug=false
configure --helpconfigure --prefix=/usrconfigure --with-aterm=/usr/libconfigure --with-optimization=trueconfigure --with-debug=false
Explicit Context Dependencies
Dependencies on build-level components are declared in configuration interfaces
> configure --help … --with-aterm=DIR use ATerm Library at DIR --with-sglr=DIR use SGLR Parser at DIR …
> configure --help … --with-aterm=DIR use ATerm Library at DIR --with-sglr=DIR use SGLR Parser at DIR …
Independent Deployment
Build-level components are deployed as packages
A package is a versioned release of a build-level component, e.g.,
firefox-1.0.tar.gz
Packages are published on web/ftp sites
Third-party Composition
A configuration interface enables late-binding of dependencies
Compositions are thus not predefined
and can be defined by a third party
> configure --with-aterm=/usr/local/aterm --with-sglr=/usr/local/sglr --with-…=
> configure --with-aterm=/usr/local/aterm --with-sglr=/usr/local/sglr --with-…=
Some advantages when using GNU autotools
The installation of a program is straightforward:
./configure; make; make install
This procedure checks for system parameters, libraries, location of programs, availability of functions and writes a Makefile
./configure supports many options to overwrite defaults settings
GNU autoconf
autoscan
configure.scan
configure.ac(configure.in)
autoconf
configure
Source Code
aclocal
GNU automake
Makefile.am
automake
Makefile.in
configure
Makefile
configure.ac dnl Comment … …
AC_INIT(main.c)
AM_INIT_AUTOMAKE(project_name, 1.2.8)
AM_PROG_LIBTOOL it supports libtool and shared libraries
AC_PROG_CC (or AC_PROG_CXX) it locates the C (C++) compiler
AC_HEADER_STDC it checks for standard headers
AC_CHECK_HEADERS(sys/time.h /header.h) it checks for headers availability
AC_CHECK_LIB(crypto SSLeay_version) it checks for libraries availability
AC_CHECK_FUNCS(ctime) it checks for functions availability
AC_PROG_INSTALL it checks for BSD compatible install utility
AC_OUTPUT([Makefile])
Makefile.am
bin_PROGRAMS = foo/configure --prefix=… (default /usr/local)
foo_PROGRAMS=foo.c foo.h
noist_PROGRAMS=test(make compiles, make install does nothing)
EXTRA_DIST=disclaimer.txt
Example foo.c :
#include <stdio.h>main(){ printf(“Cum grano salis\n");}
Makefile.am :bin_PROGRAMS = foofoo_SOURCES = foo.c
configure.ac :AC_INIT(foo.c)AM_INIT_AUTOMAKE(latin_words, 0.9)AC_PROG_CCAC_HEADER_STDCAC_PROG_INSTALLAC_OUTPUT([Makefile])
Summary
Source Code, configure.ac, Makefile.am autoscan; aclocal; autoconf Create NEWS README AUTHORS ChangeLog
automake –add-missing ./configure; make; make dist Result: project_name-2.10.tar.gz
aclocal.m4 autom4te-2.53.cache ChangeLog config.status configure.in COPYING install-sh Makefile.am missing NEWS README AUTHORS autoscan.log config.log configure configure.scan INSTALL Makefile.in mkinstalldirs code.c
References GNU Autoconf, Automake, and Libtool http://sources.redhat.com/autobook/autobook/autobook_toc
.html
GNU Autoconf Manualhttp://www.gnu.org/manual/autoconf
GNU Automake Manualhttp://www.gnu.org/manual/automake
GNU Libtool Manualhttp://www.gnu.org/manual/libtool
Learning the GNU development toolshttp://autotoolset.sourceforge.net/tutorial.html
The GNU configure and build systemhttp://www.airs.com/ian/configure/configure_toc.html
GNU macro processor (GNU m4)http://www.gnu.org/manual/m4-1.4/m4.html