Development ToolsFor HPC Applications

Deniz Savas, Michael GriffithsCorporate Information and Computing ServicesThe University of SheffieldEmail m.griffiths@sheffield.ac.uk, d.savas@sheffield.ac.uk

• Building Applications– Gnu compilers, g++, g++, g77– Portland compilers, pg++, pgf77

• Calling fortran from C and C from fortran

• Using, Building and installing libraries• Using the make utility• The Eclipse Development Environment• Links

Outline

Compilers

Language GNU Portland

C gcc pgcc

C++ g++ pgCC

Fortran 77 g77 pgf77

Fortran90/95 pgf90

Invoking the Compiler

• Compiling FORTRAN Programs– pgf77 –o mycode [options] mycode.f

• Compiling c/c++ Programs– pgcc –o mycode [options] mycode.c

Options Used with Both gnu and Portland Compilers

Option Action-c Compile, do not link.

-o exefile Specifies a name for the resulting executable.

-g Produce debugging information (no optimization).

-Ilibrary_name (lower case L)

Link the given library into the program.

e.g. include math library by using option -lm

-ldirectory-name (upper case I)

Add directory to search path for include files

-O N Set optimisation level to N

-Dmacro[=defn] Define a macro

Options Used with Portland Compilers

Option Action-tp k8-64 Specify target processor type to be opteron processor

running 64 bit system (Portland only).

-Mvect=sse2 Vectorizer option, turn on streaming SIMD extensions (SSE) and SSE2 instructions. SSE2 instructions operate on 64bit floating point data

-Mvect=prefetch Generate prefetch instructions

-Mconcur Auto parallelization option

-fastsse Optimal set of options for processors supporting SSE/SSE2, Streaming SIMD Extensions

-fast Full optimisation with function unrolling and code reordering (Portland only).

-g77 libs Link time option allowing object files generated by g77 to be linked into programs (may cause problems with parallel libraries)

-Mbounds Check arrays for out of bounds access

Options Used with gnu CompilersOption Action TO CHECK--version Return version information

-mtune=cpu-type

-march=cpu-type

I386-I64 processor specification options.

-mmmx -msse

-msse2

Options to enable sse/sse2 or mmx(windows)

-Wall Show all warnings

-Wno-deprecated Switch off warnings about use of deprecated function (c++ only)

Linking a FORTRAN application with NAG libraries• NAG best and most comprehensive library of

numerical computing routines available• Mark20 on iceberg use –lnag and –lacml with the

pgf77 or pgf90 compiler• Example

– pgf90 myprogf90 –lnag –lacml

• See comprehensive documentation at– https://iceberg.shef.ac.uk/docs/nag/index.html

The AMD core math libraries (acml)

• ACML consists of the following main components:– A full implementation of Level 1, 2 and 3 Basic Linear

Algebra Subroutines (BLAS), with key routines optimized for high performance on AMD Opteron™ processors.

– A full suite of Linear Algebra (LAPACK) routines. As well as taking advantage of the highly-tuned BLAS kernels, a key set of LAPACK routines has been further optimized to achieve considerably higher performance than standard LAPACK implementations.

– A comprehensive suite of Fast Fourier Transforms (FFTs) in both single-, double-, single-complex and double-complex data types.

Using the acml libraries

• Building an application using the portland compilers– pgcc myapp.c -I/opt/acml-pg2.6.0/pgi64/include -L/opt/acml-

pg2.6.0/pgi64/lib -lm –lacml

• Building an application using the gnu compilers– gcc myapp.c -I/opt/acml-gnu2.6.0/gnu64/include

-L/opt/acml-gnu2.6.0/gnu64/lib -lm –lacml

• Examples– Documentation at

https://iceberg.shef.ac.uk/docs/acmldoc/html/index.html– See

http://www.shef.ac.uk/wrgrid/documents/hpc/numlibs.html#acmlexamples

Building Large Applications

• Typically compile program using– g++ –o myprog myprog.c –lm –g

• Large programs– Modularized– Combine into a single executable

• Building large applications is a multi step process– Compile each source file– Link resulting objects into an executable

Example Multi Source Program:1

• To build the Monte-carlo model, mc, we do the following.– g++ –c –g mc.cpp– g++ –c –g mc_system.cpp– g++ –c –g mc_particle.cpp– g++ –c -g mc_statistics.cpp– g++ –o mc mc.o mc_system.o mc_particle.o mc_statistics.o

–lm• Note: only one of the sources has a main function

Example Multi Source Program:2

• If mc_system.cpp is edited we don’t need to recompile– mc_statistics, mc_particle or mc

• Rebuild the application as follows– g++ –c –g mc_system.cpp

– g++ –o mc mc.o mc_system.o mc_particle.o mc_statistics.o –lm

• Automate these steps using make

Libraries

• Libraries are packaged collections of object files– Standard library contains printf… etc..– Maths library contains sin, cos etc..

• Specify additional libraries with –l<name>– Only standard library is provided automatically

• To compile a program with a maths library– g++ –c myprog myprog.c -lm

Building your own library

• Benefits of building libraries– Share standardised functions with community

– Separate functionality from detailed code

– Good way of packing up your most useful routines and reusing them

• How to build– Build libraries using

– Named as lib<name>.a or lib<name>.so

– http://www-cs.canisius.edu/PL_TUTORIALS/C/C-UNIX/libraries

Example

• Example my util library– g++ -c vec.cc

• Generates vec.o– g++ -c mat.cc

• Generates mat.o• Add object files to library

– ar r myutillib.a vec.o– ar r mylibutil.a mat.o

• Don’t use –l for your own libraries link as follows– g++ myprog.cc mylib.a –o myprog

Installing a Library

• General steps– Download and uncompress source– Read documentation and build e.g. using configure– make and make install to build and install– Update your environment

• Set LD_LIBRARY_PATH

• Compile with -lMyNewLib

Using the Make Utility

• Used to compile and link programs• Makefile tells make how to perform link and

compilation• Consists of rules with the following shape

target …… : dependencies ……

command

……………

make

• target name of file generated by a program• dependency used as input to create target• Target files are created whenever a dependency has changed• Commands can include

– cc, CC, g++, f77, f95, mpf77

• make• make clean

make target

• Perform actions to obtain a target from a set of dependecies

• Make checks when dependencies last updated

target : dependencies

rule

Simple Makefile ….. almost trivial!game : game.o

gcc -o game game.o game.o : game.c

gcc -c game.c

clean :rm game game.o

Simple Makefile

• Generates executable called game from a single source file called game.c

• Has a sequence of rules– game

• Rule for building target executable file

– game.o• Rule for building object files

– clean• Rule for cleaning executable and object files

Make multiple source file projectproject : main.o data.o io.o

CC -o project main.o data.o io.o main.o : main.c io.h data.h

CC -c main.c

data.o : data.c io.h data.hCC -c data.c

io.o : io.c io.hCC -c io.c

clean :rm project main.o data.o io.o

Hints for Building Makefiles

• Use # at the start of a line for comments• Use \ at the end of a line for line continuation• The line defining the rule that follows the definition of

target and dependencies should normally be indented using a tab character and NOT whitespace characters

Makefile with implict rules for compiling a static library

objects = vec.o vecpair.o mat.o

flags = -fast -tp k8-64 libmyutil.a : $(objects)

ar -r -o myutil.a $(objects) $(flags)

vec.o : vec.cpgCC -c vec.c $(flags)

vecpair.o : vecpair.cpgCC -c vecpair.c $(flags)

mat.o : mat.cpgCC -c mat.c $(flags)

clean :rm myutil.a $(objects)

Macros Used with Makefiles

$@ Full name of the current target . $< The source file of the current (single) dependency . $* The part of a filename which matched a suffix rule. $? The names of all the dependencies newer than the target separated by spaces. $^ The names of all the dependencies separated by spaces, but with duplicate names removed.

Suffixes

• Make uses a special target, named .SUFFIXES to allow you to define your own suffixes.

• For example, the dependency line:.SUFFIXES: .foo .bar– tells make that you will be using these special suffixes to

make your own rules.

Custom Suffix Rule

• Similar to how make already knows how to make a .o file from a .c file, you can define rules in the following manner:

.foo.bar: tr '[A-Z][a-z]' '[N-Z][A-M][n-z][a-m]' < $< > $@ .c.o: $(CC) $(CFLAGS) -c $<

• The first rule allows you to create a .bar file from a .foo file. (Don't worry about what it does, it basically scrambles the file.)

• The second rule is the default rule used by make to create a .o file from a .c file.

Makefile with suffix rule

objects = blastest.o

flags = -fast -tp k8-64

mk4 : $(objects)pgCC -o mk4 $(objects) $(flags)

.c.o:pgCC -c $(flags) $<

clean :rm mk4 $(objects)

Using Eclipse

• Advantages• Starting

Using Eclipse: Advantages

• Open source• Available for many platforms

– Windows requires cygwin and gnu development tools g++, g77, gdb, gmake, stl etc..

• Use to develop wide variety of applications in a single development environment– e.g. c, c++, f77, f90, java

Eclipse features

• Perspectives for java, C/C++, debug and soon fortran development

• Multiple projects• Browsers

– Help, members, types, namespaces

• Interactive debugging• Build projects using make or ant

Starting eclipse

• Type “eclipse”• Requests directory for workspace

– N.B. Sometimes necessary to start eclipse using– eclipse -vm $JAVA_HOME/jre/bin/java

Eclipse C/C++ Perspective Layout

Creating a new project

• Managed make project– Helloworld

• Standard make project– Hellotest– Provide a make file

Creating a standard make project

Editing Project Settings

Finishing Steps

Debug WindowSwitch

between perspecti

ves

Debug window

s

Debug steppingcontrols

Running the Debugger

Setting and Modifying Breakpoint Properties

Right click here to toggle breakpoi

nt

Right click here

to edit breakpoin

t propertie

s

Links

• http://www.eclipse.org/• http://www-106.ibm.com/developerworks/library/os-e

cc/– A useful tutorial on the eclipse cdt

• http://www.cplusplus.com– Very useful reference section