gaudi - gnu automatic differentiation euroad workshop - ralf leidenb… · gaudi - gnu automatic...

GAuDi - GNU Automatic Differentiation

Ralf Leidenberger |November 27, 2009 |Faculty for Mathematics and EconomicsUniversity of Ulm

A multi programming languagesource transformation tool

Page 2 | Idea | November 23, 2009 |

Idea

Structure of GAuDiPreprocessingDifferentiationPostprocessing

ExampleFortran-ExampleC-Example

Outlook

Idea of GAuDiGAuDi stand for GNU Automatic Differntiation

Goals of GAuDiI GNU AD-ToolI mostly independent of the programming language

I transform in language independent formatI efficient derivative calculating

I function handlingI modular structure

I simple changeable differentiation & back write unit

Realization of GAuDiI written in perlI using the gcc GNU Compiler Collection

with gcc -c -fdump-tree-cfg-uid-lineno

Page 4 | Structure of GAuDi | November 23, 2009 |

Idea

Outlook

Page 5 | Structure of GAuDi | November 23, 2009 |

Main structure of GAuDi

Page 6 Preprocessing | Structure of GAuDi | November 23, 2009 |

GAuDi Preprocessing

CompileI create the programming language independent codeI using the gcc GNU Compiler CollectionI with the option -c -fdump-tree-cfg-uid-linenoI and by fortran additional with -c -fdump-parse-tree

Page 6 Preprocessing | Structure of GAuDi | November 23, 2009 |

GAuDi Preprocessing

CompileI create the programming language independent codeI using the gcc GNU Compiler CollectionI with the option -c -fdump-tree-cfg-uid-linenoI and by fortran additional with -c -fdump-parse-tree

Page 7 Differentiation | Structure of GAuDi | November 23, 2009 |

GAuDi Main I

SplitI move each function in a separate fileI fragment each function in their block(s)

GAuDi Main I

/Volumes/Data/Vortraege/Slides/GAuDi Basic/Structure/Differentation/block.cfg Page 1

# BLOCK 4, starting at line 56 # PRED: 2 (false)

...

# SUCC: 5 (true) 6 (false)

SplitI move each function in a separate fileI fragment each function in their block(s)

GAuDi Main I

Recursion detectionI build function call graphI if this graph contains cycles user handling

GAuDi Main I

GAuDi Main II

DependentI get a list of all possible independent variablesI choose independent variable

GAuDi Main II

DependentI get a list of all possible independent variablesI choose independent variable

GAuDi Main II

TargetI get a list of all possible target variablesI choose target variable

GAuDi Main III

TreeI search all target variables on the left hand side and set this

to the rootI build the tree by using the pre-treeI set the dependence of the tree elements starting at the

independent variablesI remove unused subtrees (which are not dependent)

GAuDi Main III

TreeI search all target variables on the left hand side and set this

to the rootI build the tree by using the pre-treeI set the dependence of the tree elements starting at the

independent variablesI remove unused subtrees (which are not dependent)

GAuDi Main IV

DifferentiationI add differentiated lines in the treeI each line contains only elementary operationI it is simple to change this unit for second derivatives

GAuDi Main IV

DifferentiationI add differentiated lines in the treeI each line contains only elementary operationI it is simple to change this unit for second derivatives

Page 11 Postprocessing | Structure of GAuDi | November 23, 2009 |

GAuDi Postprocessing

Back writingI separate the tree in the linesI create new source code (language dependent)I merge new lines with the original source codeI remove old function calls and add the new function calls

Page 11 Postprocessing | Structure of GAuDi | November 23, 2009 |

GAuDi Postprocessing

Back writingI separate the tree in the linesI create new source code (language dependent)I merge new lines with the original source codeI remove old function calls and add the new function calls

Page 12 | Example | November 23, 2009 |

Idea

Outlook

Page 13 Fortran-Example | Example | November 23, 2009 |

Fortran-Example I

ExampleI simple and short program with a loopI with function handlingI variable f is targetI variable t is independent

Fortran-Example I

/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/bsp_f.f90 Page 1

program bsp real t real x, x_const real delta real a,b,c real f integer i x_c =1. x=0 delta=0.02 a=1. b=1. c=1. do i=1,10 t = getX(delta,i) x = t f = quad(a,b,c,x) a = t f = quad(a,b,c,x_c) end doend program

function getX(delta,i) result(x) real, intent(in) :: delta integer, intent(in) :: i real ::f x= delta*iend function

function quad(a,b,c,x) result(f) real, intent(in) :: a,b,c,x real :: f f= a*x*x+b*x+cend function

Fortran-Example I

Fortran-Example II/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/bsp_f.f90 Page 1program bsp real t real x, x_const real delta real a,b,c real f integer i x_c =1. x=0 delta=0.02 a=1. b=1. c=1. do i=1,10 t = getX(delta,i) x = t f = quad(a,b,c,x) a = t f = quad(a,b,c,x_c) end doend program

�

First function call quadI variable x is the active

argumentI a, b, c are the inactive

arguments

?

Second function call quadI variable a is the active

argumentI b, c, x c are the inactive

arguments

�

New variablesI differentiated variablesI function help variables

Function calls

I f (x , a, b, c) = ax2 + bx + c,∂f (x,a,b,c)

∂x = 2axx′ + bx ′

∂a = a′x2

I same argument signature butdifferent differentiation in thefunction

�

arguments

?

arguments

�

Function calls

∂x = 2axx′ + bx ′

∂a = a′x2

�

arguments

?

arguments

�

Function calls

∂x = 2axx′ + bx ′

∂a = a′x2

Fortran-Example II

�

arguments

?

arguments

/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/MAIN__.f90 Page 1

program bsp real fct_quad_2 real fct_quad_2_d real fct_quad_3 real fct_quad_3_d real(kind=4) :: t_d real(kind=4) :: x_d real(kind=4) :: f_d real(kind=4) :: a_d real t real x, x_const real delta real a,b,c real f integer i x_c =1. x=0 delta=0.02 a=1. b=1. c=1. do i=1,10 t = getX(delta,i) t_d =1. x = t x_d = t_d fct_quad_2 = quad_2_d(a, b, c, x, x_d, fct_quad_2_d) f = fct_quad_2 f_d = fct_quad_2_d a = t a_d = t_d fct_quad_3 = quad_3_d(a, b, c, x_c, a_d, fct_quad_3_d) f = fct_quad_3 f_d = fct_quad_3_d end doend program

�

Function calls

∂x = 2axx′ + bx ′

∂a = a′x2

Fortran-Example II

�

arguments

?

arguments

/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/MAIN__.f90 Page 1

program bsp real fct_quad_2 real fct_quad_2_d real fct_quad_3 real fct_quad_3_d real(kind=4) :: t_d real(kind=4) :: x_d real(kind=4) :: f_d real(kind=4) :: a_d real t real x, x_const real delta real a,b,c real f integer i x_c =1. x=0 delta=0.02 a=1. b=1. c=1. do i=1,10 t = getX(delta,i) t_d =1. x = t x_d = t_d fct_quad_2 = quad_2_d(a, b, c, x, x_d, fct_quad_2_d) f = fct_quad_2 f_d = fct_quad_2_d a = t a_d = t_d fct_quad_3 = quad_3_d(a, b, c, x_c, a_d, fct_quad_3_d) f = fct_quad_3 f_d = fct_quad_3_d end doend program

�

Function calls

∂x = 2axx′ + bx ′

∂a = a′x2

Fortran-Example II

�

arguments

?

arguments

�

/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/MAIN__.f90[+] Page 1

program bsp real fct_quad_2 ... c=1. do i=1,10 t = getX(delta,i) t_d =1. x = t x_d = t_d fct_quad_2 = quad_2_d(a, b, c, x, x_d, fct_quad_2_d) f = fct_quad_2 f_d = fct_quad_2_d a = t a_d = t_d fct_quad_3 = quad_3_d(a, b, c, x_c, a_d, fct_quad_3_d) f = fct_quad_3 f_d = fct_quad_3_d end doend program

Function calls

∂x = 2axx′ + bx ′

∂a = a′x2

Fortran-Example II

�

arguments

?

arguments

�

Function callsI f (x , a, b, c) = ax2 + bx + c,

∂f (x,a,b,c)∂x = 2axx

′ + bx ′

∂a = a′x2

Fortran-Example II

�

arguments

?

arguments

�

′ + bx ′

∂a = a′x2

Fortran-Example II

�

arguments

?

arguments

�

′ + bx ′

∂a = a′x2

Fortran-Example III

Function quadI four input argumentsI return the value of the quadratic function

f (x , a, b, c) = ax2 + bx + c

Function quad 2 dI gets the derivative of the variable x as an input

parametersI store the derivative of the quadratic function respect to

x in the call by reference argument

Function quad 3 dI gets the derivative of the parameter variable a as an

input parametersI store the derivative of the quadratic function respect to

the parameter a in the call by reference argument

Fortran-Example III/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/quad 2.f90 Page 1

function quad_2_d(a, b, c, x, x_d, f_d) result(f) real(kind=4), intent(inout) :: f_d real, intent(in) :: a,b,c,x real, intent(inout) :: x_d real :: f f= a*x*x+b*x+c f_d = ( ( ( (a*x_d) *x) + ( (a*x) *x_d) ) + (b*x_d) )end function

f (x , a, b, c) = ax2 + bx + c

Fortran-Example III/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/quad 3.f90 Page 1

function quad_3_d(a, b, c, x, a_d, f_d) result(f) real(kind=4), intent(inout) :: f_d real, intent(in) :: a,b,c,x real, intent(inout) :: a_d real :: f f= a*x*x+b*x+c f_d = ( (a_d*x) *x)end function

f (x , a, b, c) = ax2 + bx + c

Page 16 C-Example | Example | November 23, 2009 |

C-Example I

ExampleI simple and short programI with function handlingI variable f is targetI variable x is independent

C-Example I

fct02.c Page 1

double add(double x, double y){ double out, a,b; a =x; b =y; out = a+b; return out;} double mul(double x, double y){ double out, a,b; a =x; b =y; out = a+b; return out;} intmain(){ double f,x,y; double a; a=2.; x=1.; y=mul(x,x); y=mul(a,y); f = x add(x,y); return 0;}

C-Example I

/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/fct02.c Page 1

double add(double x, double y){ double out, a,b; a =x; b =y; out = a+b; return out;} double mul(double x, double y){ double out, a,b; a =x; b =y; out = a*b; return out;} intmain(){ double f,x,y; double a; a=2.; x=1.; y=mul(x,x); y=mul(a,y); f = x add(x,y); return 0;}

C-Example I

ExampleI simple and short programI with function handlingI variable f is targetI variable x is independentfct02.c Page 1

C-Example II

fct02.c Page 1

�

First function call mulI 1. argument activeI 2. argument active

��

��+

Second function call mulI 1. argument inactiveI 2. argument active

�

Function calls

I y=mul(x,x);

I y=mul(a,y);

I f=mul(x,y);

C-Example II

fct02.c Page 1

�

��

��+

�

Function calls

I y=mul(x,x);

I y=mul(a,y);

I f=mul(x,y);

C-Example II

fct02.c Page 1

�

��

��+

�

Function calls

I y=mul(x,x);

I y=mul(a,y);

I f=mul(x,y);

C-Example II

�

��

��+

/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/main.c Page 1

int main() { double fct_mul_1; double fct_mul_1_d; double fct_mul_2; double fct_mul_2_d; double fct_add_3; double fct_add_3_d; double x_d; double y_d; double f_d; double f,x,y; double a; a=2.; x=1.; x_d = 1.; fct_mul_1 = mul_1_d(x, x, x_d, x_d, &fct_mul_1_d); y = fct_mul_1; y_d = fct_mul_1_d; fct_mul_2 = mul_2_d(a, y, y_d, &fct_mul_2_d); y = fct_mul_2; y_d = fct_mul_2_d; fct_add_3 = add_3_d(x, y, x_d, y_d, &fct_add_3_d); f = (x fct_add_3) ; f_d = (x_d fct_add_3_d) ; return 0;}

�

Function calls

I y=mul(x,x);

I y=mul(a,y);

I f=mul(x,y);

C-Example II

�

��

��+

/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/main.c Page 1

int main() { double fct_mul_1; double fct_mul_1_d; double fct_mul_2; double fct_mul_2_d; double fct_add_3; double fct_add_3_d; double x_d; double y_d; double f_d; double f,x,y; double a; a=2.; x=1.; x_d = 1.; fct_mul_1 = mul_1_d(x, x, x_d, x_d, &fct_mul_1_d); y = fct_mul_1; y_d = fct_mul_1_d; fct_mul_2 = mul_2_d(a, y, y_d, &fct_mul_2_d); y = fct_mul_2; y_d = fct_mul_2_d; fct_add_3 = add_3_d(x, y, x_d, y_d, &fct_add_3_d); f = (x fct_add_3) ; f_d = (x_d fct_add_3_d) ; return 0;}

�

Function calls

I y=mul(x,x);

I y=mul(a,y);

I f=mul(x,y);

C-Example II

�

��

��+

�

/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/main.c[+] Page 1

int main() { double fct_mul_1; ... double a; a=2.; x=1.; x_d = 1.; fct_mul_1 = mul_1_d(x, x, x_d, x_d, &fct_mul_1_d); y = fct_mul_1; y_d = fct_mul_1_d; fct_mul_2 = mul_2_d(a, y, y_d, &fct_mul_2_d); y = fct_mul_2; y_d = fct_mul_2_d; fct_add_3 = add_3_d(x, y, x_d, y_d, &fct_add_3_d); f = (x fct_add_3) ; f_d = (x_d fct_add_3_d) ; return 0;}

Function calls

I y=mul(x,x);

I y=mul(a,y);

I f=mul(x,y);

C-Example II

�

��

��+

�

Function callsI y=mul(x,x);

I y=mul(a,y);

I f=mul(x,y);

C-Example II

�

��

��+

�

I y=mul(a,y);

I f=mul(x,y);

C-Example II

�

��

��+

�

I y=mul(a,y);

I f=mul(x,y);

C-Example III

/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/fct02.c Page 1

double add(double x, double y){ double out, a,b; a =x; b =y; out = a+b; return out;} double mul(double x, double y){ double out, a,b; a =x; b =y; out = a*b; return out;} intmain(){ double f,x,y; double a; a=2.; x=1.; y=mul(x,x); y=mul(a,y); f = x add(x,y); return 0;}

Function mulI two input parameterI returns the product of the both arguments

Function mul 1 dI gets the derivative of the both arguments

as an input parametersI store the derivative of the product in the

call by reference argument

Function mul 2 dI gets the derivative of the second argument as an input

parameterI store the derivative of the product in the call by

reference argument

C-Example III/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/mul 1.c Page 1double mul_1_d(double x, double y, double x_d, double y_d, double *out_d){ double a_d; double b_d; double out, a,b; a =x; a_d = x_d; b =y; b_d = y_d; out = a*b; *out_d = ( (a_d*b) + (a*b_d) ) ; return out;

}

reference argument

C-Example III/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/mul 2.c Page 1double mul_2_d(double x, double y, double y_d, double *out_d){ double b_d; double out, a,b; a =x; b =y; b_d = y_d; out = a*b; *out_d = (a*b_d) ; return out;

}

reference argument

C-Example IV

fct02.c Page 1

Function addI two input parameterI returns the sum of the both arguments

Function add 3 dI gets the derivative of the

both arguments as aninput parameters

I store the derivative ofthe sum in the call byreference argument

C-Example IV/Volumes/Data/GAuDi/gaudi/branches/0.3/presentation/_ad/add 3.c[+] Page 1double add_3_d(double x, double y, double x_d, double y_d, double *out_d){ double a_d; double b_d; double out, a,b; a =x; a_d = x_d; b =y; b_d = y_d; out = a+b; *out_d = (a_d+b_d) ; return out;

}

Function addI two input parameterI returns the sum of the both arguments

Function add 3 dI gets the derivative of the

both arguments as aninput parameters

I store the derivative ofthe sum in the call byreference argument

Page 20 | Outlook | November 23, 2009 |

Idea

Outlook

Short-TermI array supportI other programming languages (ada, c++, java)I testing phaseI GUI (research student)I interface to ADOL-C

Long-TermI reverse modeI second derivativesI optional parallelization (OpenMP, CUDA)

Outlook

Short-TermI array supportI other programming languages (ada, c++, java)I testing phaseI GUI (research student)I interface to ADOL-C

Long-TermI reverse modeI second derivativesI optional parallelization (OpenMP, CUDA)

Page 22 | Contact | November 23, 2009 |

Contact

DFG Research Training Group 1100,Institute for Numerical Mathematics,

Ulm University

Ralf [email protected]

Thank you for your attention.

Page 22 | Contact | November 23, 2009 |

Contact

DFG Research Training Group 1100,Institute for Numerical Mathematics,

Ulm University

Ralf [email protected]

Thank you for your attention.

IdeaStructure of GAuDiPreprocessingDifferentiationPostprocessing

OutlookContact

gaudi - gnu automatic differentiation euroad workshop - ralf leidenb… · gaudi - gnu automatic...

Documents