Source Transformation via Operator Overloadingin MATLAB

Matthew J. WeinsteinAnil V. Rao

Department of Mechanical and Aerospace EngineeringUniversity of Florida

Gainesville, FL 32611-6250

15th EuroAD WorkshopJune 16th and 17th, 2014

1 / 16

Motivation

• Target Applications - all require repeated evaluation ofderivative

• First derivatives of stiff ODE’s• First and second derivatives of NLPs• Particular emphasis on vectorized functions which result from

collocation methods - direct collocation optimal control

• Generate Stand-Alone Derivative Files• Efficient at run-time• Can apply method recursively - allows for second and higher

order derivative computation without complexity of codingsecond and higher order derivative rules

• Pure Source Transformation in MATLAB is Hard• Interpreted nature of the language makes it difficult to

determine what is happening from a purely lexical analysis

2 / 16

Key Points of the Method

1 Fix following input information:• All variable sizes• All derivative sparsity patterns• Any reference/assignment indices

2 Offload as Much Information as Possible to File Generation Steps3 Evaluate Basic Blocks on Overloaded CADA Objects

• Utilizes sparse forward mode AD• Objects do not contain numeric values but rather

• Variable sizes• Derivative sparsity patterns• Symbolic identifiers

• Overloaded evaluations result in function and derivativecalculations being printed to a file as well as propagation ofsparsity patterns within the objects.

4 Transcribe Flow Control from Function Program to DerivativeProgram

• Requires an initial source to source transformation• Psuedo-overloading of flow control statements

5 Vectorized derivative computation of vectorized functions

3 / 16

Basics of Overloaded Class

Sparse Notation:

• f = f(x) : Rn → Rm

• dfx ∈ Rq≤mn : thenon-zero elements of5xf(x) ∈ Rm×n

• ifx ∈ Rq: the rowlocations of dfx in5xf(x)

• jfx ∈ Rq: the columnlocations of dfx in5xf(x)

Basic Object Properties:

• F.func.name = ‘f.f’ =string identifier of what thef is written to in generatedfile

• F.func.size = [m 1]

• F.deriv.name = ‘f.dx’

= string identifier of whatdfx is written to ingenerated file

• F.deriv.nzlocs = [ifx jfx]

4 / 16

Basic Overloaded Operation ExampleConsider g = g(f) : Rm → Rp, where f = f(x), and the correspondingoverloaded operation G = G(F).Procedure of G:

1 Determine size(g) given m and assign to G.func.size2 Assign a function variable name to G.func.name, call this ‘g.f’

3 Print calculation to file which will compute g(f)Ex: g.f = sin(f.f);

4 Determine igx and jgx using ifx and jfx and assign to G.deriv.nzlocs5 Assign a derivative variable name to G.deriv.name, call this‘g.dx’

6 Print derivative calculations to file which will compute dgx(f,dfx) -typically dependent upon one or more reference or assignmentindices

Ex: g.dx = cos(f.f(ifx)).*f.dx

NOTE: Non-zero derivative computations typically only valid for thefixed values of ifx

5 / 16

Issues

If simply evaluating a function file on the overloaded class

1 In general, cannot evaluate branching statements (numericvalues are free) - multiple possible branches

• is Y > 0? - maybe

2 If loops are unrollable, unrolled loop will be printed -derivative code can get quite large

3 Printed derivative code hard to read, no variable names copiedover

4 Redundant 1st - (n− 1)th derivative calculations printed in nth

derivative file

6 / 16

Handling Flow ControlBranching Statements:

• Evaluate each possible branch independently on overloadedobjects

Loops:

• Want to evaluate single loop iteration on overloaded objects andprint calculations valid for all loop iterations

Issues:

1 Calculations printed by overloaded evaluations only valid for fixedvalues of function size and non-zero derivative locations

• Different branches may result in objects containing differentnon-zero derivative locations

• Inputs to loops may be iteration dependent

2 Iteration dependent organizational operations

• Consider the reference g = f(k), derivative rule is thendgx = dfx(kx), but there does not, in general, exist a mapping fromfunction reference index k to the derivative reference index kx

7 / 16

Unions and Re-Maps of Overloaded Objects

• W = U ∪ V (W is the union of U and V) has followingproperties:

1 Row/column dimension of W is considered to be maximumrow/column dimension of U and V

2 Derivative element of W is only considered to be zero if thecorresponding derivative element of U and V are also zero

• W = U → W (re-map of U to W)

1 Append zeros to rows/columns of u to get w2 Initialize dwx to be vector of zeros3 Map dux into proper elements of dwx

• U =W → U (re-map of W to U)

1 Remove rows/columns of w to get u2 Reference proper elements of dwx to get dux

• Assume that any calculations printed by the operation g(W)are valid for g(U) and g(V)

8 / 16

Overmapped Objects

Consider a variable y which can be assigned n different objectsY(1), . . . ,Y(n)

• Define the overmap of y to be Y = ∪ni=1Y(i)

• Y(i) can be thought of as the output of the i th branch of aconditional fragment or the input to the i th iteration of a loop

• Ignoring iteration/branch dependent organizationaloperations, can assume

1 If y is a loop input, then evaluating a single iteration of a loopon Y will print valid calculations for all iterations

2 If y is a conditional fragment output, then evaluating the restof the program on Y will print valid calculations for any branch

9 / 16

The ADiGator AlgorithmInputs: user program P(x) together with information to instantiate X

1 Transform user program P to intermediate program P ′

• Augment to the original program calls to transformation routines• Replace flow control statements with calls to transformation

routines

2 Evaluate intermediate program on X three times1 Empty Parsing Evaluation

• Collect information on data and control flow (similar to controlflow graphs and data flow graphs of compilers) - no derivativecomputations performed, no calculations printed to derivative file

• Use this to determine what overmapped objects must be built andwhich objects belong to them

2 Overmapping Evaluation• Build/store required overmapped objects and collect loop

organizational operation data

3 Printing Evaluation• Use the collected data from previous two operations to print the

derivative file

10 / 16

Transformation of Basic Blocks

y1 = s1;

y2(i) = s2;

Pred (A)

Succ (A)

Ay1 = s1;

y1 = VarAnalyzer (‘y1 = s1’,y1,‘y1’,0);

y2(i) = s2;

y2 = VarAnalyzer (‘y2(i) = s2’,y2,‘y2’,1);

Pred (A′)

Succ (A′)

A′

VarAnalyzer has control over evaluating workspace

• VarAnalyzer used to track variables and manipulateevaluating workspace

11 / 16

Transformation of Conditional Fragments

if s1 elseif s2 · · · elseF F F

Bk,1 Bk,2 · · · Bk,n

T T

Pred (Ck)

end

Succ (Ck)

Ck

• Each branch evaluatedindependently

• Overmapped outputs arebuilt and brought intoevaluating workspace

• In printing evaluation,perform re-maps aftereach branch evaluating

cadacond1 = s1;

cadacond1 = VarAnalyzer (‘s1’,cadacond1,‘cadacond1’,0);...

cadacondn-1 = sn;

cadacondn-1 =

VarAnalyzer (‘sn-1’,cadacondn-1,‘cadacondn-1’,0);

IfIterStart (k,1);

IfIterEnd (k,1);

[IfEvalStr,IfEvalVar] = IfIterStart (k,2);

if not(isempty(IfEvalStr))

cellfun(@eval,(IfEvalStr);

end

IfIterEnd(k,2);

[IfEvalStr,IfEvalVar] = IfIterStart (k,3);

if not(isempty(IfEvalStr))

cellfun(@eval,(IfEvalStr);

end

...

[IfEvalStr,IfEvalVar] = IfIterEnd (k,n);

if not(isempty(IfEvalStr))

cellfun(@eval,(IfEvalStr);

end

B ′k,1

B ′k,2

...

B ′k,n

Pred (C ′k)

Succ (C ′k)

C ′k

12 / 16

Transformation of Loops

for i =

sf

Ik

end

Pred (Lk)

Succ (Lk)

Lk

• Build overmapped inputs andfinal output, collectorganizational operation databy evaluating all iterations inovermapping eval

• On printing eval, re-mapinputs to overmapped inputs,evaluate single iteration onovermaps, remap overmappedoutputs to true outputs

cadaLoopVar k = sf;

cadaLoopVar k = ...

VarAnalyzer (‘sf’,cadaLoopVar k,‘cadaLoopVar k’,0);

[adigatorForVar k, ForEvalStr, ForEvalVar] = ...

ForInitialize(k,cadaLoopVar k);

if not(isempty(ForEvalStr))

cellfun(@eval,ForEvalStr)

end

for adigatorForVar k i = adigatorForVar k;

cadaForCount k = ForIterStart(k,adigatorForVar k i);

i = cadaLoopVar k(:,cadaForCount k);

i = VarAnalyzer(‘cadaLoopVar k(:,cadaForCount k)’,i,‘i’,0);

I ′k

[ForEvalStr, ForEvalVar] = ...

ForIterEnd(k,adigatorForVar k i);

end

if not(isempty(ForEvalStr))

cellfun(@eval,ForEvalStr)

end

Pred (L′k)

Succ (L′k)

L′k

13 / 16

Questions?

https://sourceforge.net/projects/adigator/

We gratefully acknowledge support for this research from Office of Naval Research

Grant N00014-11-1-0068 U.S. Defense Advanced Research Projects Agency (DARPA)

Under Contract HR0011-12-0011

14 / 16

Vectorized Functions

Continuous Function:

• f = f(x(t)) : Rn → Rm

• dfx ∈ Rq≤mn : the non-zero elements of 5xf(x) ∈ Rm×n

• ifx ∈ Rq: the row locations of dfx in 5xf(x)

• jfx ∈ Rq: the column locations of dfx in 5xf(x)

Discretized Function:

• Let X =[X1 X2 · · · XN

]∈ Rn×N , where Xi = x(ti )

• F(X) =[f(X1) f(X2) · · · f(XN)

]∈ Rm×N

5XF(X) ∈ Rn×N×m×N defined by:

• dimensions n, m, N

• row and column locations ifx ∈ Rq and jfx ∈ Rq

• Non-zero derivatives DFX =

[dfX1

dfX2· · · dfXN

]∈ Rp×N

15 / 16

Vectorized Differentiation

• Allow vectorized dimension N to be free

• Store non-zero locations (e.g. ifx, and jfx) and size (e.g. n) ofcontinuous function (e.g. f(x(t)))

• Print calculations which compute DFX

Example:

• Scalar:g.dx = cos(f.f(ifx)).*f.dx ∈ Rq

• Vectorized:g.dX = cos(f.f(:,ifx)).*f.dX ∈ Rq×N

16 / 16