automatic differentiation in chebfun for solution of
TRANSCRIPT
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
Automatic Differentiation in Chebfunfor Solution of Nonlinear Boundary-Value Problems
Asgeir [email protected]
Collaboration with Prof. Toby Driscoll, Univ. of Delawareand the Chebfun team
University of Oxford, Mathematical Institute, Numerical Analysis Group
Eleventh European Workshop on Automatic DifferentiationDecember, 2010
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
To solve the nonlinear BVP
0.05u′′ + u′u + cos u = 0, 0 < x < 2, u(0) = 1, u′(2) = −1
2
for the function u = u(x), we only need to execute the followingfour lines of Chebfun code in Matlab :
phi = @(u) 0.05*diff(u,2) + diff(u).*u + cos(u);
alpha = @(u) u-1; beta = @(u) diff(u)+1/2;
N = chebop(domain(0,2),phi,alpha,beta);
u = N\0;
The code runs in 1.0 seconds on a tri-core 2.5 GHz workstation,and yields an accuracy of 9.7 · 10−10. How do we achieve this?
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
To solve the nonlinear BVP
0.05u′′ + u′u + cos u = 0, 0 < x < 2, u(0) = 1, u′(2) = −1
2
for the function u = u(x), we only need to execute the followingfour lines of Chebfun code in Matlab :
phi = @(u) 0.05*diff(u,2) + diff(u).*u + cos(u);
alpha = @(u) u-1; beta = @(u) diff(u)+1/2;
N = chebop(domain(0,2),phi,alpha,beta);
u = N\0;
The code runs in 1.0 seconds on a tri-core 2.5 GHz workstation,and yields an accuracy of 9.7 · 10−10. How do we achieve this?
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
To solve the nonlinear BVP
0.05u′′ + u′u + cos u = 0, 0 < x < 2, u(0) = 1, u′(2) = −1
2
for the function u = u(x), we only need to execute the followingfour lines of Chebfun code in Matlab :
phi = @(u) 0.05*diff(u,2) + diff(u).*u + cos(u);
alpha = @(u) u-1; beta = @(u) diff(u)+1/2;
N = chebop(domain(0,2),phi,alpha,beta);
u = N\0;
The code runs in 1.0 seconds on a tri-core 2.5 GHz workstation,and yields an accuracy of 9.7 · 10−10. How do we achieve this?
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
0 0.5 1 1.5 20.9
1
1.1
1.2
1.3
1.4
1.5Solution
x
u(x)
1 3 5 710
−14
10−12
10−10
10−8
10−6
10−4
10−2
100
Norm of update
Iteration number
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
Table of Contents
1 Preface
2 The Chebfun system
3 Functional Newton iteration
4 Automatic differentiation in Chebfun
5 Chebops for automatic iterations for BVPs
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Chebfun
From the Chebfun website [5] (first result for a Google search on“chebfun”):
Chebfun is a collection of algorithms, and a software system inobject-oriented Matlab , which extends familiar powerfulmethods of numerical computation involving numbers tocontinuous or piecewise-continuous functions.
It also implements continuous analogues of linear algebranotions like the QR decomposition and the SVD, and solvesordinary differential equations.
The mathematical basis of the system combines tools ofChebyshev expansions, fast Fourier transform, barycentricinterpolation, recursive zerofinding, and automaticdifferentiation.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Chebfun
From the Chebfun website [5] (first result for a Google search on“chebfun”):
Chebfun is a collection of algorithms, and a software system inobject-oriented Matlab , which extends familiar powerfulmethods of numerical computation involving numbers tocontinuous or piecewise-continuous functions.
It also implements continuous analogues of linear algebranotions like the QR decomposition and the SVD, and solvesordinary differential equations.
The mathematical basis of the system combines tools ofChebyshev expansions, fast Fourier transform, barycentricinterpolation, recursive zerofinding, and automaticdifferentiation.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Chebfun
From the Chebfun website [5] (first result for a Google search on“chebfun”):
Chebfun is a collection of algorithms, and a software system inobject-oriented Matlab , which extends familiar powerfulmethods of numerical computation involving numbers tocontinuous or piecewise-continuous functions.
It also implements continuous analogues of linear algebranotions like the QR decomposition and the SVD, and solvesordinary differential equations.
The mathematical basis of the system combines tools ofChebyshev expansions, fast Fourier transform, barycentricinterpolation, recursive zerofinding, and automaticdifferentiation.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Chebfun
From the Chebfun website [5] (first result for a Google search on“chebfun”):
Chebfun is a collection of algorithms, and a software system inobject-oriented Matlab , which extends familiar powerfulmethods of numerical computation involving numbers tocontinuous or piecewise-continuous functions.
It also implements continuous analogues of linear algebranotions like the QR decomposition and the SVD, and solvesordinary differential equations.
The mathematical basis of the system combines tools ofChebyshev expansions, fast Fourier transform, barycentricinterpolation, recursive zerofinding, and automaticdifferentiation.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Chebfun demo
In the Chebfun system, we work with functions defined on aninterval [a, b]; the functions are represented by interpolants insuitably rescaled Chebyshev points {xj}nj=0, such that
xj = cos(jπ/n), 0 ≤ j ≤ n.
By overloading common Matlab functions, (and introducingnew ones), we can carry out computations with chebfuns.
chebfun demo
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Chebfun demo
In the Chebfun system, we work with functions defined on aninterval [a, b]; the functions are represented by interpolants insuitably rescaled Chebyshev points {xj}nj=0, such that
xj = cos(jπ/n), 0 ≤ j ≤ n.
By overloading common Matlab functions, (and introducingnew ones), we can carry out computations with chebfuns.
chebfun demo
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Chebfun demo
In the Chebfun system, we work with functions defined on aninterval [a, b]; the functions are represented by interpolants insuitably rescaled Chebyshev points {xj}nj=0, such that
xj = cos(jπ/n), 0 ≤ j ≤ n.
By overloading common Matlab functions, (and introducingnew ones), we can carry out computations with chebfuns.
chebfun demo
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Linear operators in Chebfun – linops
When working with vectors, we apply matrices to form newvectors.
In the functional settings, these becomes linear operators.
In Chebfun, the class linops represents such operators [4].
Linops can amongst other uses be used to representdifferentiation (diff), integration (cumsum) andmultiplication (diag) operators.
A linop can be “expanded” to form a matrix which is thediscretization of the operator it represents on thecorresponding Chebyshev grid with the same number ofpoints.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Linear operators in Chebfun – linops
When working with vectors, we apply matrices to form newvectors.
In the functional settings, these becomes linear operators.
In Chebfun, the class linops represents such operators [4].
Linops can amongst other uses be used to representdifferentiation (diff), integration (cumsum) andmultiplication (diag) operators.
A linop can be “expanded” to form a matrix which is thediscretization of the operator it represents on thecorresponding Chebyshev grid with the same number ofpoints.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Linear operators in Chebfun – linops
When working with vectors, we apply matrices to form newvectors.
In the functional settings, these becomes linear operators.
In Chebfun, the class linops represents such operators [4].
Linops can amongst other uses be used to representdifferentiation (diff), integration (cumsum) andmultiplication (diag) operators.
A linop can be “expanded” to form a matrix which is thediscretization of the operator it represents on thecorresponding Chebyshev grid with the same number ofpoints.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Linear operators in Chebfun – linops
When working with vectors, we apply matrices to form newvectors.
In the functional settings, these becomes linear operators.
In Chebfun, the class linops represents such operators [4].
Linops can amongst other uses be used to representdifferentiation (diff), integration (cumsum) andmultiplication (diag) operators.
A linop can be “expanded” to form a matrix which is thediscretization of the operator it represents on thecorresponding Chebyshev grid with the same number ofpoints.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Linear operators in Chebfun – linops
When working with vectors, we apply matrices to form newvectors.
In the functional settings, these becomes linear operators.
In Chebfun, the class linops represents such operators [4].
Linops can amongst other uses be used to representdifferentiation (diff), integration (cumsum) andmultiplication (diag) operators.
A linop can be “expanded” to form a matrix which is thediscretization of the operator it represents on thecorresponding Chebyshev grid with the same number ofpoints.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Linop demo
Linops operate in forward mode (*) and reverse mode (\).
By appending boundary conditions, we can solve linear BVPswith linops.
Linop demo
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Linop demo
Linops operate in forward mode (*) and reverse mode (\).
By appending boundary conditions, we can solve linear BVPswith linops.
Linop demo
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
BackgroundchebfunsLinear operators in Chebfun – linops
Linop demo
Linops operate in forward mode (*) and reverse mode (\).
By appending boundary conditions, we can solve linear BVPswith linops.
Linop demo
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
1 Preface
2 The Chebfun system
3 Functional Newton iteration
4 Automatic differentiation in Chebfun
5 Chebops for automatic iterations for BVPs
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration
We can use our methods for solving linear problems as a basisfor nonlinear ones.
The method is known as Newton-Kantorovich iteration orNewton iteration in function space.
Based on similar ideas as the well-known Newton iteration forrootfinding.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration
We can use our methods for solving linear problems as a basisfor nonlinear ones.
The method is known as Newton-Kantorovich iteration orNewton iteration in function space.
Based on similar ideas as the well-known Newton iteration forrootfinding.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration
We can use our methods for solving linear problems as a basisfor nonlinear ones.
The method is known as Newton-Kantorovich iteration orNewton iteration in function space.
Based on similar ideas as the well-known Newton iteration forrootfinding.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration
Recall that we use Newton iteration when we wish to find aroot x∗ of a function f , i.e. x∗ such that f (x∗) = 0.
Starting from an initial guess x (0), we find a sequence ofguesses of the solution. Members of the sequence are definedvia
x (k+1) − x (k) = −[f ′(x (k)
)]−1f(x (k)
), (1)
where f ′(x) is the Jacobian matrix of partial derivatives∂fi/∂xj .
Rigorously justifiable using Taylor series.
For a sufficiently good initial guess, Newton iterationconverges quadratically to a root.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration
Recall that we use Newton iteration when we wish to find aroot x∗ of a function f , i.e. x∗ such that f (x∗) = 0.
Starting from an initial guess x (0), we find a sequence ofguesses of the solution. Members of the sequence are definedvia
x (k+1) − x (k) = −[f ′(x (k)
)]−1f(x (k)
), (1)
where f ′(x) is the Jacobian matrix of partial derivatives∂fi/∂xj .
Rigorously justifiable using Taylor series.
For a sufficiently good initial guess, Newton iterationconverges quadratically to a root.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration
Recall that we use Newton iteration when we wish to find aroot x∗ of a function f , i.e. x∗ such that f (x∗) = 0.
Starting from an initial guess x (0), we find a sequence ofguesses of the solution. Members of the sequence are definedvia
x (k+1) − x (k) = −[f ′(x (k)
)]−1f(x (k)
), (1)
where f ′(x) is the Jacobian matrix of partial derivatives∂fi/∂xj .
Rigorously justifiable using Taylor series.
For a sufficiently good initial guess, Newton iterationconverges quadratically to a root.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration
Recall that we use Newton iteration when we wish to find aroot x∗ of a function f , i.e. x∗ such that f (x∗) = 0.
Starting from an initial guess x (0), we find a sequence ofguesses of the solution. Members of the sequence are definedvia
x (k+1) − x (k) = −[f ′(x (k)
)]−1f(x (k)
), (1)
where f ′(x) is the Jacobian matrix of partial derivatives∂fi/∂xj .
Rigorously justifiable using Taylor series.
For a sufficiently good initial guess, Newton iterationconverges quadratically to a root.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration for BVPs
Now assume we want to solve the nonlinear BVP
φ(u) = 0,
α(u)∣∣x=a
= 0, β(u)∣∣x=b
= 0,
where u is a function (i.e. u = u(x)) and φ, α, and β areoperators between suitable normed function spaces;presumably, φ is a differential operator, and α and β may alsobe differential operators of lower order.
Letting N denote a BVP operator, we also refer to thisproblem simply as
N (u) = 0
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration for BVPs
Now assume we want to solve the nonlinear BVP
φ(u) = 0,
α(u)∣∣x=a
= 0, β(u)∣∣x=b
= 0,
where u is a function (i.e. u = u(x)) and φ, α, and β areoperators between suitable normed function spaces;presumably, φ is a differential operator, and α and β may alsobe differential operators of lower order.
Letting N denote a BVP operator, we also refer to thisproblem simply as
N (u) = 0
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration for BVPs
We generalize the Newton iteration to produce a sequence offunctions u1, u2, . . . to find a u∗ such that N (u∗) = 0.
The crucial component of this iteration is the linearization ofN about each uk .
This process requires the Frechet derivatives of φ, α, and β(derivatives with respect to functions).
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration for BVPs
We generalize the Newton iteration to produce a sequence offunctions u1, u2, . . . to find a u∗ such that N (u∗) = 0.
The crucial component of this iteration is the linearization ofN about each uk .
This process requires the Frechet derivatives of φ, α, and β(derivatives with respect to functions).
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration for BVPs
We generalize the Newton iteration to produce a sequence offunctions u1, u2, . . . to find a u∗ such that N (u∗) = 0.
The crucial component of this iteration is the linearization ofN about each uk .
This process requires the Frechet derivatives of φ, α, and β(derivatives with respect to functions).
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration for BVPs
Definition
Let V and W be Banach spaces and let U be an open subset ofV . Then for φ : U →W , the Frechet derivative of φ at u ∈ U isdefined (when possible) as the unique linear operator A = φ′(u),A : V →W , such that
limv→0
‖φ(u + v)− φ(u)− Av‖W‖v‖V
= 0.
Note in this definition that v is a function, and we require that thelimit exists as v → 0 in any manner.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration for BVPs
We obtain the linear boundary-value problem
φ′(uk)v = −φ(uk),[α′(uk)v
]x=a
= −α(uk)∣∣x=a
,[β′(uk)v
]x=b
= −β(uk)∣∣x=b
,
or using the notation introduced above,
N ′(uk)v = −N (uk).
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration for BVPs
We obtain the linear boundary-value problem
φ′(uk)v = −φ(uk),[α′(uk)v
]x=a
= −α(uk)∣∣x=a
,[β′(uk)v
]x=b
= −β(uk)∣∣x=b
,
or using the notation introduced above,
N ′(uk)v = −N (uk).
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration for BVPs
Solving the problem
N ′(uk)v = −N (uk).
for the function v to obtain the update vk , we define
uk+1 = uk + vk
and iterate until convergence criteria are satisfied.
Under suitable conditions, the Kantorovich Theoremguarantees quadratic convergence for suitably close initialguesses [3].
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionNewton iteration for rootfindingNewton iteration for BVPsFrechet derivatives
Newton iteration for BVPs
Solving the problem
N ′(uk)v = −N (uk).
for the function v to obtain the update vk , we define
uk+1 = uk + vk
and iterate until convergence criteria are satisfied.
Under suitable conditions, the Kantorovich Theoremguarantees quadratic convergence for suitably close initialguesses [3].
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
1 Preface
2 The Chebfun system
3 Functional Newton iteration
4 Automatic differentiation in Chebfun
5 Chebops for automatic iterations for BVPs
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
AD in Chebfun
Even though Frechet differentiation and normal differentiationw.r.t. a function are conceptually different, they both rely onsimilar rules from calculus (e.g. chain rule).
We can thus use automatic differentiation to calculate theFrechet derivatives of the operators involved – we chooseforward mode AD.
Since Chebfun already overloads Matlab methods, operatoroverloaded AD implementation was our choice.
Since Chebfun has many other uses from solving ODEs, weput a high emphasis on minimizing computational overheadfor all uses.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
AD in Chebfun
Even though Frechet differentiation and normal differentiationw.r.t. a function are conceptually different, they both rely onsimilar rules from calculus (e.g. chain rule).
We can thus use automatic differentiation to calculate theFrechet derivatives of the operators involved – we chooseforward mode AD.
Since Chebfun already overloads Matlab methods, operatoroverloaded AD implementation was our choice.
Since Chebfun has many other uses from solving ODEs, weput a high emphasis on minimizing computational overheadfor all uses.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
AD in Chebfun
Even though Frechet differentiation and normal differentiationw.r.t. a function are conceptually different, they both rely onsimilar rules from calculus (e.g. chain rule).
We can thus use automatic differentiation to calculate theFrechet derivatives of the operators involved – we chooseforward mode AD.
Since Chebfun already overloads Matlab methods, operatoroverloaded AD implementation was our choice.
Since Chebfun has many other uses from solving ODEs, weput a high emphasis on minimizing computational overheadfor all uses.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
AD in Chebfun
Even though Frechet differentiation and normal differentiationw.r.t. a function are conceptually different, they both rely onsimilar rules from calculus (e.g. chain rule).
We can thus use automatic differentiation to calculate theFrechet derivatives of the operators involved – we chooseforward mode AD.
Since Chebfun already overloads Matlab methods, operatoroverloaded AD implementation was our choice.
Since Chebfun has many other uses from solving ODEs, weput a high emphasis on minimizing computational overheadfor all uses.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
AD in Chebfun
Leads to delayed evaluation – derivatives are not evaluateduntil explicitly required.
Obtained using anonymous functions.
Our implementation is recursive – we assign a unique ID toevery chebfun.
We go backwards in the evaluation trace until we get a matchof IDs.
Introduce two new fields to the chebfun objects, .jac and.ID.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
AD in Chebfun
Leads to delayed evaluation – derivatives are not evaluateduntil explicitly required.
Obtained using anonymous functions.
Our implementation is recursive – we assign a unique ID toevery chebfun.
We go backwards in the evaluation trace until we get a matchof IDs.
Introduce two new fields to the chebfun objects, .jac and.ID.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
AD in Chebfun
Leads to delayed evaluation – derivatives are not evaluateduntil explicitly required.
Obtained using anonymous functions.
Our implementation is recursive – we assign a unique ID toevery chebfun.
We go backwards in the evaluation trace until we get a matchof IDs.
Introduce two new fields to the chebfun objects, .jac and.ID.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
AD in Chebfun
Leads to delayed evaluation – derivatives are not evaluateduntil explicitly required.
Obtained using anonymous functions.
Our implementation is recursive – we assign a unique ID toevery chebfun.
We go backwards in the evaluation trace until we get a matchof IDs.
Introduce two new fields to the chebfun objects, .jac and.ID.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
AD in Chebfun
Leads to delayed evaluation – derivatives are not evaluateduntil explicitly required.
Obtained using anonymous functions.
Our implementation is recursive – we assign a unique ID toevery chebfun.
We go backwards in the evaluation trace until we get a matchof IDs.
Introduce two new fields to the chebfun objects, .jac and.ID.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
Modifications to code
To enable AD in Chebfun, we only need to add a couple oflines to already overloaded methods.
Take for example the overloaded sin method.
The essential lines are
function fout = sin(fin)
fout = comp(fin, @(x) sin(x));
fout.jac = @(u) diag(cos(fin))*diff(fin,u);
fout.ID = newIDnum;
Here, the argument u has the meaning of the independentvariable we will be differentiating with respect to.
A call to diff with two chebfun arguments starts theevaluation process.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
Modifications to code
To enable AD in Chebfun, we only need to add a couple oflines to already overloaded methods.
Take for example the overloaded sin method.
The essential lines are
function fout = sin(fin)
fout = comp(fin, @(x) sin(x));
fout.jac = @(u) diag(cos(fin))*diff(fin,u);
fout.ID = newIDnum;
Here, the argument u has the meaning of the independentvariable we will be differentiating with respect to.
A call to diff with two chebfun arguments starts theevaluation process.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
Modifications to code
To enable AD in Chebfun, we only need to add a couple oflines to already overloaded methods.
Take for example the overloaded sin method.
The essential lines are
function fout = sin(fin)
fout = comp(fin, @(x) sin(x));
fout.jac = @(u) diag(cos(fin))*diff(fin,u);
fout.ID = newIDnum;
Here, the argument u has the meaning of the independentvariable we will be differentiating with respect to.
A call to diff with two chebfun arguments starts theevaluation process.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
Modifications to code
To enable AD in Chebfun, we only need to add a couple oflines to already overloaded methods.
Take for example the overloaded sin method.
The essential lines are
function fout = sin(fin)
fout = comp(fin, @(x) sin(x));
fout.jac = @(u) diag(cos(fin))*diff(fin,u);
fout.ID = newIDnum;
Here, the argument u has the meaning of the independentvariable we will be differentiating with respect to.
A call to diff with two chebfun arguments starts theevaluation process.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
Examples
Differentiate g w.r.t. fReturn −Msin(f) + M2xf = M2xf−sin(f)
Differentiate g1 w.r.t. fReturn −Msin(f)
Differentiate g2 w.r.t. fReturn M2xf
Differentiate cos(f) w.r.t. fReturn M− sin(f)I = −Msin(f)
Differentiate xf2 w.r.t. fReturn OMf2 + MxM2f = M2xf
Differentiate f w.r.t. fReturn I
Differentiate x w.r.t. fReturn O
Differentiate f2 w.r.t. fReturn M2fI = M2f
Differentiate f w.r.t. fReturn I
1
Figure: Recursive diff. of g with respect f , where f = sin(x),g1 = cos(f ), g2 = xf 2 and g = g1 + g2. In the figure, I stands for theidentity operator, M for a multiplication operator (such thatMh(x) : k(x) 7→ h(x)k(x)) and O for the zero operator on the domain ofthe functions.
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
The use of ADOur AD implementationCode modificationsEvaluation of Frechet derivativesExample of evaluation
Examples
Live demo
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionExamples
1 Preface
2 The Chebfun system
3 Functional Newton iteration
4 Automatic differentiation in Chebfun
5 Chebops for automatic iterations for BVPs
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionExamples
Chebops
Combining the ideas of automatic differentiation, Newtoniteration and object oriented programming leads to thedevelopment of the chebop class.
A chebop is an object which contains all required informationto solve a nonlinear BVP.
Performs damped Newton iteration
uk+1 = uk + λkvk
where uk is the current guess of the solution, vk is theNewton update and λk is a damping parameter.
For discussion on the damping strategy, see [2, 1].
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionExamples
Examples
Live demo
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionExamples
Thank you for your attention!
Asgeir Birkisson Automatic Frechet Differentiation
PrefaceThe Chebfun system
Functional Newton iterationAutomatic differentiation in Chebfun
Chebops for automatic iterations for BVPs
IntroductionExamples
A. Birkisson and T. Driscoll, Automatic Frechetdifferentiation for the numerical solution of boundary-valueproblems.
Submitted.
A. Birkisson and T. Driscoll, Tech. Rep. 10/15, OxfordUniversity Mathematical Institute, November 2010.
P. Deuflhard, Newton Methods for Nonlinear Problems,Springer-Verlag, Berlin/Heidelberg, 2006.
T. A. Driscoll, F. Bornemann, and L. N. Trefethen, Thechebop system for automatic solution of differential equations, BITNumerical Mathematics, 48 (2008), pp. 701–723.
L. N. Trefethen, N. Hale, R. B. Platte, T. A. Driscoll,and R. Pachon, Chebfun version 3.
Oxford University, 2009.
http://www.maths.ox.ac.uk/chebfun/.Asgeir Birkisson Automatic Frechet Differentiation