Deep learning and uncertain dynamical systems

Tenavi Nakamura-Zimmerer Daniele Venturi Qi Gong

Applied Mathematics DepartmentUniversity of California, Santa Cruz

SIAM Conference on Computational Science and EngineeringFebruary 25, 2019

1

Motivation

Nonlinear control system with random initial condition (note thatthis framework includes parameter uncertainty):

x = f (x ,u), x(0) = x0 ⇠ p0(x0), t 2 [0, tf ].

u(x , t) 2 U ⇢ Rm is the deterministic control, x0 ⇠ p0(x0) is therandom initial condition which causes uncertainty in the statex(t; x0) 2 Rn.

Want to design an optimal control which is robust to uncertaintyby solving, for example,

minu

E{F (x(tf ))} =

ZF (x)p(x, tf )dx

2

Applications

(De)synchronization of neural oscillators:

*

Trajectory planning for swarms of small satellites:

**

*From Monga et al. ACC 2018.

**From Peng and Li JGCD 2017.

3

Motivation

Existing methods (e.g. Phelps et al. SICON 2016, Ross et al.JGCD 2015) rely on massive amounts of sampling and numericalintegration from the initial PDF.

=) Curse of dimensionality, typically limited to just a fewuncertain variables.

We propose an alternative approach to the forward UQ problembased on combining PDF equations with deep learning.

4

The Liouville equation

The PDF p(x , t) for a dynamical system x = f (x ,u) with randominitial condition x(0) = x0 ⇠ p0(x0) is known to satisfy theLiouville equation:

@p(x , t)@t

+rx · [p(x , t)f (x ,u)] = 0.

Analytical solution:

p(x , t) = p0(�0(x , t;u)) exp✓�Z t

0rx · f (�(x0, ⌧ ;u),u)d⌧

◆,

where�(x0, t;u) and �0(x , t;u)

are the forward and inverse flow maps for a given control u.

5

Data driven methods for the Liouville equation

If there is an analytical solution, isn’t this problem already solved?

No! The analytical solution requires the flow map �(x0, t) andinverse flow map �0(x , t).

However, the analytical solution makes computing the density atindividual points straightforward.

=) Suggests an approach which combines data and knowledgeof physics (Liouville PDE).

=) Alternatively, develop e�cient data-driven approximators ofthe forward and inverse flow maps.

6

PDF approximation with physics informed neural nets

“Physics informed neural nets” (PINNs) augment training data byfitting the PDE at collocation points (Raissi et al. (arXiv 2017)).

Automatic di↵erentiation �! @p/@t and rx p, the partialderivatives of the NN predictions p(x , t) ⇡ p(x , t).

For the true probability p(x , t),

0 =@p (x , t)

@t+rx · [p (x , t) f (x , t)] .

=) Define the residual R(p) as

R (p (x , t)) :=@p (x , t)

@t+rx · [p (x , t) f (x , t)] .

=) Quantifies how well predictions p fit the Liouville equation.

7

PDF approximation with physics informed neural nets

Define a new loss metric for training the neural net:

lossPINN(✓) := lossp(✓) + µ lossL(✓).

✓ := {W j ,bj}Mj=1 are the parameters of the NN, µ is a scalarweight, lossp is the loss with respect to the training data (we useweighted MSE on log probability):

lossp(✓) =1

Nd

NdX

i=1

p⇣x (i), t

⌘ hlog p

⇣x (i), t

⌘� log p

⇣x (i), t

⌘i2,

lossL measures how well predictions p fit the Liouville equation ona set of Nc collocation points (we use MSE):

lossL(✓) =1

Nc

NcX

i=1

hR⇣p⇣x (i), t(i)

⌘⌘i2.

8

Training physics informed neural nets

9

Prediction with physics informed neural nets

10

PDF data generation

Example: Van der Pol oscillator

8>>><

>>>:

x = y

y = (1� x2)y � x ,

p0(x , y) = Nx(0, 0.25)Ny (0, 0.25),

t 2 [0, 4].

Plot courtesy of wikipedia.

11

PDF data generationForward sampling

1 Sample Ns pointsnx (i)0

ofrom the initial PDF p0(x).

2 Numerically integrate to get the state�x (i)(tk)

at times

{tk} ⇢ [0, tf ].3 Compute density by analytical Liouville solution,

p(x , t) = p0(�0(x , t;u)) exp⇣�R t0 rx · f (�(x0, ⌧ ;u),u)d⌧

⌘

12

PDF data generationConvex hull stratification and backward sampling

1 “Convex hull stratificationalgorithm” from Ziegelmeier etal. (SIAM REV 2017) toconstruct an approximateboundary of the trajectorytube/PDF support at final time(generally highly non-convex).

2 Weight data by its distance tothe boundary.

3 Sample new data points nearthe boundary and integratebackwards to time t0.

13

PDF data generation

14

Easy progressive batch L-BFGS

Original PINNs optimize without any weight on the PDE penaltyterm, using L-BFGS (full-batch Quasi-Newton)...

=) Physics is only encouraged by a single penalty

=) Ideally, force R(p(x , t)) ! 0 using constrained optimization.Di�cult in practice...

=) Simple idea: Train in multiple rounds:

Each round, increase the penalty weight µ

Train on random subsets of the collocation and training data;increase the size of the subsets over the training rounds

15

ResultsVan der Pol oscillator

8><

>:

x = y , y = (1� x2)y � x ,

p0(x , y) = Nx(0, 0.25)Ny (0, 0.25),

t 2 [0, 4].

554 training trajectories evaluated at Nt = 160 time snapshots

Normalized root mean square error measured with respect to 133test trajectories not seen during training

optimizer PDE training time test NRMSEfull batch L-BFGS - 210 s 7.17 e–03progressive L-BFGS - 123 s 5.27 e–03full batch L-BFGS penalty 509 s 3.74 e–03progressive L-BFGS penalty 106 s 4.75 e–03full batch IPOPT - 117 s 2.54 e–02progressive IPOPT constrained 480 s 6.86 e–03

16

ResultsVan der Pol oscillator

Approximated PDF p(x , y , t) of the Van der Pol equation. Each frame is

500⇥ 500; predicting all 1.5 million outputs takes about ⇠ 0.06 seconds.

17

ResultsForced Du�ng oscillator with parameter uncertainty

x = y , y = ��y � x�↵+ �x2

�+ � cos(!t).

Means: µx = 0, µy = 0, µ� = 0.5, µ↵ = �1, µ� = 1, µ! = 1, µ� = 0.5

Variances: �2x = �2

y = 1, �2� = �2

↵ = �2� = �2

! = �2� = 0.252 = 0.0625

1186 training trajectories evaluated at Nt = 77 time snapshots

283 test trajectories

optimizer PDE training time test NRMSEfull batch L-BFGS - 465 s 2.95 e–03progressive L-BFGS - 399 s 3.13 e–03full batch L-BFGS penalty 1586 s 2.01 e–03progressive L-BFGS penalty 802 s 4.17 e–03

18

ResultsForced Du�ng oscillator with parameter uncertainty

Approximated PDF p(x , y , t|� = 0.5,↵ = �1,� = 1,! = 1, � = 0.5) of

the uncertain forced Du�ng equation for mean parameter values.

Predicting all 1.5 million outputs below takes about ⇠ 0.12 seconds.

19

Flow map approximation

We also consider the problem of approximating the flow map andinverse flow map:

b�(x0, t) ⇡ �(x0, t), b�0(x , t) ⇡ �(x , t).

Two interpretations of the flow map:

1 “Step” (short-time) flow map with RNN, studied in e.g. Luschet al. (Nat. Commun. 2018): b�(x(t), t) ⇡ x(t +�t).

2 “Jump” (long-time) flow map with PINN: b�(x0, t) ⇡ x(t).

20

Three degree-of-freedom fixed-wing UAVModel introduction

8>>>>>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>>>>>:

x = v cos � cos�

y = v cos � sin�

z = v sin �

v = 1m (�D + T cos↵)� g sin �

� = 1mv (L cosµ+ T cosµ sin↵)� g

v cos �

� = 1mv cos � (L sinµ+ T sinµ sin↵)

T = uT

↵ = u↵

µ = uµ

Cx0 = Cxa = Cz0 = Cza = 0

Lift: L = 12⇢v

2SCL

Drag: D = 12⇢v

2SCD

Air pressure: ⇢ = 1.21e�z/8000

Wing area: S = 0.982

CL =(Cx0 + Cxa↵) sin↵� (Cz0 + Cza↵) cos↵

CD =� (Cx0 + Cxa↵) cos↵� (Cz0 + Cza↵) sin↵

21

Three degree-of-freedom fixed-wing UAVResults

Use a control designed for the deterministic system, adduncertainty in everything!!!

200 sample trajectories for training, 50 for testing. Test errormeasured using mean relative L2

approximator training time test errorforward flow map 373 s 3.81 e–04inverse flow map 220 s 1.39 e–03

22

Three degree-of-freedom fixed-wing UAVResults

23

Conclusions

If we need p(x , t) in high dimensions and high resolution,training and evaluating a NN can be more e�cient thanMonte Carlo sampling or solving Liouville PDE.

No control yet / ... but potential new tool for uncertaintyquantification and evaluation of control robustness givenlimited sample data.

24

Acknowledgments

This research was developed with funding from the DefenseAdvanced Research Projects Agency (DARPA). The views,opinions and/or findings expressed are those of the author andshould not be interpreted as representing the o�cial views orpolicies of the Department of Defense or the U.S. Government.

25

References I

[1] M. Abadi, A. Agarwal, and P. Barham, et al., TensorFlow:

Large-scale machine learning on heterogeneous systems.

https://www.tensorflow.org/, 2015.

[2] M. Ehrendorfer, The Liouville equation in atmospheric predictability, in

Seminar on predictability of weather and climate, Shinfield Park, Reading,

2003, ECMWF, pp. 47–81.

[3] B. Lusch, J. N. Kutz, and S. L. Brunton, Deep learning for universal

linear embeddings of nonlinear dynamics, Nature Communications, 9

(2018).

[4] B. Monga, G. Froyland, and J. Moehlis, Synchronizing and

desynchronizing neural populations through phase distribution control, in

American Control Conference, 06 2018, pp. 2808–2813.

[5] H. Peng and C. Li, Bound evaluation for spacecraft swarm on libration

orbits with an uncertain boundary, Journal of Guidance, Control, and

Dynamics, 40 (2017), pp. 2690–2698.

26

References II

[6] C. Phelps, I. Kaminer, and Q. Gong, Optimal control of uncertain

systems using sample average approximations, SIAM Journal on Control

and Optimization, 54 (2016), pp. 1–29.

[7] M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics informed

deep learning (part I): data-driven solutions of nonlinear partial di↵erential

equations, arXiv preprint arXiv:1711.10561 [cs.AI], (2017).

[8] I. M. Ross, R. J. Proulx, M. Karpenko, and Q. Gong,

Riemann–stieltjes optimal control problems for uncertain dynamic systems,

Journal of Guidance, Control, and Dynamics, 38 (2015), pp. 1251–1263.

[9] L. Ziegelmeier, M. Kirby, and C. Peterson, Stratifying

high-dimensional data based on proximity to the convex hull boundary,

SIAM Review, 59 (2017), pp. 346–365.

27