7/25/2019 15213 Variants of the HJB Equation

1/2

1.3 Variants of the HJB equation

/planning.cs.uiuc.edu/node816.html[5/3/2016 8:28:08 PM]

Next: 15.2.2 Linear-Quadratic Problems Up: 15.2.1 Hamilton-Jacobi-Bellman Equation Previous: 15.2.1.2 Theontinuous case

15.2.1.3 Variants of the HJB equation

Several versions of the HJB equation exist. The one presented in ( 15.14 ) is suitable for planning problems such as thosexpressed in Chapter 14. If the cost-to-go functions are time-dependent, then the HJB equation is

(15

and is a function of both and . This can be derived again using a Taylor expansion, but with and treated s the variables. Most textbooks on optimal control theory present the HJB equation in this form or in a slightly

different form by pulling outside of the and moving it to the right of the equation:

(15

n differential game theory, the HJB equation generalizes to the Hamilton-Jacobi-Isaacs (HJI) equations [ 59,477 ].Suppose that the system is given as ( 13.203 ) and a zero-sum game is defined using a cost term of the form

. The HJI equations characterize saddle equilibria and are given as

(15

and

(15

http://planning.cs.uiuc.edu/node817.htmlhttp://planning.cs.uiuc.edu/node813.htmlhttp://planning.cs.uiuc.edu/node815.htmlhttp://planning.cs.uiuc.edu/node815.htmlhttp://planning.cs.uiuc.edu/node815.html#eqn:hjbhttp://planning.cs.uiuc.edu/node713.html#cha:nplanhttp://planning.cs.uiuc.edu/node858.html#BasOls95http://planning.cs.uiuc.edu/node858.html#Isa65http://planning.cs.uiuc.edu/node710.html#eqn:diffgamehttp://planning.cs.uiuc.edu/node710.html#eqn:diffgamehttp://planning.cs.uiuc.edu/node858.html#Isa65http://planning.cs.uiuc.edu/node858.html#BasOls95http://planning.cs.uiuc.edu/node713.html#cha:nplanhttp://planning.cs.uiuc.edu/node815.html#eqn:hjbhttp://planning.cs.uiuc.edu/node815.htmlhttp://planning.cs.uiuc.edu/node815.htmlhttp://planning.cs.uiuc.edu/node813.htmlhttp://planning.cs.uiuc.edu/node817.htmlhttp://planning.cs.uiuc.edu/node859.htmlhttp://planning.cs.uiuc.edu/node1.htmlhttp://planning.cs.uiuc.edu/node815.htmlhttp://planning.cs.uiuc.edu/node813.htmlhttp://planning.cs.uiuc.edu/node817.htmlhttp://planning.cs.uiuc.edu/

7/25/2019 15213 Variants of the HJB Equation

2/2

1.3 Variants of the HJB equation

/planning.cs.uiuc.edu/node816.html[5/3/2016 8:28:08 PM]

There are clear similarities between these equations and ( 15.16 ). Also, the swapping of the and operatorsesembles the definition of saddle points in Section 9.3 .

Next: 15.2.2 Linear-Quadratic Problems Up: 15.2.1 Hamilton-Jacobi-Bellman Equation Previous: 15.2.1.2 Theontinuous case

teven M LaValle 2012-04-20

http://planning.cs.uiuc.edu/node451.html#sec:zerosumhttp://planning.cs.uiuc.edu/node817.htmlhttp://planning.cs.uiuc.edu/node813.htmlhttp://planning.cs.uiuc.edu/node815.htmlhttp://planning.cs.uiuc.edu/node815.htmlhttp://planning.cs.uiuc.edu/node815.htmlhttp://planning.cs.uiuc.edu/node815.htmlhttp://planning.cs.uiuc.edu/node813.htmlhttp://planning.cs.uiuc.edu/node817.htmlhttp://planning.cs.uiuc.edu/node859.htmlhttp://planning.cs.uiuc.edu/node1.htmlhttp://planning.cs.uiuc.edu/node815.htmlhttp://planning.cs.uiuc.edu/node813.htmlhttp://planning.cs.uiuc.edu/node817.htmlhttp://planning.cs.uiuc.edu/node451.html#sec:zerosum