stochastic perron's method: obstacle problems and dynkin … · stochastic perron’s method:...
TRANSCRIPT
Stochastic Perron’s Method: Obstacle Problemsand Dynkin Games
Erhan Bayraktar
Department of Mathematics,
University of Michigan
Joint work with Mihai Sırbu, UT Austin
Outline
Overview of DPP
Objective
Main Idea of Stochastic Perron’s method
Obstacle problems and Dynkin games
Classical approaches to solving stochastic control problems
I State equation:
dX
t
= b(t,Xt
,↵t
)dt + �(t,Xt
,↵t
)dWt
, X
s
= x .
I Reward:
J(s, x ,↵) = E [
ZT
s
R(s,Xt
,↵t
)dt + g(XT
)].
I Value function:
v(s, x) = sup↵
J(s, x ,↵).
Classical approaches to solving stochastic control problems
I State equation:
dX
t
= b(t,Xt
,↵t
)dt + �(t,Xt
,↵t
)dWt
, X
s
= x .
I Reward:
J(s, x ,↵) = E [
ZT
s
R(s,Xt
,↵t
)dt + g(XT
)].
I Value function:
v(s, x) = sup↵
J(s, x ,↵).
Characterizing v as the solution of the HJB
Two approaches:
I Analytic (direct)
I Probabilistic (analyzing the properties of the value function)
Characterizing v as the solution of the HJB
Two approaches:
I Analytic (direct)
I Probabilistic (analyzing the properties of the value function)
Probabilistic Approach
I Prove DPP. (Requires a priori regularity).
I Show that DPP ! v is a viscosity solution.
I Then comparison implies that v is the unique viscositysolution.
References:
I By discrete-time approximations (Krylov)
I Working with the value function directly (El Karoui, Borkar,Haussmann, Bouchard-Touzi (weak DPP)).
Probabilistic Approach
I Prove DPP. (Requires a priori regularity).
I Show that DPP ! v is a viscosity solution.
I Then comparison implies that v is the unique viscositysolution.
References:
I By discrete-time approximations (Krylov)
I Working with the value function directly (El Karoui, Borkar,Haussmann, Bouchard-Touzi (weak DPP)).
Probabilistic Approach
I Prove DPP. (Requires a priori regularity).
I Show that DPP ! v is a viscosity solution.
I Then comparison implies that v is the unique viscositysolution.
References:
I By discrete-time approximations (Krylov)
I Working with the value function directly (El Karoui, Borkar,Haussmann, Bouchard-Touzi (weak DPP)).
Probabilistic approach++
Some cases viscosity solutions are regular, then use
Perron ! viscosity solution ! smooth solution ! uniqueness
References: Shreve Soner (1994), Janecek and Sırbu (2012).
Probabilistic approach++
Some cases viscosity solutions are regular, then use
Perron ! viscosity solution ! smooth solution ! uniqueness
References: Shreve Soner (1994), Janecek and Sırbu (2012).
Probabilistic approach++
Some cases viscosity solutions are regular, then use
Perron ! viscosity solution ! smooth solution ! uniqueness
References: Shreve Soner (1994), Janecek and Sırbu (2012).
Perron’s method, Ishii (1987)
v
� = supw2U�,visc
w , v
+ = infw2U+,visc
w
are viscosity super- and sub-solutions.Here U�,visc and U+,visc are the collections of viscosity sub- andsuper-solutions respectively.
Question: why not inf/sup over classical super/sub-solutions?
Answer: Because one cannot prove (in general/directly) the resultis a viscosity solution. The classical solutions are not enough (theset of classical solutions is not stable under max or min).
Perron’s method, Ishii (1987)
v
� = supw2U�,visc
w , v
+ = infw2U+,visc
w
are viscosity super- and sub-solutions.
Here U�,visc and U+,visc are the collections of viscosity sub- andsuper-solutions respectively.
Question: why not inf/sup over classical super/sub-solutions?
Answer: Because one cannot prove (in general/directly) the resultis a viscosity solution. The classical solutions are not enough (theset of classical solutions is not stable under max or min).
Perron’s method, Ishii (1987)
v
� = supw2U�,visc
w , v
+ = infw2U+,visc
w
are viscosity super- and sub-solutions.Here U�,visc and U+,visc are the collections of viscosity sub- andsuper-solutions respectively.
Question: why not inf/sup over classical super/sub-solutions?
Answer: Because one cannot prove (in general/directly) the resultis a viscosity solution. The classical solutions are not enough (theset of classical solutions is not stable under max or min).
Perron’s method, Ishii (1987)
v
� = supw2U�,visc
w , v
+ = infw2U+,visc
w
are viscosity super- and sub-solutions.Here U�,visc and U+,visc are the collections of viscosity sub- andsuper-solutions respectively.
Question: why not inf/sup over classical super/sub-solutions?
Answer: Because one cannot prove (in general/directly) the resultis a viscosity solution. The classical solutions are not enough (theset of classical solutions is not stable under max or min).
Perron’s method, Ishii (1987)
v
� = supw2U�,visc
w , v
+ = infw2U+,visc
w
are viscosity super- and sub-solutions.Here U�,visc and U+,visc are the collections of viscosity sub- andsuper-solutions respectively.
Question: why not inf/sup over classical super/sub-solutions?
Answer: Because one cannot prove (in general/directly) the resultis a viscosity solution. The classical solutions are not enough (theset of classical solutions is not stable under max or min).
Objective
Provide a method to replace the first two steps in
Perron ! visc .solution ! smoothsolution ! uniqueness + (DPP)
(in case one cannot prove that the viscosity solutions are smooth)
and construct a viscosity solutions to the HJB that one candirectly compare with the value function.
Remarks on Perron’s method
v
� = supw2U�,visc
w , v
+ = supw2U+,visc
w
I Test function w does NOT compare to the value function vUNLESS one proves v is a viscosity solutions already AND theviscosity comparison
I If we ask w to be classical semi-solutions, we cannot provethat the inf/sup are viscosity solutions
Main Idea
Perform Perrons Method over a class of semi-solutions which are
I weak enough to conclude (in general/directly) that v� andv
+ are viscosity super/sub-solutions.
I strong enough to compare with the value function withoutstudying the properties of the value function
Stochastic solutions do the job! (Generalization of a solutionconcept introduced by Stroock and Varadhan for linear PDEs.)
Main Idea
Perform Perrons Method over a class of semi-solutions which are
I weak enough to conclude (in general/directly) that v� andv
+ are viscosity super/sub-solutions.
I strong enough to compare with the value function withoutstudying the properties of the value function
Stochastic solutions do the job! (Generalization of a solutionconcept introduced by Stroock and Varadhan for linear PDEs.)
Main Idea
Perform Perrons Method over a class of semi-solutions which are
I weak enough to conclude (in general/directly) that v� andv
+ are viscosity super/sub-solutions.
I strong enough to compare with the value function withoutstudying the properties of the value function
Stochastic solutions do the job! (Generalization of a solutionconcept introduced by Stroock and Varadhan for linear PDEs.)
Stochastic Perron
General statement:
I sup over stochastic sub-solutions is a viscosity
super-solution
v⇤ = supw2U�,stoch
w v .
I inf over stochastic super-solutions is a viscosity sub-solution
v
⇤ = infw2U+,stoch
.
Conclusion: v⇤ v v
⇤.
If a viscosity comparison holds, then v is the unique viscositysolution. In other words:
SP+visc. comparison ! v is the unique disc sol (DPE)+DPP.
Stochastic Perron
General statement:
I sup over stochastic sub-solutions is a viscosity
super-solution
v⇤ = supw2U�,stoch
w v .
I inf over stochastic super-solutions is a viscosity sub-solution
v
⇤ = infw2U+,stoch
.
Conclusion: v⇤ v v
⇤.
If a viscosity comparison holds, then v is the unique viscositysolution. In other words:
SP+visc. comparison ! v is the unique disc sol (DPE)+DPP.
Stochastic Perron
General statement:
I sup over stochastic sub-solutions is a viscosity
super-solution
v⇤ = supw2U�,stoch
w v .
I inf over stochastic super-solutions is a viscosity sub-solution
v
⇤ = infw2U+,stoch
.
Conclusion: v⇤ v v
⇤.
If a viscosity comparison holds, then v is the unique viscositysolution. In other words:
SP+visc. comparison ! v is the unique disc sol (DPE)+DPP.
Stochastic Perron
General statement:
I sup over stochastic sub-solutions is a viscosity
super-solution
v⇤ = supw2U�,stoch
w v .
I inf over stochastic super-solutions is a viscosity sub-solution
v
⇤ = infw2U+,stoch
.
Conclusion: v⇤ v v
⇤.
If a viscosity comparison holds, then v is the unique viscositysolution.
In other words:
SP+visc. comparison ! v is the unique disc sol (DPE)+DPP.
Stochastic Perron
General statement:
I sup over stochastic sub-solutions is a viscosity
super-solution
v⇤ = supw2U�,stoch
w v .
I inf over stochastic super-solutions is a viscosity sub-solution
v
⇤ = infw2U+,stoch
.
Conclusion: v⇤ v v
⇤.
If a viscosity comparison holds, then v is the unique viscositysolution. In other words:
SP+visc. comparison ! v is the unique disc sol (DPE)+DPP.
Remarks
I Morally, substituting Ito’s lemma with comparison principle inverification.
I Set of stochastic sub and super-solutions have to be carefullydefined (depending on the control problem) as to obtainviscosity solutions as sup/inf (and to retain the comparisonbuild in).
Obstacle Problems and Dynkin Games
State process:
dX
t
= b(t,Xt
)dt + �(t,Xt
)dWt
, Xs
= x .
Assumption: Continuous coe�cients with linear growth. [Thereexist (possibly non-unique) weak solutions of the SDE.] We willmake a selection (not necessarily Markovian).
Let g : Rd ! R l , u : [0,T ]⇥Rd ! R be bounded and measurablefunctions satisfying l u and l(T , ·) g u(T , ·).
Denote by T s,x the set of stopping times ⌧ (with respect to thefiltration (F s,x
t
)stT
) which satisfy s ⌧ T .
The first player (⇢) pays to the second player (⌧) the amount
J(s, x , ⌧, ⇢) :=
Es,x⇥I{⌧<⇢}l(⌧,X
s,x⌧ ) + I{⇢⌧,⇢<T})u(⇢,X
s,x⇢ ) + I{⌧=⇢=T}g(X
s,xT
)⇤.
Obstacle Problems and Dynkin GamesState process:
dX
t
= b(t,Xt
)dt + �(t,Xt
)dWt
, Xs
= x .
Assumption: Continuous coe�cients with linear growth. [Thereexist (possibly non-unique) weak solutions of the SDE.] We willmake a selection (not necessarily Markovian).
Let g : Rd ! R l , u : [0,T ]⇥Rd ! R be bounded and measurablefunctions satisfying l u and l(T , ·) g u(T , ·).
Denote by T s,x the set of stopping times ⌧ (with respect to thefiltration (F s,x
t
)stT
) which satisfy s ⌧ T .
The first player (⇢) pays to the second player (⌧) the amount
J(s, x , ⌧, ⇢) :=
Es,x⇥I{⌧<⇢}l(⌧,X
s,x⌧ ) + I{⇢⌧,⇢<T})u(⇢,X
s,x⇢ ) + I{⌧=⇢=T}g(X
s,xT
)⇤.
Obstacle Problems and Dynkin GamesState process:
dX
t
= b(t,Xt
)dt + �(t,Xt
)dWt
, Xs
= x .
Assumption: Continuous coe�cients with linear growth. [Thereexist (possibly non-unique) weak solutions of the SDE.] We willmake a selection (not necessarily Markovian).
Let g : Rd ! R l , u : [0,T ]⇥Rd ! R be bounded and measurablefunctions satisfying l u and l(T , ·) g u(T , ·).
Denote by T s,x the set of stopping times ⌧ (with respect to thefiltration (F s,x
t
)stT
) which satisfy s ⌧ T .
The first player (⇢) pays to the second player (⌧) the amount
J(s, x , ⌧, ⇢) :=
Es,x⇥I{⌧<⇢}l(⌧,X
s,x⌧ ) + I{⇢⌧,⇢<T})u(⇢,X
s,x⇢ ) + I{⌧=⇢=T}g(X
s,xT
)⇤.
Obstacle Problems and Dynkin GamesState process:
dX
t
= b(t,Xt
)dt + �(t,Xt
)dWt
, Xs
= x .
Assumption: Continuous coe�cients with linear growth. [Thereexist (possibly non-unique) weak solutions of the SDE.] We willmake a selection (not necessarily Markovian).
Let g : Rd ! R l , u : [0,T ]⇥Rd ! R be bounded and measurablefunctions satisfying l u and l(T , ·) g u(T , ·).
Denote by T s,x the set of stopping times ⌧ (with respect to thefiltration (F s,x
t
)stT
) which satisfy s ⌧ T .
The first player (⇢) pays to the second player (⌧) the amount
J(s, x , ⌧, ⇢) :=
Es,x⇥I{⌧<⇢}l(⌧,X
s,x⌧ ) + I{⇢⌧,⇢<T})u(⇢,X
s,x⇢ ) + I{⌧=⇢=T}g(X
s,xT
)⇤.
Obstacle Problems and Dynkin GamesState process:
dX
t
= b(t,Xt
)dt + �(t,Xt
)dWt
, Xs
= x .
Assumption: Continuous coe�cients with linear growth. [Thereexist (possibly non-unique) weak solutions of the SDE.] We willmake a selection (not necessarily Markovian).
Let g : Rd ! R l , u : [0,T ]⇥Rd ! R be bounded and measurablefunctions satisfying l u and l(T , ·) g u(T , ·).
Denote by T s,x the set of stopping times ⌧ (with respect to thefiltration (F s,x
t
)stT
) which satisfy s ⌧ T .
The first player (⇢) pays to the second player (⌧) the amount
J(s, x , ⌧, ⇢) :=
Es,x⇥I{⌧<⇢}l(⌧,X
s,x⌧ ) + I{⇢⌧,⇢<T})u(⇢,X
s,x⇢ ) + I{⌧=⇢=T}g(X
s,xT
)⇤.
Dynkin Games
The lower value of the Dynkin game
v⇤(s, x) := sup⌧2T s,x
inf⇢2T s,x
J(s, x , ⌧, ⇢),
and the upper value of the game
v
⇤(s, x) := inf⇢2T s,x
sup⌧2T s,x
J(s, x , ⌧, ⇢).
DPE for Dynkin games
⇢F (t, x , v , v
t
, vx
, vxx
) = 0,u(T , x) = g(x),
where
F (t,x , v , vt
, vx
, vxx
) :=
max{v � u,min{�v
t
� L
t
v , v � l}}= min{v � l ,max{�v
t
� L
v
, v � u}},
Stochastic super-solution
U+, is the set of functions v : [0,T ]⇥ Rd ! R which have thefollowing properties:
1. These functions are upper semicontinuous (USC) andbounded on [0,T ]⇥ Rd . In addition, they satisfy v � l andthe terminal condition v(T , ·) � g(·).
2. For each (s, x) 2 [0,T ]⇥ Rd , and any stopping times⇢1 ⇢2 2 T s,x , we have
v(⇢1,X s,x⇢1 ) � E
hv(⇢2 ^ ⇢+,X s,x
⇢2^⌧+)|Fs,x⇢1
i.
where the stopping time ⌧+(v ; s, x , ⇢1) is defined as
⌧+(v , s, x , ⌧1) := inf{t 2 [⇢1,T ] : v(t,X s,x) � u(t,X s,x)}.
Question: Why the starting stopping time? No Markov property.
Stochastic super-solution
U+, is the set of functions v : [0,T ]⇥ Rd ! R which have thefollowing properties:
1. These functions are upper semicontinuous (USC) andbounded on [0,T ]⇥ Rd . In addition, they satisfy v � l andthe terminal condition v(T , ·) � g(·).
2. For each (s, x) 2 [0,T ]⇥ Rd , and any stopping times⇢1 ⇢2 2 T s,x , we have
v(⇢1,X s,x⇢1 ) � E
hv(⇢2 ^ ⇢+,X s,x
⇢2^⌧+)|Fs,x⇢1
i.
where the stopping time ⌧+(v ; s, x , ⇢1) is defined as
⌧+(v , s, x , ⌧1) := inf{t 2 [⇢1,T ] : v(t,X s,x) � u(t,X s,x)}.
Question: Why the starting stopping time? No Markov property.
Stochastic Perron for obstacle problems
Define U�, stochastic sub-solutions, symmetrically. Let
v
� := supw2U�
w v⇤ v
⇤ v
+ := infw2U+
w .
Cannot show v
� 2 U� or v+ 2 U+, but what really matters is theclosedness under max/min .(Thanks to an argument similar to theproof of Dini’s lemma).
This is the reason we can assume continuity.
Theorem.
Iv
� is a viscosity supersoln of the DPE.
Iv
+ is a viscosity sub solution of the DPE.
Stochastic Perron for obstacle problems
Define U�, stochastic sub-solutions, symmetrically. Let
v
� := supw2U�
w v⇤ v
⇤ v
+ := infw2U+
w .
Cannot show v
� 2 U� or v+ 2 U+, but what really matters is theclosedness under max/min .(Thanks to an argument similar to theproof of Dini’s lemma).
This is the reason we can assume continuity.
Theorem.
Iv
� is a viscosity supersoln of the DPE.
Iv
+ is a viscosity sub solution of the DPE.
Stochastic Perron for obstacle problems
Define U�, stochastic sub-solutions, symmetrically. Let
v
� := supw2U�
w v⇤ v
⇤ v
+ := infw2U+
w .
Cannot show v
� 2 U� or v+ 2 U+, but what really matters is theclosedness under max/min .(Thanks to an argument similar to theproof of Dini’s lemma).
This is the reason we can assume continuity.
Theorem.
Iv
� is a viscosity supersoln of the DPE.
Iv
+ is a viscosity sub solution of the DPE.
Stochastic Perron for obstacle problems
Define U�, stochastic sub-solutions, symmetrically. Let
v
� := supw2U�
w v⇤ v
⇤ v
+ := infw2U+
w .
Cannot show v
� 2 U� or v+ 2 U+, but what really matters is theclosedness under max/min .(Thanks to an argument similar to theproof of Dini’s lemma).
This is the reason we can assume continuity.
Theorem.
Iv
� is a viscosity supersoln of the DPE.
Iv
+ is a viscosity sub solution of the DPE.
Stochastic Perron for obstacle problems
Define U�, stochastic sub-solutions, symmetrically. Let
v
� := supw2U�
w v⇤ v
⇤ v
+ := infw2U+
w .
Cannot show v
� 2 U� or v+ 2 U+, but what really matters is theclosedness under max/min .(Thanks to an argument similar to theproof of Dini’s lemma).
This is the reason we can assume continuity.
Theorem.
Iv
� is a viscosity supersoln of the DPE.
Iv
+ is a viscosity sub solution of the DPE.
Verification by comparison
Theorem
I If comparison holds, then there exist a unique, continuousviscosity solution equal to v
� = v⇤ = v
⇤ = v
+.
I The first hitting times are optimal.
Verification by comparison
Theorem
I If comparison holds, then there exist a unique, continuousviscosity solution equal to v
� = v⇤ = v
⇤ = v
+.
I The first hitting times are optimal.
Verification by comparison
Theorem
I If comparison holds, then there exist a unique, continuousviscosity solution equal to v
� = v⇤ = v
⇤ = v
+.
I The first hitting times are optimal.
Remarks about optimal stoppin, i.e. u = 1
References: El Karoui, Shiryaev, Fakeev: In the Markov case, thevalue function is the least excessive function, i.e.
v
+ = infw2U+
w = v .
The proof requires to show that v 2 U+. We avoid showing thatby using
v
� v v
+,
and use comparison.
We prove the continuity of the value function for free. (See studyof the continuity in Basan Ceci.)
Remarks about optimal stoppin, i.e. u = 1
References: El Karoui, Shiryaev, Fakeev: In the Markov case, thevalue function is the least excessive function, i.e.
v
+ = infw2U+
w = v .
The proof requires to show that v 2 U+. We avoid showing thatby using
v
� v v
+,
and use comparison.
We prove the continuity of the value function for free. (See studyof the continuity in Basan Ceci.)
Remarks about optimal stoppin, i.e. u = 1
References: El Karoui, Shiryaev, Fakeev: In the Markov case, thevalue function is the least excessive function, i.e.
v
+ = infw2U+
w = v .
The proof requires to show that v 2 U+. We avoid showing thatby using
v
� v v
+,
and use comparison.
We prove the continuity of the value function for free. (See studyof the continuity in Basan Ceci.)
Remarks about optimal stoppin, i.e. u = 1
References: El Karoui, Shiryaev, Fakeev: In the Markov case, thevalue function is the least excessive function, i.e.
v
+ = infw2U+
w = v .
The proof requires to show that v 2 U+. We avoid showing thatby using
v
� v v
+,
and use comparison.
We prove the continuity of the value function for free. (See studyof the continuity in Basan Ceci.)
References
1) Stochastic Perron’s method and verification without smoothnessusing viscosity comparison: the linear case, (EB and Mihai Sirbu),Proceedings of the American Mathematical Society, 140,3645-3654, 2012.
2) Stochastic Perron’s method and verification without smoothnessusing viscosity comparison: obstacle problems and Dynkin games,(EB and Mihai Sirbu), to appear in the Proceedings of the
American Mathematical Society.