Section II: Stochastic processes and Ito’s lemma

1 Introduction to stochastic processes

• A stochastic process is a variable that evolvesover time in a way that is at least in partrandom.

• eg. Temperature in Waterloo: a stochasticprocess with a deterministic component.

Econ 659 1

• continuous time process versus a discrete time process

• continuous variable versus a discrete variable

A stochastic process is defined by a probability law for theevolution of a variable, say x, over time. We can calculate theprobability that x1, x2, x3, ... lie in some specified range attimes t1, t2, t3, ...:

prob(a1 < x1 ≤ b1, a2 < x2 ≤ b2, ...)

Econ 659 2

• Example of a discrete-time, discrete state randomvariable: a random walk.

• Let xt denote a random variable that begins at x0 and attimes t = 1, 2, 3 takes a jump up of size ∆h withprobability p and a jump down of size ∆h withprobability q(= 1− p).

• x0 → x0 + ∆h with prob p

• x0 → x0 −∆h with prob q

• x follows a Markov process; Jumps are independent ofeach other.

• Sketch a lattice to show paths for x over time for threetime periods.

Econ 659 3

• Let ∆x be the change in x over the interval t→ t+ ∆t.x can take only one jump over this interval.

• Determine E(∆x), E((∆x)2), and Var(∆x)

Econ 659 4

• Consider the distribution of x after n moves. Note thatt = n∆t

• Consider the probability of j upward jumps. There are(nj

)ways in which exactly j upward jumps could occur.

• The probability of each of these is pjqn−j

• Pr(n, j) denotes the probability of j up moves and n− jdown moves.

• A binomial distribution.

pr(n, j) =

(n

j

)pjqn−j =

n!

j!(n− j)!pjqn−j (1)

Econ 659 5

• Let xn be the value of x after n steps:

E(xn − x0) = nE(∆x) (2)

V ar(xn − x0) = nV ar(∆x) (3)

• Sub in for E(∆x) and V ar(∆x) and note that n = t/∆t

E(xn − x0) =t

∆t(p− q)∆h =

t

∆t(2p− 1)∆h (4)

(5)

V ar(xn − x0) =t

∆t4pq∆h2 =

t

∆t4p(1− p)∆h2 (6)

• Note that the variance of xn − x0 increases with t. Anon-stationary process.

• What about the mean?

Econ 659 6

• Let the size of the jump be a continuous randomvariable, eg. ∆h ∼ N(0, σ2). This would be a discretetime, continuous state process.

• A first order AR(1) process is another example of adiscrete time, continuous state random variable

xt = δ + ρxt−1 + εt (7)

−1 < ρ < 1, ε ∼ N(0, σ)

• Random walks and AR(1) processes are Markovprocesses which means that only the current value isrelevant for predicting the future. Past history isirrelevant.

Econ 659 7

2 The Wiener Process (BrownianMotion)

• One of the most useful stochastic processes used in manyfields - social sciences, science, engineering - namedafter English botanist Robert Brown who discovered it in1827

• The motion exhibited by a small particle totallyimmersed in liquid or gas

• A concise definition was given by Norbert Wiener in aseries of papers beginning in 1918

Econ 659 8

Some useful definitions. Consider a random variable, X(t).

• X(t) has independent increments if for allt1 < t2 < ... < tn

X(tn)−X(tn−1), X(tn−1)−X(tn−2), ..., X(t1)−X(t0) (8)

are independent.

• X(t) has stationary increments if the distribution ofX(t+ s)−X(t) does not depend on t.

Econ 659 9

• A Markov process is one which satisfies the Markovproperty. X(t) satisfies the Markov property if for alls, t ≥ 0, and non-negative integers i,j,x(u), 0 ≤ u < s

P{X(t+ s) = j| X(s) = i,X(u) = x(u), 0 ≤ u < s}= P{X(ts) = j|X(s) = i}

• In other words the conditional distribution of the futureX(t+ s) given the present X(s) and the past X(u),0 ≤ u < s, depends only on the present and isindependent of the past.

Econ 659 10

• The Weiner process is a continuous time stochasticprocess

• A Markov process with mean zero and variance rate of 1

• If z(t) is a Wiener process then any ∆z satisfies∆z = ε

√∆t where ε ∼ N(0, 1).

• It follows that E(∆z) = 0 and

V ar(∆z) = V ar(ε√

∆t) (9)

= V ar(ε)(√

∆t2) = ∆t (10)

• The ∆z for any two short intervals, ∆t, are independent.

Econ 659 11

• Consider the change in z during a relatively long periodof time, t. Denote this change as z(t)− z(0).

• Can be regarded as the sum of changes in z in n smallintervals of length ∆t. n = t/∆t.

• z(t)− z(0) =∑n

i=1 εi√

∆t

• εi (i = 1, 2, ..., n) are N(0, 1) and are independent

• Then z(t)− z(0) is normally distributed with mean of 0and variance of t

• z is a Wiener process. Its variance grows linearly withthe time horizon.

Econ 659 12

• Let ∆t→ 0. Then

dz = εt√dt (11)

E(dz) = 0 (12)

V ar(dz) = E[(dz)2] = dt (13)

Econ 659 13

• A Wiener process has no time derivative. As ∆t→ 0

derivative of z with respect to t goes to infinity.

• The expected length of the path followed by z in anytime interval is infinite.

• The expected number of times z equals any particularvalue at any time interval is infinite.

Econ 659 14

Suppose z1 and z2 are Wiener processes. What is E(dz1dz2)?

Econ 659 15

• Use the Wiener process as a building block for ageneralized Wiener process that allows for adeterministic drift term.

• The drift rate is the mean change per unit of time. Thevariance rate is the variance per unit of time.

• The basic Wiener process has a drift of zero and variancerate of 1.

• Generalized Wiener Process with constants a and b

dx = adt+ bdz (14)

• Solve for the value of x if b = 0

Econ 659 16

• In a small interval ∆t, ∆x = a∆t+ bε√

∆t

• What are the mean and variance of ∆x? What are themean and variance of a change in x over a longer intervalt?

Econ 659 17

3 Linking Brownian motion &random walks

• The generalized Wiener process

dx = adt+ bdz (15)

dz = εt√dt (16)

E(dx) = adt (17)

V ar(dx) = b2dt (18)

• The discrete random walk. x = x0 at t = 0. At t = ∆t,x0 → x0 + ∆h with prob p, and x0 → x0 −∆h withprob q.

Econ 659 18

• Distribution of x after n moves (t = n∆t)

E(xn − x0) =t

∆t(p− q)∆h (19)

V ar(xn − x0) =t

∆t4pq∆h2 (20)

Econ 659 19

• We want to take the limit as ∆t→ 0 in such a a way thatthe mean and variance of x after a finite time t isindependent of ∆t, and we would like to recoverEquations (15), (17), and (18).

• From Equation (20), we should choose∆h = constant

√∆t. Why is this so?

• Show that for other choices of ∆h the variance ofxn − x0 is either zero or infinite in finite time.

• Choose ∆h = b√

∆t for some constant b and sub intoEquations (19) and (20) to get the mean and variance.

Econ 659 20

• From Equation (19) we see that for the mean of therandom walk to be independent of ∆t as ∆t→ 0 wemust have (p− q) = constant

√∆t.

• Choose p− q = ab

√∆t. Solve this for p and q using

p+ q = 1.

• We get

p =1

2

(1 +

a

b

√∆t)

(21)

q =1

2

(1− a

b

√∆t)

(22)

Econ 659 21

Sub into Equations (19) and (20) for p and q and let ∆t→ 0

E(xn − x0) =tb√∆t

a

b

√∆t = at (23)

V ar(xn − x0) = t4pqb2 = t(1− a2

b2∆t)b2 (24)

= tb2 as ∆t→ 0. (25)

Econ 659 22

• Note as ∆t→ 0, the number of steps approaches infinityand the binomial distribution converges to a normaldistribution.

• Now suppose xn − x0 becomes very small so thatxn − x0 ≈ dx and t = tn − t0 ≈ dt.

• Then E(dx) = adt and V ar(dx) = b2dt. We haveaccomplished our goal. We started with a random walkand have recovered a generalized Wiener process(Equations (15),(16) (17), (18))

Econ 659 23

4 Ito Processes

• Use the Wiener process as a building block.

dx = a(x, t)dt+ b(x, t)dz (26)

• a(x, t) and b(x, t) are known functions. a(x, t) is theinstantaneous drift rates. b2(x, t) is the instantaneousvariance rate.

• E(dx) = a(x, t)dt and V ar(dx) = b2(x, t)dt

• Notice both the expected drift rate and variance rate maychange over time.

Econ 659 24

• In the small time interval between t and t+ ∆t, xchanges to x+ ∆x where

∆x = a(x, t)∆t+ b(x, t)ε√

∆t (27)

• Assumes drift and variance rate of x remain constant andequal to a(x, t) and b(x, t)2 between t and t+ ∆t.

Econ 659 25

4.1 Geometric Brownian Motion

• Let a(x, t) = αx and b(x, t) = σx, α and σ areconstants. Then,

dx = αxdt+ σxdz (28)

• Note that dxx∼ N or d[lnx] ∼ N

• GBM is often used to model stock prices.

• Suppose x represents a stock’s price and that volatility iszero. In this case we can solve for xt by directintegration. The result is that xt = x0e

αt

• It can also be shown that for a given x0, E[xt] = x0eαt.

Econ 659 26

• Note that for GBM the variability of the proportionatereturn, dx/x, in any ∆t is the same regardless of thelevel of x.

• Example: A stock pays no dividends, has a volatility of30% per annum, and provides an expected return of 15%per year with continuous compounding.

• The stochastic differential equation describing this stockis dx

x= 0.15dt+ 0.3dz.

• A discrete time approximation would be:∆xx

= 0.15∆t+ 0.3ε√

∆t, ε ∼ N(0, 1) where ∆x is theincrease in stock price over the next interval.

Econ 659 27

Consider a time interval of 1 week and the initial x0 = 100.What are the mean and standard deviation of ∆x?

Econ 659 28

• It can be shown that the variance of x is

var(x(t)) = x20e

2αt(eσ2t − 1) (29)

We will derive this later.

• Determine a formula for the expected present discountedvalue of x(t)?

Econ 659 29

4.2 Mean reverting processes

• More appropriate for commodities than GBM. Why?

• Ornstein-Uhlenbeck process

dx = η(x− x)dt+ σdz (30)

η is the speed of mean reversion

• Equation (30) is the limit of an AR(1) process as∆t→ 0.

xt − xt−1 = x(1− e−η) + (e−η − 1)xt−1 + εt (31)

Econ 659 30

• It can be shown that

E(xt) = x+ (x0 − x)e−ηt (32)

var(xt − x) =σ2

2η(1− e−2ηt) (33)

• A Markov process, but does not have independentincrements.

Econ 659 31

• Alternative mean reverting processes

dx = η(x− x)dt+ σxdz (34)

dx = ηx(x− x)dt+ σxdz (35)

Econ 659 32

5 Ito’s lemma

5.1 Derivation

• Normal rules of calculous do not apply to stochasticdifferential equations. We need a result from stochasticcalculous.

• A general Ito process

dx = a(x, t)dt+ b(x, t)dz (36)

• Consider a function G = G(x, t). After dt,G→ G+ dG.

Econ 659 33

• Ito’s lemma

dG =(a(x, t)∂G

∂x+ b2(x,t)

2∂2G∂x2

+ ∂G∂t

)dt

+b(x, t)∂G∂xdz (37)

Econ 659 34

• It will be useful to find E(dz2).

• dz2 = ε2dt.

• We know that E(ε) = 0 and V ar(ε) = 1

• But V ar(ε) = E(ε2)− (E(ε))2 = 1. This implies thatE(ε2) = 1

• It follows that

E(dz2) = E(ε2dt) = E(ε2)E(dt) = dt (38)

• It can be shown that as dt→ 0, ε2dt becomesnon-stochastic, so that with probability 1, dz2 → dt asdt→ 0

Econ 659 35

• Use a Taylor series to approximate the function G(x, t)

dG =∂G

∂xdx+

∂G

∂tdt+

1

2

∂2G

∂x2dx2+

1

6

∂3G

∂x3dx3+... (39)

• Denoting a(x, t) ≡ a and b(x, t) ≡ b,

(dx)2 = (adt+ bdz)2 = a2dt2 + 2(adt)(bdz) + b2dz2

= a2dt2 + 2(adt)bε√dt+ b2ε2dt

= a2dt2 + 2abε(dt3/2) + b2ε2dt

(40)

• The last term is the largest for small dt.

• dt3/2 and dt2 go to zero faster than dt as it becomesinfinitesimally small. These terms can be ignored.

Econ 659 36

• We are left with

dx2 = b2dz2 = b2dt, as dt→ 0 (41)

Econ 659 37

• In Equation (39) every term associated with theexpansion of dx3 will include dt raised to a powergreater than 1 and will go to zero faster than dt in thelimit. These terms can be ignored.

• Sub in for dx and dx2 in Equation (39)

dG =∂G

∂x(a(x, t)dt+b(x, t)dz)+

∂G

∂tdt+

1

2

∂2G

∂x2b2(x, t)dt

(42)

• Rearranging give Ito’s lemma.

dG =

[aGx +Gt +Gxx

b2

2

]dt+Gxbdz (43)

Econ 659 38

• Ito’s lemma may be extended to two or more processes.Consider two stochastic variables, x and y.

dx = ax(x, t)dt+ bx(x, t)dzx (44)

dy = ay(y, t)dt+ by(y, t)dzy (45)

• For the function G(x, y) Ito’s lemma is as follows:

dG = [Gt + axGx + ayGy +Gxxb2x2

+Gyyb2y2

+ρxybxbyGxy]dt+Gxbxdzx +Gybydzy (46)

where ρxy = E[dzxdzy] is the correlation coefficientbetween x and y.

Econ 659 39

Note that:

E[dzxdzy] = E[εx√dtεy√dt] (47)

= E[εx√dt]E[εy

√dt] + cov(εxεy)dt

= cov(εxεy)dt

= ρxydt

The last two lines follow because εi and εj have standardnormal distributions.

Econ 659 40

Examples

1. x is described by dx = αxdt+ σxdz. SupposeG(x) = log(x). What stochastic process is followed byG(x)?

2. Consider correlated processes, x and y with correlationcoefficient ρ:

dx = αxxdt+ σxxdzx (48)

dy = αyydt+ σyydzy (49)

Suppose F (x, y) = xy. Use Ito’s lemma to find anexpression for dF .

Econ 659 41

5.2 An Application of Ito’s lemma

• Previously we saw that a stochastic process such asdS = µdt+ σdz where µ and σ are constants can beintegrated exactly from t = 0 to t = tn to give

S(tn)− S(0) = µtn + σ[z(tn)− z(0)]

z(tn)− z(0) ∼ N(0, tn)

• It is only when µ and σ are constants that we can exactlyintegrate the stochastic differential equation.

• In other cases we can use Ito’s lemma to derive theproperties of a stochastic variable.

Econ 659 42

• Suppose dS = µSdt+ σSdz, with µ and σ as constants.

• In this case the mean of S can be shown to be:

E[dS] = dS = E[µS]dt

S = S0eµt (50)

Econ 659 43

• Now let G(S) = S2. Then GS = 2S and GSS = 2. FromIto’s lemma:

dG = E[2µS2 + σ2S2]dt+ E[2S2σdz]

= E[2µS2 + σ2S2]dt = [2µ+ σ2]Gdt (51)

• We can integrate directly to find G:

G = G0e(2µ+σ2)t

E(S2) = S20e

(2µ+σ2)t (52)

Econ 659 44

• From Equations (50) and (52) we have:

V ar(S) = E(S2)− [E(S)]2

= S20e

(2µ+σ2)t − S20e

2µt

= S0e2µt[eσ

2t − 1]

= S2(eσ2t − 1) (53)

• So we have used Ito’s lemma to find the mean andvariance of a stochastic variable S that follow geometricBrownian motion.

Econ 659 45

6 Jump Processes

• Some economic variables of interest make suddendiscrete jumps.

• Price of oil - mixed Brownian motion-jump process

• Poisson process - a process subject to jumps of fixed orrandom size. Each jump is called an event.

• Let φ denote the mean arrival rate of an event during dt.It represents the probability of an event over theinfinitesimal interval dt

• Denote the size of the jump as u. This can be a randomvariable.

Econ 659 46

An aside on the Poisson distribution

• Let Z represent the number of events in a fixed timeinterval ∆t. If Z has a Poisson distribution theprobability density function is

f(Z,∆t) =(φ∆t)Ze−φ∆t

Z!, Z = 0, 1, 2, ...

(54)

• The number of events in non-overlapping intervals areindependent.

• Show that the expected time until the first event is 1/φ.

• The prob of Z = 1 in ∆t is φ∆t. (This can be foundusing a Taylor series approximation, ignoring terms of

Econ 659 47

∆t2 and higher.)

Econ 659 48

• q is a Poisson process

dq = 0 with probability 1− φdt= 1 with probability φdt (55)

• Let x be described by the following stochasticdifferential equation

dx = f(x, t)dt+ g(x, t)dq (56)

where f(x, t) and g(x, t) are known non-randomfunctions.

Econ 659 49

• Suppose when x jumps it jumps to Jx where J is theproportional size of the jump. Restrict J to beingnon-negative.

dxjump = Jx− x = (J − 1)x (57)

Therefore g(x, t) ≡ (J − 1)x and

dx = f(x, t)dt+ (J − 1)xdq (58)

Econ 659 50

• Assume the jump size has some known pdf, g(J). Thisimplies ∫ ∞

−∞g(J)dJ =

∫ ∞0

g(J)dJ = 1 (59)

• Suppose we have an investment whose value depends onx and t. Denote its value as V (x, t).

• The change in V due to jumps and the deterministic driftare, respectively:

dVjumps = [V (Jx, t)− V (x, t)]dq

dVdeter = [Vtdt+ Vxf(x, t)]dt

Econ 659 51

• Adding these two gives

dV = [Vt + Vxf(x, t)]dt+ [V (Jx, t)− V (x, t)]dq (60)

• Take the expected value of dV , assuming the probabilityof a jump and the jump size are independent:

E[dV ] = [Vt+Vxf(x, t)]dt+EJ [V (Jx, t)−V (x, t)]E[dq]

(61)

• We can use Equation (61) to value projects which dependon a variable that exhibit jumps. This will be shown laterin the course.

• Look at the examples from Dixit and Pindyck, pages86-87.

Econ 659 52