solution of stochastic process and modelling

31
7/23/2019 Solution of Stochastic Process and Modelling http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 1/31                     p rest  = 0.1                   m ∈ {1, 2, 3, 4}    1 prest m  =  0.9 m         2                                      

Upload: anonymous-8ze6fjrb

Post on 19-Feb-2018

222 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 1/31

❙ ♦ ❝ ❤ ❛ ✐ ❝ ♦ ❝ ❡ ❡ ❛ ♥ ❞ ▼ ♦ ❞ ❡ ❧ ✐ ♥ ❣ ✭ ▼ ❆ ❚ ❍ ✻ ✻ ✻ ✵ ✮  

❆ ✐ ❣ ♥ ♠ ❡ ♥ ✸  

● ♦ ✉ ❛ ✈ ❙ ❛ ❤ ❛ ✭ ❘ ■ ◆ ✿ ✻ ✻ ✶ ✺ ✸ ✾ ✽ ✻ ✻ ✮  

✶ ❆ ♣ ♣ ❧ ✐ ❝ ❛ ✐ ♦ ♥ ❛ ♥ ❞ ❈ ❛ ❧ ❝ ✉ ❧ ❛ ✐ ♦ ♥ ♦ ❜ ❧ ❡ ♠  

✶ ✳ ✶ ❘ ❛ ✐ ♥ ❛ ▼ ❛ ❦ ♦ ✈ ▼ ❛ ③ ❡  

✶ ✷ ✸ ✹  

✺ ✻ ✼ ✽  

✾ ✶ ✵ ✶ ✶ ✶ ✷  

✶ ✸ ✶ ✹ ✶ ✺ ✶ ✻  

❚ ❛ ❜ ❧ ❡ ✶ ✿ ❋ ✐ ❣ ✉ ❡ ❤ ♦ ✇ ✐ ♥ ❣ ❤ ❡ ♠ ❛ ③ ❡ ❛ ❧ ♦ ♥ ❣ ✇ ✐ ❤ ✐ ♥ ❞ ✐ ❝ ❡ ❡ ♣ ❡ ❡ ♥ ✐ ♥ ❣ ❤ ❡ ❛ ❡ ✳ ❘ ❛ ❤ ❜ ❡ ❛ ❢ ♦ ♠ ❙ ❛ ❡ ✲ ✶ ✻ ✱ ❤ ❡ ❤ ♦ ❝ ❦ ✐  

❧ ♦ ❝ ❛ ❡ ❞ ✐ ♥ ❙ ❛ ❡ ✲ ✽ ❛ ♥ ❞ ❙ ❛ ❡ ✲ ✾ ✱ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ✐ ❧ ♦ ❝ ❛ ❡ ❞ ✐ ♥ ❙ ❛ ❡ ✲ ✸ ❛ ♥ ❞ ❙ ❛ ♠ ❛ ❢ ♦ ♠ ❙ ❛ ❡ ✲ ✶ ✳

✭ ❛ ✮  ❚ ❤ ❡ ✇ ❛ ② ❘ ❛ ❤ ❜ ❡ ♠ ♦ ✈ ❡ ✐ ♥ ❤ ❡ ♠ ❛ ③ ❡ ✐ ❣ ✉ ✐ ❞ ❡ ❞ ❜ ② ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ❤ ❡ ❡ ✉ ❧ ❡ ✿

• ❘ ✉ ❧ ❡ ✲ ✶ ✿   ■ ♥ ❡ ✈ ❡ ② ❡ ♣ ♦ ❝ ❤ ❤ ❡ ❡ ✐ ❛ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②    prest   = 0.1 ❤ ❛ ❘ ❛ ❤ ❜ ❡ ✇ ♦ ♥ ✬ ♠ ♦ ✈ ❡ ❜ ❡ ❝ ❛ ✉ ❡ ❤ ❡ ✴ ❤ ❡ ✐ ✐ ❡ ❞ ❛ ♥ ❞  

✇ ❛ ♥ ♦ ❡ ✳

• ❘ ✉ ❧ ❡ ✲ ✷ ✿   ■ ♥ ❡ ✈ ❡ ② ❡ ♣ ♦ ❝ ❤ ✱ ❘ ❛ ❤ ❜ ❡ ❝ ❛ ♥ ♠ ♦ ✈ ❡ ♦ ♥ ❧ ②   ♦ ♥ ❡ ❡ ♣   ❡ ✐ ❤ ❡   ✉ ♣ ♦ ❞ ♦ ✇ ♥ ♦ ❧ ❡ ❢ ♦ ✐ ❣ ❤   ♣ ♦ ✈ ✐ ❞ ❡ ❞ ❤ ❛ ❤ ❡ ❡ ✐  

♥ ♦ ♦ ❜ ❛ ❝ ❧ ❡ ✐ ♥ ❤ ❛ ❞ ✐ ❡ ❝ ✐ ♦ ♥ ✳ ❉ ✐ ❛ ❣ ♦ ♥ ❛ ❧ ♠ ♦ ✈ ❡ ♠ ❡ ♥ ✐ ◆ ❖ ❚ ❛ ❧ ❧ ♦ ✇ ❡ ❞ ✳

• ❘ ✉ ❧ ❡ ✲ ✸ ✿   ❙ ❛ ② ❤ ❛ ✐ ♥ ❛ ❣ ✐ ✈ ❡ ♥ ❛ ❡ ❘ ❛ ❤ ❜ ❡ ❝ ❛ ♥ ♠ ♦ ✈ ❡ ✐ ♥    m  ∈ {1, 2, 3, 4} ♣ ♦ ✐ ❜ ❧ ❡ ❞ ✐ ❡ ❝ ✐ ♦ ♥ ✳ ❚ ❤ ❡ ♥ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②  

❤ ❛ ✐ ✇ ✐ ❧ ❧ ♠ ♦ ✈ ❡ ✐ ♥ ♦ ♥ ❡ ♦ ❢ ❤ ❡ ❡ ❞ ✐ ❡ ❝ ✐ ♦ ♥ ✐  

1− prestm

  =   0.9m

✱ ✐ ✳ ❡ ✳ ❘ ❛ ❤ ❜ ❡ ❤ ❛ ❡ ✉ ❛ ❧ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♦ ♠ ♦ ✈ ❡ ✐ ♥ ❛ ♥ ②  

❞ ✐ ❡ ❝ ✐ ♦ ♥ ✳

■ ▼ ❖ ❘ ❚ ❆ ◆ ❚ ✿  ❲ ✐ ❤ ♦ ✉   ❘ ✉ ❧ ❡ ✲ ✶   ❤ ❡ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ✇ ✐ ❧ ❧ ❜ ❡ ♣ ❡ ✐ ♦ ❞ ✐ ❝ ✇ ✐ ❤ ♣ ❡ ✐ ♦ ❞   2✳ ❚ ❤ ✐ ♠ ❛ ② ❧ ❡ ❛ ❞ ♦ ❝ ♦ ♠ ♣ ❧ ✐ ❝ ❛ ✐ ♦ ♥ ✐ ♥  

❝ ♦ ♠ ♣ ✉ ❛ ✐ ♦ ♥ ✳ ❍ ❡ ♥ ❝ ❡ ✇ ❡ ✐ ♠ ♣ ♦ ❡ ❞   ❘ ✉ ❧ ❡ ✲ ✶   ♦ ♠ ❛ ❦ ❡ ❤ ❡ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ❛ ♣ ❡ ✐ ♦ ❞ ✐ ❝ ✳

❚ ❤ ❡ ❛ ❜ ♦ ✈ ❡ ✉ ❧ ❡ ❧ ❡ ❛ ❞ ♦ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①   P ✳

✶ ✷ ✸ ✹ ✺ ✻ ✼ ✽ ✾ ✶ ✵ ✶ ✶ ✶ ✷ ✶ ✸ ✶ ✹ ✶ ✺ ✶ ✻

✶  ✵ ✳ ✶ ✵ ✵ ✵ ✵ ✳ ✾ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵

✷  ✵ ✵ ✳ ✶ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✸  ✵ ✵ ✳ ✹ ✺ ✵ ✳ ✶ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✹  ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✳ ✶ ✵ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✺  ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✵ ✳ ✶ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✻  ✵ ✵ ✳ ✷ ✷ ✺ ✵ ✵ ✵ ✳ ✷ ✷ ✺ ✵ ✳ ✶ ✵ ✳ ✷ ✷ ✺ ✵ ✵ ✵ ✳ ✷ ✷ ✺ ✵ ✵ ✵ ✵ ✵ ✵  

✼  ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✳ ✶ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✽  ✵ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✳ ✶ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✾  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✶ ✵ ✵ ✵ ✵ ✳ ✾ ✵ ✵ ✵

✶ ✵  ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✵ ✳ ✶ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✵ ✵  

✶ ✶  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✳ ✶ ✵ ✵ ✵ ✵ ✳ ✹ ✺ ✵  

✶ ✷  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✶ ✵ ✵ ✵ ✵ ✳ ✾

✶ ✸  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✵ ✳ ✶ ✵ ✳ ✹ ✺ ✵ ✵  

✶ ✹  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✳ ✶ ✵ ✳ ✹ ✺ ✵  

✶ ✺  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✸ ✵ ✵ ✳ ✸ ✵ ✳ ✶ ✵ ✳ ✸

✶ ✻  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✵ ✵ ✳ ✹ ✺ ✵ ✳ ✶

❚ ❛ ❜ ❧ ❡ ✷ ✿ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①   P  ❣ ♦ ✈ ❡ ♥ ✐ ♥ ❣ ❤ ❡ ♠ ♦ ✈ ❡ ♠ ❡ ♥ ♦ ❢ ❘ ❛ ❤ ❜ ❡ ✐ ♥ ❤ ❡ ♠ ❛ ③ ❡ ✳

✶ 

Page 2: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 2/31

▲ ❊ ❆ ❙ ❊ ◆ ❖ ❚ ❊ ✿  

✶ ✮  ❚ ❤ ❡ ▼ ❛ ❦ ♦ ✈ ❈ ❤ ❛ ✐ ♥ ❣ ♦ ✈ ❡ ♥ ❡ ❞ ❜ ② ❤ ❡ ❛ ❜ ♦ ✈ ❡ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ① ✐   ✐ ❡ ❞ ✉ ❝ ✐ ❜ ❧ ❡   ❛ ♥ ❞ ❢ ♦ ♠ ❛   ❝ ❧ ♦ ❡ ❞   ❝ ♦ ♠ ♠ ✉ ♥ ✐ ❝ ❛ ✲  

✐ ♦ ♥ ❝ ❧ ❛ ✳

✷ ✮  ❙ ✐ ♥ ❝ ❡ ❤ ❡ ▼ ❛ ❦ ♦ ✈ ❈ ❤ ❛ ✐ ♥ ❢ ♦ ♠ ❛ ❝ ❧ ♦ ❡ ❞ ❝ ♦ ♠ ♠ ✉ ♥ ✐ ❝ ❛ ✐ ♦ ♥ ❝ ❧ ❛ ✱ ✇ ❡ ❝ ❛ ♥ ✉ ❡ ❤ ❡ ❝ ♦ ♥ ❝ ❡ ♣ ❢ ♦ ♠   ▲ ❡ ❝ ✉ ❡ ✲ ✶ ✷   ✭  19th ❖ ❝ ♦ ❜ ❡ ✱

✷ ✵ ✶ ✺ ✮ ♦ ❛ ♥ ✇ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ♣ ❛ ✳ ❚ ♦ ❛ ♥ ✇ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ♣ ❛ ✇ ❡ ♠ ✐ ❣ ❤ ♥ ♦ ❣ ✐ ✈ ❡ ❛ ❞ ❡ ❛ ✐ ❧ ❡ ❞ ❡ ① ♣ ❧ ❛ ♥ ❛ ✐ ♦ ♥ ♦ ❢ ❤ ❡ ❢ ♦ ♠ ✉ ❧ ❛  

✉ ❡ ❞ ✳ ❘ ❛ ❤ ❡ ✱ ✇ ❡ ✇ ✐ ❧ ❧ ❥ ✉ ♠ ❛ ♣ ❤ ❡ ✉ ❡ ✐ ♦ ♥ ♦ ✇ ❤ ❛ ✇ ❡ ❧ ❡ ❛ ♥ ❡ ❞ ✐ ♥   ▲ ❡ ❝ ✉ ❡ ✲ ✶ ✷   ❛ ♥ ❞ ❞ ✐ ❡ ❝ ❧ ② ✉ ❡ ❤ ❡ ❢ ♦ ♠ ✉ ❧ ❛ ❢ ♦ ♠   ▲ ❡ ❝ ✉ ❡ ✲ ✶ ✷   ✳

✸ ✮  ❲ ❡ ♠ ✐ ❣ ❤ ❤ ❛ ✈ ❡ ❞ ♦ ♥ ❡ ♦ ♠ ❡   ♥ ♦ ❛ ✐ ♦ ♥ ♦ ✈ ❡ ❧ ♦ ❛ ❞ ✐ ♥ ❣  

✶ 

♦ ❛ ♥ ✇ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ♣ ❛ ♦ ❢ ❤ ❡ ✉ ❡ ✐ ♦ ♥ ✳

✹ ✮  ❚ ❤ ❡ ▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣ ♦ ❛ ❧ ❧ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ♣ ❛ ✐ ✐ ♥ ❝ ❧ ✉ ❞ ❡ ❞ ✐ ♥   ❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ✲ ❆   ✳

✭ ❜ ✮  ❚ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ❝ ❛ ♥ ❜ ❡ ❛ ❜ ❛ ❝ ❡ ❞ ❛ ✿ ❙ ❛ ✐ ♥ ❣ ❢ ♦ ♠ ❛ ❡   i✱ ✇ ❤ ❛ ✐ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❡ ♣ ♦ ❝ ❤ ✉ ♥ ✐ ❧ ❛ ❡    j ✐  

❡ ❛ ❝ ❤ ❡ ❞  ❄ ■ ♥ ❤ ✐ ✉ ❡ ✐ ♦ ♥   i = 16 ❛ ♥ ❞   j  = 3✳ ❚ ♦ ❛ ♥ ✇ ❡ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✇ ❡ ❛ ❞ ♦ ♣ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ❡ ♣ ✿

❙ ❡ ♣ ✲ ✶ ✿   ❲ ❡ ♠ ❛ ❦ ❡ ❛ ❡   j  = 3 ❛ ❜ ♦ ❜ ✐ ♥ ❣ ✳ ❚ ♦ ❞ ♦ ❤ ✐ ✇ ❡ ♠ ❛ ❦ ❡ ❛ ♠ ♦ ❞ ✐ ❢ ❡ ❞ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①  

  P  ❛ ❢ ♦ ❧ ❧ ♦ ✇ ✿ ✶ ✮ 

❙ ❡  

P   = P  ✳ ✷ ✮  ❙ ❡ ❛ ❧ ❧ ❤ ❡ ❡ ❧ ❡ ♠ ❡ ♥ ♦ ❢ ❤ ❡    3rd ♦ ✇ ♦ ❢  

  P  ❛    0 ❡ ① ❝ ❡ ♣     p33 ✇ ❤ ✐ ❝ ❤ ✐ ❡ ♦    1 ✳

❙ ❡ ♣ ✲ ✷ ✿   ❲ ❡ ❝ ♦ ♥ ✉ ❝ ❛ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①   Q ✭ ❛ ✐ ❝ ❧ ② ✉ ❜ ✲ ♦ ❝ ❤ ❛ ✐ ❝ ♠ ❛ ✐ ① ✮ ✇ ❤ ✐ ❝ ❤ ❝ ♦ ♥ ❛ ✐ ♥ ❤ ❡ ♦ ♥ ❡ ✲ ❡ ♣  

❛ ♥ ✐ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❢ ♦ ♠ ♦ ♥ ❡ ❛ ♥ ✐ ❡ ♥ ❛ ❡ ♦ ❛ ♥ ♦ ❤ ❡ ❛ ♥ ✐ ❡ ♥ ❛ ❡ ✳ ■ ♥ ❤ ❡ ♠ ♦ ❞ ✐ ✜ ❡ ❞ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①  

P ✱ ❛ ❧ ❧ ❛ ❡ ❛ ❡ ❛ ♥ ✐ ❡ ♥ ❡ ① ❝ ❡ ♣ ❙ ❛ ❡ ✲ ✸ ✳ ❍ ❡ ♥ ❝ ❡ ✇ ❡ ❞ ❡ ❧ ❡ ❡ ❤ ❡  

  3rd

♦ ✇ ❛ ♥ ❞  

3rd

❝ ♦ ❧ ✉ ♠ ♥ ❢ ♦ ♠    P  ♦ ❣ ❡  

  Q✳

❙ ❡ ♣ ✲ ✸ ✿   ▲ ❡ ❛ ② ❤ ❛ ❤ ❡ ✐ ♠ ❡ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣ ♦ ❡ ❛ ❝ ❤ ❡ ♣ ♦ ❝ ❤ ✐   1 ✳ ❲ ❡ ❤ ❡ ❡ ❢ ♦ ❡ ❡ ❤ ❡ ❛ ❞ ❞ ✐ ✐ ✈ ❡ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ❛ ❧    f   ≡ 1 ✳ ▼ ♦ ❡  

♣ ❡ ❝ ✐ ❡ ❧ ② ✇ ❡ ❞ ❡ ✜ ♥ ❡ ❛ ❝ ♦ ❧ ✉ ♠ ♥ ✈ ❡ ❝ ♦   f  ∈ R15❛ ♥ ❞ ❡ ❛ ❧ ❧ ✐ ❡ ❧ ❡ ♠ ❡ ♥ ❛   1 ✳

❙ ❡ ♣ ✲ ✹ ✿   ❲ ❡ ❝ ♦ ♠ ♣ ✉ ❡ ❤ ❡ ✈ ❡ ❝ ♦  

µτ   = (I  − Q)−1

✇ ❤ ✐ ❝ ❤ ❤ ♦ ✇ ❤ ❡ ❛ ✈ ❡ ❛ ❣ ❡ ✐ ♠ ❡ ♦ ❡ ❛ ❝ ❤ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ❛ ✐ ♥ ❣ ❢ ♦ ♠ ❛ ♥ ② ❛ ♥ ✐ ❡ ♥ ❛ ❡ ✳

❙ ❡ ♣ ✲ ✺ ✿   ❲ ❡ ❛ ❡ ✐ ♥ ❡ ❡ ❡ ❞ ✐ ♥ ❤ ❡ ❛ ✈ ❡ ❛ ❣ ❡ ✐ ♠ ❡ ♦ ❡ ❛ ❝ ❤ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ❣ ✐ ✈ ❡ ♥ ❤ ❛ ❘ ❛ ❤ ❜ ❡ ❛ ❢ ♦ ♠ ❙ ❛ ❡ ✲ ✶ ✻ ✳ ❚ ❤ ✐ ✐ ❣ ✐ ✈ ❡ ♥  

❜ ②  µτ  (15)✱ ❤ ❡    15th ♦ ✇ ♦ ❢  µτ ✳

❚ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ✐ ♠ ❡ ✐ ❢ ♦ ✉ ♥ ❞ ♦ ❜ ❡   85.1852 ✳

✭ ❝ ✮  ❚ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ❝ ❛ ♥ ❜ ❡ ❛ ❜ ❛ ❝ ❡ ❞ ❛ ✿ ❙ ❛ ✐ ♥ ❣ ❢ ♦ ♠ ❛ ❡   i✱ ✇ ❤ ❛ ✐ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❡ ♣ ♦ ❝ ❤ ♣ ❡ ♥ ✐ ♥ ❛ ❡    k ✭   k

❝ ❛ ♥ ❜ ❡ ❛ ✈ ❡ ❝ ♦ ❝ ♦ ♥ ✐ ✐ ♥ ❣ ♦ ❢ ♠ ❛ ♥ ② ❛ ❡ ✮ ✱ ❜ ❡ ❢ ♦ ❡ ❛ ❡    j ✐ ✈ ✐ ✐ ❡ ❞ ❄   ■ ♥ ❤ ✐ ✉ ❡ ✐ ♦ ♥    i  = 16 ✱   k  =

8 9

❛ ♥ ❞   j  = 3 ✳ ❚ ♦ 

❛ ♥ ✇ ❡ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✇ ❡ ❛ ❞ ♦ ♣ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ❡ ♣ ✿

❙ ❡ ♣ ✲ ✶ ✿   ❲ ❡ ♠ ❛ ❦ ❡ ❛ ❡   j  = 3 ❛ ❜ ♦ ❜ ✐ ♥ ❣ ❧ ✐ ❦ ❡ ✇ ❡ ❞ ✐ ❞ ✐ ♥   ❛ ✭ ❜ ✮  ✳

❙ ❡ ♣ ✲ ✷ ✿   ❲ ❡ ❝ ♦ ♥ ✉ ❝ ❤ ❡ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①   Q ❧ ✐ ❦ ❡ ✇ ❡ ❞ ✐ ❞ ✐ ♥   ❛ ✭ ❜ ✮  ✳

❙ ❡ ♣ ✲ ✸ ✿   ❉ ❡ ✜ ♥ ❡ ❤ ❡ ❛ ❞ ❞ ✐ ✐ ✈ ❡ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ❛ ❧ ❛   f  ≡ δ km ✇ ❤ ❡ ❡ ❤ ❡ ❞ ✐ ❛ ❝ ❞ ❡ ❧ ❛ ❢ ✉ ♥ ❝ ✐ ♦ ♥    δ km ❢ ♦ ❛ ✈ ❡ ❝ ♦    k =

k1   k2   · · ·   kn

✐ ❞ ❡ ✜ ♥ ❡ ❞ ❛  

δ km =1 ;  m  =  k1 ♦    m =  k2   · · · ♦    m =  kn

0 ; ♦ ✳ ✇ ✳

▼ ♦ ❡ ♣ ❡ ❝ ✐ ❡ ❧ ②   f  ∈ R15✐ ❛ ❝ ♦ ❧ ✉ ♠ ♥ ✈ ❡ ❝ ♦ ✇ ❤ ♦ ❡   7th ❛ ♥ ❞  8th ♦ ✇ ✭ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣ ♦ ❤ ❡   8th ❛ ♥ ❞ ❤ ❡   9th ❛ ❡ ❡ ♣ ❡ ❝ ✐ ✈ ❡ ❧ ② ✮  

✐ ❡ ♦   1 ❛ ♥ ❞ ❛ ❧ ❧ ♦ ❤ ❡ ❡ ❧ ❡ ♠ ❡ ♥ ✐ ❡ ♦    0 ✳

❙ ❡ ♣ ✲ ✹ ✿   ❲ ❡ ❝ ♦ ♠ ♣ ✉ ❡ ❤ ❡ ✈ ❡ ❝ ♦  

µshock = (I  − Q)−1

✶ 

■ ✐ ❥ ✉ ❛ ❡ ♠ ✇ ❡ ❝ ♦ ✐ ♥ ❡ ❞ ✇ ❤ ✐ ❝ ❤ ✐ ♠ ♣ ❧ ✐ ❡ ❤ ❡ ✉ ❡ ♦ ❢ ❛ ♠ ❡ ♠ ❛ ❤ ❡ ♠ ❛ ✐ ❝ ❛ ❧ ♥ ♦ ❛ ✐ ♦ ♥ ✐ ♥ ♠ ✉ ❧ ✐ ♣ ❧ ❡ ❝ ♦ ♥ ❡ ① ❤ ❛ ✈ ✐ ♥ ❣ ❞ ✐ ✛ ❡ ❡ ♥ ♠ ❡ ❛ ♥ ✐ ♥ ❣ ✦ ✦ ✦

✷ 

Page 3: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 3/31

✇ ❤ ✐ ❝ ❤ ❤ ♦ ✇ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ❘ ❛ ❜ ❡ ❡ ❝ ❡ ✐ ✈ ❡ ❜ ❡ ❢ ♦ ❡ ❡ ❛ ❝ ❤ ✐ ♥ ❣ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ✱ ❛ ✐ ♥ ❣ ❢ ♦ ♠ ❛ ♥ ② ❛ ♥ ✐ ❡ ♥ ❛ ❡ ✳

❙ ❡ ♣ ✲ ✺ ✿   ❲ ❡ ❛ ❡ ✐ ♥ ❡ ❡ ❡ ❞ ✐ ♥ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ❣ ✐ ✈ ❡ ♥ ❤ ❛ ❘ ❛ ❤ ❜ ❡ ❛ ❢ ♦ ♠ ❙ ❛ ❡ ✲ ✶ ✻ ✳ ❚ ❤ ✐ ✐ ❣ ✐ ✈ ❡ ♥ ❜ ②  

µshock (15)✱ ❤ ❡    15th ♦ ✇ ♦ ❢  µshock ✳

❚ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ❘ ❛ ❜ ❡ ❡ ❝ ❡ ✐ ✈ ❡ ❜ ❡ ❢ ♦ ❡ ❡ ❛ ❝ ❤ ✐ ♥ ❣ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ✐   6.2963✳

✭ ❞ ✮  ❚ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ❝ ❛ ♥ ❜ ❡ ❛ ❜ ❛ ❝ ❡ ❞ ❛ ✿ ❙ ❛ ✐ ♥ ❣ ❢ ♦ ♠ ❛ ❡   i ✱ ✇ ❤ ❛ ✐ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ❛ ❡    j ✐ ✈ ✐ ✐ ❡ ❞ ❜ ❡ ❢ ♦ ❡ ❛ ❡ ❦  

✭ ❝ ❛ ♥ ❜ ❡ ❛ ✈ ❡ ❝ ♦ ✮  ❄ ■ ♥ ❤ ✐ ✉ ❡ ✐ ♦ ♥   i = 16 ✱   j  = 3 ❛ ♥ ❞   k = 8 9✳ ❚ ♦ ❛ ♥ ✇ ❡ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✇ ❡ ❛ ❞ ♦ ♣ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ❡ ♣ ✿

❙ ❡ ♣ ✲ ✶ ✿   ❲ ❡ ♠ ❛ ❦ ❡ ❛ ❡   j  = 3 ❛ ♥ ❞   k =

8 9

❛ ❜ ♦ ❜ ✐ ♥ ❣ ✳ ❚ ♦ ❞ ♦ ❤ ✐ ✇ ❡ ♠ ❛ ❦ ❡ ❛ ♠ ♦ ❞ ✐ ❢ ❡ ❞ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①  

  P 

❛ ❢ ♦ ❧ ❧ ♦ ✇ ✿ ✶ ✮  ❙ ❡  

P   = P ✳ ✷ ✮  ❙ ❡ ❛ ❧ ❧ ❡ ❧ ❡ ♠ ❡ ♥ ♦ ❢ ❤ ❡    3rd ✱ 8th ❛ ♥ ❞  9th ♦ ✇ ♦ ❢  

  P  ❛    0 ✳ ✸ ✮  ❙ ❡     p33  =  p88  =  p99  = 1 ✳

❙ ❡ ♣ ✲ ✷ ✿   ❲ ❡ ❝ ♦ ♥ ✉ ❝ ❛ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①   Q ✇ ❤ ✐ ❝ ❤ ❝ ♦ ♥ ❛ ✐ ♥ ❤ ❡ ♦ ♥ ❡ ✲ ❡ ♣ ❛ ♥ ✐ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❢ ♦ ♠ ♦ ♥ ❡ ❛ ♥ ✲  

✐ ❡ ♥ ❛ ❡ ♦ ❛ ♥ ♦ ❤ ❡ ❛ ♥ ✐ ❡ ♥ ❛ ❡ ✳ Q ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❢ ♦ ♠  

  P  ❜ ② ❞ ❡ ❧ ❡ ✐ ♥ ❣ ✐    3rd ✱ 8th ❛ ♥ ❞  9th ♦ ✇ ❛ ♥ ❞   3rd ✱ 8th ❛ ♥ ❞  9th ❝ ♦ ❧ ✉ ♠ ♥ ✳

❙ ❡ ♣ ✲ ✸ ✿   ❲ ❡ ❝ ♦ ♥ ✉ ❝ ❛ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①   R ✇ ❤ ✐ ❝ ❤ ❝ ♦ ♥ ❛ ✐ ♥ ❤ ❡ ♦ ♥ ❡ ✲ ❡ ♣ ❛ ♥ ✐ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❢ ♦ ♠ ❛ ❛ ♥ ✲  

✐ ❡ ♥ ❛ ❡ ♦ ❛ ❡ ❝ ✉ ❡ ♥ ❛ ❡ ✳ Q ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❢ ♦ ♠  

  P  ❜ ② ❞ ❡ ❧ ❡ ✐ ♥ ❣ ✐    3rd ✱ 8th ❛ ♥ ❞  9th ♦ ✇ ❛ ♥ ❞ ❞ ❡ ❧ ❡ ✐ ♥ ❣ ❛ ❧ ❧ ❝ ♦ ❧ ✉ ♠ ♥ ❡ ① ❝ ❡ ♣  

❤ ❡  3rd ✱ 8th ❛ ♥ ❞  9th ✳

❙ ❡ ♣ ✲ ✹ ✿   ❲ ❡ ❝ ♦ ♠ ♣ ✉ ❡ ❤ ❡ ♠ ❛ ✐ ①  

U  = (I  − Q)−1

R

✇ ❤ ♦ ❡   U ij ❡ ❧ ❡ ♠ ❡ ♥ ✐ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ❛ ✐ ♥ ❣ ❢ ♦ ♠ ❛ ❛ ♥ ✐ ❡ ♥ ❛ ❡    i✱ ❤ ❡ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ✇ ✐ ❧ ❧ ❣ ❡ ❛ ❜ ♦ ❜ ❡ ❞ ✐ ♥ ❡ ❝ ✉ ❡ ♥  

❛ ❡    j ✳

❙ ❡ ♣ ✲ ✺ ✿   ❚ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ❘ ❛ ❤ ❜ ❡ ❡ ❛ ❝ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ✇ ✐ ❤ ♦ ✉ ❣ ❡ ✐ ♥ ❣ ❤ ♦ ❝ ❦ ❡ ❞ ✐ ❣ ✐ ✈ ❡ ♥ ❜ ② ❤ ❡   (13, 1)th

❡ ❧ ❡ ♠ ❡ ♥ ♦ ❢   U ✳ ■ ♥ 

❤ ❡  U  ♠ ❛ ✐ ① ♦ ✇    13 ❝ ♦ ❡ ♣ ♦ ♥ ❞ ♦ ❙ ❛ ❡ ✲ ✶ ✻ ❛ ♥ ❞ ❝ ♦ ❧ ✉ ♠ ♥    1 ❝ ♦ ❡ ♣ ♦ ♥ ❞ ♦ ❙ ❛ ❡ ✲ ✸ ✳

❚ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ❘ ❛ ❜ ❡ ✜ ♥ ❞ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ❜ ❡ ❢ ♦ ❡ ❣ ❡ ✐ ♥ ❣ ❤ ♦ ❝ ❦ ❡ ❞ ✐ ❢ ♦ ✉ ♥ ❞ ♦ ❜ ❡   0.2143 ✳

✭ ❡ ✮  ❲ ❡ ✇ ✐ ❧ ❧ ❞ ✐ ✈ ✐ ❞ ❡ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✐ ♥ ✇ ♦ ♣ ❛ ✿ ✐ ✮  ❋ ♦ ♠ ✉ ❧ ❛ ✐ ♥ ❣ ❤ ❡ ▼ ❛ ❦ ♦ ✈ ❈ ❤ ❛ ✐ ♥ ▼ ♦ ❞ ❡ ❧ ♦ ❢ ❤ ❡ ❘ ❛ ❜ ❡ ✲ ❙ ❛ ♠ ② ❡ ♠ ✳ ✐ ✐ ✮ 

❈ ❛ ❧ ❝ ✉ ❧ ❛ ✐ ♥ ❣ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ✐ ♠ ❡ ♦ ✇ ❛ ✐ ✉ ♥ ✐ ❧ ❙ ❛ ♠ ❛ ♥ ❞ ❘ ❛ ❜ ❡ ✜ ♥ ❞ ❡ ❛ ❝ ❤ ♦ ❤ ❡ ✳

❋ ♦ ♠ ✉ ❧ ❛ ✐ ♥ ❣ ❤ ❡ ▼ ❛ ❦ ♦ ✈ ❈ ❤ ❛ ✐ ♥ ▼ ♦ ❞ ❡ ❧ ♦ ❢ ❤ ❡ ❘ ❛ ❜ ❡ ✲ ❙ ❛ ♠ ② ❡ ♠   ✿

❚ ❤ ✐ ❡ ♣ ❝ ❛ ♥ ❜ ❡ ❢ ✉ ❤ ❡ ✉ ❜ ✲ ❞ ✐ ✈ ✐ ❞ ❡ ❞ ✐ ♥ ♦ ✇ ♦ ❡ ♣ ✿ ■ ✮  ▼ ♦ ❞ ❡ ❧ ❧ ✐ ♥ ❣ ❤ ❡ ♠ ♦ ✈ ❡ ♠ ❡ ♥ ♦ ❢   ❙ ❛ ♠ ❛ ❧ ♦ ♥ ❡   ✉ ✐ ♥ ❣ ❛ ▼ ❛ ❦ ♦ ✈ ❈ ❤ ❛ ✐ ♥  

♠ ♦ ❞ ❡ ❧ ✳ ■ ■ ✮  ▼ ♦ ❞ ❡ ❧ ❧ ✐ ♥ ❣ ❤ ❡ ❝ ♦ ♠ ❜ ✐ ♥ ❡ ❞ ♠ ♦ ✈ ❡ ♠ ❡ ♥ ♦ ❢   ❘ ❛ ❜ ❡ ✲ ❙ ❛ ♠ ② ❡ ♠   ✉ ✐ ♥ ❣ ▼ ❛ ❦ ♦ ✈ ❈ ❤ ❛ ✐ ♥ ♠ ♦ ❞ ❡ ❧ ✳

❲ ❡ ✇ ✐ ❧ ❧ ✜ ♠ ♦ ❞ ❡ ❧ ❤ ❡ ♠ ♦ ✈ ❡ ♠ ❡ ♥ ♦ ❢ ❙ ❛ ♠ ❛ ❧ ♦ ♥ ❡ ✳ ▼ ♦ ✈ ❡ ♠ ❡ ♥ ♦ ❢ ❙ ❛ ♠ ✐ ♥ ❤ ❡ ♠ ❛ ③ ❡ ✐ ❣ ✉ ✐ ❞ ❡ ❞ ❜ ② ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ❤ ❡ ❡ ✉ ❧ ❡ ✿

• ❘ ✉ ❧ ❡ ✲ ✶ ✿   ■ ♥ ❡ ✈ ❡ ② ❡ ♣ ♦ ❝ ❤ ❤ ❡ ❡ ✐ ❛ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♦ ❢    0.2 ❤ ❛ ❙ ❛ ♠ ✇ ♦ ♥ ✬ ♠ ♦ ✈ ❡ ❜ ❡ ❝ ❛ ✉ ❡ ❤ ❡ ✴ ❤ ❡ ✐ ✐ ❡ ❞ ❛ ♥ ❞ ✇ ❛ ♥ ♦ ❡ ✳

• ❘ ✉ ❧ ❡ ✲ ✷ ✿   ■ ♥ ❡ ✈ ❡ ② ❡ ♣ ♦ ❝ ❤ ✱ ❙ ❛ ♠ ❝ ❛ ♥ ♠ ♦ ✈ ❡ ♦ ♥ ❧ ②   ♦ ♥ ❡ ❡ ♣   ❡ ✐ ❤ ❡   ✉ ♣ ♦ ❞ ♦ ✇ ♥ ♦ ❧ ❡ ❢ ♦ ✐ ❣ ❤  ✳ ❉ ✐ ❛ ❣ ♦ ♥ ❛ ❧ ♠ ♦ ✈ ❡ ♠ ❡ ♥ ✐ ◆ ❖ ❚  

❛ ❧ ❧ ♦ ✇ ❡ ❞ ✳

• ❘ ✉ ❧ ❡ ✲ ✸ ✿   ❙ ✐ ♥ ❝ ❡ ❙ ❛ ♠ ✐   ❜ ❧ ✐ ♥ ❞   ✱ ✐ ❝ ❛ ♥ ② ♦ ♠ ♦ ✈ ❡ ✐ ♥ ❛ ♥ ② ♦ ♥ ❡ ♦ ❢ ❤ ❡ ❢ ♦ ✉ ❞ ✐ ❡ ❝ ✐ ♦ ♥ ✇ ✐ ❤   ❡ ✉ ❛ ❧ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②   ✳ ■ ❢ ❛ ❛ ❣ ✐ ✈ ❡ ♥  

❡ ♣ ♦ ❝ ❤ ❙ ❛ ♠ ✐ ❡ ♦ ♠ ♦ ✈ ❡ ✐ ♥ ❤ ❡ ❞ ✐ ❡ ❝ ✐ ♦ ♥ ✇ ❤ ✐ ❝ ❤ ❤ ❛ ❛ ♥ ♦ ❜ ❛ ❝ ❧ ❡ ✱ ❤ ❡ ♥ ✐ ✇ ✐ ❧ ❧ ❣ ❡ ❜ ♦ ✉ ♥ ❝ ❡ ❞ ❜ ❛ ❝ ❦ ❛ ♥ ❞ ❤ ❡ ♥ ❝ ❡ ✇ ✐ ❧ ❧ ❡ ♠ ❛ ✐ ♥  

✐ ♥ ❤ ❡ ❛ ♠ ❡ ❛ ❡ ✳

❙ ❛ ② ❤ ❛ ✐ ♥ ❛ ❣ ✐ ✈ ❡ ♥ ❛ ❡ ✱ ❙ ❛ ♠ ❝ ❛ ♥ ♠ ♦ ✈ ❡ ✐ ♥   m ∈ {1, 2, 3, 4} ♣ ♦ ✐ ❜ ❧ ❡ ❞ ✐ ❡ ❝ ✐ ♦ ♥ ✳ ❚ ❤ ❡ ♥ ✉ ♥ ❞ ❡ ❤ ❡ ✉ ❧ ❡ ❞ ❡ ✜ ♥ ❡ ❞ ❛ ❜ ♦ ✈ ❡ ✱ ❤ ❡  

♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ❙ ❛ ♠ ❡ ♠ ❛ ✐ ♥ ✐ ♥ ❤ ❡ ❛ ♠ ❡ ❛ ❡ ✐   0.2 +   (1−0.2)4   (4 − m) = 1 − 0.2m ❛ ♥ ❞ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ❙ ❛ ♠ ♠ ♦ ✈ ❡  

✐ ♥ ♦ ♥ ❡ ♦ ❢ ❤ ❡   m ♣ ♦ ✐ ❜ ❧ ❡ ❞ ✐ ❡ ❝ ✐ ♦ ♥ ✐  

  (1−0.2)4   = 0.2 ✳ ❚ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❛ ♥ ✐ ✐ ♦ ♥ ♠ ❛ ✐ ①    W  ❣ ♦ ✈ ❡ ♥ ✐ ♥ ❣ ❤ ❡ ♠ ♦ ✈ ❡ ♠ ❡ ♥ ♦ ❢ ❙ ❛ ♠  

✐ ❤ ♦ ✇ ♥ ❜ ❡ ❧ ♦ ✇ ✳

✸ 

Page 4: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 4/31

✶ ✷ ✸ ✹ ✺ ✻ ✼ ✽ ✾ ✶ ✵ ✶ ✶ ✶ ✷ ✶ ✸ ✶ ✹ ✶ ✺ ✶ ✻  

✶  ✵ ✳ ✽ ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✷  ✵ ✵ ✳ ✻ ✵ ✳ ✷ ✵ ✵ ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✸  ✵ ✵ ✳ ✷ ✵ ✳ ✻ ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✹  ✵ ✵ ✵ ✳ ✷ ✵ ✳ ✻ ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✺  ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✳ ✻ ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✻  ✵ ✵ ✳ ✷ ✵ ✵ ✵ ✳ ✷ ✵ ✳ ✷ ✵ ✳ ✷ ✵ ✵ ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✵ ✵  

✼  ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✳ ✻ ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✽  ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✵ ✵ ✳ ✷ ✵ ✳ ✻ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵  

✾  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✽ ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✵ ✵  

✶ ✵  ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✳ ✻ ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✵  

✶ ✶  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✳ ✻ ✵ ✵ ✵ ✵ ✳ ✷ ✵  

✶ ✷  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✽ ✵ ✵ ✵ ✵ ✳ ✷  

✶ ✸  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✵ ✵ ✵ ✳ ✻ ✵ ✳ ✷ ✵ ✵  

✶ ✹  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✳ ✻ ✵ ✳ ✷ ✵  

✶ ✺  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✵ ✳ ✷ ✵ ✳ ✹ ✵ ✳ ✷  

✶ ✻  ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✵ ✳ ✷ ✵ ✵ ✵ ✳ ✷ ✵ ✳ ✻  

❚ ❛ ❜ ❧ ❡ ✸ ✿ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①   W  ❣ ♦ ✈ ❡ ♥ ✐ ♥ ❣ ❤ ❡ ♠ ♦ ✈ ❡ ♠ ❡ ♥ ♦ ❢ ❙ ❛ ♠ ✐ ♥ ❤ ❡ ♠ ❛ ③ ❡ ✳

◆ ♦ ✇ ❝ ♦ ♥ ✐ ❞ ❡ ❤ ❡ ❝ ♦ ♠ ❜ ✐ ♥ ❡ ❞ ♠ ♦ ✈ ❡ ♠ ❡ ♥ ♦ ❢ ❘ ❛ ❜ ❡ ❛ ♥ ❞ ❙ ❛ ♠ ✳ ❋ ♦ ❡ ❛ ❝ ❤ ♦ ❢ ❤ ❡   16 ❛ ❡ ♦ ❢ ❘ ❛ ❜ ❡ ✱ ❙ ❛ ♠ ❝ ❛ ♥ ❜ ❡ ✐ ♥    16 ♣ ♦ ✐ ❜ ❧ ❡  

❛ ❡ ✳ ❍ ❡ ♥ ❝ ❡ ❤ ❡ ❡ ✉ ✐ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❛ ❡ ✐ ♥ ❤ ❡ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ♦ ❝ ❛ ♣ ✉ ❡ ❤ ✐ ♠ ♦ ❞ ❡ ❧ ✐  

16 × 16 = 256✳ ❲ ❡ ❤ ❛ ✈ ❡ ♦ ✜  

❣ ✐ ✈ ❡ ♣ ❤ ② ✐ ❝ ❛ ❧ ♠ ❡ ❛ ♥ ✐ ♥ ❣ ♦ ❤ ❡ ❡ ❛ ❡ ✳ ■ ❢ ❘ ❛ ❜ ❡ ✐ ✐ ♥ ❛ ❡   sr  ∈ {1, 2, . . . , 16} ❛ ♥ ❞ ❙ ❛ ♠ ✐ ✐ ♥ ❛ ❡    ss ∈ {1, 2, . . . , 16}✱ ❤ ❡ ♥  

✐ ♥ ❤ ❡ ❝ ♦ ♠ ❜ ✐ ♥ ❡ ❞ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ❤ ❡ ❛ ❡ ♦ ❢ ❤ ❡ ❘ ❛ ❜ ❡ ✲ ❙ ❛ ♠ ② ❡ ♠ ✐  

i = 16 (sr − 1) + (ss − 1) ✭ ✶ ✮ 

◆ ♦ ❡ ❤ ❛   i   ∈ {0, 1, 2, . . . , 255}✳ ❲ ❡ ✇ ✐ ❧ ❧ ♥ ♦ ✇ ❝ ♦ ♥ ❝ ❡ ♥ ❛ ❡ ♦ ♥ ❞ ❡ ✐ ❣ ♥ ✐ ♥ ❣ ❤ ❡ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①    M  ♦ ❢ ❤ ❡  

❝ ♦ ♠ ❜ ✐ ♥ ❡ ❞ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ✳ ❆ ✉ ♠ ✐ ♥ ❣ ❤ ❛ ❙ ❛ ♠ ❛ ♥ ❞ ❘ ❛ ❜ ❡ ✬ ♠ ♦ ✈ ❡ ♠ ❡ ♥ ✐   ✐ ♥ ❞ ❡ ♣ ❡ ♥ ❞ ❡ ♥   ♦ ❢ ❡ ❛ ❝ ❤ ♦ ❤ ❡ ✱ ❤ ❡ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②  

❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①   M  ✐ ❣ ✐ ✈ ❡ ♥ ❜ ②  

M ij  = P sirs

jr

W siss

js

✭ ✷ ✮ 

✇ ❤ ❡ ❡   sir ❛ ♥ ❞   sjr ✭   sis ❛ ♥ ❞   sjs ✮ ✐ ❤ ❡ ❛ ❡ ♦ ❢ ❘ ❛ ❜ ❡ ✭ ❙ ❛ ♠ ✮ ✇ ❤ ❡ ♥ ❤ ❡ ❛ ❡ ♦ ❢ ❤ ❡ ❘ ❛ ❜ ❡ ✲ ❙ ❛ ♠ ② ❡ ♠ ✐    i ❛ ♥ ❞   j ❡ ♣ ❡ ❝ ✐ ✈ ❡ ❧ ② ✳

▼ ❛ ❤ ❡ ♠ ❛ ✐ ❝ ❛ ❧ ❧ ② ✱

sir   = ✢ ♦ ♦  

  i

16

+ 1 ✭ ✸ ✮ 

sis   =   i − 16

sir − 1

+ 1 ✭ ✹ ✮ 

sjr   = ✢ ♦ ♦  

  j

16

+ 1 ✭ ✺ ✮ 

sjs   =   j − 16

sjr − 1

+ 1 ✭ ✻ ✮ 

❊ ✉ ❛ ✐ ♦ ♥ ✷ ❛ ❧ ♦ ♥ ❣ ✇ ✐ ❤ ❡ ✉ ❛ ✐ ♦ ♥ ✸ ✱ ✹ ✱ ✺ ❛ ♥ ❞ ✻ ❝ ♦ ♠ ♣ ❧ ❡ ❡ ❧ ② ❝ ❛ ♣ ✉ ❡ ❤ ❡ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ♠ ♦ ❞ ❡ ❧ ♦ ❢ ❤ ❡ ❘ ❛ ❜ ❡ ✲ ❙ ❛ ♠ ② ❡ ♠ ✳

❈ ❛ ❧ ❝ ✉ ❧ ❛ ✐ ♥ ❣ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ✐ ♠ ❡ ♦ ✇ ❛ ✐ ✉ ♥ ✐ ❧ ❙ ❛ ♠ ❛ ♥ ❞ ❘ ❛ ❜ ❡ ✜ ♥ ❞ ❡ ❛ ❝ ❤ ♦ ❤ ❡  ✿

❋ ✐ ♦ ❢ ❛ ❧ ❧ ❧ ❡ ✜ ♥ ❞ ❤ ❡ ✐ ♥ ✐ ✐ ❛ ❧ ❛ ❡ ♦ ❢ ❤ ❡ ❝ ♦ ♠ ❜ ✐ ♥ ❡ ❞ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ✳ ❲ ❡ ❦ ♥ ♦ ✇ ❤ ❡ ❢ ♦   n = 0 ✱   sr  = 16 ❛ ♥ ❞   ss  = 1 ✳ ❲ ❤ ❡ ♥  

✉ ❜ ✐ ✉ ❡ ❞ ✐ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✇ ❡ ✜ ♥ ❞ ❤ ❛ ❤ ❡ ✐ ♥ ✐ ✐ ❛ ❧ ❛ ❡ ♦ ❢ ❤ ❡ ❝ ♦ ♠ ❜ ✐ ♥ ❡ ❞ ▼ ❛ ❦ ♦ ✈ ❈ ❤ ❛ ✐ ♥ ✐   i  = 240 ✳ ❖ ✉ ♦ ❢ ❤ ❡    256 ❛ ❡  

❤ ❡ ❡ ❛ ❡ ♦ ♥ ❧ ②   16 ❛ ❡ ✇ ❤ ✐ ❝ ❤ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ♦ ❙ ❛ ♠ ❛ ♥ ❞ ❘ ❛ ❜ ❡ ♠ ❡ ❡ ✐ ♥ ❣ ❡ ❛ ❝ ❤ ♦ ❤ ❡ ✳ ❚ ❤ ❡ ❡ ❛ ❡ ❛ ❡ ❝ ❤ ❛ ❛ ❝ ❡ ✐ ❡ ❞ ❜ ②  

sr   =  ss ✳ ❯ ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✱ ✇ ❡ ❝ ❛ ♥ ❛ ② ❤ ❛ ❙ ❛ ♠ ❛ ♥ ❞ ❘ ❛ ❜ ❡ ♠ ❡ ❡ ❡ ❛ ❝ ❤ ♦ ❤ ❡ ✐ ❢ ❛ ❡    j ♦ ❢ ❤ ❡ ❝ ♦ ♠ ❜ ✐ ♥ ❡ ❞ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥  

❜ ❡ ❧ ♦ ♥ ❣ ♦ ❤ ❡ ❡   C  =  {0, 17, 34, . . . , 255}✳

■ ♥ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✇ ❡ ❛ ❡ ✐ ♥ ❡ ❡ ❡ ❞ ✐ ♥ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ✐ ♠ ❡ ♦ ❡ ❛ ❝ ❤   ❛ ♥ ② ♦ ♥ ❡   ♦ ❢ ❤ ❡ ❛ ❡ ✐ ♥   C ✳ ❚ ♦ ❝ ❛ ❧ ❝ ✉ ❧ ❛ ❡ ❤ ✐ ✇ ❡ ❝ ❛ ♥ ♠ ❛ ❦ ❡  

❡ ❛ ❝ ❤ ❛ ❡ ✐ ♥   C  ❛ ❛ ❝ ❧ ♦ ❡ ❞ ✭ ❡ ❝ ✉ ❡ ♥ ✮ ❝ ♦ ♠ ♠ ✉ ♥ ✐ ❝ ❛ ✐ ♦ ♥ ❝ ❧ ❛ ❛ ♥ ❞ ❤ ❡ ♥ ❝ ❛ ❧ ❝ ✉ ❧ ❛ ❡ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ✐ ♠ ❡ ♣ ❡ ♥ ✐ ♥ ❤ ❡ ❛ ♥ ✐ ❡ ♥  

❛ ❡ ✳ ❲ ❡ ❝ ❛ ♥ ✉ ♠ ♠ ❛ ✐ ③ ❡ ❤ ❡ ❡ ✉ ✐ ❡ ❞ ❡ ♣ ❛ ❢ ♦ ❧ ❧ ♦ ✇ ✿

❙ ❡ ♣ ✲ ✶ ✿   ▼ ❛ ❦ ❡ ❛ ♠ ♦ ❞ ✐ ✜ ❡ ❞ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①  

M  ✉ ❝ ❤ ❤ ❛ ❛ ❧ ❧ ❤ ❡ ❛ ❡ ✐ ♥ ❡    C  ❛ ❡ ❛ ❜ ♦ ❜ ✐ ♥ ❣ ✳

✹ 

Page 5: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 5/31

❙ ❡ ♣ ✲ ✷ ✿   ❡ ❢ ♦ ♠ ❤ ❡ ❝ ❛ ♥ ♦ ♥ ✐ ❝ ❛ ❧ ❞ ❡ ❝ ♦ ♠ ♣ ♦ ✐ ✐ ♦ ♥ ♦ ❢  

M  ♦ ❣ ❡ ♥ ❡ ❛ ❡ ❤ ❡    Q ♠ ❛ ✐ ① ✳ ● ✐ ✈ ❡ ♥ ❤ ❛ ❤ ❡ ❡ ❛ ❡    16 ❛ ❜ ♦ ❜ ✐ ♥ ❣ ❛ ❡ ✱

❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❛ ♥ ✐ ❡ ♥ ❛ ❡ ✐   256 − 16 = 240 ✳ ❍ ❡ ♥ ❝ ❡    Q ∈ R240×240

❙ ❡ ♣ ✲ ✸ ✿   ❈ ❛ ❧ ❝ ✉ ❧ ❛ ❡   µτ  = (I  − Q)−1

f  ✇ ❤ ❡ ❡    f  ∈ R240×1✐ ❛ ❝ ♦ ❧ ✉ ♠ ♥ ✈ ❡ ❝ ♦ ✇ ✐ ❤ ❛ ❧ ❧ ❡ ♥ ✐ ❡ ❛   1 ✳

❙ ❡ ♣ ✲ ✹ ✿   ■ ♥ ❤ ❡ ❝ ♦ ❧ ✉ ♠ ♥ ✈ ❡ ❝ ♦   µτ  ✱ ❤ ❡ ✐ ♥ ✐ ✐ ❛ ❧ ❛ ❡ ♦ ❢ ❤ ❡ ② ❡ ♠    i = 240 ✱ ✐ ❛ ♦ ❝ ✐ ❛ ❡ ❞ ✇ ✐ ❤ ♦ ✇ ♥ ✉ ♠ ❜ ❡    226✳ ❍ ❡ ♥ ❝ ❡ ❤ ❡  

❡ ✉ ✐ ❡ ❞ ❛ ♥ ✇ ❡ ✐   µτ  (226)✳

❚ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ✐ ♠ ❡ ♦ ✇ ❛ ✐ ✉ ♥ ✐ ❧ ❙ ❛ ♠ ❛ ♥ ❞ ❘ ❛ ❜ ❡ ✜ ♥ ❞ ❡ ❛ ❝ ❤ ♦ ❤ ❡ ✐ ❡ ✉ ❛ ❧ ♦   54.4488 ✳

✭ ❢ ✮  ■ ♥ ❛ ✭ ❜ ✮ ✱ ✇ ❡ ❢ ♦ ✉ ♥ ❞ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ✐ ♠ ❡ ❢ ♦ ❘ ❛ ❜ ❡ ♦ ✜ ♥ ❞ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ❣ ✐ ✈ ❡ ♥ ❤ ❛ ✐ ❛ ❛ ❙ ❛ ❡ ✶ ✻ ✳ ❚ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞  

✐ ♠ ❡ ✐   85.1852✳ ❆ ❢ ❡ ❡ ❛ ✐ ♥ ❣ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ❢ ♦ ❤ ❡    1st ✐ ♠ ❡ ✱ ❘ ❛ ❜ ❡ ❤ ❛ ♦ ❡ ❛ ❝ ❤ ❡ ❡ ❡   C − 1 ✐ ♠ ❡ ❜ ❡ ❢ ♦ ❡ ✐ ❣ ♦ ❡ ♦ ❡ ✳ ❚ ❤ ❡  

❡ ① ♣ ❡ ❝ ❡ ❞ ✐ ♠ ❡ ♦ ✜ ♥ ❞ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ❢ ♦ ❤ ❡ ❧ ❛   C  − 1 ✐ ♠ ❡ ❛ ❡ ❡ ✉ ❛ ❧ ✳ ▲ ❡ ❤ ✐ ✐ ♠ ❡ ❜ ❡    τ ✳ ❖ ♥ ❡ ✐ ❝ ❦ ♦ ✜ ♥ ❞    τ  ✐ ♦ ♦ ❜ ❡ ✈ ❡  

❤ ❛ ✐ ✐ ❤ ❡   ♠ ❡ ❛ ♥ ✜ ❡ ✉ ♥ ✐ ♠ ❡   ♦ ❢ ❛ ❡ ✲ ✸ ✳ ● ✐ ✈ ❡ ♥ ❤ ❛ ❤ ✐ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ✐ ♥ ✐ ❡ ❞ ✉ ❝ ✐ ❜ ❧ ❡ ✱ ✐ ❤ ❛ ❛ ✜ ♥ ✐ ❡ ♠ ❡ ❛ ♥ ❡ ✉ ♥  

✐ ♠ ❡ ✳ τ  ✐ ✐ ♥ ❞ ❡ ❡ ❞ ❣ ✐ ✈ ❡ ♥ ❜ ② ❛ ✈ ❡ ② ✐ ♠ ♣ ❧ ❡ ❢ ♦ ♠ ✉ ❧ ❛  

τ  =  1

π3

✇ ❤ ❡ ❡   π3 ✐ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣ ♦ ❛ ❡ ✸ ♦ ❢ ❤ ❡ ❛ ♦ ❝ ✐ ❛ ❡ ❞ ❛ ✐ ♦ ♥ ❛ ② ❞ ✐ ✐ ❜ ✉ ✐ ♦ ♥    π ✳ ❲ ❡ ❝ ❛ ♥ ✜ ♥ ❞    π ✉ ✐ ♥ ❣ ❤ ❡  

❢ ♦ ♠ ✉ ❧ ❛  

π =  1T  (I  − P  + ONES )

−1

❯ ✐ ♥ ❣ ▼ ❆ ❚ ▲ ❆ ❇ ✇ ❡ ❢ ♦ ✉ ♥ ❞ ❤ ❛   τ  = 16 ✳ ❍ ❡ ♥ ❝ ❡ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ✐ ♠ ❡ ❜ ❡ ❢ ♦ ❡ ❘ ❛ ❜ ❡ ❝ ❛ ♥ ❛ ❦ ❡ ❡ ✐    85.1852 + 16 (C − 1) ✳

✭ ❣ ✮  ❚ ❤ ❡ ❡ ♣ ♦ ♦ ❧ ✈ ❡ ❤ ✐ ♣ ♦ ❜ ❧ ❡ ♠ ❛ ❡ ❛ ❢ ♦ ❧ ❧ ♦ ✇ ✿

❙ ❡ ♣ ✲ ✶ ✿   ▼ ❛ ❦ ❡ ❛ ♠ ♦ ❞ ✐ ✜ ❡ ❞ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ①  

P  ✐ ♥ ✇ ❤ ✐ ❝ ❤ ❛ ❡ ✸ ✭ ❝ ❤ ❡ ❡ ❡ ❛ ❡ ✮ ✱ ✽ ❛ ♥ ❞ ✾ ✭ ❤ ♦ ❝ ❦ ❛ ❡ ✮ ❛ ❡  

❛ ❜ ♦ ❜ ✐ ♥ ❣ ❛ ❡ ✳

❙ ❡ ♣ ✲ ✷ ✿   ❈ ❛ ❧ ❝ ✉ ❧ ❛ ❡ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♦ ❢ ❣ ❡ ✐ ♥ ❣ ❛ ❜ ♦ ❜ ❡ ❞ ✐ ♥ ❛ ❡ ✸ ✱ ✽ ♦ ✾ ❛ ✐ ♥ ❣ ❢ ♦ ♠ ❛ ❡ ✶ ✻ ✳ ❚ ❤ ✐ ❝ ❛ ♥ ❜ ❡ ❞ ♦ ♥ ❡ ❜ ② ✉ ✐ ♥ ❣  

❤ ❡ ❢ ♦ ♠ ✉ ❧ ❛   U  = (I  − Q)−1

R ✇ ❤ ❡ ❡    Q ❛ ♥ ❞   R ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❜ ② ❝ ❛ ♥ ♦ ♥ ✐ ❝ ❛ ❧ ❞ ❡ ❝ ♦ ♠ ♣ ♦ ✐ ✐ ♦ ♥ ♦ ❢  

  P ✳ ▲ ❡ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♦ ❢ ❣ ❡ ✐ ♥ ❣  

❛ ❜ ♦ ❜ ❡ ❞ ✐ ♥ ❛ ❡ ✸ ✱ ✽ ❛ ♥ ❞ ✾ ❜ ❡   p3 ✱   p8 ❛ ♥ ❞   p9 ✳ ❖ ❜ ❡ ✈ ❡ ❤ ❛    p3 +  p8 +  p9  = 1 ✳

❙ ❡ ♣ ✲ ✸ ✿   ■ ❢ ❤ ❡ ❛ ❡ ♦ ❢ ❤ ❡ ♠ ♦ ❞ ✐ ✜ ❡ ❞ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ❣ ❡ ❛ ❜ ♦ ❜ ❡ ❞ ✐ ♥ ❛ ❡ ✸ ❤ ❡ ♥ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ✐   0 ✳ ❚ ❤ ✐ ❤ ❛ ♣ ♣ ❡ ♥  

✇ ✐ ❤ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②    p3 ✳ ❍ ❡ ♥ ❝ ❡ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣ ♦ ❤ ✐ ❝ ❛ ❡ ✐    p3 · 0✳

❙ ❡ ♣ ✲ ✹ ✿   ❙ ❛ ② ❤ ❛ ❤ ❡ ❛ ❡ ♦ ❢ ❤ ❡ ♠ ♦ ❞ ✐ ✜ ❡ ❞ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ❣ ❡ ❛ ❜ ♦ ❜ ❡ ❞ ✐ ♥ ❛ ❡ ✽ ✳ ❚ ❤ ✐ ❤ ❛ ♣ ♣ ❡ ♥ ✇ ✐ ❤ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②    p8 ✳ ❚ ♦ 

❤ ✐ ❡ ♥ ❞ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ❡ ❝ ❡ ✐ ✈ ❡ ❞ ✐   1 ❛ ♥ ❞ ❤ ❡ ❤ ♦ ❝ ❦ ❣ ❡ ♥ ❡ ❛ ♦ ✐ ♥ ❛ ❡ ✽ ✇ ✐ ❧ ❧ ❜ ❡ ✉ ♥ ❡ ❞ ♦ ✛ ✳ ◆ ♦ ✇ ✇ ❡ ❛ ❦ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣  

✉ ❜ ❡ ♣ ✿

• ❲ ❡ ♠ ❛ ❦ ❡ ❛ ❡ ✽ ❛ ♥ ❞    9 ✉ ♥ ❛ ❜ ♦ ❜ ✐ ♥ ❣ ✳ ❚ ❤ ✐ ✐ ❞ ♦ ♥ ❡ ❜ ② ❡ ✐ ♥ ❣ ❤ ❡    8th ❛ ♥ ❞ ❤ ❡   9th ♦ ✇ ♦ ❢  

  P  ❡ ✉ ❛ ❧ ♦ ❤ ❡    8th ❛ ♥ ❞ ❤ ❡  

9th ♦ ✇ ♦ ❢  P ✳ ❲ ❡ ✐ ❧ ❧ ❦ ❡ ❡ ♣ ❛ ❡ ✸ ❛ ❜ ♦ ❜ ✐ ♥ ❣ ✳

• ◆ ♦ ✇ ✇ ❡ ❤ ❛ ✈ ❡ ♦ ✜ ♥ ❞ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ❡ ❝ ❡ ✐ ✈ ❡ ❞ ✐ ♥ ❛ ❡ ✾ ❜ ❡ ❢ ♦ ❡ ❡ ❛ ❝ ❤ ✐ ♥ ❣ ❤ ❡ ❝ ❤ ❡ ❡ ❡ ✳ ❚ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ❝ ❛ ♥ ❜ ❡  

❛ ❜ ❛ ❝ ❡ ❞ ❛ ✿ ❙ ❛ ✐ ♥ ❣ ❢ ♦ ♠ ❛ ❡ ✽ ✱ ✇ ❤ ❛ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❡ ♣ ♦ ❝ ❤ ♣ ❡ ♥ ✐ ♥ ❛ ❡ ✾ ✱ ❜ ❡ ❢ ♦ ❡ ❛ ❡ ✸ ✐ ✈ ✐ ✐ ❡ ❞ ❄   ❚ ♦ 

❛ ♥ ✇ ❡ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✇ ❡ ✇ ✐ ❧ ❧ ❛ ❦ ❡ ❡ ♣ ✐ ♠ ✐ ❧ ❛ ♦ ❛ ✭ ❝ ✮ ✳

• ▲ ❡ ❤ ❡ ❛ ♥ ✇ ❡ ♦ ❤ ❡ ❛ ❜ ♦ ✈ ❡ ✉ ❡ ✐ ♦ ♥ ❜ ❡    N 8 ✳ ❆ ❧ ♦ ✐ ❡ ❝ ❡ ✐ ✈ ❡ ❞    1 ❤ ♦ ❝ ❦ ✐ ♥ ❤ ❡ ❜ ❡ ❣ ✐ ♥ ♥ ✐ ♥ ❣ ✳ ❍ ❡ ♥ ❝ ❡ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡  

♦ ❢ ❤ ♦ ❝ ❦ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣ ♦ ❤ ✐ ❝ ❛ ❡ ✐  

 p8 (N 8 + 1)✳

❙ ❡ ♣ ✲ ✺ ✿   ❙ ❛ ② ❤ ❛ ❤ ❡ ❛ ❡ ♦ ❢ ❤ ❡ ♠ ♦ ❞ ✐ ✜ ❡ ❞ ♠ ❛ ❦ ♦ ✈ ❝ ❤ ❛ ✐ ♥ ❣ ❡ ❛ ❜ ♦ ❜ ❡ ❞ ✐ ♥ ❛ ❡ ✾ ✳ ❚ ❤ ✐ ❤ ❛ ♣ ♣ ❡ ♥ ✇ ✐ ❤ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②    p9 ✳ ◆ ♦ ✇  

✇ ❡ ❡ ♣ ❡ ❛ ❤ ❡ ❛ ♠ ❡ ❡ ♣ ❛ ❙ ❡ ♣ ✹ ✳ ❚ ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣ ♦ ❤ ✐ ❝ ❛ ❡ ✐   p9 (N 9 + 1) ✇ ❤ ❡ ❡    N 9 ✐  

❤ ❡ ✏  ❊ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❡ ♣ ♦ ❝ ❤ ♣ ❡ ♥ ✐ ♥ ❛ ❡ ✽ ✱ ❜ ❡ ❢ ♦ ❡ ❛ ❡ ✸ ✐ ✈ ✐ ✐ ❡ ❞ ✱ ❣ ✐ ✈ ❡ ♥ ❤ ❛ ✇ ❡ ❛ ❢ ♦ ♠ ❛ ❡ ✾   ✑ ✳

❋ ✐ ♥ ❛ ❧ ❧ ② ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ✐   p3 · 0 + p8 (N 8 + 1) + p9 (N 9 + 1) =  p8 (N 8 + 1) + p9 (N 9 + 1) ✳ ❲ ❡ ❝ ❛ ❧ ❝ ✉ ❧ ❛ ❡ ❞    p8 ✱

 p9 ✱   N 8 ❛ ♥ ❞   N 9 ✉ ✐ ♥ ❣ ▼ ❆ ❚ ▲ ❆ ❇ ❛ ♥ ❞ ✜ ♥ ❛ ❧ ❧ ② ❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ✇ ❛ ❝ ❛ ❧ ❝ ✉ ❧ ❛ ❡ ❞ ♦ ❜ ❡    1.791✳ ❆ ♦ ♥ ❡ ♠ ❛ ② ❡ ① ♣ ❡ ❝ ✱  

❤ ❡ ❡ ① ♣ ❡ ❝ ❡ ❞ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❤ ♦ ❝ ❦ ✐ ♥ ❤ ✐ ❝ ❛ ❡ ❤ ♦ ✉ ❧ ❞ ❜ ❡ ❧ ❡ ❤ ❛ ♥ ❤ ❛ ✐ ♥ ❛ ✭ ❝ ✮ ❜ ❡ ❝ ❛ ✉ ❡ ✐ ♥ ❤ ✐ ❝ ❛ ❡ ♦ ♥ ❡ ❤ ♦ ❝ ❦ ❣ ❡ ♥ ❡ ❛ ♦  

❣ ❡ ✇ ✐ ❝ ❤ ❡ ❞ ♦ ✛ ✳ ❚ ❤ ✐ ✐ ♥ ✉ ✐ ✐ ♦ ♥ ✐ ❢ ♦ ✉ ♥ ❞ ♦ ❜ ❡ ❝ ♦ ❡ ❝ ❛   1.791 <  6.2963✳ 

❆ ❣ ❛ ✐ ♥ ❛ ❧ ❧ ❤ ❡ ▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ✐ ✐ ♥ ❝ ❧ ✉ ❞ ❡ ❞ ✐ ♥ ❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ❆ ✳  

✺ 

Page 6: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 6/31

✶ ✳ ✷ ■ ❲ ❛ ❛ ❙ ♥ ♦ ✇ ❜ ❛ ❧ ❧ ✐ ♥ ❍ ❡ ❧ ❧  

❋ ✐ ❣ ✉ ❡ ✶ ✿ ❋ ✐ ❣ ✉ ❡ ❤ ♦ ✇ ✐ ♥ ❣ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ❛ ♥ ❞ ✐ ♥ ❝ ♦ ♠ ✐ ♥ ❣ ❡ ♣ ♦ ❝ ❤ ✳

▲ ❊ ❆ ❙ ❊ ◆ ❖ ❚ ❊ ✿   ❛ ✭ ❛ ✮ ✱ ✭ ❜ ✮ ✱ ✭ ❝ ✮ ❛ ♥ ❞ ✭ ❞ ✮ ❡ ✉ ✐ ❡ ✉ ♦ ✐ ♠ ♣ ❧ ❡ ♠ ❡ ♥ ❤ ❡ ❝ ♦ ❞ ❡ ❛ ♥ ❞ ❡ ❤ ❡ ❝ ♦ ❞ ❡ ✉ ✐ ♥ ❣ ❛ ♥ ❡ ① ❛ ♠ ♣ ❧ ❡ ✳ ❚ ❤ ❡  

▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ❢ ♦ ❤ ❡ ❡ ♣ ❛ ✐ ✐ ♥ ❝ ❧ ✉ ❞ ❡ ❞ ✐ ♥ ❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ❇ ✳ ❆ ❢ ♦ ❤ ❡ ❡ ① ❛ ♠ ♣ ❧ ❡ ✱ ✇ ❡ ❤ ❛ ✈ ❡ ♠ ❛ ❞ ❡ ❛ ❡ ♣ ❡ ❛ ❡ ❡ ❝ ✐ ♦ ♥ ✐ ♥ ❤ ❡ ❡ ♥ ❞  

♦ ❢ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✇ ❤ ❡ ❡ ✇ ❡ ❝ ✉ ♠ ✉ ❧ ❛ ✐ ✈ ❡ ❧ ② ❤ ❛ ♥ ❞ ❧ ❡ ❛ ❧ ❧ ❤ ❡ ❡ ① ❛ ♠ ♣ ❧ ❡ ✳

✭ ❛ ✮  ❋ ✐ ❣ ✉ ❡ ✶ ❤ ♦ ✇ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ❛ ♥ ❞ ✐ ♥ ❝ ♦ ♠ ✐ ♥ ❣ ❡ ♣ ♦ ❝ ❤ ✳ ❖ ✉ ❣ ♦ ✐ ♥ ❣ ❡ ♣ ♦ ❝ ❤   ✐ ❤ ❡ ❡ ♣ ♦ ❝ ❤ ✐ ♥ ✇ ❤ ✐ ❝ ❤ ■ ❡ ♥ ❞   X n ♠ ❛ ✐ ❧ ✳ ■ ♥ ❝ ♦ ♠ ✐ ♥ ❣ ❡ ♣ ♦ ❝ ❤  

✐ ❤ ❡ ❡ ♣ ♦ ❝ ❤ ✐ ♥ ✇ ❤ ✐ ❝ ❤ ■ ❡ ❝ ❡ ✐ ✈ ❡   Z n ♠ ❛ ✐ ❧ ✳ ◆ ♦ ✇ ✇ ❡ ✇ ✐ ❧ ❧ ❞ ❡ ✈ ❡ ❧ ♦ ♣ ❤ ❡   ♦ ❝ ❤ ❛ ✐ ❝ ✉ ♣ ❞ ❛ ❡ ✉ ❧ ❡   ♦ ❢ ❤ ✐ ② ❡ ♠ ✳

Z n+1   =

Xni=1

M I n,i ✭ ✼ ✮ 

X n+1   =

Z n+1j=1

M On+1 , j ✭ ✽ ✮ 

■ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✼ ✱ M I n,i ✐ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ✐ ♥ ❝ ♦ ♠ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣ ♦    ith ♠ ❛ ✐ ❧ ♦ ❢ ❤ ❡   nth ♦ ✉ ❣ ♦ ✐ ♥ ❣ ❡ ♣ ♦ ❝ ❤ ✳ M I n,i ∼  pI  ❛ ♥ ❞ 

P I  (s) ✐ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❣ ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ♦ ❢    pI ✳ ■ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✼ ✱ M On+1 , j ✐ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣  

♦   jth ♠ ❛ ✐ ❧ ♦ ❢ ❤ ❡   (n + 1)th

✐ ♥ ❝ ♦ ♠ ✐ ♥ ❣ ❡ ♣ ♦ ❝ ❤ ✳ M On+1 , j   ∼  pO ❛ ♥ ❞  P O (s) ✐ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❣ ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ♦ ❢    pO ✳ ▲ ❡  

P Xn (s) ❛ ♥ ❞   P Z n (s) ❜ ❡ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❣ ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ♦ ❢ ❤ ❡ ❛ ♥ ❞ ♦ ♠ ✈ ❛ ✐ ❛ ❜ ❧ ❡    X n ❛ ♥ ❞   Z n ❡ ♣ ❡ ❝ ✐ ✈ ❡ ❧ ② ✳ ❋ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣  

▲ ❡ ❝ ✉ ❡ ✶ ✺   ✭  5th ◆ ♦ ✈ ❡ ♠ ❜ ❡ ✮ ✇ ❡ ❝ ❛ ♥ ❞ ✐ ❡ ❝ ❧ ② ✇ ✐ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ❡ ❧ ❛ ✐ ♦ ♥ ❤ ✐ ♣  

P Z n+1 (s) =   P Xn (P I  (s)) ✭ ✾ ✮ 

P Xn+1 (s) =   P Z n+1 (P O (s)) ✭ ✶ ✵ ✮  

❯ ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✾ ❛ ♥ ❞ ✶ ✵ ✇ ❡ ❣ ❡  

P Xn+1 (s) =   P Xn (P I  (P O (s))) ✭ ✶ ✶ ✮  

▲ ❡   P Y   (s) =  P I  (P O (s)) ✭ ♦    P Y   = P I  ◦ P O ✮ ✳ ❆ ❧ ♦ ❧ ❡ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♠ ❛ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣ ♦    P Y  (s) ❜ ❡   pY  ✳ ❚ ❤ ❡ ♥  

❤ ❡ ❡ ❧ ❛ ✐ ♦ ♥ ❜ ❡ ✇ ❡ ❡ ♥   X n ❛ ♥ ❞   X n+1 ❝ ❛ ♥ ❜ ❡ ✐ ♠ ♣ ❧ ② ❝ ❛ ♣ ✉ ❡ ❞ ❜ ② ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ♦ ❝ ❤ ❛ ✐ ❝ ✉ ♣ ❞ ❛ ❡ ✉ ❧ ❡  

X n+1  =

Xnk=1

Y n,k ✭ ✶ ✷ ✮  

✻ 

Page 7: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 7/31

✇ ❤ ❡ ❡   Y n,k  ∼  pY  ✳ ❚ ❤ ✐ ✐ ❛ ❛ ♥ ❞ ❛ ❞   ● ❛ ❧ ♦ ♥ ✲ ❲ ❛ ♦ ♥ ❇ ❛ ♥ ❝ ❤ ✐ ♥ ❣ ♦ ❝ ❡   ❛ ♥ ❞ ❤ ❡ ♥ ❝ ❡ ✇ ❡ ❝ ❛ ♥ ❛ ♣ ♣ ❧ ② ❤ ❡ ❢ ♦ ♠ ✉ ❧ ❛ ✇ ❤ ✐ ❝ ❤ ✇ ❡  

❧ ❡ ❛ ♥ ❡ ❞ ✐ ♥   ▲ ❡ ❝ ✉ ❡ ✶ ✺ ✱ ✶ ✻ ❛ ♥ ❞ ✶ ✼  ✳ ◆ ♦ ✇ ✇ ❡ ♣ ♦ ❝ ❡ ❡ ❞ ✇ ✐ ❤ ❝ ❛ ❧ ❝ ✉ ❧ ❛ ✐ ♥ ❣ ♠ ❡ ❛ ♥ ❛ ♥ ❞ ✈ ❛ ✐ ❛ ♥ ❝ ❡ ♦ ❢  X n ✳

E [X n+1] =

s

 d

ds

P Xn+1 (s)

s=1

=

s

 d

ds

P Xn

 (P I  (P O (s)))

s=1

=   sP ′

O (s) P ′

I  (P O (s)) P ′

Xn (P I  (P O (s)))s=1

=   P ′

O (1) P ′

I  (P O (1)) P ′

Xn (P I  (P O (1)))

=   P ′

O (1) P ′

I  (1) P ′

Xn (P I  (1))

=   P ′

O (1) P ′

O (1) P ′

Xn (1)

=   E [O] E [I ] E [X n]

=   µOµI E [X n] ✭ ✶ ✸ ✮  

(X n+1)

2

  =

s

 d

ds2

P Xn+1 (s)

s=1

=

s d

ds

sP 

O (s) P ′

I  (P O (s)) P ′

Xn (P I  (P O (s)))

s=1

=   sP O′

(s) P I ′ 

P O (s)

P ′

Xn

P I  

P O (s)

+s2P ′′

O (s) P ′

I  (P O (s)) P ′

Xn (P I  (P O (s)))

+s2P ′

O (s) P ′

O (s) P ′′

I   (P O (s)) P ′

Xn (P I  (P O (s)))

+s2P ′

O (s) P ′

I  (P O (s)) P ′

O (s) P ′

I  (P O (s)) P ′′

Xn (P I  (P O (s)))

s=1

=   P ′

O (1) P ′

O (1) P ′

Xn (1) + P 

′′

O (1) P ′

I  (1) P ′

Xn (1)

+

P ′

O (1)

2

P ′′

I   (1) P ′

Xn (1) +

P ′

I  (1) P ′

O (1)

2

P ′′

Xn (1)

= P ′O (1) + P ′′O (1)P ′I  (1) P ′Xn (1) + P ′O (1)2 P ′′I   (1) P ′Xn

 (1)

+

P ′

I  (1) P ′

O (1)2

P ′′

Xn (1)

=   E 

O2

E [I ] E [X n] + E [O]2

I 2

− E [I ]

E [X n] + (E [I ] E [O])2

(X n)2

− E [X n]

=

O2

− E [O]2

E [I ] +

I 2

− E [I ]2

E [O]2

E [X n] + (E [I ] E [O])2

(X n)2

=

❱ ❛    [O] E [I ] + ❱ ❛    [I ] E [O]2

E [X n] + (E [I ] E [O])2

(X n)2

=

σ2OµI  + σ2

I µ2O

E [X n] + (µOµI )

2E 

(X n)2

✭ ✶ ✹ ✮  

■ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✸ ❛ ♥ ❞ ✶ ✹ ✱ µI  ❛ ♥ ❞   σI  ✭   µO ❛ ♥ ❞   σO ✮ ✐ ❤ ❡ ♠ ❡ ❛ ♥ ❛ ♥ ❞ ❛ ♥ ❞ ❛ ❞ ❞ ❡ ✈ ✐ ❛ ✐ ♦ ♥ ♦ ❢ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♠ ❛ ❢ ✉ ♥ ❝ ✐ ♦ ♥    pI  ✭   pO ✮ 

❡ ♣ ❡ ❝ ✐ ✈ ❡ ❧ ② ✳ ❲ ❡ ❝ ❛ ♥ ✇ ✐ ❡ ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✸ ❛ ♥ ❞ ✶ ✹ ✐ ♥ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ♠ ❛ ✐ ① ❢ ♦ ♠    E [X n+1]

(X n+1)2

 =

  µOµI    0

σ2OµI  + σ2

I µ2O   (µOµI )

2

  E [X n]

(X n)2

✭ ✶ ✺ ✮  

❯ ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✺ ✐ ♥ ❛ ❡ ❝ ✉ ✐ ✈ ❡ ♠ ❛ ♥ ♥ ❡ ✇ ❡ ❣ ❡    E [X n]

(X n)2

 =

  µOµI    0

σ2OµI  + σ2

I µ2O   (µOµI )

2

n   E [X 0]

(X 0)2

 =

  µOµI    0

σ2OµI  + σ2

I µ2O   (µOµI )

2

n m

m2

✭ ✶ ✻ ✮  

❙ ♦ ❧ ✈ ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✻ ✇ ❡ ❣ ❡ ✱

✼ 

Page 8: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 8/31

E [X n] =   m (µOµI )n

❱ ❛   [X n] =   E 

(X n)2

− (E [X n])2

=   m

σ2OµI  + σ2

I µ2O

(µOµI )

n−1  (µOµI )n

− 1

µOµI  − 1

▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ♦ ❝ ♦ ♠ ♣ ✉ ❡   E [X n] ❛ ♥ ❞ ❱ ❛    [X n] ✐ ✐ ♥ ❝ ❧ ✉ ❞ ❡ ❞ ✐ ♥   ❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ✲ ❇   ✳

✭ ❜ ✮  ❲ ❡ ✇ ✐ ❧ ❧ ❞ ✐ ❡ ❝ ❧ ② ✉ ❡ ❝ ♦ ♥ ❝ ❡ ♣ ❢ ♦ ♠   ▲ ❡ ❝ ✉ ❡ ✶ ✻ ❛ ♥ ❞ ✶ ✼   ♦ ❛ ♥ ✇ ❡ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✳ ● ✐ ✈ ❡ ♥ ❤ ❛ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ♦ ✉ ❣ ♦ ✐ ♥ ❣  

♠ ❛ ✐ ❧ ✐ ♥ ❤ ❡   0th ❡ ♣ ♦ ❝ ❤ ✐   m ✱ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②    pne ❤ ❛ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ✇ ✐ ❧ ❧ ❜ ❡ ❡ ① ✐ ♥ ❝ ❛ ❤ ❡    nth ❡ ① ❝ ❤ ❛ ♥ ❣ ❡ ✐ ❣ ✐ ✈ ❡ ♥ ❜ ②  

 pne   = am ✇ ❤ ❡ ❡  

a =  P Y   ◦ P Y   · · · ◦ P  Y    (0)

n ❢ ♦ ❧ ❞ ❝ ♦ ♠ ♣ ♦ ✐ ✐ ♦ ♥  

❆ ❧ ♦   P Y   = P I  ◦ P O ✳ ■ ♥ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ♣ ❛ ♦ ❢ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✇ ❡ ✜ ❧ ❧ ✉ ❡ ❤ ❡ ❤ ♦ ❤ ❛ ♥ ❞ ♥ ♦ ❛ ✐ ♦ ♥    P nY  ♦ ❡ ♣ ❡ ❡ ♥ ❤ ❡    n ❢ ♦ ❧ ❞  

❝ ♦ ♠ ♣ ♦ ✐ ✐ ♦ ♥   P Y   ◦ P Y  · · · ◦ P  Y  ✳

▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ♦ ❝ ♦ ♠ ♣ ✉ ❡   pne ✐ ✐ ♥ ❝ ❧ ✉ ❞ ❡ ❞ ✐ ♥   ❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ✲ ❇   ✳

✭ ❝ ✮ 

♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ■ ❛ ♠ ❢ ♦ ❡ ✈ ❡ ❞ ❡ ❛ ❧ ✐ ♥ ❣ ✇ ✐ ❤ ❡ ♠ ❛ ✐ ❧ ❂ ✶ ✲ ❊ ① ✐ ♥ ❝ ✐ ♦ ♥ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♦ ❢ ❤ ❡ ❡ ♠ ❛ ✐ ❧  

▲ ❡ ❞ ❡ ♥ ♦ ❡ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❜ ②    pe ✳ ❯ ✐ ♥ ❣ ❝ ♦ ♥ ❝ ❡ ♣ ❢ ♦ ♠   ▲ ❡ ❝ ✉ ❡ ✶ ✻ ❛ ♥ ❞ ✶ ✼   ✇ ❡ ❝ ❛ ♥ ✇ ✐ ❡  

 pe  =  αm✇ ❤ ❡ ❡  

α =  P Y   (α)

❲ ❡ ❝ ❛ ♥ ❛ ❧ ♦ ♦ ❜ ❛ ✐ ♥   α ✉ ✐ ♥ ❣ ❤ ❡ ❢ ♦ ♠ ✉ ❧ ❛  

α  = limn→∞

P nY   (0)

❋ ✐ ♥ ❛ ❧ ❧ ② ✱ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ■ ❛ ♠ ❢ ♦ ❡ ✈ ❡ ❞ ❡ ❛ ❧ ✐ ♥ ❣ ✇ ✐ ❤ ❡ ♠ ❛ ✐ ❧ ✐   1 − pe ✳ ▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ✐ ✐ ♥ ❝ ❧ ✉ ❞ ❡ ❞ ✐ ♥   ❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ✲ ❇   ✳

✭ ❞ ✮  ■ ♥ ❤ ✐ ❝ ❛ ❡ ❤ ❡ ♦ ❝ ❤ ❛ ✐ ❝ ✉ ♣ ❞ ❛ ❡ ✉ ❧ ❡ ❣ ✐ ✈ ❡ ♥ ❜ ② ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✷ ✇ ✐ ❧ ❧ ❣ ❡ ♠ ♦ ❞ ✐ ✜ ❡ ❞ ❛  

X n+1   =

Xnk=1

Y n,k + N n

✇ ❤ ❡ ❡   N n  ∼  pN ✳ N n ✐ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❛ ❞ ❞ ✐ ✐ ♦ ♥ ❛ ❧ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ✐ ♥ ❤ ❡    nth ♦ ✉ ❣ ♦ ✐ ♥ ❣ ❡ ♣ ♦ ❝ ❤ ✳ N n  ∼  pN  ❛ ♥ ❞  P N  (s) ✐ ❤ ❡  

♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❣ ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ♦ ❢   pN ✳ ◆ ♦ ✇  

P Xn+1 (s) =   E sXn+1=   E 

Xnk=1

sY n,k · sN n

=   E 

Xnk=1

sY n,k

sN n

✭ ✶ ✼ ✮  

=   P Xn (P Y   (s)) P N  (s) ✭ ✶ ✽ ✮  

❊ ✉ ❛ ✐ ♦ ♥ ✶ ✼ ✐ ♣ ♦ ✐ ❜ ❧ ❡ ❞ ✉ ❡ ♦ ❤ ❡   ✐ ♥ ❞ ❡ ♣ ❡ ♥ ❞ ❡ ♥ ❝ ❡   ♦ ❢ ❤ ❡ ❛ ❞ ❞ ✐ ✐ ♦ ♥ ❛ ❧ ♠ ❛ ✐ ❧   N n ❛ ♥ ❞ ❤ ❡ ❡ ❣ ✉ ❧ ❛ ♠ ❛ ✐ ❧    Y n,k ✳

❍ ♦ ✇ ✇ ✐ ❧ ❧ ❛ ♥ ✇ ❡ ♦ ❛ ✭ ❛ ✮ ❝ ❤ ❛ ♥ ❣ ❡  ✿

✽ 

Page 9: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 9/31

❋ ✐ ❧ ❡ ♣ ♦ ❝ ❡ ❡ ❞ ✇ ✐ ❤ ❤ ❡ ♠ ❡ ❛ ♥ ♦ ❢  X n ✳

E [X n+1] =

s

 d

ds

P Xn+1 (s)

s=1

=

s

 d

ds

P Xn

 (P Y  (s)) P N  (s)

s=1

=   sP N  (s)  d

ds

 (P Xn (P Y   (s))) + sP Xn

 (P Y  (s))  d

ds

 (P N  (s))s=1

✭ ✶ ✾ ✮  

=   sP N  (s) P ′

Y   (s) P ′

Xn (P Y  (s)) + sP Xn

 (P Y  (s)) P ′

N  (s)s=1

=   sP N  (s) P ′

Y   (s) P ′

Xn (P Y  (s))

s=1

+  sP Xn (P Y  (s)) P 

N  (s)s=1

=   P N  (1)  sP ′

Y  (s) P ′

Xn (P Y   (s))

s=1

+ 1 · P Xn (P Y  (1)) P 

N  (1)

= 1 ·  sP ′

Y   (s) P ′

Xn (P Y  (s))

s=1

+ 1 · P Xn (1) · P 

N  (1)

=   µOµI E [X n] + 1 · 1 · P ′

N  (1) ✭ ✷ ✵ ✮  

=   µOµI E [X n] + µN  ✭ ✷ ✶ ✮  

❊ ✉ ❛ ✐ ♦ ♥ ✶ ✾ ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ✉ ✐ ♥ ❣   ❝ ❤ ❛ ✐ ♥ ✉ ❧ ❡   ❛ ♥ ❞ ❡ ✉ ❛ ✐ ♦ ♥ ✷ ✵ ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❢ ♦ ♠ ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✸ ✳ ◆ ❡ ① ✇ ❡ ❝ ♦ ♠ ♣ ✉ ❡ ❡ ❝ ♦ ♥ ❞ ♠ ♦ ♠ ❡ ♥  

♦ ❢  X n ✇ ❤ ✐ ❝ ❤ ✇ ✐ ❧ ❧ ❜ ❡ ✉ ❡ ❢ ✉ ❧ ♦ ✜ ♥ ❞ ✈ ❛ ✐ ❛ ♥ ❝ ❡ ♦ ❢    X n ✳

(X n+1)2

  =

s

 d

ds

2

P Xn+1 (s)

s=1

=

s

 d

ds

sP N  (s) P 

Y   (s) P ′

Xn (P Y  (s)) + sP Xn

 (P Y  (s)) P ′

N  (s)

s=1

=

s

 d

ds

P N  (s) sP 

Y   (s) P ′

Xn (P Y  (s))

s=1

+

s

 d

ds

sP Xn

 (P Y  (s)) P ′

N  (s)

s=1

=   s

P N  (s)

  d

ds

sP 

Y   (s) P ′

Xn (P Y  (s))

+ P 

N  (s) sP ′

Y  (s) P ′

Xn (P Y   (s))

s=1

+ s P Xn (P Y   (s)) P ′

N  (s) + sP ′

Y   (s) P ′

Xn (P Y   (s)) P ′

N  (s) + sP Xn (P Y  (s)) P ′′

N  (s)s=1

✭ ✷ ✷ ✮  

=   P N  (1)

s

 d

ds

sP 

Y   (s) P ′

Xn (P Y   (s))

s=1

+ 1 · P ′

N  (1) · 1 · P ′

Y   (1) P ′

Xn (P Y  (1))

1 · P Xn (P Y  (1)) P 

N  (1) + 12P ′

Y   (1) P ′

Xn (P Y  (1)) P 

N  (1) + 12P Xn (P Y  (1)) P 

′′

N  (1)

=

s

 d

ds

sP 

Y   (s) P ′

Xn (P Y  (s))

s=1

+ E [N ] E [Y ] E [X n]

+E [N ] + E [N ] E [Y ] E [X n] +

N 2

− E [N ]

=

s

 d

ds

sP 

Y   (s) P ′

Xn (P Y  (s))

s=1

+ 2µN µOµI E [X n] +

σ2N  + (µN )

2

✭ ✷ ✸ ✮  

= σ2OµI  + σ2

I µ2OE [X n] + (µOµI )

2E (X n)

2

✭ ✷ ✹ ✮  

+2µN µOµI E [X n] +

σ2N  + (µN )

2=

σ2OµI  + σ2

I µ2O + 2µN µOµI 

E [X n] + (µOµI )

2E 

(X n)2

+

σ2N  + (µN )

2

✭ ✷ ✺ ✮  

❊ ✉ ❛ ✐ ♦ ♥ ✷ ✷ ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ✉ ✐ ♥ ❣   ❝ ❤ ❛ ✐ ♥ ✉ ❧ ❡  ✳ ■ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✷ ✸ ✇ ❡ ✉ ❡ ❞ ❤ ❡ ❢ ❛ ❝ ❤ ❛   E [Y ] = E [O] E [I ] = µOµI ✳ ❊ ✉ ❛ ✐ ♦ ♥ ✷ ✹  

✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❢ ♦ ♠ ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✹ ✳ ❊ ① ♣ ❡ ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✷ ✶ ❛ ♥ ❞ ✷ ✺ ✐ ♥ ♠ ❛ ✐ ① ❢ ♦ ♠  

  E [X n+1]

(X n+1)2

  =

  µOµI    0

σ2OµI  + σ2

I µ2O + 2µN µOµI    (µOµI )

2

  E [X n]

(X n)2

+

  µN 

σ2N  + (µN )

2

✭ ✷ ✻ ✮  

✾ 

Page 10: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 10/31

❊ ✉ ❛ ✐ ♦ ♥ ✷ ✻ ✐ ❜ ❛ ✐ ❝ ❛ ❧ ❧ ② ❛ ❞ ✐ ❝ ❡ ❡ ✐ ♠ ❡ ▲ ❚ ■ ② ❡ ♠ ✳ ❚ ♦ ❣ ❡   E [X n] ❛ ♥ ❞   E 

(X n)2

✇ ❡ ❝ ❛ ♥ ❡ ❝ ✉ ✐ ✈ ❡ ❧ ② ♦ ❧ ✈ ❡ ❡ ✉ ❛ ✐ ♦ ♥ ✷ ✻  

✐ ♥ ▼ ❆ ❚ ▲ ❆ ❇ ❛ ✐ ♥ ❣ ❢ ♦ ♠   E [X 0] =  m ❛ ♥ ❞   E 

(X 0)2

 =  m2✳ ◗ ✉ ❛ ❧ ✐ ❛ ✐ ✈ ❡ ❧ ② ♣ ❡ ❛ ❦ ✐ ♥ ❣ ✱   E [X n]   ❢ ♦ ❤ ❡ ❝ ❛ ❡ ✇ ✐ ❤ ❛ ❞ ❞ ✐ ✐ ♦ ♥ ❛ ❧ 

♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ✐ ❞ ❡ ✜ ♥ ✐ ❡ ❧ ② ❧ ❛ ❣ ❡ ❤ ❛ ♥   E [X n] ✇ ❤ ❡ ♥ ❤ ❡ ❡ ✇ ❤ ❡ ❡ ♥ ♦ ❛ ❞ ❞ ✐ ✐ ♦ ♥ ❛ ❧ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧  ✳

❍ ♦ ✇ ✇ ✐ ❧ ❧ ❛ ♥ ✇ ❡ ♦ ❛ ✭ ❜ ✮ ❝ ❤ ❛ ♥ ❣ ❡  ✿

❚ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ❞ ❡ ✐ ✈ ❛ ✐ ♦ ♥ ✐ ✐ ♠ ♣ ♦ ❛ ♥ ♦ ❛ ♥ ✇ ❡ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✳ ❋ ♦ ♠ ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✽ ✇ ❡ ❤ ❛ ✈ ❡  

P Xn (s) =   P Xn−1 (P Y  (s)) P N  (s)

P Xn−1 (s) =   P Xn−2 (P Y  (s)) P N  (s)

P Xn−1 (P Y   (s)) =   P Xn−2 (P Y  (P Y   (s))) P N  (P Y   (s))

P Xn (s) =   P Xn−2 (P Y  (P Y   (s))) P N  (P Y   (s)) P N  (s)

P Xn−2 (s) =   P Xn−3 (P Y  (s)) P N  (s)

P Xn−2 (P Y   (P Y  (s))) =   P Xn−3 (P Y  (P Y   (P Y   (s)))) P N  (P Y   (P Y  (s)))

P Xn (s) =   P Xn−3 (P Y  (P Y   (P Y   (s)))) P N  (P Y   (P Y  (s))) P N  (P Y   (s)) P N  (s)

P Xn (s

) =   P X0 (P 

n

Y  (s

))

n−1

k=0 P N  P 

k

Y  (s

)P Xn

 (s) = (P nY  (s))m

n−1k=0

P N 

P kY  (s)

✭ ✷ ✼ ✮  

❧ ❡ ❛ ❡ ◆ ♦ ❡   ✿ ■ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✷ ✼ ✱ P kY   (s) ❡ ♣ ❡ ❡ ♥ ❤ ❡    k ❢ ♦ ❧ ❞ ❝ ♦ ♠ ♣ ♦ ✐ ✐ ♦ ♥ ♦ ❢ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❣ ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❢ ✉ ♥ ❝ ✐ ♦ ♥    P Y   (s)✳

◆ ♦ ✇ ✇ ❡ ✇ ✐ ❧ ❧ ♣ ♦ ❝ ❡ ❡ ❞ ✇ ✐ ❤ ❤ ❡ ♠ ❛ ✐ ♥ ✉ ❡ ✐ ♦ ♥ ✳ ❚ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②    pne ❤ ❛ ❤ ❡ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ✇ ✐ ❧ ❧ ❜ ❡ ❡ ① ✐ ♥ ❝ ❛ ❤ ❡    nth

❡ ① ❝ ❤ ❛ ♥ ❣ ❡ ✐ ❡ ✉ ❛ ❧ ♦   P Xn (0) ✳ ❲ ❡ ❝ ❛ ♥ ❝ ❛ ❧ ❝ ✉ ❧ ❛ ❡ ❤ ✐ ❜ ② ✉ ❜ ✐ ✉ ✐ ♥ ❣    s = 0 ✐ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✷ ✼ ✳ ◗ ✉ ❛ ❧ ✐ ❛ ✐ ✈ ❡ ❧ ② ✱    pne ✇ ✐ ❤ ❛ ❞ ❞ ✐ ✐ ♦ ♥ ❛ ❧  

♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ✇ ✐ ❧ ❧ ❞ ❡ ❢ ♥ ✐ ❡ ❧ ② ❜ ❡ ❧ ❡ ❤ ❛ ♥ ✭ ♦ ❡ ✉ ❛ ❧ ♦ ✮ ❤ ❡ ❝ ❛ ❡ ✇ ❤ ❡ ♥ ❤ ❡ ❡ ❛ ❡ ♥ ♦ ✇ ❛ ❞ ❞ ✐ ✐ ♦ ♥ ❛ ❧ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧  ✳

❚ ❤ ❡ ▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ✐ ❣ ✐ ✈ ❡ ♥ ✐ ♥   ❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ✲ ❇   ✳

❍ ♦ ✇ ✇ ✐ ❧ ❧ ❛ ♥ ✇ ❡ ♦ ❛ ✭ ❝ ✮ ❝ ❤ ❛ ♥ ❣ ❡  ✿

♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ■ ❛ ♠ ❢ ♦ ❡ ✈ ❡ ❞ ❡ ❛ ❧ ✐ ♥ ❣ ✇ ✐ ❤ ❡ ♠ ❛ ✐ ❧ ❂ ✶ ✲ ❊ ① ✐ ♥ ❝ ✐ ♦ ♥ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♦ ❢ ❤ ❡ ❡ ♠ ❛ ✐ ❧  

❚ ❤ ❡ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②    pe ✐ ❣ ✐ ✈ ❡ ♥ ❜ ②    limn→∞

P Xn (0)

 pe   = limn→∞

P Xn (0)

= limn→∞

(P nY  (0))m

n−1k=0

P N 

P kY   (0)

❚ ♦ ✜ ♥ ❞   pe ✇ ❡ ❥ ✉ ❛ ❦ ❡ ❛ ❧ ❛ ❣ ❡    n ❛ ♥ ❞ ❝ ♦ ♠ ♣ ✉ ❡    P Xn (0) ✉ ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✷ ✼ ✳ ❚ ❤ ❡ ▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ✐ ❣ ✐ ✈ ❡ ♥ ✐ ♥   ❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ✲ ❇   ✳

✭ ❡ ✮  ❚ ♦ ❛ ♥ ✇ ❡ ❤ ✐ ♣ ❛ ✇ ❡ ❣ ♦ ❜ ❛ ❝ ❦ ♦ ❡ ✉ ❛ ✐ ♦ ♥ ✶ ✽  

P Xn+1 (s) =  P Xn (P Y   (s)) P N  (s) ✭ ✷ ✽ ✮  

P Xn (s) ✐ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❣ ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ♦ ❢ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ✐ ♥ ❤ ❡    nth ❡ ♣ ♦ ❝ ❤ ✳ ■ ❢ ❛ ✐ ♦ ♥ ❛ ② ❞ ✐ ✐ ❜ ✉ ✐ ♦ ♥  

❡ ① ✐ ❤ ❡ ♥ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♠ ❛ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ♦ ❢   X n ✇ ✐ ❧ ❧ ❝ ♦ ♥ ✈ ❡ ❣ ❡ ♦ ❤ ❡ ❛ ✐ ♦ ♥ ❛ ② ❞ ✐ ✐ ❜ ✉ ✐ ♦ ♥ ❢ ♦ ❧ ❛ ❣ ❡    n✳ ❙ ✐ ♥ ❝ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②  

♠ ❛ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ❝ ♦ ♥ ✈ ❡ ❣ ❡ ♦ ❤ ❡ ❛ ✐ ♦ ♥ ❛ ② ❞ ✐ ✐ ❜ ✉ ✐ ♦ ♥ ✱ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❣ ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ✇ ✐ ❧ ❧ ❛ ❧ ♦ ❝ ♦ ♥ ✈ ❡ ❣ ❡ ♦   P π (s)✭ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❣ ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ♦ ❢ ❤ ❡ ❛ ✐ ♦ ♥ ❛ ② ❞ ✐ ✐ ❜ ✉ ✐ ♦ ♥ ✮ ✳ ❚ ❤ ✐ ✐ ❜ ❡ ❝ ❛ ✉ ❡ ❛ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❣ ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ❛ ♥ ❞  

❛ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♠ ❛ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ❤ ❛ ❛   ♦ ♥ ❡ ✲ ♦ ✲ ♦ ♥ ❡   ❡ ❧ ❛ ✐ ♦ ♥ ❤ ✐ ♣ ✳ ▼ ❛ ❤ ❡ ♠ ❛ ✐ ❝ ❛ ❧ ❧ ② ✱ P Xn+1 (s) → P π (s) ❛ ♥ ❞   P Xn (s) → P π (s) ❢ ♦  

❧ ❛ ❣ ❡   n ✳ ❙ ✉ ❜ ✐ ✉ ✐ ♥ ❣ ❤ ✐ ✐ ❡ ✉ ❛ ✐ ♦ ♥ ✷ ✽ ✇ ❡ ❣ ❡  

✶ ✵ 

Page 11: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 11/31

P π (s) =   P π (P Y   (s)) P N  (s)

P π (s) =   P π (P I  (P O (s))) P N  (s) ✭ ✷ ✾ ✮  

❊ ✉ ❛ ✐ ♦ ♥ ✷ ✾ ✐ ❤ ❡ ❡ ✉ ✐ ❡ ❞ ❡ ❧ ❛ ✐ ♦ ♥ ❤ ✐ ♣ ✳

✭ ❢ ✮  ■ ♠ ❛ ② ♥ ♦ ❜ ❡ ♣ ♦ ✐ ❜ ❧ ❡ ♦ ❞ ✐ ❡ ❝ ❧ ② ♦ ❧ ✈ ❡ ❡ ✉ ❛ ✐ ♦ ♥ ✷ ✾ ♦ ❣ ❡   P π (s)✳ ❲ ❡ ✐ ♥ ❡ ❛ ❞ ✉ ❡ ❤ ❡ ✐ ❞ ❡ ❛ ❤ ❛ ✐ ❢    P π (s) ❡ ① ✐ ✱ ❤ ❡ ♥ ✇ ❡  

❝ ❛ ♥ ❡ ① ♣ ❡ ✐ ❛ ❛ ❧ ✐ ♠ ✐ ♦ ❢  P Xn (s) ✳ ❯ ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✷ ✼ ✇ ❡ ❝ ❛ ♥ ❛ ❧ ❡ ❛ ❞ ② ❡ ① ♣ ❡    P Xn

 (s) ✐ ♥ ❡ ♠ ♦ ❢ ❡ ♣ ❡ ❛ ❡ ❞ ❝ ♦ ♠ ♣ ♦ ✐ ✐ ♦ ♥  

♦ ❢  P N  (s) ✱   P O (s) ❛ ♥ ❞   P I  (s)✳ ❍ ❡ ♥ ❝ ❡  

P π (s) = limn→∞

P Xn (s)

= limn→∞

(P nY   (s))m

n−1k=0

P N 

P kY   (s)

✭ ✸ ✵ ✮  

✇ ❤ ❡ ❡   P Y  (s) =  P I  (P O (s))✳

❊ ❳ ❆ ▼ ▲ ❊  

Epoch0 5 10 15 20

   M  e  a  n

0

1

2

3

4

5 Without Extra Mails

With Extra Mails

Epoch

0 5 10 15 20

   V  a  r   i  a  n  c  e

0

2

4

6

8

10

12

14 Without Extra Mails

With Extra Mails

Epoch0 5 10 15 20

   E  x   t   i  n  c   t   i  o  n   P  r  o   b  a   b   i   l   i   t  y

0

0.2

0.4

0.6

0.8

1

Without Extra Mails

With Extra Mails

Epoch

0 5 10 15 20

   M  e  a  n

×104

0

1

2

3

4

5

6

7

8Without Extra Mails

With Extra Mails

Epoch0 5 10 15 20

   V  a  r   i  a  n  c  e

×109

0

0.5

1

1.5

2

2.5Without Extra Mails

With Extra Mails

Epoch0 5 10 15 20

   E  x   t   i  n  c   t   i  o  n   P  r  o   b  a   b   i   l   i   t  y

0

0.01

0.02

0.03

0.04

0.05Without Extra Mails

With Extra Mails

µI µO=0.77µN   = 0.67

µN   = 0.67

µI µO=1.6

µN   = 0.67   µN   = 0.67

µI µO=1.6

µN   = 0.67

µI µO=0.77µN   = 0.67

µI µO=0.77

µI µO=1.6

❋ ✐ ❣ ✉ ❡ ✷ ✿ ❋ ✐ ❣ ✉ ❡ ♦ ✉ ❞ ② ❤ ❡ ❡ ✛ ❡ ❝ ♦ ❢   µOµI  ❛ ♥ ❞ ❡ ① ❛ ♠ ❛ ✐ ❧ ♦ ♥ ❤ ❡ ♠ ❡ ❛ ♥ ✱ ✈ ❛ ✐ ❛ ♥ ❝ ❡ ❛ ♥ ❞ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♦ ❢ ❤ ❡  

♥ ✉ ♠ ❜ ❡ ♦ ❢ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ✳

❲ ❡ ❛ ♥ ❞ ♦ ♠ ❧ ② ❣ ❡ ♥ ❡ ❛ ❡ ❞ ✇ ♦ ❡ ♦ ❢   pO ❛ ♥ ❞   pI ✱ ♦ ♥ ❡ ✇ ✐ ❤    µI µO   ≈  0.8  <  1 ❛ ♥ ❞ ❤ ❡ ♦ ❤ ❡ ✇ ✐ ❤    µI µO   ≈  1.5  >  1 ✳ ❲ ❡ ❛ ❧ ♦  

❣ ❡ ♥ ❡ ❛ ❡ ❞ ❛ ❝ ♦ ♠ ♠ ♦ ♥   pN  ❢ ♦ ❜ ♦ ❤ ❤ ❡ ❡ ❡ ✇ ✐ ❤    µN   ≈ 0.7 ✳ ❲ ❡ ♥ ♦ ✇ ❞ ♦ ✇ ♦ ❦ ✐ ♥ ❞ ♦ ❢ ❝ ♦ ♠ ♣ ❛ ❛ ✐ ✈ ❡ ✉ ❞ ② ✿ ■ ✮  ❚ ❤ ❡ ❡ ✛ ❡ ❝ ♦ ❢  

µOµI ✳ ■ ■ ✮  ❚ ❤ ❡ ❡ ✛ ❡ ❝ ♦ ❢ ❡ ① ❛ ♠ ❛ ✐ ❧ ✳

❚ ❤ ❡ ❡ ✛ ❡ ❝ ♦ ❢  µOµI ✿  ❲ ❤ ❡ ♥    µI µO  <  1 ✱ ❤ ❡ ♠ ❡ ❛ ♥ ❛ ♥ ❞ ✈ ❛ ✐ ❛ ♥ ❝ ❡ ♦ ❢ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ❝ ♦ ♥ ✈ ❡ ❣ ❡ ♦ ❛ ✜ ♥ ✐ ❡ ✈ ❛ ❧ ✉ ❡  

❛ ✐ ♠ ❡ ✐ ♥ ❝ ❡ ❛ ❡ ✳ ■ ❢   µI µO   >   1 ✱ ❜ ♦ ❤ ♠ ❡ ❛ ♥ ❛ ♥ ❞ ✈ ❛ ✐ ❛ ♥ ❝ ❡ ♦ ❢ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ✐ ♥ ❝ ❡ ❛ ❡ ❡ ① ♣ ♦ ♥ ❡ ♥ ✐ ❛ ❧ ❧ ② ✇ ✐ ❤  

✐ ♠ ❡ ✳ ❚ ❤ ❡ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ✐ ♥ ❝ ❡ ❛ ❡ ✇ ✐ ❤ ❞ ❡ ❝ ❡ ❛ ❡ ✐ ♥   µOµI ✱ ■ ♥ ❞ ❡ ❡ ❞ ✱ ✐ ❢ ❤ ❡ ❡ ❛ ❡ ♥ ♦ ❡ ① ❛ ♠ ❛ ✐ ❧ ❛ ♥ ❞    µOµI   < 1 ❤ ❡ ♥  

❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❝ ♦ ♥ ✈ ❡ ❣ ❡ ♦   1 ✇ ❤ ✐ ❧ ❡ ✐ ❢    µOµI   > 1 ❤ ❡ ♥ ❤ ❡ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❝ ♦ ♥ ✈ ❡ ❣ ❡ ♦ ❛ ✜ ♥ ✐ ❡ ✈ ❛ ❧ ✉ ❡ ✳

❚ ❤ ❡ ❡ ✛ ❡ ❝ ♦ ❢ ❡ ① ❛ ♠ ❛ ✐ ❧ ✿   ❲ ✐ ❤ ❡ ① ❛ ♠ ❛ ✐ ❧ ♠ ❡ ❛ ♥ ❛ ♥ ❞ ✈ ❛ ✐ ❛ ♥ ❝ ❡ ♦ ❢ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ♦ ✉ ❣ ♦ ✐ ♥ ❣ ♠ ❛ ✐ ❧ ✐ ♥ ❝ ❡ ❛ ❡ ✳ ❲ ✐ ❤ ♥ ♦  

❡ ① ❛ ♠ ❛ ✐ ❧ ✱ ❤ ❡ ♠ ❡ ❛ ♥ ❛ ♥ ❞ ❤ ❡ ✈ ❛ ✐ ❛ ♥ ❝ ❡ ✇ ❡ ❡ ❝ ♦ ♥ ✈ ❡ ❣ ✐ ♥ ❣ ♦   0 ✐ ❢   µI µO   <  1 ✳ ❍ ♦ ✇ ❡ ✈ ❡ ✇ ✐ ❤ ❡ ① ❛ ♠ ❛ ✐ ❧ ✐ ✐ ❝ ♦ ♥ ✈ ❡ ❣ ✐ ♥ ❣ ♦  

❛ ✜ ♥ ✐ ❡ ✈ ❛ ❧ ✉ ❡ ✳ ■ ❢   µOµI   <  1 ❤ ❡ ♥ ✇ ✐ ❤ ♥ ♦ ❡ ① ❛ ♠ ❛ ✐ ❧ ❤ ❡ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❝ ♦ ♥ ✈ ❡ ❣ ❡ ♦    1 ❜ ✉ ✇ ✐ ❤ ❡ ① ❛ ♠ ❛ ✐ ❧ ✐  

❝ ♦ ♥ ✈ ❡ ❣ ❡ ♦ ❛ ✜ ♥ ✐ ❡ ✈ ❛ ❧ ✉ ❡ ✳ ■ ❢   µOµI   > 1 ❛ ♥ ❞ ❤ ❡ ❡ ❛ ❡ ❡ ① ❛ ♠ ❛ ✐ ❧ ✱ ❤ ❡ ♥ ❤ ❡ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❝ ♦ ♥ ✈ ❡ ❣ ❡ ♦    0 ✳

✶ ✶ 

Page 12: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 12/31

✷ ◆ ✉ ♠ ❡ ✐ ❝ ❛ ❧ ❈ ♦ ♠ ♣ ✉ ❛ ✐ ♦ ♥  

✷ ✳ ✶ ❇ ❛ ♥ ❝ ❤ ✐ ♥ ❣ ❖ ✉ ✐ ♥ ❚ ✇ ♦ ❲ ❛ ②  

✭ ❛ ✮  ❚ ❤ ❡  ❙ ♦ ❝ ❤ ❛ ✐ ❝ ❯ ♣ ❞ ❛ ❡ ❘ ✉ ❧ ❡   ❢ ♦ ❤ ❡ ✇ ♦ ✇ ❛ ② ❜ ❛ ♥ ❝ ❤ ✐ ♥ ❣ ♣ ♦ ❝ ❡ ✐  

X An+1  =

XAn

kA=1

Y (AA)n,kA

+

XBn

kB=1

Y (BA)n,kB

;   X Bn+1  =

XAn

kA=1

Y (AB)n,kA

+

XBn

kB=1

Y (BB)n,kB

✭ ✸ ✶ ✮  

✇ ❤ ❡ ❡   X An ❛ ♥ ❞   X Bn ❛ ❡ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞ ❚ ② ♣ ❡ ✲ ❇ ❡ ♣ ❡ ❝ ✐ ✈ ❡ ❧ ② ✐ ♥ ❤ ❡    nth ❡ ♣ ♦ ❝ ❤ ✳ X An ❛ ♥ ❞   X Bn ❛ ❡ ❛ ❧ ♦  

❤ ❡ ❛ ❡ ♦ ❢ ❤ ❡ ▼ ❛ ❦ ♦ ✈ ❈ ❤ ❛ ✐ ♥ ✳ Y (AA)n,kA

✭  Y (AB)n,kA

✮ ✐ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ✭ ❚ ② ♣ ❡ ✲ ❇ ✮ ♣ ♦ ❞ ✉ ❝ ❡ ❞ ❛ ❣ ❡ ♥   kA ♦ ❢ ❚ ② ♣ ❡ ✲ ❆  

✐ ♥ ❤ ❡   nth ❡ ♣ ♦ ❝ ❤ ✳ Y (AA)n,kA

❛ ♥ ❞  Y (AB)n,kA

❛ ❡ ❣ ♦ ✈ ❡ ♥ ❡ ❞ ❜ ② ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❞ ✐ ✐ ❜ ✉ ✐ ♦ ♥    p(A) ✱ ✐ ✳ ❡ ✳

(AA)n,kA

, Y (AB)n,kA

 ∼  p(A)

✳ ❙ ✐ ♠ ✐ ❧ ❛ ❧ ②  

Y (BA)n,kB

✭  Y (BB)n,kB

✮ ✐ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ✭ ❚ ② ♣ ❡ ✲ ❇ ✮ ♣ ♦ ❞ ✉ ❝ ❡ ❞ ❛ ❣ ❡ ♥   kB ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ✐ ♥ ❤ ❡    nth ❡ ♣ ♦ ❝ ❤ ✳ Y (BA)n,kB

❛ ♥ ❞ 

Y (BB)n,kB

❛ ❡ ❣ ♦ ✈ ❡ ♥ ❡ ❞ ❜ ② ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❞ ✐ ✐ ❜ ✉ ✐ ♦ ♥    p(B)✱ ✐ ✳ ❡ ✳

(BA)n,kB

, Y (BB)n,kB

∼ p(B)

❚ ❤ ❡ ▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ❢ ♦ ❤ ❡ ▼ ♦ ♥ ❡ ❈ ❛ ❧ ♦ ✐ ♠ ✉ ❧ ❛ ✐ ♦ ♥ ♦ ❢ ❚ ✇ ♦ ❲ ❛ ② ❇ ❛ ♥ ❝ ❤ ✐ ♥ ❣ ♦ ❝ ❡ ✐ ❣ ✐ ✈ ❡ ♥ ✐ ♥   ❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ✲ ❈   ✳

▲ ❊ ❆ ❙ ❊ ◆ ❖ ❚ ❊ ✿  ❘ ❛ ❤ ❡ ❤ ❛ ♥ ❛ ♥ ❞ ♦ ♠ ❧ ② ❞ ♦ ✐ ♥ ❣ ▼ ♦ ♥ ❡ ❈ ❛ ❧ ♦ ✐ ♠ ✉ ❧ ❛ ✐ ♦ ♥ ❢ ♦ ✈ ❛ ✐ ♦ ✉   p(A) ❛ ♥ ❞   p(B) ♦ ❣ ❡ ♥ ❡ ❛ ❡ ❞ ✐ ✛ ❡ ❡ ♥  

❝ ❡ ♥ ❛ ✐ ♦ ✱ ✇ ❡ ❤ ♦ ✉ ❣ ❤ ❤ ❛ ✐ ✐ ♠ ♦ ❡ ❛ ❡ ❣ ✐ ❝ ♦ ✜ ❞ ❡ ✈ ❡ ❧ ♦ ♣ ❤ ❡ ❤ ❡ ♦ ② ❜ ② ♦ ❧ ✈ ✐ ♥ ❣   ❛ ✭ ❞ ✮ ✱ ✭ ❡ ✮ ❛ ♥ ❞ ✭ ❢ ✮   ❛ ♥ ❞ ❤ ❡ ♥ ✉ ❡  

❤ ❡ ❤ ❡ ♦ ② ♦ ❣ ❡ ♥ ❡ ❛ ❡ ❞ ✐ ✛ ❡ ❡ ♥ ❝ ❡ ♥ ❛ ✐ ♦ ❛ ♥ ❞ ❡ ✐ ✉ ✐ ♥ ❣ ▼ ♦ ♥ ❡ ❈ ❛ ❧ ♦ ✐ ♠ ✉ ❧ ❛ ✐ ♦ ♥ ✳

✭ ❞ ✮  ▲ ❡   P XAn X

Bn

❜ ❡ ❤ ❡ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❉ ✐ ✐ ❜ ✉ ✐ ♦ ♥ ❋ ✉ ♥ ❝ ✐ ♦ ♥ ♦ ❢ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ✭  X An ✮ ❛ ♥ ❞ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇  

❛ ❣ ❡ ♥ ✭  X Bn ❛ ✮ ✐ ♥ ❤ ❡    nth ❡ ♣ ♦ ❝ ❤ ✳ ▲ ❡ ❤ ❡ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ● ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❋ ✉ ♥ ❝ ✐ ♦ ♥  

✷ 

♦ ❢  P XAn X

Bn

❜ ❡  P XAn+1X

Bn+1

(u, v) ✳ ◆ ♦ ✇ ✱

P XAn+1X

Bn+1

(u, v)

=   E 

uXAn+1vX

Bn+1

=   E 

XAn

kA=1

uY  (AA)n,kA

XBn

kB=1

uY  (BA)n,kB

XAn

kA=1

vY  (AB)n,kA

XBn

kB=1

vY  (BB)n,kB

✭ ✸ ✷ ✮  

=   E XAn

kA=1 uY  (AA)n,kA v

Y  (AB)n,kA

  XBn

kB=1 uY  (BA)n,kB  v

Y  (BB)n,kB

=∞

xB=0

∞xA=0

xAkA=1

uY  (AA)n,kA v

Y  (AB)n,kA

  xBkB=1

uY  (BA)n,kB v

Y  (BB)n,kB

|X An   = xA, X Bn   = xB

X An   = xA, X Bn   = xB

✭ ✸ ✸ ✮  

=∞

xB=0

∞xA=0

xAkA=1

uY  (AA)n,kA v

Y  (AB)n,kA

  xBkB=1

uY  (BA)n,kB v

Y  (BB)n,kB

X An   = xA, X Bn   = xB

✭ ✸ ✹ ✮  

=∞

xB=0

∞xA=0

xAkA=1

uY  (AA)n,kA v

Y  (AB)n,kA

  xBkB=1

uY  (BA)n,kB v

Y  (BB)n,kB

X An   = xA, X Bn   = xB

✭ ✸ ✺ ✮  

=∞

xB=0

xA=0 xA

kA=1

E uY  (AA)kA   v

Y  (AB)kA

  xB

kB=1

E uY  (BA)kB   v

Y  (BB)kB P X A

n  = xA, X B

n  = xB ✭ ✸ ✻ ✮  

=∞

xB=0

∞xA=0

uY  (AA)

vY  (AB)

xA E 

uY  (BA)

vY  (BB)

xBP 

X An   = xA, X Bn   = xB

✭ ✸ ✼ ✮  

=∞

xB=0

∞xA=0

P (A) (u, v)

xA P (B) (u, v)

xBP 

X An   = xA, X Bn   = xB

=   P XAn X

Bn

P (A) (u, v)   ,  P (B) (u, v)

✷ 

❙ ✐ ♥ ❝ ❡ ❤ ❡ ❡ ❛ ❡ ✇ ♦ ❛ ♥ ❞ ♦ ♠ ✈ ❛ ✐ ❛ ❜ ❧ ❡  

X An ❛ ♥ ❞ 

  X Bn ✱ ❤ ❡ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ● ❡ ♥ ❡ ❛ ✐ ♥ ❣ ❋ ✉ ♥ ❝ ✐ ♦ ♥ ✇ ✐ ❧ ❧ ❜ ❡ ❝ ❤ ❛ ❛ ❝ ❡ ✐ ③ ❡ ❞ ❜ ② ✇ ♦ ♣ ❛ ❛ ♠ ❡  

  u❛ ♥ ❞ 

  v

✐ ♥ ❡ ❛ ❞ ♦ ❢ ❛ ✐ ♥ ❣ ❧ ❡ ♣ ❛ ❛ ♠ ❡ ❡  s ✭ ❧ ✐ ❦ ❡ ✇ ❡ ❞ ✐ ❞ ✐ ♥ ❝ ❧ ❛ ✮ ✳

✶ ✷ 

Page 13: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 13/31

❊ ✉ ❛ ✐ ♦ ♥ ✸ ✷ ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❜ ② ✉ ❜ ✐ ✉ ✐ ♥ ❣   X An+1 ❛ ♥ ❞   X Bn+1 ❢ ♦ ♠ ❡ ✉ ❛ ✐ ♦ ♥ ✸ ✶ ✳ ❚ ♦ ❣ ❡ ❡ ✉ ❛ ✐ ♦ ♥ ✸ ✸ ✱ ✇ ❡ ✉ ❡ ❤ ❡   ▲ ❛ ✇ ♦ ❢ ❚ ♦ ❛ ❧  

❊ ① ♣ ❡ ❝ ❛ ✐ ♦ ♥   ❝ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❡ ❞ ♦ ♥ ❤ ❡ ❢ ❛ ❝ ❤ ❛   X An   = xA ❛ ♥ ❞  X Bn   = xB ✳ ■ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✸ ✹ ✇ ❡ ❝ ♦ ✉ ❧ ❞ ❡ ♠ ♦ ✈ ❡ ❤ ❡ ❝ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❜ ❡ ❝ ❛ ✉ ❡  

❤ ❡ ❛ ♥ ❞ ♦ ♠ ✈ ❛ ✐ ❛ ❜ ❧ ❡   Y (AA)n,kA

✱ Y (AB)n,kA

✱ Y (BA)n,kB

❛ ♥ ❞  Y (BB)n,kB

❛ ❡ ✐ ♥ ❞ ❡ ♣ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ❤ ❡ ❛ ♥ ❞ ♦ ♠ ✈ ❛ ✐ ❛ ❜ ❧ ❡   X An ❛ ♥ ❞   X Bn ✳ ■ ♥ ❡ ✉ ❛ ✐ ♦ ♥  

✸ ✺ ✇ ❡ ❝ ♦ ✉ ❧ ❞ ❡ ♣ ❡ ❛ ❡ ❤ ❡ ✇ ♦ ❡ ① ♣ ❡ ❝ ❛ ✐ ♦ ♥ ❜ ❡ ❝ ❛ ✉ ❡ ❤ ❡ ❛ ♥ ❞ ♦ ♠ ✈ ❛ ✐ ❛ ❜ ❧ ❡   Y (AA)n,kA

❛ ♥ ❞  Y (AB)n,kA

✐ ✐ ♥ ❞ ❡ ♣ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ❛ ♥ ❞ ♦ ♠  

✈ ❛ ✐ ❛ ❜ ❧ ❡   Y (BA)n,kB

❛ ♥ ❞  Y (BB)n,kB

✳ ❊ ✉ ❛ ✐ ♦ ♥ ✸ ✻ ❛ ♥ ❞ ✸ ✼ ✐ ❤ ❡ ❡ ✉ ❧ ♦ ❢ ❤ ❡ ❢ ❛ ❝ ❤ ❛ ❞ ✐ ✐ ❜ ✉ ✐ ♦ ♥ ✭ ❛ ♥ ❞ ❤ ❡ ♥ ❝ ❡ ❤ ❡ ❡ ① ♣ ❡ ❝ ❛ ✐ ♦ ♥ ✮ ♦ ❢  

❤ ❡ ❛ ♥ ❞ ♦ ♠ ✈ ❛ ✐ ❛ ❜ ❧ ❡   Y (AA)n,kA

❛ ♥ ❞  Y (AB)n,kA

✭  Y (BA)n,kB

❛ ♥ ❞  Y (BB)n,kB

✮ ✐ ✐ ♥ ❞ ❡ ♣ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ❤ ❡ ❡ ♣ ♦ ❝ ❤   n ❛ ♥ ❞ ❤ ❡ ❛ ❣ ❡ ♥ ♥ ✉ ♠ ❜ ❡    kA ✭   kB ✮ ✳

❋ ✐ ♥ ❛ ❧ ❧ ② ✇ ❡ ❤ ❛ ✈ ❡ ✱

P XAn+1X

Bn+1

(u, v) =   P XAn X

BnP (A) (u, v)   ,  P (B) (u, v) ✭ ✸ ✽ ✮  

♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ❤ ❡ ❞ ❡ ❝ ❡ ♥ ❞ ❡ ♥ ✇ ✐ ❧ ❧ ❣ ♦ ❡ ① ✐ ♥ ❝ ❛ ❢ ❡   n ❡ ♣ ♦ ❝ ❤ ✐    P XAn X

Bn

(0, 0)✳ ❚ ♦ ✜ ♥ ❞    P XAn X

Bn

(0, 0) ✇ ❡ ❝ ❛ ♥ ❞ ♦ ❤ ❡  

❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣   ❡ ❝ ✉ ✐ ♦ ♥  

P XAn X

Bn

(0, 0) =   P XAn−1X

Bn−1

P (A) (0, 0)   ,  P (B) (0, 0)

P XA

n−1XBn−1

(u, v) =   P XAn−2X

Bn−2

P (A) (u, v)   ,  P (B) (u, v)

P XA

n XBn

(0, 0) =   P XAn−2X

Bn−2

P (A)

P (A) (0, 0) , P (B) (0, 0)

  ,  P (B)

P (A) (0, 0) , P (B) (0, 0)

P XA

n XB

n

(0, 0) =   P XA

0  XB

0

(α, β )

✇ ❤ ❡ ❡   α ❛ ♥ ❞   β  ❛ ❡ ♦ ♠ ❡ ❝ ♦ ♥ ❛ ♥ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❛ ❢ ❡    n ❝ ♦ ♠ ♣ ♦ ✐ ✐ ♦ ♥ ♦ ❢ ❢ ✉ ♥ ❝ ✐ ♦ ♥    P (A) (u, v) ❛ ♥ ❞   P (B) (u, v) ✳ ❆ ❤ ✐ ♣ ♦ ✐ ♥ ✐  

✐ ✐ ♥ ❡ ❡ ✐ ♥ ❣ ♦ ♥ ♦ ❡ ❛ ✈ ❡ ② ✐ ♠ ♣ ♦ ❛ ♥ ❡ ❧ ❛ ✐ ♦ ♥ ✇ ❤ ✐ ❝ ❤ ✇ ❡ ✇ ✐ ❧ ❧ ✉ ❡ ✐ ♠ ♠ ❡ ❞ ✐ ❛ ❡ ❧ ② ✳

P XAn X

Bn

(0, 0) =  P XA0  X

B0

(α, β )   ⇒ P XAn+1X

Bn+1

(0, 0) =  P XA1  X

B1

(α, β ) ✭ ✸ ✾ ✮  

❚ ❤ ✐ ✐ ❜ ❡ ❝ ❛ ✉ ❡ ♦ ❣ ❡ ❢ ♦ ♠ ❡ ♣ ♦ ❝ ❤   n + 1 ♦ ❡ ♣ ♦ ❝ ❤    1 ✇ ❡ ❤ ❛ ✈ ❡ ♦ ❞ ♦    n ❝ ♦ ♠ ♣ ♦ ✐ ✐ ♦ ♥ ♦ ❢ ❤ ❡ ❛ ♠ ❡ ❢ ✉ ♥ ❝ ✐ ♦ ♥    P (A) (u, v) ❛ ♥ ❞ 

P (B) (u, v) ❧ ✐ ❦ ❡ ✇ ❡ ❞ ✐ ❞ ♦ ❣ ❡ ❢ ♦ ♠ ❡ ♣ ♦ ❝ ❤    n ♦ ❡ ♣ ♦ ❝ ❤    0 ✳ ▲ ❡ ❞ ❡ ✜ ♥ ❡ ✇ ♦ ✈ ❛ ✐ ❛ ❜ ❧ ❡  

an10   =   P 

X An   = 0 , X Bn   = 0|X A0   = 1, X B0   = 0

✭ ✹ ✵ ✮  

an01   =   P 

X An   = 0 , X Bn   = 0|X A0   = 0, X B0   = 1

✭ ✹ ✶ ✮  

◗ ✉ ❛ ❧ ✐ ❛ ✐ ✈ ❡ ❧ ② ✱ an10 ✭   an01 ✮ ✐ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❤ ❛ ❤ ❡ ❞ ❡ ❝ ❡ ♥ ❞ ❛ ♥ ♣ ♦ ❞ ✉ ❝ ❡ ❞ ❜ ② ❛ ✐ ♥ ❣ ❧ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ✭ ❚ ② ♣ ❡ ✲ ❇ ✮ ✇ ✐ ❧ ❧ ❜ ❡  

❡ ① ✐ ♥ ❝ ❛ ❢ ❡   n ❡ ♣ ♦ ❝ ❤ ✳ ❧ ❡ ❛ ❡ ♥ ♦ ❡ ❤ ❛    an10 ❛ ♥ ❞   an01 ❛ ❡ ❡ ① ❛ ❝ ❧ ② ❤ ❡ ✉ ❛ ♥ ✐ ✐ ❡ ❛ ❦ ❡ ❞ ✐ ♥ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✳   ◆ ♦ ✇ ✐ ❢  

X A0   = 1 ❛ ♥ ❞   X B0   = 0 ✱ ✇ ❡ ❤ ❛ ✈ ❡  

an10   =   P XAn X

Bn

(0, 0)

=   P XA0  X

B0

(α, β )

=xB

xA

αxAβ xBP 

X A0   = xA, X B0   = xB

=   α1β 0P 

X A0   = 1, X B0   = 0

=   α1β 0 · 1 =  α ✭ ✹ ✷ ✮  

an01   =   β  ♦ ♦ ❢ ✐ ✐ ♠ ✐ ❧ ❛ ♦    an10  =  α ✭ ✹ ✸ ✮  

❙ ✉ ❜ ✐ ✉ ✐ ♥ ❣  

α❛ ♥ ❞ 

  β ❢ ♦ ♠ ❡ ✉ ❛ ✐ ♦ ♥ ✹ ✷ ❛ ♥ ❞ ✹ ✸ ✐ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✸ ✾ ✇ ❡ ❣ ❡ ✱

P XAn+1X

Bn+1

(0, 0) =   P XA1  X

B1

(an10, an01)

P XA1  X

B1

(u, v) =   P XA0  X

B0

P (A) (u, v)   ,  P (B) (u, v)

P XA

n+1XBn+1

(0, 0) =   P XA0  X

B0

P (A) (an10, an01)   ,  P (B) (an10, an01)

✭ ✹ ✹ ✮  

❆ ♣ ♣ ❧ ② ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✹ ✷ ❛ ♥ ❞ ✹ ✸ ✐ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✹ ✹ ✇ ❡ ❣ ❡  

an+110   =   P (A) (an10, an01) ✭ ✹ ✺ ✮  

an+101   =   P (B) (an10, an01) ✭ ✹ ✻ ✮  

✶ ✸ 

Page 14: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 14/31

❊ ✉ ❛ ✐ ♦ ♥ ✹ ✺ ❛ ♥ ❞ ✹ ✻ ❞ ❡ ✜ ♥ ❡ ❛ ❡ ❝ ✉ ✐ ✈ ❡ ❡ ❧ ❛ ✐ ♦ ♥ ❤ ✐ ♣ ✇ ❤ ✐ ❝ ❤ ✇ ❡ ❝ ❛ ♥ ✉ ❡ ♦ ❣ ❡   an10 ❛ ♥ ❞   an01 ✳ ❍ ♦ ✇ ❡ ✈ ❡ ♦ ❛ ❤ ❡ ❡ ❝ ✉ ✐ ♦ ♥  

✇ ❡ ♥ ❡ ❡ ❞   a010 ❛ ♥ ❞   a001 ✳ ■ ✐ ✐ ✈ ✐ ❛ ❧ ♦ ♦ ❜ ❡ ✈ ❡ ❤ ❛    a010   = 0 ✭   a001   = 0 ✮ ❛    X A0   = 1 ❛ ♥ ❞   X B0   = 0 ✭   X A0   = 0 ❛ ♥ ❞   X B0   = 1✮ ✳

❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ✲ ❉   ❝ ♦ ♥ ❛ ✐ ♥ ❤ ❡ ▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ✇ ❤ ✐ ❝ ❤ ✐ ♠ ♣ ❧ ❡ ♠ ❡ ♥ ❤ ✐ ❡ ❝ ✉ ✐ ♦ ♥ ♦ ❝ ❛ ❧ ❝ ✉ ❧ ❛ ❡   an10 ❛ ♥ ❞   an01 ✳

▲ ❊ ❆ ❙ ❊ ◆ ❖ ❚ ❊ ✿ ❚ ❤ ✐ ♣ ❛ ♦ ❢ ❤ ❡ ✉ ❡ ✐ ♦ ♥ ❛ ❧ ♦ ❛ ❦ ✉ ♦ ✏ ❝ ♦ ♠ ♣ ✉ ❡ ❤ ❡ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ✐ ❡ ❢ ♦ ❤ ❡  

❡ ① ❛ ♠ ♣ ❧ ❡ ✇ ❡ ✐ ♠ ✉ ❧ ❛ ❡ ❞ ✐ ♥ ❛ ✭ ❜ ✮ ✑ ✳ ❲ ❡ ✇ ✐ ❧ ❧ ❞ ❡ ❢ ❡ ❤ ✐ ♦ ❛ ✭ ❜ ✮ ✇ ❤ ✐ ❝ ❤ ✇ ❡ ✇ ✐ ❧ ❧ ❛ ♥ ✇ ❡ ❜ ❡ ❧ ♦ ✇ ✳  

✭ ❡ ✮  ▲ ❡ ✜ ❞ ❡ ✜ ♥ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ✈ ❛ ✐ ❛ ❜ ❧ ❡ ✱

a pq  = P  X An   = 0 , X Bn   = 0 ❢ ♦ ♦ ♠ ❡    n > 0|X A0   = p, X B0   = q ◗ ✉ ❛ ❧ ✐ ❛ ✐ ✈ ❡ ❧ ② ♣ ❡ ❛ ❦ ✐ ♥ ❣ ✱ a pq ✐ ❤ ❡   ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ②   ❛ ✐ ♥ ❣ ❢ ♦ ♠    p ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞    q  ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ✳ ❚ ❤ ❡ ❦ ❡ ②  

❞ ✐ ✛ ❡ ❡ ♥ ❝ ❡ ❜ ❡ ✇ ❡ ❡ ♥   a10 ✭   a01 ✮ ❛ ♥ ❞    an10 ✭   an01 ✮ ✐ ❤ ❛    a10 ✭   a01 ✮ ❞ ♦ ❡ ♥ ♦ ♣ ❡ ❝ ✐ ❢ ② ❤ ❡ ✐ ♠ ❡ ♦ ❢ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥   ✳ ■ ♥ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ✇ ❡ ✇ ✐ ❧ ❧  

✜ ❤ ♦ ✇ ❤ ❛   a pq  = (a10) p

(a01)q

a pq   =   P 

  pi=1

{❉ ❡ ❝ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ✐ ♥ ✐ ✐ ❛ ❧ ❛ ❣ ❡ ♥ ★ ✐ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❣ ♦ ❡ ① ✐ ♥ ❝    }

❛ ♥ ❞ 

qj=1

{❉ ❡ ❝ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ✐ ♥ ✐ ✐ ❛ ❧ ❛ ❣ ❡ ♥ ★ ❥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ❣ ♦ ❡ ① ✐ ♥ ❝    }

=   P 

  p

i=1

{❉ ❡ ❝ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ✐ ♥ ✐ ✐ ❛ ❧ ❛ ❣ ❡ ♥ ★ ✐ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❣ ♦ ❡ ① ✐ ♥ ❝    }· P 

qj=1

{❉ ❡ ❝ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ✐ ♥ ✐ ✐ ❛ ❧ ❛ ❣ ❡ ♥ ★ ❥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ❣ ♦ ❡ ① ✐ ♥ ❝    }

✭ ✹ ✼ ✮  

=

 pi=1

P  ( ❉ ❡ ❝ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ✐ ♥ ✐ ✐ ❛ ❧ ❛ ❣ ❡ ♥ ★ ✐ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❣ ♦ ❡ ① ✐ ♥ ❝    )

·

qj=1

P  ( ❉ ❡ ❝ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ✐ ♥ ✐ ✐ ❛ ❧ ❛ ❣ ❡ ♥ ★ ❥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ❣ ♦ ❡ ① ✐ ♥ ❝    ) ✭ ✹ ✽ ✮  

=

  p

i=1

a10

q

j=1

a01

 = (a10) p

(a01)q

✭ ✹ ✾ ✮  

❊ ✉ ❛ ✐ ♦ ♥ ✹ ✼ ❛ ♥ ❞ ✹ ✽ ✉ ❡ ✐ ♥ ❞ ❡ ♣ ❡ ♥ ❞ ❡ ♥ ❝ ❡ ✱ ♠ ♦ ❡ ♣ ❡ ❝ ✐ ❡ ❧ ② ✱ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ❞ ❡ ❝ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ♦ ♥ ❡ ❛ ❣ ❡ ♥ ✐ ✐ ♥ ❞ ❡ ♣ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ❤ ❡  

♥ ✉ ♠ ❜ ❡ ♦ ❢ ❞ ❡ ❝ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ❛ ♥ ♦ ❤ ❡ ❛ ❣ ❡ ♥ ✳ ❊ ✉ ❛ ✐ ♦ ♥ ✹ ✾ ✉ ❡ ❤ ❡ ❢ ❛ ❝ ❤ ❛ ❤ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❞ ✐ ✐ ❜ ✉ ✐ ♦ ♥ ♦ ❢ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢  

❞ ❡ ❝ ❡ ♥ ❞ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ✭ ❚ ② ♣ ❡ ✲ ❇ ✮ ❛ ❣ ❡ ♥ ❞ ♦ ❡ ♥ ♦ ✈ ❛ ② ✇ ✐ ❤ ❛ ❣ ❡ ♥ ♥ ✉ ♠ ❜ ❡ ✳ ◆ ♦ ✇ ❝ ♦ ♥ ✐ ❞ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ✱

a10   =   P 

 ∞n=1

X An   = 0, X Bn   = 0

|X A0   = 1, X B0   = 0

✭ ✺ ✵ ✮  

=∞j=0

∞i=0

 ∞n=1

X An   = 0, X Bn   = 0

|X A1   = i, X B1   = j, X A0   = 1, X B0   = 0

·   P X A1   = i, X B1   = j |X A0   = 1, X B0   = 0 ✭ ✺ ✶ ✮  

=∞j=0

∞i=0

 ∞n=1

X An   = 0, X Bn   = 0

|X A1   = i, X B1   = j, X A0   = 1, X B0   = 0

 p(A) (i, j)

=∞j=0

∞i=0

 ∞n=1

X An   = 0, X Bn   = 0

|X A1   = i, X B1   = j

 p(A) (i, j)

✭ ✺ ✷ ✮  

=∞j=0

∞i=0

aij p

(A) (i, j)

✭ ✺ ✸ ✮  

=∞j=0

∞i=0

(a10)

i(a01)

j p(A) (i, j)

 =  P (A) (a10, a01) ✭ ✺ ✹ ✮  

✶ ✹ 

Page 15: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 15/31

❚ ♦ ❣ ❡ ❡ ✉ ❛ ✐ ♦ ♥ ✺ ✶ ✱ ✇ ❡ ✉ ❡ ❤ ❡   ▲ ❛ ✇ ♦ ❢ ❚ ♦ ❛ ❧ ❊ ① ♣ ❡ ❝ ❛ ✐ ♦ ♥   ❝ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❡ ❞ ♦ ♥ ❤ ❡ ❢ ❛ ❝ ❤ ❛   X A1   = i ❛ ♥ ❞   X B1   = j ✳ ❊ ✉ ❛ ✐ ♦ ♥ ✺ ✷  

✐ ❤ ❡ ❡ ✉ ❧ ♦ ❢ ❤ ❡   ▼ ❛ ❦ ♦ ✈ ♦ ♣ ❡ ②  ✳ ❊ ✉ ❛ ✐ ♦ ♥ ✺ ✸ ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❜ ②   ✐ ♠ ❡ ❤ ✐ ❢ ✐ ♥ ❣   n  →   n + 1 ✳ ❊ ✉ ❛ ✐ ♦ ♥ ✺ ✹ ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❜ ②  

✉ ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✹ ✾ ✳ ❙ ✐ ♠ ✐ ❧ ❛ ❧ ② ✇ ❡ ❝ ❛ ♥ ♣ ♦ ✈ ❡ ❤ ❛   a01  =  P (B) (a10, a01)✳ ❲ ❡ ✜ ♥ ❛ ❧ ❧ ② ❤ ❛ ✈ ❡ ✱

a10   =   P (A) (a10, a01) ✭ ✺ ✺ ✮  

a01   =   P (B) (a10, a01) ✭ ✺ ✻ ✮  

❊ ✉ ❛ ✐ ♦ ♥ ✺ ✺ ❛ ♥ ❞ ✺ ✻ ✐ ❛ ② ❡ ♠ ♦ ❢ ♥ ♦ ♥ ❧ ✐ ♥ ❡ ❛ ❡ ✉ ❛ ✐ ♦ ♥ ✇ ✐ ❤ ✇ ♦ ✈ ❛ ✐ ❛ ❜ ❧ ❡ ✇ ❤ ✐ ❝ ❤ ❝ ❛ ♥ ❜ ❡ ♦ ❧ ✈ ❡ ❞ ♦ ♦ ❜ ❛ ✐ ♥   a10 ❛ ♥ ❞   a01 ✳

❧ ❡ ❛ ❡ ♥ ♦ ❡ ❤ ❛   a10 ❛ ♥ ❞   a01 ❛ ❡ ❡ ① ❛ ❝ ❧ ② ❤ ❡ ✉ ❛ ♥ ✐ ✐ ❡ ❛ ❦ ❡ ❞ ✐ ♥ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✳   ❖ ♥ ❡ ✇ ❛ ② ♦ ♦ ❧ ✈ ❡ ❡ ✉ ❛ ✐ ♦ ♥ ✺ ✺  

❛ ♥ ❞ ✺ ✻ ✐ ♦ ✉ ❡ ❤ ❡   ♦ ❧ ✈ ❡   ❢ ✉ ♥ ❝ ✐ ♦ ♥ ♦ ❢   ▼ ❛ ❤ ❡ ♠ ❛ ✐ ❝ ❛  ✳ ❍ ♦ ✇ ❡ ✈ ❡ ❤ ❡ ❡ ② ❡ ♠ ♦ ❢ ❡ ✉ ❛ ✐ ♦ ♥ ♠ ❛ ② ❤ ❛ ✈ ❡ ♠ ✉ ❧ ✐ ♣ ❧ ❡ ♦ ❧ ✉ ✐ ♦ ♥ ✳ ❖ ♥ ❡  

✐ ✈ ✐ ❛ ❧ ♦ ❧ ✉ ✐ ♦ ♥ ✐   a10  =  a01  = 1 ✳ ❚ ❤ ✐ ✐ ❜ ❡ ❝ ❛ ✉ ❡    P (A) (1, 1) =  P (B) (1, 1) = 1 ✳ ❚ ❤ ❡ ❡ ❝ ❛ ♥ ❜ ❡ ♦ ❤ ❡ ♥ ♦ ♥ ✲ ✐ ✈ ✐ ❛ ❧ ♦ ❧ ✉ ✐ ♦ ♥  

♦ ♦ ✳ ● ✐ ✈ ❡ ♥ ❤ ❛ ✐ ✐ ❛ ② ❡ ♠ ♦ ❢ ♥ ♦ ♥ ❧ ✐ ♥ ❡ ❛ ❡ ✉ ❛ ✐ ♦ ♥ ✐ ♥ ✇ ♦ ✈ ❛ ✐ ❛ ❜ ❧ ❡ ✱ ✐ ✐ ❞ ✐ ✣ ❝ ✉ ❧ ♦ ✉ ♥ ❞ ❡ ❛ ♥ ❞ ❤ ❡ ❣ ❡ ♦ ♠ ❡ ② ♦ ❢ ❤ ❡  

❛ ♦ ❝ ✐ ❛ ❡ ❞ ❣ ❛ ♣ ❤ ❡ ✉ ✐ ❡ ❞ ♦ ♦ ❧ ✈ ❡ ❤ ❡ ♣ ♦ ❜ ❧ ❡ ♠ ✳ ■ ♥ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ✇ ❡ ✇ ✐ ❧ ❧ ✉ ❣ ❣ ❡ ❛ ♥ ♦ ❤ ❡ ✇ ❛ ② ♦ ♦ ❧ ✈ ❡ ❡ ✉ ❛ ✐ ♦ ♥ ✺ ✺ ❛ ♥ ❞  

✺ ✻ ❜ ② ❞ ❡ ✈ ❡ ❧ ♦ ♣ ✐ ♥ ❣ ❛ ❡ ❧ ❛ ✐ ♦ ♥ ❜ ❡ ✇ ❡ ❡ ♥   a10 ❛ ♥ ❞   an10 ✭   a01 ❛ ♥ ❞   an01 ✮ ✳

❊ ✉ ❛ ✐ ♦ ♥ ✺ ✵ ❝ ❛ ♥ ❛ ❧ ♦ ❜ ❡ ❡ ① ♣ ❡ ❡ ❞ ❛  

a10  = limN →∞

  N n=1

X An   = 0, X Bn   = 0

|X A0   = 1, X B0   = 0

✭ ✺ ✼ ✮  

■ ❢  X A

n

  = 0 ❛ ♥ ❞   X B

n

  = 0 ❢ ♦ ❛ ❣ ✐ ✈ ❡ ♥    n✱ ❤ ❡ ♥ ❢ ♦ ❛ ♥ ②    n ≥  n ✇ ❡ ❝ ❛ ♥ ✐ ♠ ♠ ❡ ❞ ✐ ❛ ❡ ❧ ② ❛ ②    X A

n

  = 0 ❛ ♥ ❞   X B

n

  = 0 ✳ ❚ ❤ ✐ ✐ ❜ ❡ ❝ ❛ ✉ ❡  

(0, 0) ✐ ❛ ♥ ❛ ❜ ♦ ❜ ✐ ♥ ❣ ❛ ❡ ✳ ❚ ❤ ✐ ✐ ♠ ❛ ❤ ❡ ♠ ❛ ✐ ❝ ❛ ❧ ❧ ② ❡ ① ♣ ❡ ❡ ❞ ❛  

X An   = 0, X Bn   = 0

 ⊆

X An   = 0, X Bn   = 0

  ;   n ≥  n

⇒N n=1

X An   = 0, X Bn   = 0

 =

X AN   = 0, X BN   = 0

✭ ✺ ✽ ✮  

❙ ✉ ❜ ✐ ✉ ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✺ ✽ ✐ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✺ ✼ ✇ ❡ ❣ ❡  

a10   = limN →∞

X AN  = 0, X BN   = 0

|X A0   = 1, X B0   = 0

= limN →∞ aN 

10✭ ✺ ✾ ✮  

❊ ✉ ❛ ✐ ♦ ♥ ✺ ✾ ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❢ ♦ ♠ ❤ ❡ ❞ ❡ ✜ ♥ ✐ ✐ ♦ ♥ ♦ ❢   an10 ✭ ❡ ❢ ❡ ❡ ✉ ❛ ✐ ♦ ♥ ✹ ✵ ✮ ✳ ❙ ✐ ♠ ✐ ❧ ❛ ❧ ② ✇ ❡ ❝ ❛ ♥ ♣ ♦ ✈ ❡ ❤ ❛  

a01  = limN →∞

aN 01 ✭ ✻ ✵ ✮  

❊ ✉ ❛ ✐ ♦ ♥ ✺ ✾ ❛ ♥ ❞ ✻ ✵ ✉ ❣ ❣ ❡ ❤ ❛ ✇ ❡ ❝ ❛ ♥ ✜ ♥ ❞   a10 ❛ ♥ ❞   a01 ❜ ② ✐ ♠ ♣ ❧ ② ❝ ❛ ❧ ❝ ✉ ❧ ❛ ✐ ♥ ❣    aN 10 ❛ ♥ ❞   aN 01 ❡ ♣ ❡ ❝ ✐ ✈ ❡ ❧ ② ❢ ♦ ❧ ❛ ❣ ❡    n✳ ❲ ❡ 

❝ ❛ ♥ ❞ ♦ ❤ ✐ ❜ ② ✉ ✐ ♥ ❣ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ❡ ❝ ✉ ✐ ✈ ❡ ❡ ❧ ❛ ✐ ♦ ♥ ❞ ❡ ✈ ❡ ❧ ♦ ♣ ❡ ❞ ✐ ♥   ❛ ✭ ❞ ✮  

an+110   =   P (A) (an10, an01) ✭ ✻ ✶ ✮  

an+101   =   P (B) (an10, an01) ✭ ✻ ✷ ✮  

❖ ❜ ❡ ✈ ❡ ❤ ❛ ❡ ✉ ❛ ✐ ♦ ♥ ✻ ✶ ❛ ♥ ❞ ✻ ✷ ❞ ❡ ✜ ♥ ❡ ❛ ♥ ♦ ♥ ❧ ✐ ♥ ❡ ❛ ❞ ✐ ❝ ❡ ❡ ✐ ♠ ❡ ❞ ② ♥ ❛ ♠ ✐ ❝ ❛ ❧ ② ❡ ♠ ✇ ✐ ❤ ❛ ❡   (an10, an01)✳ ❖ ♥ ❡ ❝ ❛ ♥  

✉ ❡ ✐ ♦ ♥ ❤ ❡ ❛ ❜ ✐ ❧ ✐ ② ❛ ♥ ❞ ❝ ♦ ♥ ✈ ❡ ❣ ❡ ♥ ❝ ❡ ♦ ❢ ❤ ✐ ❞ ② ♥ ❛ ♠ ✐ ❝ ❛ ❧ ② ❡ ♠ ✳ ❚ ♦ ♣ ♦ ✈ ❡ ❛ ❜ ✐ ❧ ✐ ② ✇ ❡ ❝ ❛ ♥ ❤ ♦ ✇ ❤ ❛ ❤ ❡ ❡  

S  =  {(x, y) : 0 ≤ x  ≤  1 ; 0 ≤ y  ≤  1}

✐   ✐ ♥ ✈ ❛ ✐ ❛ ♥  ✳ ❚ ♦ ♣ ♦ ✈ ❡ ❤ ✐ ❝ ♦ ♥ ✐ ❞ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ❛ ❣ ✉ ♠ ❡ ♥ ✳ ❋ ✐ ♥ ♦ ❡ ❤ ❛   a010  = 0 ❛ ♥ ❞   a001  = 0 ✱ ✐ ✳ ❡  

a010, a001

 ∈ S ✳ ❙ ❛ ②  

❤ ❛   (an10, an01) ∈ S ✳ ◆ ♦ ✇  

an+110   =   P (A) (an10, an01)

=   E 

(an10)Y  (AA)

(an01)Y  (AB)

✭ ✻ ✸ ✮  

✶ ✺ 

Page 16: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 16/31

◆ ♦ ✇  0   ≤   an10   ≤   1 ❛ ♥ ❞   0   ≤   an01   ≤   1 ✐ ♥ ❝ ❡    (an10, an01)   ∈ S ✳ ❆ ❧ ♦    Y (AA) ≥   0 ❛ ♥ ❞   Y (AB) ≥   0 ✳ ❚ ❤ ✐ ✐ ♠ ♣ ❧ ✐ ❡ ❤ ❛  

0   ≤   (an10)Y  (AA)

·  (an01)Y  (AB)

≤   1 ❛ ♥ ❞ ❤ ❡ ♥ ❝ ❡    0   ≤   an+110   =   E 

(an10)

Y  (AA)

(an01)Y  (AB)

  ≤   1 ✳ ❙ ✐ ♠ ✐ ❧ ❛ ❧ ② ✇ ❡ ❝ ❛ ♥ ♣ ♦ ✈ ❡ ❤ ❛  

0  ≤   an+101   ≤  1 ✳ ❍ ❡ ♥ ❝ ❡  

an+110   , an+1

01

 ∈ S ✳ ❲ ❡ ✜ ♥ ❛ ❧ ❧ ② ❝ ♦ ♥ ❝ ❧ ✉ ❞ ❡ ❤ ❛ ✐ ❢  

a010, a001

 ∈ S  ❤ ❡ ♥    (an10, an01)  ∈ S ;   ∀n  ≥  0✳ ❚ ❤ ✐  

♣ ♦ ✈ ❡ ❤ ❛   S  ✐ ❛ ♥ ✐ ♥ ✈ ❛ ✐ ❛ ♥ ❡ ❛ ♥ ❞ ❤ ❡ ♥ ❝ ❡    an10 ❛ ♥ ❞   an01 ✐ ❧ ♦ ✇ ❡ ❛ ♥ ❞ ✉ ♣ ♣ ❡ ❜ ♦ ✉ ♥ ❞ ❡ ❞ ❜ ②    0 ❛ ♥ ❞   1 ❡ ♣ ❡ ❝ ✐ ✈ ❡ ❧ ② ✳ ❚ ❤ ✐ ❝ ♦ ♠ ♣ ❧ ❡ ❡  

❤ ❡ ♣ ♦ ♦ ❢ ❤ ❛ ❤ ❡ ❞ ② ♥ ❛ ♠ ✐ ❝ ❛ ❧ ② ❡ ♠ ❞ ❡ ✜ ♥ ❡ ❞ ❜ ② ❡ ✉ ❛ ✐ ♦ ♥ ✻ ✶ ❛ ♥ ❞ ✻ ✷ ✐ ❛ ❜ ❧ ❡ ✳

❍ ♦ ✇ ❡ ✈ ❡ ✱ ❡ ✈ ❡ ♥ ✐ ❢ ❤ ❡ ❞ ② ♥ ❛ ♠ ✐ ❝ ❛ ❧ ② ❡ ♠ ✐ ❛ ❜ ❧ ❡ ✐ ♠ ❛ ② ♥ ♦ ❝ ♦ ♥ ✈ ❡ ❣ ❡ ✳ ❘ ❛ ❤ ❡ ✱ ✐ ♠ ❛ ② ❣ ♦ ✐ ♥ ♦ ❛   ❧ ✐ ♠ ✐ ❝ ② ❝ ❧ ❡  ✳ ■ ❤ ♦ ✉ ❧ ❞ ❜ ❡  

♣ ♦ ✐ ❜ ❧ ❡ ♦ ♣ ♦ ✈ ❡ ❤ ❛ ❤ ✐ ❞ ② ♥ ❛ ♠ ✐ ❝ ❛ ❧ ② ❡ ♠ ❞ ♦ ❡ ❝ ♦ ♥ ✈ ❡ ❣ ❡ ✱ ❜ ✉ ✇ ❡ ❝ ♦ ✉ ❧ ❞ ♥ ♦ ♣ ♦ ✈ ❡ ✐ ✳

❚ ❤ ❡ ▼ ❆ ❚ ▲ ❆ ❇ ❝ ♦ ❞ ❡ ♦ ❝ ❛ ❧ ❝ ✉ ❧ ❛ ❡   a10 ❛ ♥ ❞   a01 ✐ ✐ ♥ ❝ ❧ ✉ ❞ ❡ ❞ ✐ ♥   ❆ ♣ ♣ ❡ ♥ ❞ ✐ ① ✲ ❉   ✳

✭ ❢ ✮  ■ ♥ ▲ ❡ ❝ ✉ ❡ ✶ ✻ ✱ ✇ ❡ ❧ ❡ ❛ ♥ ❡ ❞ ❛ ❜ ♦ ✉ ❤ ♦ ✇ ♦ ✐ ♥ ❢ ❡ ✱ ✐ ❢ ❛ ❜ ❛ ♥ ❝ ❤ ✐ ♥ ❣ ♣ ♦ ❝ ❡ ✐ ❣ ♦ ✐ ♥ ❣ ♦ ❡ ① ✐ ♥ ❝ ✱ ❜ ② ❝ ❤ ❡ ❝ ❦ ✐ ♥ ❣ ❤ ❡ ❡ ① ♣ ❡ ❝ ❛ ♥ ❝ ②  

♦ ❢ ❤ ❡ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ♦ ✛ ♣ ✐ ♥ ❣ ✳ ❲ ❡ ✇ ♦ ✉ ❧ ❞ ❧ ✐ ❦ ❡ ♦ ❞ ♦ ❤ ❡ ❛ ♠ ❡ ❤ ❡ ❡ ✳ ❙ ♦ ✇ ❡ ❝ ❛ ❧ ❝ ✉ ❧ ❛ ❡ ❤ ❡ ❡ ① ♣ ❡ ❝ ❛ ♥ ❝ ② ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞ ❚ ② ♣ ❡ ✲ ❇  

❛ ❣ ❡ ♥ ❛ ❤ ❡   nth ❡ ♣ ♦ ❝ ❤  

X An+1

  =

u

  d

du

 P XA

n+1XBn+1

(u, 1)u=1

=   u  d

du

P XA

n XBn

P (A) (u, 1) P (A) (1, 1)   ,  P (B) (u, 1) P (B) (1, 1)

u=1

✭ ✻ ✹ ✮  

=   u

  d

du P XAn X

Bn P 

(A)

(u, 1)   ,  P 

(B)

(u, 1)u=1

=   u  d

du

P XA

n XBn

(X  (u)   ,  Y (u))u=1

✇ ❤ ❡ ❡    X  (u) =  P (A) (u, 1) ,  Y (u) =  P (B) (u, 1)

=

u

dX 

du  · P X XA

n XBn

(X  (u) , Y (u)) + udY 

du  · P Y 

XAn X

Bn

(X  (u) , Y (u))

u=1

✭ ✻ ✺ ✮  

=   E 

Y (AA)

· P X XAn X

Bn

(1, 1) + E 

Y (BA)

· P Y XAn X

Bn

(1, 1)

=   E 

Y (AA)

· E 

X An

+ E 

Y (BA)

· E 

X Bn

✭ ✻ ✻ ✮  

❊ ✉ ❛ ✐ ♦ ♥ ✻ ✹ ✐ ♦ ❜ ❛ ✐ ♥ ❡ ❞ ❜ ② ✉ ❜ ✐ ✉ ✐ ♥ ❣   (u, v) = (u, 1) ✐ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✸ ✽ ✳ ■ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✻ ✺    P X XAn X

Bn

(X  (u) , Y (u)) ✭   P Y XAn X

Bn

(X  (u) ,

♠ ❡ ❛ ♥ ♣ ❛ ✐ ❛ ❧ ❞ ✐ ✛ ❡ ❡ ♥ ✐ ❛ ✐ ♦ ♥ ♦ ❢   P XAn X

Bn

✇ ✐ ❤ ❡ ♣ ❡ ❝ ♦   X  ✭   Y ✮ ✳ ❙ ✐ ♠ ✐ ❧ ❛ ❧ ② ✇ ❡ ❝ ❛ ♥ ♣ ♦ ✈ ❡  

E X Bn+1 =  E Y (AB)E X An + E Y (BB)E X Bn ✭ ✻ ✼ ✮  

❈ ♦ ♠ ❜ ✐ ♥ ✐ ♥ ❣ ❡ ✉ ❛ ✐ ♦ ♥ ✻ ✻ ❛ ♥ ❞ ✻ ✼ ✇ ❡ ❣ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ♠ ❛ ✐ ① ❡ ✉ ❛ ✐ ♦ ♥  E 

X An+1

X Bn+1

 =

Y (AA)

  E 

Y (BA)

Y (AB)

  E 

Y (BB) E 

X An

X Bn

▲ ❡   µAA =  E 

Y (AA)

✱   µAB =  E 

Y (AB)

✱   µBA =  E 

Y (BA)

❛ ♥ ❞   µBB =  E 

Y (BB)

✳ ❙ ✉ ❜ ✐ ✉ ✐ ♥ ❣ ✐ ♥ ❤ ❡ ❛ ❜ ♦ ✈ ❡ ❡ ✉ ❛ ✐ ♦ ♥  

✇ ❡ ❣ ❡   E 

X An+1

X Bn+1

 =

µAA µBA

µAB µBB

X An

X Bn

✭ ✻ ✽ ✮  

❊ ✉ ❛ ✐ ♦ ♥ ✻ ✽ ❞ ❡ ✜ ♥ ❡ ❛ ▲ ❚ ■ ② ❡ ♠ ✇ ✐ ❤ ✇ ♦ ❛ ❡ ✳ ❚ ❤ ❡ ❡ ✐ ❣ ❡ ♥ ✈ ❛ ❧ ✉ ❡ ♦ ❢ ❤ ❡ ② ❡ ♠ ❛ ❡  

λM    =

µAA + µBB

+

 (µAA − µBB)

2+ 4µBAµAB

2✭ ✻ ✾ ✮  

λm   =

µAA + µBB

 (µAA − µBB)

2+ 4µBAµAB

2✭ ✼ ✵ ✮  

λM  ❛ ♥ ❞   λm ❞ ❡ ✜ ♥ ❡ ❞ ❜ ② ❡ ✉ ❛ ✐ ♦ ♥ ✻ ✾ ❛ ♥ ❞ ✼ ✵ ❤ ❛ ✈ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ♣ ♦ ♣ ❡ ✐ ❡ ✿

✶ ✮  λM  ❛ ♥ ❞   λm ❛ ❡ ❛ ❧ ✇ ❛ ② ❡ ❛ ❧ ✳

✷ ✮  λM  ≥ 0✸ ✮  λM  ≥ |λm| ✳

✶ ✻ 

Page 17: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 17/31

▲ ❡   X A0   =  p ❛ ♥ ❞   X B0   =  q  ✐ ♠ ♣ ❧ ② ✐ ♥ ❣    E 

X A0

 =  p ❛ ♥ ❞   E 

X B0

 =  q ✳ ❖ ♥ ❡ ❝ ❛ ♥ ❤ ♦ ✇ ❤ ❛ ❤ ❡ ❣ ❡ ♥ ❡ ❛ ❧ ✉ ❝ ✉ ❡ ♦ ❢ ❤ ❡ ✐ ♠ ❡  

❡ ✈ ♦ ❧ ✉ ✐ ♦ ♥ ♦ ❢   E 

X An

❛ ♥ ❞   E 

X Bn

✐  

X An

  =   α1 (λM )n

 p + α2 (λm)n

q  ✭ ✼ ✶ ✮  

X Bn

  =   β 1 (λM )n

 p + β 2 (λm)n

q  ✭ ✼ ✷ ✮  

❚ ❛ ❦ ✐ ♥ ❣ ❤ ❡ ❧ ♣ ❢ ♦ ♠   ❧ ❡ ❝ ✉ ❡ ✶ ✻ ✭ ♣ ❛ ❣ ❡ ✸ ✮   ✇ ❡ ❞ ❡ ✈ ❡ ❧ ♦ ♣ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ✉ ❧ ❡ ✿

✶ ✮  ■ ❢  limn→∞

E X An  = 0 ✭    limn→∞

E X Bn  = 0 ✮ ❤ ❡ ♥ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ✭ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ✮ ❲ ■ ▲ ▲ ❜ ❡ ❝ ♦ ♠ ❡ ❡ ① ✐ ♥ ❝ ✳

✷ ✮  ■ ❢  limn→∞

X An

 =  θ ✭   limn→∞

X Bn

 =  θ ✮ ✱ ✇ ❤ ❡ ❡    θ ✐ ❛ ✜ ♥ ✐ ❡ ❝ ❛ ❧ ❛ ✉ ❛ ♥ ✐ ② ✱ ❤ ❡ ♥ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ✭ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ✮ ❲ ■ ▲ ▲  

❜ ❡ ❝ ♦ ♠ ❡ ❡ ① ✐ ♥ ❝ ✳

✸ ✮  ■ ❢  limn→∞

X An

 =  ∞ ✭    limn→∞

X Bn

 =  ∞ ✮ ❤ ❡ ♥ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ✭ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ✮ ▼ ❆ ❨ ❜ ❡ ❝ ♦ ♠ ❡ ❡ ① ✐ ♥ ❝ ✳ ■ ✐ ✏ ▼ ❆ ❨ ✑ ❜ ❡ ❝ ❛ ✉ ❡  

✐ ♥ ❤ ✐ ❝ ❛ ❡ ❤ ❡ ❡ ✐ ❛ ✜ ♥ ✐ ❡ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ♦ ❢ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ✳

❈ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❢ ♦ ✇ ❤ ✐ ❝ ❤ ❜ ♦ ❤ ❤ ❡ ❛ ❣ ❡ ♥ ❲ ■ ▲ ▲ ❜ ❡ ❝ ♦ ♠ ❡ ❡ ① ✐ ♥ ❝ ❢ ♦ ❛ ♥ ② ✐ ♥ ✐ ✐ ❛ ❧ ♣ ♦ ♣ ✉ ❧ ❛ ✐ ♦ ♥ ✿  

❚ ❤ ❡ ❝ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❢ ♦ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ❢ ♦ ❛ ♥ ② ✐ ♥ ✐ ✐ ❛ ❧ ♣ ♦ ♣ ✉ ❧ ❛ ✐ ♦ ♥  

λM  ≤ 1   ⇒µAA + µBB+ (µAA − µBB)2 + 4µBAµAB

2  ≤ 1 ✭ ✼ ✸ ✮  

■ ❢  λM   < 1 ❤ ❡ ♥ ❢ ♦ ♠ ❡ ✉ ❛ ✐ ♦ ♥ ✼ ✶ ❛ ♥ ❞ ✼ ✷ ✇ ❡ ❝ ❛ ♥ ❡ ❡ ❤ ❛    limn→∞

X An

 = 0 ❛ ♥ ❞   limn→∞

X Bn

 = 0✳ ❍ ❡ ♥ ❝ ❡ ❜ ♦ ❤ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞  

❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ✇ ✐ ❧ ❧ ❜ ❡ ❝ ♦ ♠ ❡ ❡ ① ✐ ♥ ❝ ✳

■ ❢  λM  = 1 ❤ ❡ ♥    limn→∞

X An

 =  θ ❛ ♥ ❞   limn→∞

X Bn

 =  φ ✱ ✇ ❤ ❡ ❡    θ ❛ ♥ ❞   φ ❛ ❡ ✜ ♥ ✐ ❡ ❝ ❛ ❧ ❛ ✉ ❛ ♥ ✐ ② ✳ ❍ ❡ ♥ ❝ ❡ ❜ ♦ ❤ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞  

❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ✇ ✐ ❧ ❧ ❜ ❡ ❝ ♦ ♠ ❡ ❡ ① ✐ ♥ ❝ ✳

❧ ❡ ❛ ❡ ♥ ♦ ❡ ❤ ❛ ✐ ♥ ❡ ✉ ❛ ❧ ✐ ② ✼ ✸ ✐ ❤ ❡ ❝ ✐ ❡ ✐ ❛ ❢ ♦ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ✇ ❤ ✐ ❝ ❤ ✐ ❛ ❦ ❡ ❞ ✐ ♥ ❛ ✭ ❢ ✮  ✳ ❲ ❡ ❛ ❧ ♦ ❞ ❡ ✐ ✈ ❡ ❢ ❡ ✇  

♦ ❤ ❡ ❝ ✐ ❡ ✐ ❛ ❜ ❡ ❧ ♦ ✇ ♦ ❤ ❡ ❧ ♣ ✉ ✇ ✐ ❤ ❤ ❡ ▼ ♦ ♥ ❡ ❈ ❛ ❧ ♦ ✐ ♠ ✉ ❧ ❛ ✐ ♦ ♥ ✐ ♥   ❛ ✭ ❜ ✮  ✳

❈ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❢ ♦ ✇ ❤ ✐ ❝ ❤ ❜ ♦ ❤ ❤ ❡ ❛ ❣ ❡ ♥ ▼ ❆ ❨ ✢ ♦ ✉ ✐ ❤ ❢ ♦ ❛ ♥ ② ✐ ♥ ✐ ✐ ❛ ❧ ♣ ♦ ♣ ✉ ❧ ❛ ✐ ♦ ♥ ✿  

❈ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❤ ❛ ❜ ♦ ❤ ❛ ❣ ❡ ♥ ▼ ❆ ❨ ✢ ♦ ✉ ✐ ❤ ❢ ♦ ❛ ♥ ② ✐ ♥ ✐ ✐ ❛ ❧ ♣ ♦ ♣ ✉ ❧ ❛ ✐ ♦ ♥ ✐  

λM   > 1   or   |λm| >  1 ✭ ✼ ✹ ✮  

❯ ♥ ❞ ❡ ❛ ♥ ② ♦ ❢ ❤ ❡ ❡ ❝ ♦ ♥ ❞ ✐ ✐ ♦ ♥   limn→∞

X An

 =  ∞ ❛ ♥ ❞   limn→∞

X Bn

 =  ∞ ✳ ❍ ❡ ♥ ❝ ❡ ❜ ♦ ❤ ❛ ❣ ❡ ♥ ♠ ❛ ② ✢ ♦ ✉ ✐ ❤ ✳

❈ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❢ ♦ ✇ ❤ ✐ ❝ ❤ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ❲ ■ ▲ ▲ ❡ ① ✐ ♥ ❝ ❜ ✉ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ▼ ❆ ❨ ✢ ♦ ✉ ✐ ❤ ❢ ♦ ❛ ♥ ② ✐ ♥ ✐ ✐ ❛ ❧ ♣ ♦ ♣ ✉ ❧ ❛ ✐ ♦

❈ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❤ ❛ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ✇ ✐ ❧ ❧ ❡ ① ✐ ♥ ❝ ❛ ♥ ❞ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ♠ ❛ ② ✢ ♦ ✉ ✐ ❤ ✐  

µBA = 0   and µAA ≤ 1   and µBB > 1 ✭ ✼ ✺ ✮  

❇ ② ❛ ♣ ♣ ❧ ② ✐ ♥ ❣ ❤ ❡ ❡ ❝ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ✐ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✻ ✽ ✇ ❡ ❝ ❛ ♥ ❝ ♦ ♥ ❝ ❧ ✉ ❞ ❡ ❤ ❛ ✿

✶ ✮  limn→∞

X An

 = 0 ✐ ❢   µAA < 1 ✐ ♠ ♣ ❧ ② ✐ ♥ ❣ ❤ ❛ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ✇ ✐ ❧ ❧ ❜ ❡ ❝ ♦ ♠ ❡ ❡ ① ✐ ♥ ❝ ✳ ■ ❢    µAA = 1 ❤ ❡ ♥    limn→∞

X An

 =  θ ❛ ❣ ❛ ✐ ♥  

✐ ♠ ♣ ② ❧ ✐ ♥ ❣ ❤ ❛ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ✇ ✐ ❧ ❧ ❜ ❡ ❝ ♦ ♠ ❡ ❡ ① ✐ ♥ ❝ ✳

✷ ✮  limn→∞

X Bn

 =  ∞ ✐ ♠ ♣ ❧ ② ✐ ♥ ❣ ❤ ❛ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ♠ ❛ ② ✢ ♦ ✉ ✐ ❤ ✳

◆ ❖ ❚ ❊ ✿  µBA = 0 ✐ ♠ ♣ ❧ ✐ ❡ ❤ ❛ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ❞ ♦ ❡ ♥ ♦ ❣ ❡ ♥ ❡ ❛ ❡ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ✳

❈ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❢ ♦ ✇ ❤ ✐ ❝ ❤ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ❲ ■ ▲ ▲ ❡ ① ✐ ♥ ❝ ❜ ✉ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ▼ ❆ ❨ ✢ ♦ ✉ ✐ ❤ ❢ ♦ ❛ ♥ ② ✐ ♥ ✐ ✐ ❛ ❧ ♣ ♦ ♣ ✉ ❧ ❛ ✐ ♦

❚ ❤ ✐ ✐ ✐ ♠ ✐ ❧ ❛ ♦ ❤ ❡ ♣ ❡ ✈ ✐ ♦ ✉ ❝ ❛ ❡ ✳ ❚ ❤ ❡ ❡ ✉ ✐ ❡ ❞ ❝ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ✐  

✶ ✼ 

Page 18: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 18/31

µAB = 0   and µBB ≤ 1   and µAA > 1 ✭ ✼ ✻ ✮  

◆ ❖ ❚ ❊ ✿  µAB = 0 ✐ ♠ ♣ ❧ ✐ ❡ ❤ ❛ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ❞ ♦ ❡ ♥ ♦ ❣ ❡ ♥ ❡ ❛ ❡ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ✳

✭ ❜ ✮  ■ ♥ ❤ ✐ ♣ ❛ ♦ ❢ ❤ ❡ ✉ ❡ ✐ ♦ ♥ ✇ ❡ ❞ ♦ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ✿

• ❉ ♦ ▼ ♦ ♥ ❡ ❈ ❛ ❧ ♦ ✐ ♠ ✉ ❧ ❛ ✐ ♦ ♥ ♦ ❣ ❡ ♥ ❡ ❛ ❡ ❢ ♦ ✉ ✉ ❛ ❧ ✐ ❛ ✐ ✈ ❡ ❧ ② ❞ ✐ ✛ ❡ ❡ ♥ ❝ ❛ ❡ ✿ ✐ ✮  ❇ ♦ ❤ ❆ ❣ ❡ ♥ ❡ ① ✐ ♥ ❝ ✳ ✐ ✐ ✮  ❇ ♦ ❤ ❛ ❣ ❡ ♥  

✢ ♦ ✉ ✐ ❤ ❡ ✳ ✐ ✐ ✐ ✮   ❚ ② ♣ ❡ ✲ ❆ ❡ ① ✐ ♥ ❝ ❛ ♥ ❞ ❚ ② ♣ ❡ ✲ ❇ ✢ ♦ ✉ ✐ ❤ ❡ ✳ ✐ ✈ ✮  ❚ ② ♣ ❡ ✲ ❇ ❡ ① ✐ ♥ ❝ ❛ ♥ ❞ ❚ ② ♣ ❡ ✲ ❆ ✢ ♦ ✉ ✐ ❤ ❡ ✳

• ❈ ♦ ♠ ♠ ❡ ♥ ❤ ♦ ✇ ❤ ❡ ♠ ❡ ❛ ♥ ♥ ✉ ♠ ❜ ❡ ♦ ❢ ♦ ✛ ♣ ✐ ♥ ❣ ♦ ❢ ❡ ❛ ❝ ❤ ② ♣ ❡ ✇ ✐ ❧ ❧ ❛ ✛ ❡ ❝ ✇ ❤ ✐ ❝ ❤ ❝ ❛ ❡ ✇ ✐ ❧ ❧ ❜ ❡ ♦ ❜ ❡ ✈ ❡ ❞ ✳ ❲ ❡ ❤ ❛ ✈ ❡ ❛ ❧ ❡ ❛ ❞ ②  

❛ ♥ ✇ ❡ ❡ ❞ ❤ ✐ ✐ ♥   ❛ ✭ ❢ ✮   ♦ ❢ ❤ ❡ ✉ ❡ ✐ ♦ ♥ ✳ ■ ♥ ❞ ❡ ❡ ❞ ✱ ✇ ❡ ✉ ❡ ❤ ❡ ❡ ✉ ❧ ❢ ♦ ♠   ♣ ❛ ✭ ❢ ✮   ♦ ❣ ❡ ♥ ❡ ❛ ❡ ❤ ❡ ❢ ♦ ✉ ❞ ✐ ✛ ❡ ❡ ♥  

❝ ❛ ❡ ✳

• ❈ ♦ ♠ ♣ ✉ ❡ ❤ ❡ ❡ ① ✐ ♥ ❝ ✐ ♦ ♥ ♣ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ✐ ❡ ❢ ♦ ❤ ❡ ❢ ♦ ✉ ❞ ✐ ✛ ❡ ❡ ♥ ❝ ❛ ❡ ✳ ❚ ❤ ✐ ✇ ❛ ❛ ❦ ❡ ❞ ✐ ♥   ❛ ✭ ❞ ✮   ♦ ❢ ❤ ❡ ✉ ❡ ✐ ♦ ♥ ✳

❲ ❡ ❛ ♥ ❞ ♦ ♠ ❧ ② ❣ ❡ ♥ ❡ ❛ ❡ ❞ ❢ ♦ ✉ ❡ ♦ ❢   p(A) ❛ ♥ ❞   p(B)❡ ❛ ❝ ❤ ❝ ♦ ❡ ♣ ♦ ♥ ❞ ✐ ♥ ❣ ♦ ♦ ♥ ❡ ♦ ❢ ❤ ❡ ❢ ♦ ✉ ❝ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ✇ ❤ ✐ ❝ ❤ ✇ ❡ ❞ ❡ ✐ ✈ ❡ ❞ ✐ ♥  

❛ ✭ ❢ ✮ ♦ ❢ ❤ ✐ ✉ ❡ ✐ ♦ ♥ ✳ ❍ ♦ ✇ ❡ ✈ ❡ ✇ ❡ ❞ ✐ ❞ ✐ ♠ ♣ ♦ ❡ ♦ ♠ ❡ ❡ ✐ ❝ ✐ ♦ ♥ ♦ ♥ ❤ ❡ ✉ ❝ ✉ ❡ ♦ ❢   p(A) ❛ ♥ ❞   p(B)✱ ✐ ✳ ❡ ✳ ♦ ♥ ❡ ❚ ② ♣ ❡ ✲ ❆  

❛ ❣ ❡ ♥ ❝ ❛ ♥ ♣ ♦ ❞ ✉ ❝ ❡ ❛ ♠ ❛ ① ✐ ♠ ✉ ♠ ♦ ❢ ✷ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ❛ ♥ ❞ ✶ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ❆ ◆ ❉ ❙ ■ ▼ ■ ▲ ❆ ❘ ▲ ❨ ♦ ♥ ❡ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ❝ ❛ ♥ ♣ ♦ ❞ ✉ ❝ ❡ ❛  

♠ ❛ ① ✐ ♠ ✉ ♠ ♦ ❢ ✷ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ❛ ♥ ❞ ✶ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ✳ ❲ ❡ ❤ ❛ ❞ ♦ ✐ ♠ ♣ ♦ ❡ ❤ ❡ ❡ ❝ ♦ ♥ ❞ ✐ ✐ ♦ ♥ ❜ ❡ ❝ ❛ ✉ ❡ ♦ ❤ ❡ ✇ ✐ ❡ ✐ ✇ ❛ ❜ ❡ ❝ ♦ ♠ ✐ ♥ ❣  

✈ ❡ ② ❞ ✐ ✣ ❝ ✉ ❧ ♦ ❛ ♥ ❞ ♦ ♠ ❧ ② ❣ ❡ ♥ ❡ ❛ ❡    p(A) ❛ ♥ ❞   p(B)✇ ❤ ✐ ❝ ❤ ❤ ❛ ✈ ❡ ♦ ♠ ❡ ❞ ❡ ✐ ❡ ❞ ♣ ♦ ♣ ❡ ✐ ❡ ✳ ❉ ✉ ❡ ♦ ❤ ✐ ❡ ✐ ❝ ✐ ♦ ♥ ♦ ♥ ❤ ❡  

✉ ❝ ✉ ❡ ♦ ❢   p(A) ❛ ♥ ❞   p(B)✱ ❤ ❡ ✐ ♠ ✉ ❧ ❛ ✐ ♦ ♥ ❡ ✉ ❧ ❛ ❡ ♥ ♦ ✈ ❡ ② ✐ ♥ ❡ ❡ ✐ ♥ ❣ ✳

❚ ❤ ❡ ✐ ♠ ✉ ❧ ❛ ✐ ♦ ♥ ❡ ✉ ❧ ❛ ❡ ❤ ♦ ✇ ♥ ✐ ♥ ❤ ❡ ♥ ❡ ① ♣ ❛ ❣ ❡ ✳ ✳

✶ ✽ 

Page 19: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 19/31

Epoch

0 2 4 6 8 10 12 14 16 18 20

   N  o .  o   f   T  y  p  e  -   A  a  g  e  n   t

0

0.5

1

1.5

2A

B

A & B

Epoch

0 2 4 6 8 10 12 14 16 18 20

   N  o .  o   f   T  y  p  e  -   B  a  g  e  n   t

0

0.5

1

1.5

2A

B

A & B

Epoch

0 2 4 6 8 10 12 14 16 18 20

   E  x   t   i  n  c   t   i  o  n   P  r  o   b  a   b   i   l   i   t  y

0

0.2

0.4

0.6

0.8

1Extinction probability starting with one Type-A agent

Epoch

0 2 4 6 8 10 12 14 16 18 20

   E  x   t   i  n  c   t   i  o  n   P  r  o   b  a   b   i   l   i   t  y

0

0.2

0.4

0.6

0.8

1Extinction probability starting with one Type-B agent

Both Type-A and Type-B agent goes extinct

❋ ✐ ❣ ✉ ❡ ✸ ✿ ❇ ♦ ❤ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ❣ ♦ ❡ ❡ ① ✐ ♥ ❝ ✳ ❆ ❢ ♦ ❤ ❡ ❧ ❡ ❣ ❡ ♥ ❞ ✬ ❆ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❧ ② ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ✱

✬ ❇ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❧ ② ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ❛ ♥ ❞ ✬ ❆ ✫ ❇ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞ ♦ ♥ ❡ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ✳

Epoch

0 2 4 6 8 10 12 14 16 18 20

   N  o .  o   f   T  y  p  e  -   A  a  g  e  n   t

0

100

200

300

400

500A

B

A & B

Epoch0 2 4 6 8 10 12 14 16 18 20

   N  o .  o   f   T  y  p  e  -   B  a  g  e  n   t

0

200

400

600A

B

A & B

Epoch

0 2 4 6 8 10 12 14 16 18 20

   E  x   t   i  n  c   t   i  o  n   P  r  o   b  a   b   i   l   i   t  y

0

0.1

0.2

0.3

0.4Extinction probability starting with one Type-A agent

Epoch

0 5 10 15 20

   E  x   t   i  n  c   t   i  o  n   P  r  o   b  a   b   i   l   i   t  y

0

0.1

0.2

0.3

0.4

0.5Extinction probability starting with one Type-B agent

Both Type-A and Type-B agent flourishes

❋ ✐ ❣ ✉ ❡ ✹ ✿ ❇ ♦ ❤ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ✢ ♦ ✉ ✐ ❤ ❡ ✳ ❆ ❢ ♦ ❤ ❡ ❧ ❡ ❣ ❡ ♥ ❞ ✬ ❆ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❧ ② ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ✱

✬ ❇ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❧ ② ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ❛ ♥ ❞ ✬ ❆ ✫ ❇ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞ ♦ ♥ ❡ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ✳

✶ ✾ 

Page 20: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 20/31

Epoch0 2 4 6 8 10 12 14 16 18 20

   N  o .  o   f   T  y  p  e  -   A  a  g  e  n   t

0

0.2

0.4

0.6

0.8

1A

B

A & B

Epoch0 2 4 6 8 10 12 14 16 18 20

   N  o .  o   f   T  y  p  e  -   B  a  g  e  n   t

0

500

1000

1500A

B

A & B

Epoch

0 2 4 6 8 10 12 14 16 18 20

   E  x   t   i  n  c   t   i  o  n   P  r  o   b  a   b   i   l   i   t  y

0

0.05

0.1

0.15

0.2Extinction probability starting with one Type-A agent

Epoch

0 2 4 6 8 10 12 14 16 18 20

   E  x   t   i  n  c   t   i  o  n   P  r  o   b  a   b   i   l   i   t  y

0

0.05

0.1

0.15Extinction probability starting with one Type-B agent

Type-A extincts and Type-B flourishes

❋ ✐ ❣ ✉ ❡ ✺ ✿ ❚ ② ♣ ❡ ✲ ❆ ❛ ❣ ❡ ♥ ❣ ♦ ❡ ❡ ① ✐ ♥ ❝ ❜ ✉ ❚ ② ♣ ❡ ✲ ❇ ✢ ♦ ✉ ✐ ❤ ❡ ✳ ❆ ❢ ♦ ❤ ❡ ❧ ❡ ❣ ❡ ♥ ❞ ✬ ❆ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❧ ② ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢  

❚ ② ♣ ❡ ✲ ❆ ✱ ✬ ❇ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❧ ② ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ❛ ♥ ❞ ✬ ❆ ✫ ❇ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞ ♦ ♥ ❡  

♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ✳

Epoch

0 2 4 6 8 10 12 14 16 18 20

   N  o .  o   f   T  y  p  e  -   A  a  g  e  n   t

0

500

1000

1500

2000A

B

A & B

Epoch

0 2 4 6 8 10 12 14 16 18 20

   N  o .  o

   f   T  y  p  e  -   B  a  g  e  n   t

0

0.2

0.4

0.6

0.8

1A

B

A & B

Epoch0 2 4 6 8 10 12 14 16 18 20

   E  x   t   i  n  c   t   i  o  n   P  r  o   b  a   b   i   l   i   t  y

0

0.1

0.2

0.3

0.4Extinction probability starting with one Type-A agent

Epoch0 2 4 6 8 10 12 14 16 18 20

   E  x   t   i  n  c   t   i  o  n   P  r  o   b  a   b   i   l   i   t  y

0

0.1

0.2

0.3

0.4Extinction probability starting with one Type-B agent

Type-B extincts and Type-A flourishes

❋ ✐ ❣ ✉ ❡ ✻ ✿ ❚ ② ♣ ❡ ✲ ❇ ❛ ❣ ❡ ♥ ❣ ♦ ❡ ❡ ① ✐ ♥ ❝ ❜ ✉ ❚ ② ♣ ❡ ✲ ❆ ✢ ♦ ✉ ✐ ❤ ❡ ✳ ❆ ❢ ♦ ❤ ❡ ❧ ❡ ❣ ❡ ♥ ❞ ✬ ❆ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❧ ② ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢  

❚ ② ♣ ❡ ✲ ❆ ✱ ✬ ❇ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❧ ② ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ❛ ♥ ❞ ✬ ❆ ✫ ❇ ✬ ♠ ❡ ❛ ♥ ❛ ✐ ♥ ❣ ✇ ✐ ❤ ♦ ♥ ❡ ❛ ❣ ❡ ♥ ♦ ❢ ❚ ② ♣ ❡ ✲ ❆ ❛ ♥ ❞ ♦ ♥ ❡  

♦ ❢ ❚ ② ♣ ❡ ✲ ❇ ✳

✷ ✵ 

Page 21: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 21/31

✸ ❚ ❤ ❡ ♦ ❡ ✐ ❝ ❛ ❧ ♦ ❜ ❧ ❡ ♠  

✸ ✳ ✶ ❆ ♥ ♦ ❤ ❡ ❈ ✐ ❡ ✐ ♦ ♥ ❢ ♦ ❚ ❛ ♥ ✐ ❡ ♥ ❝ ❡  

✭ ❛ ✮  ♦ ✈ ✐ ♥ ❣ ❤ ❛   f ∗ ∈ A✱ ♠ ❡ ❧ ❞ ♦ ✇ ♥ ♦ ♣ ♦ ✈ ✐ ♥ ❣ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ✿

♦ ♦ ❢ ❤ ❛   f ∗ (k) = 0 ✿

f ∗ (k) = supf ∈A

f  (k) = supf ∈A

0   ∵ f (k)=0

❛  

  f ∈A

= 0

♦ ♦ ❢ ❤ ❛   0 ≤ f ∗ ( j) ≤  1 ;  ∀ j  ∈  S ✿

f ∗ ( j) = supf ∈A

f  ( j) ≤   supf ∈A

1   ∵ f (j)≤1 ; ∀j ❛  

  f ∈A

= 1

f ∗ ( j) = supf ∈A

f  ( j) ≥   supf ∈A

0   ∵ f (j)≥0 ; ∀j

❛  

  f ∈A

= 0

❍ ❡ ♥ ❝ ❡   0 ≤ f ∗ ( j) ≤  1 ;  ∀ j  ∈  S ✳

♦ ♦ ❢ ❤ ❛   f ∗ ✐ ✉ ❜ ❤ ❛ ♠ ♦ ♥ ✐ ❝ ✇ ✐ ❤ ❡ ♣ ❡ ❝ ♦   P ✿

❈ ♦ ♥ ✐ ❞ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣ ❞ ❡ ✜ ♥ ✐ ✐ ♦ ♥  

f i ≡  arg supf ∈A

f  (i) ✭ ✼ ✼ ✮  

◗ ✉ ❛ ❧ ✐ ❛ ✐ ✈ ❡ ❧ ② ♣ ❡ ❛ ❦ ✐ ♥ ❣ ✱ f i ✐ ❛ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ❜ ❡ ❧ ♦ ♥ ❣ ✐ ♥ ❣ ♦ ❤ ❡ ❡    A ✇ ❤ ✐ ❝ ❤ ♠ ❛ ① ✐ ♠ ✐ ③ ❡    f  (i) ✳ ■ ♥ ♦ ❤ ❡ ✇ ♦ ❞    f ∗ (i) =  f i (i) ✳ ◆ ♦ ✇  

❝ ♦ ♥ ✐ ❞ ❡ ❤ ❡ ❢ ♦ ❧ ❧ ♦ ✇ ✐ ♥ ❣  

j∈S P ijf ∗ ( j)   ≥

j∈S P ijf i ( j) ✭ ✼ ✽ ✮  

≥   f i (i) ✭ ✼ ✾ ✮  

=   f ∗ (i)

■ ♥ ❡ ✉ ❛ ❧ ✐ ② ✼ ✽ ❡ ✉ ❧ ❢ ♦ ♠ ❤ ❡ ❞ ❡ ✜ ♥ ✐ ✐ ♦ ♥ ♦ ❢   f ∗ ( j) ✱ ✐ ✳ ❡ ✳  f ∗ ( j) = supf ∈A

f  ( j)   ≥   h ( j) ;   ∀h   ∈ A✳ ■ ♥ ✐ ♥ ❡ ✉ ❛ ❧ ✐ ② ✼ ✽ ✱  h   =   f i ✳

■ ♥ ❡ ✉ ❛ ❧ ✐ ② ✼ ✾ ❡ ✉ ❧ ❢ ♦ ♠ ❤ ❡ ❢ ❛ ❝ ❤ ❛   f i   ∈ A ❛ ♥ ❞ ❤ ❡ ♥ ❝ ❡    f i ✐ ❛ ✉ ❜ ❤ ❛ ♠ ♦ ♥ ✐ ❝ ❢ ✉ ♥ ❝ ✐ ♦ ♥ ✇ ✐ ❤ ❡ ♣ ❡ ❝ ♦    P ✳ ❚ ❤ ❡ ❛ ❜ ♦ ✈ ❡  

♣ ♦ ♦ ❢ ❤ ♦ ✇ ❤ ❛   f ∗ ✐ ✉ ❜ ❤ ❛ ♠ ♦ ♥ ✐ ❝ ✇ ✐ ❤ ❡ ♣ ❡ ❝ ♦    P ✳

✭ ❜ ✮  ■ ♥  ❛ ✭ ❛ ✮   ✇ ❡ ❛ ❧ ❡ ❛ ❞ ② ♣ ♦ ✈ ❡ ❞ ❤ ❛   f ∗ ✐ ✉ ❜ ❤ ❛ ♠ ♦ ♥ ✐ ❝ ✇ ✐ ❤ ❡ ♣ ❡ ❝ ♦   P ✱ ✐ ✳ ❡ ✳j∈S 

P ijf ∗ ( j) ≥  f ∗ (i) ;  ∀i = k

◆ ♦ ✇ ✇ ❡ ❤ ❛ ✈ ❡ ♦ ♣ ♦ ✈ ❡ ❤ ❛   f ∗ ✐ ❤ ❛ ♠ ♦ ♥ ✐ ❝ ✇ ✐ ❤ ❡ ♣ ❡ ❝ ♦   P ✳ ❚ ❤ ✐ ♠ ❡ ❧ ❞ ♦ ✇ ♥ ♦ ♣ ♦ ✈ ✐ ♥ ❣ ❤ ❛   ❤ ❡ ❡ ❡ ① ✐ ♥ ♦    i  ∈  S  ❢ ♦  

✇ ❤ ✐ ❝ ❤   j∈S 

P ijf ∗ ( j) > f ∗ (i) ✭ ✽ ✵ ✮  

❛ ♥ ❞ ❤ ❡ ♥ ❝ ❡ ❤ ❡ ♦ ♥ ❧ ② ♣ ♦ ✐ ❜ ✐ ❧ ✐ ② ✐  

j∈S 

P ijf ∗ ( j) =  f ∗ (i) ;  ∀i  =  k ✳ ❲ ❡ ✇ ✐ ❧ ❧ ♣ ♦ ✈ ❡ ❤ ✐ ❜ ② ✉ ✐ ♥ ❣   ❝ ♦ ♥ ❛ ❞ ✐ ❝ ✐ ♦ ♥   ✳ ▲ ❡ ❞ ❡ ✜ ♥ ❡ ❛  

❢ ✉ ♥ ❝ ✐ ♦ ♥   g ✉ ❝ ❤ ❤ ❛  

g (i) =j∈S 

P ijf ∗ ( j) ;  ∀i ∈  S  ✭ ✽ ✶ ✮  

✷ ✶ 

Page 22: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 22/31

❆ ❝ ❝ ♦ ❞ ✐ ♥ ❣ ♦ ✐ ♥ ❡ ✉ ❛ ❧ ✐ ② ✽ ✵ ✱ g (i)  > f ∗ (i) ✳ ■ ❢ ✇ ❡ ❝ ❛ ♥ ❤ ♦ ✇ ❤ ❛    g  ∈ A✱ ❤ ❡ ♥ ✐ ✇ ✐ ❧ ❧ ❝ ♦ ♥ ❛ ❞ ✐ ❝ ❤ ❡ ❞ ❡ ✜ ♥ ✐ ✐ ♦ ♥ ♦ ❢    f ∗ ✱ ❤ ❡ ❡ ❜ ②  

♣ ♦ ✈ ✐ ♥ ❣ ❤ ❛ ✐ ♥ ❡ ✉ ❛ ❧ ✐ ② ✽ ✵ ✐ ♥ ♦ ♣ ♦ ✐ ❜ ❧ ❡ ✳ ◆ ♦ ✇ ❛ ❧ ❧ ✇ ❡ ❤ ❛ ✈ ❡ ♦ ♣ ♦ ✈ ❡ ✐   g ∈ A✳ 

♦ ♦ ❢ ❤ ❛   g (k) = 0 ✿

g (k) =j∈S 

P kjf ∗ ( j) =  f ∗ (k) = 0   ∵   f ∗ ∈ A

♦ ♦ ❢ ❤ ❛   0 ≤ g ( j) ≤  1 ;  ∀ j  ∈  S ✿

g ( j) = u∈S 

P juf ∗ (u) ≤ u∈S 

P ju · 1   ∵ f ∗(u)≤1 ❛  

  f ∗∈A

= 1 ✭ ❘ ♦ ✇ ✉ ♠ ♦ ❢ ♦ ❜ ❛ ❜ ✐ ❧ ✐ ② ❚ ❛ ♥ ✐ ✐ ♦ ♥ ▼ ❛ ✐ ① ✐    1)

g ( j) =u∈S 

P juf ∗ (u) ≥u∈S 

P ju · 0   ∵ f ∗(u)≥0 ❛    f ∗∈A

= 0

❍ ❡ ♥ ❝ ❡   0 ≤ g ( j) ≤  1 ;  ∀ j  ∈  S ✳

♦ ♦ ❢ ❤ ❛   g ✐ ✉ ❜ ❤ ❛ ♠ ♦ ♥ ✐ ❝ ✇ ✐ ❤ ❡ ♣ ❡ ❝ ♦    P ✿

j∈S 

P ijg ( j)

=j∈S 

P ij

u∈S 

P juf ∗ (u)

=j∈S 

P ij

u∈S 

P juf u (u)

✭ ✽ ✷ ✮  

≥j∈S 

P ij

u∈S 

P juf j (u)

✭ ✽ ✸ ✮  

≥ j∈S 

P ijf j ( j) ✭ ✽ ✹ ✮  

=j∈S 

P ijf ∗ ( j) ✭ ✽ ✺ ✮  

=   g (i)

❊ ✉ ❛ ✐ ♦ ♥ ✽ ✷ ❛ ♥ ❞ ✽ ✺ ✐ ❛ ❞ ✐ ❡ ❝ ❝ ♦ ♥ ❡ ✉ ❡ ♥ ❝ ❡ ♦ ❢ ❤ ❡ ❞ ❡ ✜ ♥ ✐ ✐ ♦ ♥ ♦ ❢  f  ✐ ♥ ❡ ✉ ❛ ✐ ♦ ♥ ✼ ✼ ✳ ■ ♥ ❡ ✉ ❛ ❧ ✐ ② ✽ ✸ ❡ ✉ ❧ ❢ ♦ ♠ ❤ ❡ ❢ ❛ ❝ ❤ ❛  

f u (u) ≥  f j (u) ;  ∀ j  ∈  S ✳ ■ ♥ ❡ ✉ ❛ ❧ ✐ ② ✽ ✹ ✐ ✈ ❛ ❧ ✐ ❞ ❜ ❡ ❝ ❛ ✉ ❡    f ∗ ✐ ❛ ✉ ❜ ❤ ❛ ♠ ♦ ♥ ✐ ❝ ✇ ✐ ❤ ❡ ♣ ❡ ❝ ♦   P ✳

❚ ❤ ❡ ❛ ❜ ♦ ✈ ❡ ❤ ❡ ❡ ♣ ♦ ♦ ❢ ❤ ♦ ✇ ❤ ❛   g ∈ A✳ ❚ ❤ ✐ ❝ ♦ ♥ ❝ ❧ ✉ ❞ ❡ ❤ ❡ ♣ ♦ ♦ ❢ ✳

✷ ✷ 

Page 23: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 23/31

11/25/15 10:50 AM C:\Users\sahah_000\Desk...\Markov_Maze.m 1 of 4

clear

clc

load(!ro"a"#l#$%_&ra's#$#o'_Ma$r#.ma$)*

 

+++++++++++++++++++++++++ Code for 1.1.". : ,&A-& ++++++++++++++++++++++

 

!_$#lde!*

!_$#lde(:)0*

!_$#lde()1*

 

!_$#lde*

(:)3*

(:)3*

 

fo'es(151)*

 

m_$a#'v(e%e(15))6f*

s$rAvera7e &#me "efore -a$h"er$ reaches cheese 'm2s$r(m_$a(15))3*

d#s8(s$r)*

$em8m_$a(15)*

 

++++++++++++++++++++++++++ Code for 1.1.". : 9D +++++++++++++++++++++++

 

+++++++++++++++++++++++++ Code for 1.1.c. : ,&A-& ++++++++++++++++++++++

 

clearvars ece8$ ! ; $em8

 

!_$#lde!*

!_$#lde(:)0*

!_$#lde()1*

 

!_$#lde*

(:)3*

(:)3*

 

fzeros(151)*

f(<1)1*

f(=1)1*

 

m_shock#'v(e%e(15))6f*

s$rAvera7e 'm"er of shock "efore reach#'7 cheese 'm2s$r(m_shock(15))3*

d#s8(s$r)*

 

++++++++++++++++++++++++++ Code for 1.1.c. : 9D +++++++++++++++++++++++

 

+++++++++++++++++++++++++ Code for 1.1.d. : ,&A-& ++++++++++++++++++++++

 

clearvars ece8$ ! ; $em8

 

!_$#lde!*

!_$#lde(:)0*

Appendix A: Markov Maze

Page 24: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 24/31

11/25/15 10:50 AM C:\Users\sahah_000\Desk...\Markov_Maze.m 2 of 4

!_$#lde(=:)0*

!_$#lde(>:)0*

!_$#lde()1*

!_$#lde(==)1*

!_$#lde(>>)1*

 

ro?_dele$e = >3*

col_dele$e = >3*

!_$#lde*

(ro?_dele$e:)3*

(:col_dele$e)3*

 

clear ro?_dele$e col_dele$e

ro?_dele$e = >3*

col_dele$e1 2 4 5 @ < 10 11 12 1 14 15 1@3*

-!_$#lde*

-(ro?_dele$e:)3*

-(:col_dele$e)3*

 

U#'v(e%e(1))6-*

s$r!ro"a"#l#$% of reach#'7 cheese ?#$ho$ 7e$$#'7 shocked 'm2s$r(U(11))3*

d#s8(s$r)*

 

++++++++++++++++++++++++++ Code for 1.1.d. : 9D +++++++++++++++++++++++

 

+++++++++++++++++++++++++ Code for 1.1.e. : ,&A-& ++++++++++++++++++++++

 

clearvars ece8$ ! ; $em8

 

Mzeros(25@25@)*

for #0:1:255

  for 0:1:255

  s_r_#floor(#/1@)B1*

  s_s_##1@6(s_r_#1)B1*

  s_r_floor(/1@)B1*

  s_s_1@6(s_r_1)B1*

  M(#B1B1)!(s_r_#s_r_)6;(s_s_#s_s_)*

  e'd

e'd

 

M_$#ldeM*

for #0:1<:255

  M_$#lde(#B1:)zeros(125@)*

  M_$#lde(#B1#B1)1*

e'd

 

M_$#lde*

ro?_dele$e0:1<:255*

col_dele$e0:1<:255*

ro?_dele$ero?_dele$eB1*

col_dele$ecol_dele$eB1*

Page 25: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 25/31

11/25/15 10:50 AM C:\Users\sahah_000\Desk...\Markov_Maze.m of 4

(ro?_dele$e:)3*

(:col_dele$e)3*

 

fo'es(2401)*

 

m_$a#'v(e%e(240))6f*

s$rAvera7e &#me "efore -a$"er$ a'd ,am mee$ each o$her 'm2s$r(m_$a(22@))3*

d#s8(s$r)*

 

++++++++++++++++++++++++++ Code for 1.1.e. : 9D +++++++++++++++++++++++

 

+++++++++++++++++++++++++ Code for 1.1.f. : ,&A-& ++++++++++++++++++++++

 

clearvars ece8$ ! $em8

 

8_s$'r%o'es(11@)6#'v(e%e(1@)!Bo'es(1@1@))* + -es'#ck ormla

$a1/8_s$'r%()*

 

s$rAvera7e &#me "efore -a$"er$ ca' res$ 'm2s$r($em8)B'm2s$r($a)(C1)3*

d#s8(s$r)*

 

++++++++++++++++++++++++++ Code for 1.1.f. : 9D +++++++++++++++++++++++

 

+++++++++++++++++++++++++ Code for 1.1.7. : ,&A-& ++++++++++++++++++++++

 

clearvars ece8$ !

 

!_$#lde!*

!_$#lde(:)0*

!_$#lde(=:)0*

!_$#lde(>:)0*

!_$#lde()1*

!_$#lde(==)1*

!_$#lde(>>)1*

 

ro?_dele$e = >3*

col_dele$e = >3*

!_$#lde*

(ro?_dele$e:)3*

(:col_dele$e)3*

 

clear ro?_dele$e col_dele$e

ro?_dele$e = >3*

col_dele$e1 2 4 5 @ < 10 11 12 1 14 15 1@3*

-!_$#lde*

-(ro?_dele$e:)3*

-(:col_dele$e)3*

 

U#'v(e%e(1))6-*

8=U(12)*

8>U(1)*

Page 26: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 26/31

11/25/15 10:50 AM C:\Users\sahah_000\Desk...\Markov_Maze.m 4 of 4

 

!_$#lde(=:)!(=:)*

!_$#lde(>:)!(>:)*

 

clear

!_$#lde*

(:)3*

(:)3*

 

fzeros(151)*

f(=1)1*

m_shock#'v(e%e(15))6f*

=m_shock(<)*

 

fzeros(151)*

f(<1)1*

m_shock#'v(e%e(15))6f*

>m_shock(=)*

 

s$r98ec$ed 'm"er of shock (mod#f#ed) 'm2s$r(8=6(=B1)B8>6(>B1))3*

d#s8(s$r)*

 

++++++++++++++++++++++++++ Code for 1.1.7. : 9D +++++++++++++++++++++++

 

clear

 

Page 27: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 27/31

11/25/15 10:50 AM C:\Users\sah...\Email_Sharing_Analysis.m 1 of 2

funcion !mean"mean_a##iional"$ariance"$ariance_a##iional"E%incion&ro'"

E%incion&ro'_a##iional( ) Email_Sharing_Analysis*m"+",-",",

 

ecor Creaion: S+A3+

 

mean)4eros*1"+16

mean_a##iional)4eros*1"+16

$ariance)4eros*1"+16

$ariance_a##iional)4eros*1"+16

E%incion&ro')4eros*1"+16

E%incion&ro'_a##iional)4eros*1"+16

 

ecor Creaion: E7

 

-niiali4aion for 0h E,och: S+A3+

 

mean*1)m6

mean_a##iional*1)m6

$ariance*1)06

$ariance_a##iional*1)06

E%incion&ro'*1)06

E%incion&ro'_a##iional*1)06

 

-niiali4aion for 0h E,och: E7

 

&reliminary Calculaion: S+A3+

 

i)numel*,-6

mu_-)sum**0:1:*i819,-6

$ar_-)sum****0:1:*i818mu_-.9**0:1:*i818mu_-.9,-6

 

o)numel*,6

mu_)sum**0:1:*o819,6

$ar_)sum****0:1:*o818mu_.9**0:1:*o818mu_.9,6

 

a##)numel*,6

mu_)sum**0:1:*a##819,6

$ar_)sum****0:1:*a##818mu_.9**0:1:*a##818mu_.9,6

 

A)!mu_9mu_- 06 $ar_9mu_-$ar_-9*mu_;2 *mu_9mu_-;2(6

 

A_a##)A6

A_a##*2"1)A_a##*2"129mu_9mu_9mu_-6

<_a##)!mu_6 $ar_mu_;2(6

 

&reliminary Calculaion: E7

 

al,ha)06

'ea)16

 

for n)1:1:+

Appendix B: Email Sharing

Page 28: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 28/31

11/25/15 10:50 AM C:\Users\sah...\Email_Sharing_Analysis.m 2 of 2

 

=)!mean*n6mean*n;2$ariance*n(6

  =)A9=6

  mean*n1)=*16

  $ariance*n1)=*28=*1;26

 

=a##)!mean_a##iional*n6mean_a##iional*n;2$ariance_a##iional*n(6

  =a##)A_a##9=a##<_a##6

  mean_a##iional*n1)=a##*16

  $ariance_a##iional*n1)=a##*28=a##*1;26

 

'ea)'ea9&ro'_>en_?unc*,"al,ha6

  al,ha)&ro'_>en_?unc*,-"&ro'_>en_?unc*,"al,ha6

 

E%incion&ro'*n1)al,ha;m6

  E%incion&ro'_a##iional*n1)*al,ha;m9'ea6

 

en#

 

en#

 

funcion !$al( ) &ro'_>en_?unc*,#f"s

 

$al)06

)numel*,#f6

for i)0:1:*81

  $al)$al,#f*i19*s;i6

en#

 

en#

 

Page 29: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 29/31

11/25/15 10:51 AM C:\Use...\Branching_TwoWays_MonteCarlo. 1 o! 2

!"nction #$_A%$_B& ' Branching_TwoWays_MonteCarlo(T%)_A%)_B%$_A_0%$_B_0*

 

+ ,n)"ts:

+ T -- "er o! i"lation te)s (incl"ing the 0th e)och*

+ )_A -- 34 o! o!!s)rings o! Ty)e-A agent

+ )_B -- 34 o! o!!s)rings o! Ty)e-B agent

+ $_A_0 -- 6 o! Ty)e-A agents in the 0th e)och

+ $_B_0 -- 6 o! Ty)e-B agents in the 0th e)och

 

+ 7"t)"ts:

+ $_A -- 6 o! Ty)e-A agents at i!!erent e)ochs

+ $_B -- 6 o! Ty)e-B agents at i!!erent e)ochs

 

$_A'8eros(1%T91*

$_B'8eros(1%T91*

$_A(1*'$_A_0 + 6 o! Ty)e-A agents in the 0th e)och

$_B(1*'$_B_0 + 6 o! Ty)e-B agents in the 0th e)och

 

!or n'1:1:T

 

+ Contri"tion o! Ty)e-A agent !or the ne;t e)och: TA<T

  !or =A'1:1:$_A(n*

  #>_AA%>_AB&'<an"?en_24()_A* + Ty)e-A an Ty)e-B agent )ro"ce y =Ath agent

o! Ty)e-A in the c"rrent e)och

  $_A(n91*'$_A(n91*9>_AA + U)ation o! Ty)e-A agent !or the ne;t e)och

  $_B(n91*'$_B(n91*9>_AB + U)ation o! Ty)e-B agent !or the ne;t e)och

  en

  + Contri"tion o! Ty)e-A agent !or the ne;t e)och: @4

 

+ Contri"tion o! Ty)e-B agent !or the ne;t e)och: TA<T

  !or =B'1:1:$_B(n*

  #>_BA%>_BB&'<an"?en_24()_B* + Ty)e-A an Ty)e-B agent )ro"ce y =Bth agent

o! Ty)e-B in the c"rrent e)och

  $_A(n91*'$_A(n91*9>_BA + U)ation o! Ty)e-A agent !or the ne;t e)och

  $_B(n91*'$_B(n91*9>_BB + U)ation o! Ty)e-B agent !or the ne;t e)och

  en

  + Contri"tion o! Ty)e-B agent !or the ne;t e)och: @4

 

en

 

en

 

+ 24 <ano "er ?enerator: TA<T

 

!"nction #col%row&'<an"?en_24()*

 

+ col is the A agent

+ row is the B agent

 

_col'n"el()(1%:**

_row'n"el()(:%1**

ppendix C: Monte Carlo Simulation of 2-Way Branching Proce

Page 30: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 30/31

11/25/15 10:51 AM C:\Use...\Branching_TwoWays_MonteCarlo. 2 o! 2

 

te)'ran

acc')(1%1*

 

i! (te)''1*

  col'_col

  row'_row

else

  col'1

  row'1

  while(te)'acc*

  col'col91

  i! (col_col*

  col'1

  row'row91

  en

  acc'acc9)(row%col*

  en

en

 

col'col-1

row'row-1

 

en

 

+ 24 <ano "er ?enerator: @4

 

Page 31: Solution of Stochastic Process and Modelling

7/23/2019 Solution of Stochastic Process and Modelling

http://slidepdf.com/reader/full/solution-of-stochastic-process-and-modelling 31/31

11/25/15 10:51 AM C:\Users...\Branching_TwoWays_Analysis.m 1 o 1

!nc"ion #$ro%_A&$ro%_B' ( Branching_TwoWays_Analysis)$A&$B&T*

 

+ ,n$!":

+ $A -- ro%a%ili"y is"ri%!"ion o s$rings o Ty$e-A agen"

+ $B -- ro%a%ili"y is"ri%!"ion o s$rings o Ty$e-B agen"

+ T -- im!la"ion Time

 

+ !"$!":

+ $ro%_A -- 34"inc"ion ro%a%ili"y s"ar"ing wi"h one Ty$e-A agen"

+ $ro%_B -- 34"inc"ion ro%a%ili"y s"ar"ing wi"h one Ty$e-B agen"

 

$ro%_A(eros)1&T61*7

$ro%_B(eros)1&T61*7

 

$ro%_A)1*(07 + 34"inc"ion ro%a%ili"y a" 0"h e$och is 0.

$ro%_B)1*(07 + 34"inc"ion ro%a%ili"y a" 0"h e$och is 0.

 

or n(1:1:T

  $ro%_A)n61*(ro%_8en_9!nc)$A&$ro%_A)n*&$ro%_B)n**7 + ec!rsion 9orm!la

$ro%_B)n61*(ro%_8en_9!nc)$B&$ro%_A)n*&$ro%_B)n**7 + ec!rsion 9orm!la

en;

 

en;

 

+ ro%a%ili"y 8enera"ing 9!nc"ion o Two <aria%les: TAT

 

!nc"ion #=al' ( ro%_8en_9!nc)$;&!&=*

 

>a(n!mel)$;)1&:**7

>%(n!mel)$;):&1**7

 

=al(07

or ?(0:1:)>%-1*

  or i(0:1:)>a-1*

  =al(=al6$;)?61&i61*@)!i*@)=?*7

  en;

en;

 

en;

 

+ ro%a%ili"y 8enera"ing 9!nc"ion o Two <aria%les: 3>

 

Appendix D: Extinction Probability of 2-Way Branching Process