Markov Process and Excel Spreadsheets

A Computational Adventure

Our bags are packed, we are ready to go…

Three Great Workbooks

Equation Solversolves system of linear equations using matrix inversionUse to solve systems of linear equations, find a matrix inverse, or compute a determinant

MatrixPowerExcel workbook for raising a matrix to higher powersGreat for finding the n-step transition matrix P(n)

AbsorptionExcel workbook for absorption type Markov processesFind the fundamental matrix F and the absorption probabilities FR

Go the course website homepage and select Software then download the filesSoftware CommentsLindo/What's Best/LINGO solves linear, integer, and nonlinear programs

Equation Solver solves system of linear equations using matrix inversion

Frontline Systems Linear, Integer, and Nonlinear Optimization (Excel Solver enhancements)

Excel Solver Excel Add-on solves LP, IP, and NLP problems

Courseware (Text CD ) AMPL -solves linear, integer, and nonlinear programs; TORA - optimization system

Win QSB OR Software from Wiley $84.95Queue Models.xls Excel workbook for solving M/M/s queues

MatrixPower.xls Excel workbook for raising a matrix to higher powers

Absorption.xls Excel workbook for absorption type Markov processes

Queue.xls Alternate queuing spreadsheet for M/M/s and M/G/1 queues

Solving the Steady-State Equations

( ) ( )1 2 3 4 1 2 3 4

.2 .4 0 .4

.1 .1 .5 .30 .3 .3 .4.2 0 .7 .1

.2 .4 0 .4

.1 .1 .5 .30 .3 .3 .4.2 0 .7 .1

P

π π π π π π π π

⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎣ ⎦

⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎣ ⎦

A common everyday transition matrixthat is irreducible + ergodic

The algebra

( ) ( )1 2 3 4 1 2 3 4

1 1 2 4

2 1 2 3

3 2 3 4

1 2 3 4

.2 .4 0 .4

.1 .1 .5 .30 .3 .3 .4.2 0 .7 .1

.2 .1 .2.4 .1 .3.5 .3 .7

1

π π π π π π π π

π π π ππ π π ππ π π ππ π π π

⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎣ ⎦

= + += + += + ++ + + = 1 2 4

1 2 3

2 3 4

1 2 3 4

.8 .1 .2 0.4 .9 .3 0

.5 .7 .7 01

π π ππ π π

π π ππ π π π

− − =− + − =

− + − =+ + + =

Carefully enter data into the spreadsheet – Equation_Solver…

Instructions: Enter data including matrix size N then Run Macro "Inverse"

To run Macro enter ALT+F8 or select Tools then Macro from the menu bar.

1 2 4

1 2 3

2 3 4

1 2 3 4

.8 .1 .2 0.4 .9 .3 0

.5 .7 .7 01

π π ππ π π

π π ππ π π π

− − =− + − =

− + − =+ + + =

With great skill, execute the inverse macro… ( )1 2 3 4π π π π

The Absorption State Markov Process

.1 .2 .5 .1 .1

.2 .3 .2 .1 .20 .4 .1 .3 .20 0 0 1 00 0 0 0 1

.1 .1 .1 .2 .5

.1 .2 .2 .3 .2

.3 .2 0 .4 .1

P

R Q

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

I would like to work this problem for you Chuck. However, my

hands are tied.

With assured (untied) hands, enter the R and Q matrices into absorptions.xls

.1 .1 .1 .2 .5

.1 .2 .2 .3 .2

.3 .2 0 .4 .1R Q

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

Absorption Type Markov ProcessEnter Q matrix - Transient States to Transient States

Dim = 3row/col 1 2 3 4 5 6 7 8 9

1 0.1 0.2 0.52 0.2 0.3 0.23 0 0.4 0.14

Entering the R matrix…

Now Run the Macro called Compute

Enter R matrix - Transient to Absorption StatesNbr cols

= 2row/col 1 2 3 4 5 6 7 8 9

1 0.1 0.12 0.1 0.23 0.3 0.24

Look at the results….Fundamental matrix (I-Q)^-1 - Expected Nbr Transitions in Each Transient State

row/col 1 2 3 4 19 20 sum1 1.313 0.907 0.931 3.15042 0.43 1.933 0.668 3.03103 0.191 0.859 1.408 2.45824 0.00005 0.0000

FR matrix - Probability of absorption

row/col1 0.501 0.4992 0.437 0.5633 0.527 0.4734

I still like to help you with this one Chuck

but I am all tied up at the moment.

This sounds great but what if I need the n-step

transition matrix?

The n-step transition matrixusing MatrixPower!

.1 .2 .5 .1 .1

.2 .3 .2 .1 .20 .4 .1 .3 .20 0 0 1 00 0 0 0 1

P

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

Let’s do that again…

Matrix Multiplication – the Excel way… MMULT(B3:D5,F3:G5)

Note The formula in the example must be entered as an array formula. Select the range of cells for the resulting matrix starting with the formula cell. Press F2, and then press CTRL+SHIFT+ENTER.

But what if we do not want to use your homegrown

spreadsheets. There must be alternatives?

Students may use one of the matrix calculators found on the internet, purchase WinQSB or another OR software package, program Excel

themselves, or use a calculator that will perform matrix operations.

The Bottom Line

I like to do matrix

operations in my head.