Operations Research

Hiroshi Toyoizumi 1

June 11 2007

1E-mail toyoizumiwasedajp

Contents

1 Introduction of Operations Research 511 A Simple Example 512 What is OR 613 Decision Making 714 History 715 Examples of OR 8

2 Basic Probability Theory 1321 Why Probability 1322 Probability Space 1323 Conditional Probability and Independence 1524 Random Variables 1725 Expectation Variance and Standard Deviation 1926 How to Make a Random Variable 2127 News-vendor Problem ldquoHow many should you buyrdquo 2228 Covariance and Correlation 2329 Value at Risk 24210 References 27211 Exercises 27

3 Markov chain 2931 Discrete-time Markov chain 2932 Stationary State 3033 Matrix Representation 3134 Stock Price Dynamics Evaluation Based on Markov Chain 3135 Googlersquos PageRank and Markov Chain 32

2

CONTENTS 3

4 Birth and Death process and Poisson Process 3641 Definition of Birth and Death Process 3642 Infinitesimal Generator 3643 System Dynamics 3744 Poisson Process 3845 Why we use Poisson process 3846 Z-trnasform of Poisson process 3947 Independent Increment 3948 Interearrival Time 4049 Memory-less property of Poisson process 40410 PASTA Poisson Arrival See Time Average 40411 Excercise 40

5 Introduction of Queueing Systems 4251 Foundation of Performance Evaluation 4252 Starbucks 4353 Specification of queueing systems 4354 Littlersquos formula 4455 Lindley equation and Loynes variable 4656 Exercise 47

6 MM1 queue 4961 MM1 queue as a birth and death process 4962 Utilization 5063 Waiting Time Estimation 50

631 Waiting Time by Littlelsquos Formula 51632 Waiting Time Distribution of MM1 Queues 51

64 Example 5265 Excercise 53

7 Reversibility 5471 Output from Queue 5472 Reversibibility 54

721 Definition of Reversibility 54722 Local Balance for Markov Chain 54

73 Output from MM1 queue 5674 Excercise 56

4 CONTENTS

8 Network of Queues 5781 Open Network of queues 5782 Global Balance of Network 5783 Traffic Equation 5884 Product Form Solution 5885 Exercises 60

9 Repairman Problem in Manufacturing System 6191 Repairman Problem 6192 Average Analysis 6293 Single Repairman in Markov Model 65

10 Production Systems 68101 Produce-to-Order System 68102 Produce-to-Stock System 69

11 Decision Making 72

12 Analytic Hierarchy Processes 73

Chapter 1

Introduction of Operations Research

This handout can be fond at httpwwwfwasedajptoyoizumiHerersquos the brief introduction of the concept of Operations Research This

chapter is mainly consisted by quotations

11 A Simple Example

Herersquos a simple example from Mori[5]

Example 11 (Behind Monty Hallrsquos Doors) There is a famous quiz show in UScalled Letrsquos Make a Deal The MC of the show is Monty Hall

Figure 11 Monty Hall appeared in Letrsquos Make a Deal adopted fromhttpwwwcurtalliaumecomlmadhtml

5

Contents

1 Introduction of Operations Research 511 A Simple Example 512 What is OR 613 Decision Making 714 History 715 Examples of OR 8

2 Basic Probability Theory 1321 Why Probability 1322 Probability Space 1323 Conditional Probability and Independence 1524 Random Variables 1725 Expectation Variance and Standard Deviation 1926 How to Make a Random Variable 2127 News-vendor Problem ldquoHow many should you buyrdquo 2228 Covariance and Correlation 2329 Value at Risk 24210 References 27211 Exercises 27

3 Markov chain 2931 Discrete-time Markov chain 2932 Stationary State 3033 Matrix Representation 3134 Stock Price Dynamics Evaluation Based on Markov Chain 3135 Googlersquos PageRank and Markov Chain 32

2

CONTENTS 3

4 Birth and Death process and Poisson Process 3641 Definition of Birth and Death Process 3642 Infinitesimal Generator 3643 System Dynamics 3744 Poisson Process 3845 Why we use Poisson process 3846 Z-trnasform of Poisson process 3947 Independent Increment 3948 Interearrival Time 4049 Memory-less property of Poisson process 40410 PASTA Poisson Arrival See Time Average 40411 Excercise 40

5 Introduction of Queueing Systems 4251 Foundation of Performance Evaluation 4252 Starbucks 4353 Specification of queueing systems 4354 Littlersquos formula 4455 Lindley equation and Loynes variable 4656 Exercise 47

6 MM1 queue 4961 MM1 queue as a birth and death process 4962 Utilization 5063 Waiting Time Estimation 50

631 Waiting Time by Littlelsquos Formula 51632 Waiting Time Distribution of MM1 Queues 51

64 Example 5265 Excercise 53

7 Reversibility 5471 Output from Queue 5472 Reversibibility 54

721 Definition of Reversibility 54722 Local Balance for Markov Chain 54

73 Output from MM1 queue 5674 Excercise 56

4 CONTENTS

8 Network of Queues 5781 Open Network of queues 5782 Global Balance of Network 5783 Traffic Equation 5884 Product Form Solution 5885 Exercises 60

9 Repairman Problem in Manufacturing System 6191 Repairman Problem 6192 Average Analysis 6293 Single Repairman in Markov Model 65

10 Production Systems 68101 Produce-to-Order System 68102 Produce-to-Stock System 69

11 Decision Making 72

12 Analytic Hierarchy Processes 73

Chapter 1

Introduction of Operations Research

This handout can be fond at httpwwwfwasedajptoyoizumiHerersquos the brief introduction of the concept of Operations Research This

chapter is mainly consisted by quotations

11 A Simple Example

Herersquos a simple example from Mori[5]

Example 11 (Behind Monty Hallrsquos Doors) There is a famous quiz show in UScalled Letrsquos Make a Deal The MC of the show is Monty Hall

Figure 11 Monty Hall appeared in Letrsquos Make a Deal adopted fromhttpwwwcurtalliaumecomlmadhtml

5

CONTENTS 3

4 Birth and Death process and Poisson Process 3641 Definition of Birth and Death Process 3642 Infinitesimal Generator 3643 System Dynamics 3744 Poisson Process 3845 Why we use Poisson process 3846 Z-trnasform of Poisson process 3947 Independent Increment 3948 Interearrival Time 4049 Memory-less property of Poisson process 40410 PASTA Poisson Arrival See Time Average 40411 Excercise 40

5 Introduction of Queueing Systems 4251 Foundation of Performance Evaluation 4252 Starbucks 4353 Specification of queueing systems 4354 Littlersquos formula 4455 Lindley equation and Loynes variable 4656 Exercise 47

6 MM1 queue 4961 MM1 queue as a birth and death process 4962 Utilization 5063 Waiting Time Estimation 50

631 Waiting Time by Littlelsquos Formula 51632 Waiting Time Distribution of MM1 Queues 51

64 Example 5265 Excercise 53

7 Reversibility 5471 Output from Queue 5472 Reversibibility 54

721 Definition of Reversibility 54722 Local Balance for Markov Chain 54

73 Output from MM1 queue 5674 Excercise 56

4 CONTENTS

8 Network of Queues 5781 Open Network of queues 5782 Global Balance of Network 5783 Traffic Equation 5884 Product Form Solution 5885 Exercises 60

9 Repairman Problem in Manufacturing System 6191 Repairman Problem 6192 Average Analysis 6293 Single Repairman in Markov Model 65

10 Production Systems 68101 Produce-to-Order System 68102 Produce-to-Stock System 69

11 Decision Making 72

12 Analytic Hierarchy Processes 73

Chapter 1

Introduction of Operations Research

This handout can be fond at httpwwwfwasedajptoyoizumiHerersquos the brief introduction of the concept of Operations Research This

chapter is mainly consisted by quotations

11 A Simple Example

Herersquos a simple example from Mori[5]

Example 11 (Behind Monty Hallrsquos Doors) There is a famous quiz show in UScalled Letrsquos Make a Deal The MC of the show is Monty Hall

Figure 11 Monty Hall appeared in Letrsquos Make a Deal adopted fromhttpwwwcurtalliaumecomlmadhtml

5

4 CONTENTS

8 Network of Queues 5781 Open Network of queues 5782 Global Balance of Network 5783 Traffic Equation 5884 Product Form Solution 5885 Exercises 60

9 Repairman Problem in Manufacturing System 6191 Repairman Problem 6192 Average Analysis 6293 Single Repairman in Markov Model 65

10 Production Systems 68101 Produce-to-Order System 68102 Produce-to-Stock System 69

11 Decision Making 72

12 Analytic Hierarchy Processes 73

Chapter 1

Introduction of Operations Research

This handout can be fond at httpwwwfwasedajptoyoizumiHerersquos the brief introduction of the concept of Operations Research This

chapter is mainly consisted by quotations

11 A Simple Example

Herersquos a simple example from Mori[5]

Example 11 (Behind Monty Hallrsquos Doors) There is a famous quiz show in UScalled Letrsquos Make a Deal The MC of the show is Monty Hall

Figure 11 Monty Hall appeared in Letrsquos Make a Deal adopted fromhttpwwwcurtalliaumecomlmadhtml

5

Chapter 1

Introduction of Operations Research

This handout can be fond at httpwwwfwasedajptoyoizumiHerersquos the brief introduction of the concept of Operations Research This

chapter is mainly consisted by quotations

11 A Simple Example

Herersquos a simple example from Mori[5]

Example 11 (Behind Monty Hallrsquos Doors) There is a famous quiz show in UScalled Letrsquos Make a Deal The MC of the show is Monty Hall

Figure 11 Monty Hall appeared in Letrsquos Make a Deal adopted fromhttpwwwcurtalliaumecomlmadhtml

5

6 CHAPTER 1 INTRODUCTION OF OPERATIONS RESEARCH

Suppose yoursquore on a game show and yoursquore given the choice of threedoors Behind one door is a car behind the others goats You picka door say No 1 and the host who knows whatrsquos behind the doorsopens another door say No 3 which has a goat He then says to yourdquoDo you want to pick door No 2rdquo Is it to your advantage to switchyour choice

12 What is ORbull INFORMS httpwwwinformsorg The Institute for Operations Research

and the Management Sciences (INFORMS)

Since its inception more than 50 years ago OR has contributedbillions of dollars in benefits and savings to corporations govern-ment and the nonprofit sector Because the public is largely un-acquainted with the sophisticated techniques used by operationsresearchers INFORMS established a public relations departmentto educate those outside the profession1

bull Operations Reseach The Science of Better httpwwwscienceofbetterorg

In a nutshell operations research (OR) is the discipline of ap-plying advanced analytical methods to help make better deci-sions By using techniques such as mathematical modeling toanalyze complex situations operations research gives executivesthe power to make more effective decisions and build more pro-ductive systems based on

ndash More complete data

1 httpwwwinformsorg

13 DECISION MAKING 7

ndash Consideration of all available optionsndash Careful predictions of outcomes and estimates of riskndash The latest decision tools and techniques2

13 Decision MakingA uniquely powerful approach to decision making Yoursquove proba-

bly seen dozens of articles and ads about solutions that claim to en-hance your decision-making capabilities OR is unique Itrsquos bestof breed employing highly developed methods practiced by speciallytrained professionals Itrsquos powerful using advanced tools and tech-nologies to provide analytical power that no ordinary software orspreadsheet can deliver out of the box And itrsquos tailored to you be-cause an OR professional offers you the ability to define your spe-cific challenge in ways that make the most of your data and uncoveryour most beneficial options To achieve these results OR profes-sionals draw upon the latest analytical technologies including

bull Simulation Giving you the ability to try out approaches and testideas for improvement

bull Optimization Narrowing your choices to the very best whenthere are virtually innumerable feasible options and comparingthem is difficult

bull Probability and Statistics Helping you measure risk mine datato find valuable connections and insights test conclusions andmake reliable forecasts3

Decision making is hard for human beings What is important for OR is that wecan use scientific approach towards OUR decision making process

14 HistoryHerersquos the brief history how OR was born[5]

1 In 1939 AP Rowe started to use the terminology ldquooperational research4rdquo2httpwwwscienceofbetterorgwhatindexhtm3httpwwwscienceofbetterorgwhatindexhtm4Not operations research

8 CHAPTER 1 INTRODUCTION OF OPERATIONS RESEARCH

for the development of the sophisticated system of radar defense systempreventing from Nazi attack to England

2 Around 1940s the concept of OR was brought to the United States Amer-icans P M Morse and G E Kimball published the first book about OR(Morse MP Kimbal GE Methods of operations research Cambridge MAMIT Press)

3 After the world war II OR researchers are starting to use their method tomanagement science

15 Examples of OR

Example 12 (More Efficient Planning and Delivery at UPS5) The worldrsquoslargest package delivery company UPS relies on the efficient design andoperation of its hub-and-spoke air network - seven hubs and nearly 100 ad-ditional airports in the US - to move over a million domestic Next DayAir packages every night Bringing greater efficiency to such an enormoussystem was a challenge Known methods for solving large-scale networkdesign problems were inadequate for planning the UPS air network Theprimary obstacles were the complexity and immense size of the air oper-ation which involves over 17000 origin-destination flows and more than160 aircraft of nine different types Solving these problems required oper-ations research expertise in large-scale optimization integer programmingand routing

bull The OR Solution

UPS Air Group worked with an MIT specialist in transportation The jointresearch and development effort resulted in an optimization-based planningsystem for designing the UPS aircraft network To ensure overnight deliv-ery the approach simultaneously determined minimal-cost aircraft routesfleet assignments and allocation of packages to routes The project teambuilt integer programming formulations that were equivalent to conven-tional network design formulations but yielded greatly improved linear

5httpwwwscienceofbetterorg

15 EXAMPLES OF OR 9

programming-based bounds This enabled realistic instances of the orig-inal planning problem to be solved typically in less than six hours withmany instances solving in less than one hour substantial time savings

bull The Value

UPS planners now use solutions and insights generated by the system tocreate improved plans UPS management credits the system with identi-fying operational changes that have saved over $ 87 million to date andis anticipating additional savings of $189 million over the next ten yearsOther benefits include reduced planning time reduced peak and non-peakcosts reduced aircraft fleet requirements and improved planning

Example 13 (Seizing Marketplace Initiative with Merrill Lynch IntegratedChoice6) The Private Client Group at Merrill Lynch handles brokerage andlending services In late 1998 Merrill Lynch and other full-service financialservice firms were under assault Electronic trading and the commoditiza-tion of trading threatened Merrill Lynchrsquos value proposition providing ad-vice and guidance through a financial advisor Management decided to offerinvestors more choices for doing business with Merrill Lynch by developingan online investment site The firm had to balance carefully the needs of itsbrokers and its clients while planning for considerable financial risk Doingso would require operations research expertise in data mining and strategicplanning

bull The OR Solution

A cross-functional team evaluated alternative product and service struc-tures and pricing and constructed models to assess individual client be-havior The models showed that revenue at risk to Merrill Lynch rangedfrom $200 million to $1 billion The Merrill Lynch Management ScienceGroup worked with executive management and a special pricing task forceto

ndash Determine the total revenue at risk if the only clients choosing the newpricing options were those who would pay less to Merrill Lynch

ndash Determine a more realistic revenue impact based on a clientrsquos likeli-hood of adopting one of the new services

6httpwwworchampionsorgprovesuccess storiessmimlichtm

10 CHAPTER 1 INTRODUCTION OF OPERATIONS RESEARCH

Figure 12 The Hub-and-Spoke Structure of UPS adopted fromhttppeoplehofstraedugeotransengch5enappl5enhubspokeupshtml

15 EXAMPLES OF OR 11

ndash Assess the revenue impact for various pricing schedules minimum feelevels product combinations and product features

ndash Assess the impact on each financial consultant and identify those whowould be potentially most affectedAll in all they assessed more than 40 combinations of offer architec-tures and pricing Most significantly they were able to analyze newscenarios with a new set of offerings quickly enough to meet demand-ing self-imposed deadlines

Figure 13 Merrill Lynch Asset Allocation adopted from Duffy et al [2]

12 CHAPTER 1 INTRODUCTION OF OPERATIONS RESEARCH

bull The Value The resulting Integrated Choice strategy enabled Merrill Lynchto seize the marketplace initiative changed the financial services landscapeand mitigated the revenue risk At year-end 2000 client assets reached$83 billion in the new offer net new assets to the firm totaled $22 billionand incremental revenue reached $80 million David Komansky CEO ofMerrill Lynch and Chairman of the Board called Integrated Choice rdquoThemost important decision we as a firm have made since we introduced thefirst cash-management account in the 1970srdquo

Chapter 2

Basic Probability Theory

21 Why ProbabilityExample 21 Herersquos examples where we use probability

bull Lottery

bull Weathers forecast

bull Gamble

bull Baseball

bull Life insurance

bull Finance

Problem 21 Name a couple of other examples you could use probability theory

Since our intuition sometimes leads us mistake in those random phenomenawe need to handle them using extreme care in rigorous mathematical frameworkcalled probability theory (See Exercise 21)

22 Probability SpaceBe patient to learn the basic terminology in probability theory To determine theprobabilistic structure we need a probability space which is consisted by a sam-ple space a probability measure and a family of (good) set of events

13

14 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 21 (Sample Space) The set of all events is called sample space andwe write it as Ω Each element ω isinΩ is called an event

Example 22 (Lottery) Herersquos the example of Lottery

bull The sample space Ω is first prize second prize lose

bull An event ω can be first prize second prize lose and so on

Sometimes it is easy to use sets of events in sample space Ω

Example 23 (Sets in Lottery) The followings are examples in Ω of Example 22

W = win= first prize second prize sixth prize (21)L = lose (22)

Thus we can say that ldquowhat is the probability of winrdquo instead of sayingldquowhat is the probability that we have either first prize second prize or sixthprizerdquo

Definition 21 (Probability measure) The probability of A P(A) is defined foreach set of the sample space Ω if the followings are satisfyed

1 0le P(A)le 1 for all AsubΩ

2 P(Ω) = 1

3 For any sequence of mutually exclusive A1A2

P(infin⋃

i=1

Ai) =infin

sumi=1

P(Ai) (23)

In addition P is said to be the probability measure on Ω

Mathematically all function f which satisfies Definition 21 can regarded asprobability Thus we need to be careful to select which function is suitable forprobability

23 CONDITIONAL PROBABILITY AND INDEPENDENCE 15

Example 24 (Probability Measures in Lottery) Suppose we have a lottery suchas 10 first prizes 20 second prizes middot middot middot 60 sixth prizes out of total 1000 ticketsthen we have a probability measure P defined by

P(n) = P(win n-th prize) =n

100(24)

P(0) = P(lose) =79

100 (25)

It is easy to see that P satisfies Definition 21 According to the definition P wecan calculate the probability on a set of events

P(W ) = the probability of win= P(1)+P(2)+ middot middot middot+P(6)

=21100

Of course you can cheat your customer by saying you have 100 first prizesinstead of 10 first prizes Then your customer might have a different P satisfyingDefinition 21 Thus it is pretty important to select an appropriate probabilitymeasure Selecting the probability measure is a bridge between physical worldand mathematical world Donrsquot use wrong bridge

Remark 21 There is a more rigorous way to define the probability measure In-deed Definition 21 is NOT mathematically satisfactory in some cases If you arefamiliar with measure theory and advanced integral theory you may proceed toread [3]

23 Conditional Probability and IndependenceNow we introduce the most uselful and probably most difficult concepts of prob-ability theory

Definition 22 (Conditional Probability) Define the probability of B given A by

P(B | A) =P(B amp A)

P(A)=

P(BcapA)P(A)

(26)

We can use the conditional probability to calculate complex probability It isactually the only tool we can rely on Be sure that the conditional probabilityP(B|A) is different with the regular probability P(B)

16 CHAPTER 2 BASIC PROBABILITY THEORY

Example 25 (Lottery) Let W = win and F = first prize in Example 24Then we have the conditional probability that

P(F |W ) = the probability of winning 1st prize given you win the lottery

=P(F capW )

P(W )=

P(F)P(W )

=101000

2101000=

121

6= 101000

= P(F)

Remark 22 Sometimes we may regard Definition 22 as a theorem and callBayse rule But here we use this as a definition of conditional probability

Problem 22 (False positives1) Answer the followings

1 Suppose there are illegal acts in one in 10000 companies You as a ac-countant audit companies The auditing contains some uncertainty Thereis a 1 chance that a normal company is declared to have some problemFind the probability that the company declared to have a problem is actuallyillegal

2 Suppose you are tested by a disease that strikes 11000 population Thistest has 5 false positives that mean even if you are not affected by thisdisease you have 5 chance to be diagnosed to be suffered by it A medicaloperation will cure the disease but of course there is a mis-operation Giventhat your result is positive what can you say about your situation

Definition 23 (Independence) Two sets of events A and B are said to be indepen-dent if

P(AampB) = P(AcapB) = P(A)P(B) (27)

Theorem 21 (Conditional of Probability of Independent Events) Suppose A andB are independent then the conditional probability of B given A is equal to theprobability of B

1Modified from [8 p207]

24 RANDOM VARIABLES 17

Proof By Definition 22 we have

P(B | A) =P(BcapA)

P(A)=

P(B)P(A)P(A)

= P(B)

where we used A and B are independent

Example 26 (Independent two dices) Of course two dices are independent So

P(The number on the first dice is even while the one on the second is odd)= P(The number on the first dice is even)P(The number on the second dice is odd)

=12middot 1

2

Example 27 (More on two dice) Even though the two dices are independentyou can find dependent events For example

P(The first dice is bigger than second dice even while the one on the second is even) =

How about the following

P(The sum of two dice is even while the one on the second is odd ) =

See Exercise 24 for the detail

24 Random VariablesThe name random variable has a strange and stochastic history2 Although itsfragile history the invention of random variable certainly contribute a lot to theprobability theory

Definition 24 (Random Variable) The random variable X = X(ω) is a real-valued function on Ω whose value is assigned to each outcome of the experiment(event)

2J Doob quoted in Statistical Science (One of the great probabilists who established probabil-ity as a branch of mathematics) While writing my book [Stochastic Processes] I had an argumentwith Feller He asserted that everyone said ldquorandom variablerdquo and I asserted that everyone saidldquochance variablerdquo We obviously had to use the same name in our books so we decided the issueby a stochastic procedure That is we tossed for it and he won

18 CHAPTER 2 BASIC PROBABILITY THEORY

Remark 23 Note that probability and random variables is NOT same Randomvariables are function of events while the probability is a number To avoid theconfusion we usually use the capital letter to random variables

Example 28 (Lottery) A random variable X can be designed to formulate a lot-tery

bull X = 1 when we get the first prize

bull X = 2 when we get the second prize

Example 29 (Bernouilli random variable) Let X be a random variable with

X =

1 with probability p0 with probability 1minus p

(28)

for some p isin [01] The random variable X is said to be a Bernouilli randomvariable

Sometimes we use random variables to indicate the set of events For exampleinstead of saying the set that we win first prize ω isin Ω X(ω) = 1 or simplyX = 1

Definition 25 (Probability distribution) The probability distribution function F(x)is defined by

F(x) = PX le x (29)

The probability distribution function fully-determines the probability structureof a random variable X Sometimes it is convenient to consider the probabilitydensity function instead of the probability distribution

Definition 26 (probability density function) The probability density functionf (t) is defined by

f (x) =dF(x)

dx=

dPX le xdx

(210)

Sometimes we use dF(x) = dPX le x= P(X isin (xx+dx]) even when F(x)has no derivative

25 EXPECTATION VARIANCE AND STANDARD DEVIATION 19

Lemma 21 For a (good) set A

PX isin A=int

AdPX le x=

intA

f (x)dx (211)

Problem 23 Let X be an uniformly-distributed random variable on [100200]Then the distribution function is

F(x) = PX le x=xminus100

100 (212)

for x isin [100200]

bull Draw the graph of F(x)

bull Find the probability function f (x)

25 Expectation Variance and Standard DeviationLet X be a random variable Then we have some basic tools to evaluate randomvariable X First we have the most important measure the expectation or mean ofX

Definition 27 (Expectation)

E[X ] =int

infin

minusinfin

xdPX le x=int

infin

minusinfin

x f (x)dx (213)

Remark 24 For a discrete random variable we can rewrite (213) as

E[X ] = sumn

xnP[X = xn] (214)

Lemma 22 Let (Xn)n=1N be the sequence of random variables Then we canchange the order of summation and the expectation

E[X1 + middot middot middot+XN ] = E[X1]+ middot middot middot+E[XN ] (215)

Proof See Exercise 26

E[X ] gives you the expected value of X but X is fluctuated around E[X ] Sowe need to measure the strength of this stochastic fluctuation The natural choicemay be X minusE[X ] Unfortunately the expectation of X minusE[X ] is always equal tozero Thus we need the variance of X which is indeed the second moment aroundE[X ]

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

15 EXAMPLES OF OR 11

ndash Assess the revenue impact for various pricing schedules minimum feelevels product combinations and product features

ndash Assess the impact on each financial consultant and identify those whowould be potentially most affectedAll in all they assessed more than 40 combinations of offer architec-tures and pricing Most significantly they were able to analyze newscenarios with a new set of offerings quickly enough to meet demand-ing self-imposed deadlines

Figure 13 Merrill Lynch Asset Allocation adopted from Duffy et al [2]

12 CHAPTER 1 INTRODUCTION OF OPERATIONS RESEARCH

bull The Value The resulting Integrated Choice strategy enabled Merrill Lynchto seize the marketplace initiative changed the financial services landscapeand mitigated the revenue risk At year-end 2000 client assets reached$83 billion in the new offer net new assets to the firm totaled $22 billionand incremental revenue reached $80 million David Komansky CEO ofMerrill Lynch and Chairman of the Board called Integrated Choice rdquoThemost important decision we as a firm have made since we introduced thefirst cash-management account in the 1970srdquo

Chapter 2

Basic Probability Theory

21 Why ProbabilityExample 21 Herersquos examples where we use probability

bull Lottery

bull Weathers forecast

bull Gamble

bull Baseball

bull Life insurance

bull Finance

Problem 21 Name a couple of other examples you could use probability theory

Since our intuition sometimes leads us mistake in those random phenomenawe need to handle them using extreme care in rigorous mathematical frameworkcalled probability theory (See Exercise 21)

22 Probability SpaceBe patient to learn the basic terminology in probability theory To determine theprobabilistic structure we need a probability space which is consisted by a sam-ple space a probability measure and a family of (good) set of events

13

14 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 21 (Sample Space) The set of all events is called sample space andwe write it as Ω Each element ω isinΩ is called an event

Example 22 (Lottery) Herersquos the example of Lottery

bull The sample space Ω is first prize second prize lose

bull An event ω can be first prize second prize lose and so on

Sometimes it is easy to use sets of events in sample space Ω

Example 23 (Sets in Lottery) The followings are examples in Ω of Example 22

W = win= first prize second prize sixth prize (21)L = lose (22)

Thus we can say that ldquowhat is the probability of winrdquo instead of sayingldquowhat is the probability that we have either first prize second prize or sixthprizerdquo

Definition 21 (Probability measure) The probability of A P(A) is defined foreach set of the sample space Ω if the followings are satisfyed

1 0le P(A)le 1 for all AsubΩ

2 P(Ω) = 1

3 For any sequence of mutually exclusive A1A2

P(infin⋃

i=1

Ai) =infin

sumi=1

P(Ai) (23)

In addition P is said to be the probability measure on Ω

Mathematically all function f which satisfies Definition 21 can regarded asprobability Thus we need to be careful to select which function is suitable forprobability

23 CONDITIONAL PROBABILITY AND INDEPENDENCE 15

Example 24 (Probability Measures in Lottery) Suppose we have a lottery suchas 10 first prizes 20 second prizes middot middot middot 60 sixth prizes out of total 1000 ticketsthen we have a probability measure P defined by

P(n) = P(win n-th prize) =n

100(24)

P(0) = P(lose) =79

100 (25)

It is easy to see that P satisfies Definition 21 According to the definition P wecan calculate the probability on a set of events

P(W ) = the probability of win= P(1)+P(2)+ middot middot middot+P(6)

=21100

Of course you can cheat your customer by saying you have 100 first prizesinstead of 10 first prizes Then your customer might have a different P satisfyingDefinition 21 Thus it is pretty important to select an appropriate probabilitymeasure Selecting the probability measure is a bridge between physical worldand mathematical world Donrsquot use wrong bridge

Remark 21 There is a more rigorous way to define the probability measure In-deed Definition 21 is NOT mathematically satisfactory in some cases If you arefamiliar with measure theory and advanced integral theory you may proceed toread [3]

23 Conditional Probability and IndependenceNow we introduce the most uselful and probably most difficult concepts of prob-ability theory

Definition 22 (Conditional Probability) Define the probability of B given A by

P(B | A) =P(B amp A)

P(A)=

P(BcapA)P(A)

(26)

We can use the conditional probability to calculate complex probability It isactually the only tool we can rely on Be sure that the conditional probabilityP(B|A) is different with the regular probability P(B)

16 CHAPTER 2 BASIC PROBABILITY THEORY

Example 25 (Lottery) Let W = win and F = first prize in Example 24Then we have the conditional probability that

P(F |W ) = the probability of winning 1st prize given you win the lottery

=P(F capW )

P(W )=

P(F)P(W )

=101000

2101000=

121

6= 101000

= P(F)

Remark 22 Sometimes we may regard Definition 22 as a theorem and callBayse rule But here we use this as a definition of conditional probability

Problem 22 (False positives1) Answer the followings

1 Suppose there are illegal acts in one in 10000 companies You as a ac-countant audit companies The auditing contains some uncertainty Thereis a 1 chance that a normal company is declared to have some problemFind the probability that the company declared to have a problem is actuallyillegal

2 Suppose you are tested by a disease that strikes 11000 population Thistest has 5 false positives that mean even if you are not affected by thisdisease you have 5 chance to be diagnosed to be suffered by it A medicaloperation will cure the disease but of course there is a mis-operation Giventhat your result is positive what can you say about your situation

Definition 23 (Independence) Two sets of events A and B are said to be indepen-dent if

P(AampB) = P(AcapB) = P(A)P(B) (27)

Theorem 21 (Conditional of Probability of Independent Events) Suppose A andB are independent then the conditional probability of B given A is equal to theprobability of B

1Modified from [8 p207]

24 RANDOM VARIABLES 17

Proof By Definition 22 we have

P(B | A) =P(BcapA)

P(A)=

P(B)P(A)P(A)

= P(B)

where we used A and B are independent

Example 26 (Independent two dices) Of course two dices are independent So

P(The number on the first dice is even while the one on the second is odd)= P(The number on the first dice is even)P(The number on the second dice is odd)

=12middot 1

2

Example 27 (More on two dice) Even though the two dices are independentyou can find dependent events For example

P(The first dice is bigger than second dice even while the one on the second is even) =

How about the following

P(The sum of two dice is even while the one on the second is odd ) =

See Exercise 24 for the detail

24 Random VariablesThe name random variable has a strange and stochastic history2 Although itsfragile history the invention of random variable certainly contribute a lot to theprobability theory

Definition 24 (Random Variable) The random variable X = X(ω) is a real-valued function on Ω whose value is assigned to each outcome of the experiment(event)

2J Doob quoted in Statistical Science (One of the great probabilists who established probabil-ity as a branch of mathematics) While writing my book [Stochastic Processes] I had an argumentwith Feller He asserted that everyone said ldquorandom variablerdquo and I asserted that everyone saidldquochance variablerdquo We obviously had to use the same name in our books so we decided the issueby a stochastic procedure That is we tossed for it and he won

18 CHAPTER 2 BASIC PROBABILITY THEORY

Remark 23 Note that probability and random variables is NOT same Randomvariables are function of events while the probability is a number To avoid theconfusion we usually use the capital letter to random variables

Example 28 (Lottery) A random variable X can be designed to formulate a lot-tery

bull X = 1 when we get the first prize

bull X = 2 when we get the second prize

Example 29 (Bernouilli random variable) Let X be a random variable with

X =

1 with probability p0 with probability 1minus p

(28)

for some p isin [01] The random variable X is said to be a Bernouilli randomvariable

Sometimes we use random variables to indicate the set of events For exampleinstead of saying the set that we win first prize ω isin Ω X(ω) = 1 or simplyX = 1

Definition 25 (Probability distribution) The probability distribution function F(x)is defined by

F(x) = PX le x (29)

The probability distribution function fully-determines the probability structureof a random variable X Sometimes it is convenient to consider the probabilitydensity function instead of the probability distribution

Definition 26 (probability density function) The probability density functionf (t) is defined by

f (x) =dF(x)

dx=

dPX le xdx

(210)

Sometimes we use dF(x) = dPX le x= P(X isin (xx+dx]) even when F(x)has no derivative

25 EXPECTATION VARIANCE AND STANDARD DEVIATION 19

Lemma 21 For a (good) set A

PX isin A=int

AdPX le x=

intA

f (x)dx (211)

Problem 23 Let X be an uniformly-distributed random variable on [100200]Then the distribution function is

F(x) = PX le x=xminus100

100 (212)

for x isin [100200]

bull Draw the graph of F(x)

bull Find the probability function f (x)

25 Expectation Variance and Standard DeviationLet X be a random variable Then we have some basic tools to evaluate randomvariable X First we have the most important measure the expectation or mean ofX

Definition 27 (Expectation)

E[X ] =int

infin

minusinfin

xdPX le x=int

infin

minusinfin

x f (x)dx (213)

Remark 24 For a discrete random variable we can rewrite (213) as

E[X ] = sumn

xnP[X = xn] (214)

Lemma 22 Let (Xn)n=1N be the sequence of random variables Then we canchange the order of summation and the expectation

E[X1 + middot middot middot+XN ] = E[X1]+ middot middot middot+E[XN ] (215)

Proof See Exercise 26

E[X ] gives you the expected value of X but X is fluctuated around E[X ] Sowe need to measure the strength of this stochastic fluctuation The natural choicemay be X minusE[X ] Unfortunately the expectation of X minusE[X ] is always equal tozero Thus we need the variance of X which is indeed the second moment aroundE[X ]

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

12 CHAPTER 1 INTRODUCTION OF OPERATIONS RESEARCH

bull The Value The resulting Integrated Choice strategy enabled Merrill Lynchto seize the marketplace initiative changed the financial services landscapeand mitigated the revenue risk At year-end 2000 client assets reached$83 billion in the new offer net new assets to the firm totaled $22 billionand incremental revenue reached $80 million David Komansky CEO ofMerrill Lynch and Chairman of the Board called Integrated Choice rdquoThemost important decision we as a firm have made since we introduced thefirst cash-management account in the 1970srdquo

Chapter 2

Basic Probability Theory

21 Why ProbabilityExample 21 Herersquos examples where we use probability

bull Lottery

bull Weathers forecast

bull Gamble

bull Baseball

bull Life insurance

bull Finance

Problem 21 Name a couple of other examples you could use probability theory

Since our intuition sometimes leads us mistake in those random phenomenawe need to handle them using extreme care in rigorous mathematical frameworkcalled probability theory (See Exercise 21)

22 Probability SpaceBe patient to learn the basic terminology in probability theory To determine theprobabilistic structure we need a probability space which is consisted by a sam-ple space a probability measure and a family of (good) set of events

13

14 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 21 (Sample Space) The set of all events is called sample space andwe write it as Ω Each element ω isinΩ is called an event

Example 22 (Lottery) Herersquos the example of Lottery

bull The sample space Ω is first prize second prize lose

bull An event ω can be first prize second prize lose and so on

Sometimes it is easy to use sets of events in sample space Ω

Example 23 (Sets in Lottery) The followings are examples in Ω of Example 22

W = win= first prize second prize sixth prize (21)L = lose (22)

Thus we can say that ldquowhat is the probability of winrdquo instead of sayingldquowhat is the probability that we have either first prize second prize or sixthprizerdquo

Definition 21 (Probability measure) The probability of A P(A) is defined foreach set of the sample space Ω if the followings are satisfyed

1 0le P(A)le 1 for all AsubΩ

2 P(Ω) = 1

3 For any sequence of mutually exclusive A1A2

P(infin⋃

i=1

Ai) =infin

sumi=1

P(Ai) (23)

In addition P is said to be the probability measure on Ω

Mathematically all function f which satisfies Definition 21 can regarded asprobability Thus we need to be careful to select which function is suitable forprobability

23 CONDITIONAL PROBABILITY AND INDEPENDENCE 15

Example 24 (Probability Measures in Lottery) Suppose we have a lottery suchas 10 first prizes 20 second prizes middot middot middot 60 sixth prizes out of total 1000 ticketsthen we have a probability measure P defined by

P(n) = P(win n-th prize) =n

100(24)

P(0) = P(lose) =79

100 (25)

It is easy to see that P satisfies Definition 21 According to the definition P wecan calculate the probability on a set of events

P(W ) = the probability of win= P(1)+P(2)+ middot middot middot+P(6)

=21100

Of course you can cheat your customer by saying you have 100 first prizesinstead of 10 first prizes Then your customer might have a different P satisfyingDefinition 21 Thus it is pretty important to select an appropriate probabilitymeasure Selecting the probability measure is a bridge between physical worldand mathematical world Donrsquot use wrong bridge

Remark 21 There is a more rigorous way to define the probability measure In-deed Definition 21 is NOT mathematically satisfactory in some cases If you arefamiliar with measure theory and advanced integral theory you may proceed toread [3]

23 Conditional Probability and IndependenceNow we introduce the most uselful and probably most difficult concepts of prob-ability theory

Definition 22 (Conditional Probability) Define the probability of B given A by

P(B | A) =P(B amp A)

P(A)=

P(BcapA)P(A)

(26)

We can use the conditional probability to calculate complex probability It isactually the only tool we can rely on Be sure that the conditional probabilityP(B|A) is different with the regular probability P(B)

16 CHAPTER 2 BASIC PROBABILITY THEORY

Example 25 (Lottery) Let W = win and F = first prize in Example 24Then we have the conditional probability that

P(F |W ) = the probability of winning 1st prize given you win the lottery

=P(F capW )

P(W )=

P(F)P(W )

=101000

2101000=

121

6= 101000

= P(F)

Remark 22 Sometimes we may regard Definition 22 as a theorem and callBayse rule But here we use this as a definition of conditional probability

Problem 22 (False positives1) Answer the followings

1 Suppose there are illegal acts in one in 10000 companies You as a ac-countant audit companies The auditing contains some uncertainty Thereis a 1 chance that a normal company is declared to have some problemFind the probability that the company declared to have a problem is actuallyillegal

2 Suppose you are tested by a disease that strikes 11000 population Thistest has 5 false positives that mean even if you are not affected by thisdisease you have 5 chance to be diagnosed to be suffered by it A medicaloperation will cure the disease but of course there is a mis-operation Giventhat your result is positive what can you say about your situation

Definition 23 (Independence) Two sets of events A and B are said to be indepen-dent if

P(AampB) = P(AcapB) = P(A)P(B) (27)

Theorem 21 (Conditional of Probability of Independent Events) Suppose A andB are independent then the conditional probability of B given A is equal to theprobability of B

1Modified from [8 p207]

24 RANDOM VARIABLES 17

Proof By Definition 22 we have

P(B | A) =P(BcapA)

P(A)=

P(B)P(A)P(A)

= P(B)

where we used A and B are independent

Example 26 (Independent two dices) Of course two dices are independent So

P(The number on the first dice is even while the one on the second is odd)= P(The number on the first dice is even)P(The number on the second dice is odd)

=12middot 1

2

Example 27 (More on two dice) Even though the two dices are independentyou can find dependent events For example

P(The first dice is bigger than second dice even while the one on the second is even) =

How about the following

P(The sum of two dice is even while the one on the second is odd ) =

See Exercise 24 for the detail

24 Random VariablesThe name random variable has a strange and stochastic history2 Although itsfragile history the invention of random variable certainly contribute a lot to theprobability theory

Definition 24 (Random Variable) The random variable X = X(ω) is a real-valued function on Ω whose value is assigned to each outcome of the experiment(event)

2J Doob quoted in Statistical Science (One of the great probabilists who established probabil-ity as a branch of mathematics) While writing my book [Stochastic Processes] I had an argumentwith Feller He asserted that everyone said ldquorandom variablerdquo and I asserted that everyone saidldquochance variablerdquo We obviously had to use the same name in our books so we decided the issueby a stochastic procedure That is we tossed for it and he won

18 CHAPTER 2 BASIC PROBABILITY THEORY

Remark 23 Note that probability and random variables is NOT same Randomvariables are function of events while the probability is a number To avoid theconfusion we usually use the capital letter to random variables

Example 28 (Lottery) A random variable X can be designed to formulate a lot-tery

bull X = 1 when we get the first prize

bull X = 2 when we get the second prize

Example 29 (Bernouilli random variable) Let X be a random variable with

X =

1 with probability p0 with probability 1minus p

(28)

for some p isin [01] The random variable X is said to be a Bernouilli randomvariable

Sometimes we use random variables to indicate the set of events For exampleinstead of saying the set that we win first prize ω isin Ω X(ω) = 1 or simplyX = 1

Definition 25 (Probability distribution) The probability distribution function F(x)is defined by

F(x) = PX le x (29)

The probability distribution function fully-determines the probability structureof a random variable X Sometimes it is convenient to consider the probabilitydensity function instead of the probability distribution

Definition 26 (probability density function) The probability density functionf (t) is defined by

f (x) =dF(x)

dx=

dPX le xdx

(210)

Sometimes we use dF(x) = dPX le x= P(X isin (xx+dx]) even when F(x)has no derivative

25 EXPECTATION VARIANCE AND STANDARD DEVIATION 19

Lemma 21 For a (good) set A

PX isin A=int

AdPX le x=

intA

f (x)dx (211)

Problem 23 Let X be an uniformly-distributed random variable on [100200]Then the distribution function is

F(x) = PX le x=xminus100

100 (212)

for x isin [100200]

bull Draw the graph of F(x)

bull Find the probability function f (x)

25 Expectation Variance and Standard DeviationLet X be a random variable Then we have some basic tools to evaluate randomvariable X First we have the most important measure the expectation or mean ofX

Definition 27 (Expectation)

E[X ] =int

infin

minusinfin

xdPX le x=int

infin

minusinfin

x f (x)dx (213)

Remark 24 For a discrete random variable we can rewrite (213) as

E[X ] = sumn

xnP[X = xn] (214)

Lemma 22 Let (Xn)n=1N be the sequence of random variables Then we canchange the order of summation and the expectation

E[X1 + middot middot middot+XN ] = E[X1]+ middot middot middot+E[XN ] (215)

Proof See Exercise 26

E[X ] gives you the expected value of X but X is fluctuated around E[X ] Sowe need to measure the strength of this stochastic fluctuation The natural choicemay be X minusE[X ] Unfortunately the expectation of X minusE[X ] is always equal tozero Thus we need the variance of X which is indeed the second moment aroundE[X ]

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 2

Basic Probability Theory

21 Why ProbabilityExample 21 Herersquos examples where we use probability

bull Lottery

bull Weathers forecast

bull Gamble

bull Baseball

bull Life insurance

bull Finance

Problem 21 Name a couple of other examples you could use probability theory

Since our intuition sometimes leads us mistake in those random phenomenawe need to handle them using extreme care in rigorous mathematical frameworkcalled probability theory (See Exercise 21)

22 Probability SpaceBe patient to learn the basic terminology in probability theory To determine theprobabilistic structure we need a probability space which is consisted by a sam-ple space a probability measure and a family of (good) set of events

13

14 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 21 (Sample Space) The set of all events is called sample space andwe write it as Ω Each element ω isinΩ is called an event

Example 22 (Lottery) Herersquos the example of Lottery

bull The sample space Ω is first prize second prize lose

bull An event ω can be first prize second prize lose and so on

Sometimes it is easy to use sets of events in sample space Ω

Example 23 (Sets in Lottery) The followings are examples in Ω of Example 22

W = win= first prize second prize sixth prize (21)L = lose (22)

Thus we can say that ldquowhat is the probability of winrdquo instead of sayingldquowhat is the probability that we have either first prize second prize or sixthprizerdquo

Definition 21 (Probability measure) The probability of A P(A) is defined foreach set of the sample space Ω if the followings are satisfyed

1 0le P(A)le 1 for all AsubΩ

2 P(Ω) = 1

3 For any sequence of mutually exclusive A1A2

P(infin⋃

i=1

Ai) =infin

sumi=1

P(Ai) (23)

In addition P is said to be the probability measure on Ω

Mathematically all function f which satisfies Definition 21 can regarded asprobability Thus we need to be careful to select which function is suitable forprobability

23 CONDITIONAL PROBABILITY AND INDEPENDENCE 15

Example 24 (Probability Measures in Lottery) Suppose we have a lottery suchas 10 first prizes 20 second prizes middot middot middot 60 sixth prizes out of total 1000 ticketsthen we have a probability measure P defined by

P(n) = P(win n-th prize) =n

100(24)

P(0) = P(lose) =79

100 (25)

It is easy to see that P satisfies Definition 21 According to the definition P wecan calculate the probability on a set of events

P(W ) = the probability of win= P(1)+P(2)+ middot middot middot+P(6)

=21100

Of course you can cheat your customer by saying you have 100 first prizesinstead of 10 first prizes Then your customer might have a different P satisfyingDefinition 21 Thus it is pretty important to select an appropriate probabilitymeasure Selecting the probability measure is a bridge between physical worldand mathematical world Donrsquot use wrong bridge

Remark 21 There is a more rigorous way to define the probability measure In-deed Definition 21 is NOT mathematically satisfactory in some cases If you arefamiliar with measure theory and advanced integral theory you may proceed toread [3]

23 Conditional Probability and IndependenceNow we introduce the most uselful and probably most difficult concepts of prob-ability theory

Definition 22 (Conditional Probability) Define the probability of B given A by

P(B | A) =P(B amp A)

P(A)=

P(BcapA)P(A)

(26)

We can use the conditional probability to calculate complex probability It isactually the only tool we can rely on Be sure that the conditional probabilityP(B|A) is different with the regular probability P(B)

16 CHAPTER 2 BASIC PROBABILITY THEORY

Example 25 (Lottery) Let W = win and F = first prize in Example 24Then we have the conditional probability that

P(F |W ) = the probability of winning 1st prize given you win the lottery

=P(F capW )

P(W )=

P(F)P(W )

=101000

2101000=

121

6= 101000

= P(F)

Remark 22 Sometimes we may regard Definition 22 as a theorem and callBayse rule But here we use this as a definition of conditional probability

Problem 22 (False positives1) Answer the followings

1 Suppose there are illegal acts in one in 10000 companies You as a ac-countant audit companies The auditing contains some uncertainty Thereis a 1 chance that a normal company is declared to have some problemFind the probability that the company declared to have a problem is actuallyillegal

2 Suppose you are tested by a disease that strikes 11000 population Thistest has 5 false positives that mean even if you are not affected by thisdisease you have 5 chance to be diagnosed to be suffered by it A medicaloperation will cure the disease but of course there is a mis-operation Giventhat your result is positive what can you say about your situation

Definition 23 (Independence) Two sets of events A and B are said to be indepen-dent if

P(AampB) = P(AcapB) = P(A)P(B) (27)

Theorem 21 (Conditional of Probability of Independent Events) Suppose A andB are independent then the conditional probability of B given A is equal to theprobability of B

1Modified from [8 p207]

24 RANDOM VARIABLES 17

Proof By Definition 22 we have

P(B | A) =P(BcapA)

P(A)=

P(B)P(A)P(A)

= P(B)

where we used A and B are independent

Example 26 (Independent two dices) Of course two dices are independent So

P(The number on the first dice is even while the one on the second is odd)= P(The number on the first dice is even)P(The number on the second dice is odd)

=12middot 1

2

Example 27 (More on two dice) Even though the two dices are independentyou can find dependent events For example

P(The first dice is bigger than second dice even while the one on the second is even) =

How about the following

P(The sum of two dice is even while the one on the second is odd ) =

See Exercise 24 for the detail

24 Random VariablesThe name random variable has a strange and stochastic history2 Although itsfragile history the invention of random variable certainly contribute a lot to theprobability theory

Definition 24 (Random Variable) The random variable X = X(ω) is a real-valued function on Ω whose value is assigned to each outcome of the experiment(event)

2J Doob quoted in Statistical Science (One of the great probabilists who established probabil-ity as a branch of mathematics) While writing my book [Stochastic Processes] I had an argumentwith Feller He asserted that everyone said ldquorandom variablerdquo and I asserted that everyone saidldquochance variablerdquo We obviously had to use the same name in our books so we decided the issueby a stochastic procedure That is we tossed for it and he won

18 CHAPTER 2 BASIC PROBABILITY THEORY

Remark 23 Note that probability and random variables is NOT same Randomvariables are function of events while the probability is a number To avoid theconfusion we usually use the capital letter to random variables

Example 28 (Lottery) A random variable X can be designed to formulate a lot-tery

bull X = 1 when we get the first prize

bull X = 2 when we get the second prize

Example 29 (Bernouilli random variable) Let X be a random variable with

X =

1 with probability p0 with probability 1minus p

(28)

for some p isin [01] The random variable X is said to be a Bernouilli randomvariable

Sometimes we use random variables to indicate the set of events For exampleinstead of saying the set that we win first prize ω isin Ω X(ω) = 1 or simplyX = 1

Definition 25 (Probability distribution) The probability distribution function F(x)is defined by

F(x) = PX le x (29)

The probability distribution function fully-determines the probability structureof a random variable X Sometimes it is convenient to consider the probabilitydensity function instead of the probability distribution

Definition 26 (probability density function) The probability density functionf (t) is defined by

f (x) =dF(x)

dx=

dPX le xdx

(210)

Sometimes we use dF(x) = dPX le x= P(X isin (xx+dx]) even when F(x)has no derivative

25 EXPECTATION VARIANCE AND STANDARD DEVIATION 19

Lemma 21 For a (good) set A

PX isin A=int

AdPX le x=

intA

f (x)dx (211)

Problem 23 Let X be an uniformly-distributed random variable on [100200]Then the distribution function is

F(x) = PX le x=xminus100

100 (212)

for x isin [100200]

bull Draw the graph of F(x)

bull Find the probability function f (x)

25 Expectation Variance and Standard DeviationLet X be a random variable Then we have some basic tools to evaluate randomvariable X First we have the most important measure the expectation or mean ofX

Definition 27 (Expectation)

E[X ] =int

infin

minusinfin

xdPX le x=int

infin

minusinfin

x f (x)dx (213)

Remark 24 For a discrete random variable we can rewrite (213) as

E[X ] = sumn

xnP[X = xn] (214)

Lemma 22 Let (Xn)n=1N be the sequence of random variables Then we canchange the order of summation and the expectation

E[X1 + middot middot middot+XN ] = E[X1]+ middot middot middot+E[XN ] (215)

Proof See Exercise 26

E[X ] gives you the expected value of X but X is fluctuated around E[X ] Sowe need to measure the strength of this stochastic fluctuation The natural choicemay be X minusE[X ] Unfortunately the expectation of X minusE[X ] is always equal tozero Thus we need the variance of X which is indeed the second moment aroundE[X ]

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

14 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 21 (Sample Space) The set of all events is called sample space andwe write it as Ω Each element ω isinΩ is called an event

Example 22 (Lottery) Herersquos the example of Lottery

bull The sample space Ω is first prize second prize lose

bull An event ω can be first prize second prize lose and so on

Sometimes it is easy to use sets of events in sample space Ω

Example 23 (Sets in Lottery) The followings are examples in Ω of Example 22

W = win= first prize second prize sixth prize (21)L = lose (22)

Thus we can say that ldquowhat is the probability of winrdquo instead of sayingldquowhat is the probability that we have either first prize second prize or sixthprizerdquo

Definition 21 (Probability measure) The probability of A P(A) is defined foreach set of the sample space Ω if the followings are satisfyed

1 0le P(A)le 1 for all AsubΩ

2 P(Ω) = 1

3 For any sequence of mutually exclusive A1A2

P(infin⋃

i=1

Ai) =infin

sumi=1

P(Ai) (23)

In addition P is said to be the probability measure on Ω

Mathematically all function f which satisfies Definition 21 can regarded asprobability Thus we need to be careful to select which function is suitable forprobability

23 CONDITIONAL PROBABILITY AND INDEPENDENCE 15

Example 24 (Probability Measures in Lottery) Suppose we have a lottery suchas 10 first prizes 20 second prizes middot middot middot 60 sixth prizes out of total 1000 ticketsthen we have a probability measure P defined by

P(n) = P(win n-th prize) =n

100(24)

P(0) = P(lose) =79

100 (25)

It is easy to see that P satisfies Definition 21 According to the definition P wecan calculate the probability on a set of events

P(W ) = the probability of win= P(1)+P(2)+ middot middot middot+P(6)

=21100

Of course you can cheat your customer by saying you have 100 first prizesinstead of 10 first prizes Then your customer might have a different P satisfyingDefinition 21 Thus it is pretty important to select an appropriate probabilitymeasure Selecting the probability measure is a bridge between physical worldand mathematical world Donrsquot use wrong bridge

Remark 21 There is a more rigorous way to define the probability measure In-deed Definition 21 is NOT mathematically satisfactory in some cases If you arefamiliar with measure theory and advanced integral theory you may proceed toread [3]

23 Conditional Probability and IndependenceNow we introduce the most uselful and probably most difficult concepts of prob-ability theory

Definition 22 (Conditional Probability) Define the probability of B given A by

P(B | A) =P(B amp A)

P(A)=

P(BcapA)P(A)

(26)

We can use the conditional probability to calculate complex probability It isactually the only tool we can rely on Be sure that the conditional probabilityP(B|A) is different with the regular probability P(B)

16 CHAPTER 2 BASIC PROBABILITY THEORY

Example 25 (Lottery) Let W = win and F = first prize in Example 24Then we have the conditional probability that

P(F |W ) = the probability of winning 1st prize given you win the lottery

=P(F capW )

P(W )=

P(F)P(W )

=101000

2101000=

121

6= 101000

= P(F)

Remark 22 Sometimes we may regard Definition 22 as a theorem and callBayse rule But here we use this as a definition of conditional probability

Problem 22 (False positives1) Answer the followings

1 Suppose there are illegal acts in one in 10000 companies You as a ac-countant audit companies The auditing contains some uncertainty Thereis a 1 chance that a normal company is declared to have some problemFind the probability that the company declared to have a problem is actuallyillegal

2 Suppose you are tested by a disease that strikes 11000 population Thistest has 5 false positives that mean even if you are not affected by thisdisease you have 5 chance to be diagnosed to be suffered by it A medicaloperation will cure the disease but of course there is a mis-operation Giventhat your result is positive what can you say about your situation

Definition 23 (Independence) Two sets of events A and B are said to be indepen-dent if

P(AampB) = P(AcapB) = P(A)P(B) (27)

Theorem 21 (Conditional of Probability of Independent Events) Suppose A andB are independent then the conditional probability of B given A is equal to theprobability of B

1Modified from [8 p207]

24 RANDOM VARIABLES 17

Proof By Definition 22 we have

P(B | A) =P(BcapA)

P(A)=

P(B)P(A)P(A)

= P(B)

where we used A and B are independent

Example 26 (Independent two dices) Of course two dices are independent So

P(The number on the first dice is even while the one on the second is odd)= P(The number on the first dice is even)P(The number on the second dice is odd)

=12middot 1

2

Example 27 (More on two dice) Even though the two dices are independentyou can find dependent events For example

P(The first dice is bigger than second dice even while the one on the second is even) =

How about the following

P(The sum of two dice is even while the one on the second is odd ) =

See Exercise 24 for the detail

24 Random VariablesThe name random variable has a strange and stochastic history2 Although itsfragile history the invention of random variable certainly contribute a lot to theprobability theory

Definition 24 (Random Variable) The random variable X = X(ω) is a real-valued function on Ω whose value is assigned to each outcome of the experiment(event)

2J Doob quoted in Statistical Science (One of the great probabilists who established probabil-ity as a branch of mathematics) While writing my book [Stochastic Processes] I had an argumentwith Feller He asserted that everyone said ldquorandom variablerdquo and I asserted that everyone saidldquochance variablerdquo We obviously had to use the same name in our books so we decided the issueby a stochastic procedure That is we tossed for it and he won

18 CHAPTER 2 BASIC PROBABILITY THEORY

Remark 23 Note that probability and random variables is NOT same Randomvariables are function of events while the probability is a number To avoid theconfusion we usually use the capital letter to random variables

Example 28 (Lottery) A random variable X can be designed to formulate a lot-tery

bull X = 1 when we get the first prize

bull X = 2 when we get the second prize

Example 29 (Bernouilli random variable) Let X be a random variable with

X =

1 with probability p0 with probability 1minus p

(28)

for some p isin [01] The random variable X is said to be a Bernouilli randomvariable

Sometimes we use random variables to indicate the set of events For exampleinstead of saying the set that we win first prize ω isin Ω X(ω) = 1 or simplyX = 1

Definition 25 (Probability distribution) The probability distribution function F(x)is defined by

F(x) = PX le x (29)

The probability distribution function fully-determines the probability structureof a random variable X Sometimes it is convenient to consider the probabilitydensity function instead of the probability distribution

Definition 26 (probability density function) The probability density functionf (t) is defined by

f (x) =dF(x)

dx=

dPX le xdx

(210)

Sometimes we use dF(x) = dPX le x= P(X isin (xx+dx]) even when F(x)has no derivative

25 EXPECTATION VARIANCE AND STANDARD DEVIATION 19

Lemma 21 For a (good) set A

PX isin A=int

AdPX le x=

intA

f (x)dx (211)

Problem 23 Let X be an uniformly-distributed random variable on [100200]Then the distribution function is

F(x) = PX le x=xminus100

100 (212)

for x isin [100200]

bull Draw the graph of F(x)

bull Find the probability function f (x)

25 Expectation Variance and Standard DeviationLet X be a random variable Then we have some basic tools to evaluate randomvariable X First we have the most important measure the expectation or mean ofX

Definition 27 (Expectation)

E[X ] =int

infin

minusinfin

xdPX le x=int

infin

minusinfin

x f (x)dx (213)

Remark 24 For a discrete random variable we can rewrite (213) as

E[X ] = sumn

xnP[X = xn] (214)

Lemma 22 Let (Xn)n=1N be the sequence of random variables Then we canchange the order of summation and the expectation

E[X1 + middot middot middot+XN ] = E[X1]+ middot middot middot+E[XN ] (215)

Proof See Exercise 26

E[X ] gives you the expected value of X but X is fluctuated around E[X ] Sowe need to measure the strength of this stochastic fluctuation The natural choicemay be X minusE[X ] Unfortunately the expectation of X minusE[X ] is always equal tozero Thus we need the variance of X which is indeed the second moment aroundE[X ]

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

23 CONDITIONAL PROBABILITY AND INDEPENDENCE 15

Example 24 (Probability Measures in Lottery) Suppose we have a lottery suchas 10 first prizes 20 second prizes middot middot middot 60 sixth prizes out of total 1000 ticketsthen we have a probability measure P defined by

P(n) = P(win n-th prize) =n

100(24)

P(0) = P(lose) =79

100 (25)

It is easy to see that P satisfies Definition 21 According to the definition P wecan calculate the probability on a set of events

P(W ) = the probability of win= P(1)+P(2)+ middot middot middot+P(6)

=21100

Of course you can cheat your customer by saying you have 100 first prizesinstead of 10 first prizes Then your customer might have a different P satisfyingDefinition 21 Thus it is pretty important to select an appropriate probabilitymeasure Selecting the probability measure is a bridge between physical worldand mathematical world Donrsquot use wrong bridge

Remark 21 There is a more rigorous way to define the probability measure In-deed Definition 21 is NOT mathematically satisfactory in some cases If you arefamiliar with measure theory and advanced integral theory you may proceed toread [3]

23 Conditional Probability and IndependenceNow we introduce the most uselful and probably most difficult concepts of prob-ability theory

Definition 22 (Conditional Probability) Define the probability of B given A by

P(B | A) =P(B amp A)

P(A)=

P(BcapA)P(A)

(26)

We can use the conditional probability to calculate complex probability It isactually the only tool we can rely on Be sure that the conditional probabilityP(B|A) is different with the regular probability P(B)

16 CHAPTER 2 BASIC PROBABILITY THEORY

Example 25 (Lottery) Let W = win and F = first prize in Example 24Then we have the conditional probability that

P(F |W ) = the probability of winning 1st prize given you win the lottery

=P(F capW )

P(W )=

P(F)P(W )

=101000

2101000=

121

6= 101000

= P(F)

Remark 22 Sometimes we may regard Definition 22 as a theorem and callBayse rule But here we use this as a definition of conditional probability

Problem 22 (False positives1) Answer the followings

1 Suppose there are illegal acts in one in 10000 companies You as a ac-countant audit companies The auditing contains some uncertainty Thereis a 1 chance that a normal company is declared to have some problemFind the probability that the company declared to have a problem is actuallyillegal

2 Suppose you are tested by a disease that strikes 11000 population Thistest has 5 false positives that mean even if you are not affected by thisdisease you have 5 chance to be diagnosed to be suffered by it A medicaloperation will cure the disease but of course there is a mis-operation Giventhat your result is positive what can you say about your situation

Definition 23 (Independence) Two sets of events A and B are said to be indepen-dent if

P(AampB) = P(AcapB) = P(A)P(B) (27)

Theorem 21 (Conditional of Probability of Independent Events) Suppose A andB are independent then the conditional probability of B given A is equal to theprobability of B

1Modified from [8 p207]

24 RANDOM VARIABLES 17

Proof By Definition 22 we have

P(B | A) =P(BcapA)

P(A)=

P(B)P(A)P(A)

= P(B)

where we used A and B are independent

Example 26 (Independent two dices) Of course two dices are independent So

P(The number on the first dice is even while the one on the second is odd)= P(The number on the first dice is even)P(The number on the second dice is odd)

=12middot 1

2

Example 27 (More on two dice) Even though the two dices are independentyou can find dependent events For example

P(The first dice is bigger than second dice even while the one on the second is even) =

How about the following

P(The sum of two dice is even while the one on the second is odd ) =

See Exercise 24 for the detail

24 Random VariablesThe name random variable has a strange and stochastic history2 Although itsfragile history the invention of random variable certainly contribute a lot to theprobability theory

Definition 24 (Random Variable) The random variable X = X(ω) is a real-valued function on Ω whose value is assigned to each outcome of the experiment(event)

2J Doob quoted in Statistical Science (One of the great probabilists who established probabil-ity as a branch of mathematics) While writing my book [Stochastic Processes] I had an argumentwith Feller He asserted that everyone said ldquorandom variablerdquo and I asserted that everyone saidldquochance variablerdquo We obviously had to use the same name in our books so we decided the issueby a stochastic procedure That is we tossed for it and he won

18 CHAPTER 2 BASIC PROBABILITY THEORY

Remark 23 Note that probability and random variables is NOT same Randomvariables are function of events while the probability is a number To avoid theconfusion we usually use the capital letter to random variables

Example 28 (Lottery) A random variable X can be designed to formulate a lot-tery

bull X = 1 when we get the first prize

bull X = 2 when we get the second prize

Example 29 (Bernouilli random variable) Let X be a random variable with

X =

1 with probability p0 with probability 1minus p

(28)

for some p isin [01] The random variable X is said to be a Bernouilli randomvariable

Sometimes we use random variables to indicate the set of events For exampleinstead of saying the set that we win first prize ω isin Ω X(ω) = 1 or simplyX = 1

Definition 25 (Probability distribution) The probability distribution function F(x)is defined by

F(x) = PX le x (29)

The probability distribution function fully-determines the probability structureof a random variable X Sometimes it is convenient to consider the probabilitydensity function instead of the probability distribution

Definition 26 (probability density function) The probability density functionf (t) is defined by

f (x) =dF(x)

dx=

dPX le xdx

(210)

Sometimes we use dF(x) = dPX le x= P(X isin (xx+dx]) even when F(x)has no derivative

25 EXPECTATION VARIANCE AND STANDARD DEVIATION 19

Lemma 21 For a (good) set A

PX isin A=int

AdPX le x=

intA

f (x)dx (211)

Problem 23 Let X be an uniformly-distributed random variable on [100200]Then the distribution function is

F(x) = PX le x=xminus100

100 (212)

for x isin [100200]

bull Draw the graph of F(x)

bull Find the probability function f (x)

25 Expectation Variance and Standard DeviationLet X be a random variable Then we have some basic tools to evaluate randomvariable X First we have the most important measure the expectation or mean ofX

Definition 27 (Expectation)

E[X ] =int

infin

minusinfin

xdPX le x=int

infin

minusinfin

x f (x)dx (213)

Remark 24 For a discrete random variable we can rewrite (213) as

E[X ] = sumn

xnP[X = xn] (214)

Lemma 22 Let (Xn)n=1N be the sequence of random variables Then we canchange the order of summation and the expectation

E[X1 + middot middot middot+XN ] = E[X1]+ middot middot middot+E[XN ] (215)

Proof See Exercise 26

E[X ] gives you the expected value of X but X is fluctuated around E[X ] Sowe need to measure the strength of this stochastic fluctuation The natural choicemay be X minusE[X ] Unfortunately the expectation of X minusE[X ] is always equal tozero Thus we need the variance of X which is indeed the second moment aroundE[X ]

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

16 CHAPTER 2 BASIC PROBABILITY THEORY

Example 25 (Lottery) Let W = win and F = first prize in Example 24Then we have the conditional probability that

P(F |W ) = the probability of winning 1st prize given you win the lottery

=P(F capW )

P(W )=

P(F)P(W )

=101000

2101000=

121

6= 101000

= P(F)

Remark 22 Sometimes we may regard Definition 22 as a theorem and callBayse rule But here we use this as a definition of conditional probability

Problem 22 (False positives1) Answer the followings

1 Suppose there are illegal acts in one in 10000 companies You as a ac-countant audit companies The auditing contains some uncertainty Thereis a 1 chance that a normal company is declared to have some problemFind the probability that the company declared to have a problem is actuallyillegal

2 Suppose you are tested by a disease that strikes 11000 population Thistest has 5 false positives that mean even if you are not affected by thisdisease you have 5 chance to be diagnosed to be suffered by it A medicaloperation will cure the disease but of course there is a mis-operation Giventhat your result is positive what can you say about your situation

Definition 23 (Independence) Two sets of events A and B are said to be indepen-dent if

P(AampB) = P(AcapB) = P(A)P(B) (27)

Theorem 21 (Conditional of Probability of Independent Events) Suppose A andB are independent then the conditional probability of B given A is equal to theprobability of B

1Modified from [8 p207]

24 RANDOM VARIABLES 17

Proof By Definition 22 we have

P(B | A) =P(BcapA)

P(A)=

P(B)P(A)P(A)

= P(B)

where we used A and B are independent

Example 26 (Independent two dices) Of course two dices are independent So

P(The number on the first dice is even while the one on the second is odd)= P(The number on the first dice is even)P(The number on the second dice is odd)

=12middot 1

2

Example 27 (More on two dice) Even though the two dices are independentyou can find dependent events For example

P(The first dice is bigger than second dice even while the one on the second is even) =

How about the following

P(The sum of two dice is even while the one on the second is odd ) =

See Exercise 24 for the detail

24 Random VariablesThe name random variable has a strange and stochastic history2 Although itsfragile history the invention of random variable certainly contribute a lot to theprobability theory

Definition 24 (Random Variable) The random variable X = X(ω) is a real-valued function on Ω whose value is assigned to each outcome of the experiment(event)

2J Doob quoted in Statistical Science (One of the great probabilists who established probabil-ity as a branch of mathematics) While writing my book [Stochastic Processes] I had an argumentwith Feller He asserted that everyone said ldquorandom variablerdquo and I asserted that everyone saidldquochance variablerdquo We obviously had to use the same name in our books so we decided the issueby a stochastic procedure That is we tossed for it and he won

18 CHAPTER 2 BASIC PROBABILITY THEORY

Remark 23 Note that probability and random variables is NOT same Randomvariables are function of events while the probability is a number To avoid theconfusion we usually use the capital letter to random variables

Example 28 (Lottery) A random variable X can be designed to formulate a lot-tery

bull X = 1 when we get the first prize

bull X = 2 when we get the second prize

Example 29 (Bernouilli random variable) Let X be a random variable with

X =

1 with probability p0 with probability 1minus p

(28)

for some p isin [01] The random variable X is said to be a Bernouilli randomvariable

Sometimes we use random variables to indicate the set of events For exampleinstead of saying the set that we win first prize ω isin Ω X(ω) = 1 or simplyX = 1

Definition 25 (Probability distribution) The probability distribution function F(x)is defined by

F(x) = PX le x (29)

The probability distribution function fully-determines the probability structureof a random variable X Sometimes it is convenient to consider the probabilitydensity function instead of the probability distribution

Definition 26 (probability density function) The probability density functionf (t) is defined by

f (x) =dF(x)

dx=

dPX le xdx

(210)

Sometimes we use dF(x) = dPX le x= P(X isin (xx+dx]) even when F(x)has no derivative

25 EXPECTATION VARIANCE AND STANDARD DEVIATION 19

Lemma 21 For a (good) set A

PX isin A=int

AdPX le x=

intA

f (x)dx (211)

Problem 23 Let X be an uniformly-distributed random variable on [100200]Then the distribution function is

F(x) = PX le x=xminus100

100 (212)

for x isin [100200]

bull Draw the graph of F(x)

bull Find the probability function f (x)

25 Expectation Variance and Standard DeviationLet X be a random variable Then we have some basic tools to evaluate randomvariable X First we have the most important measure the expectation or mean ofX

Definition 27 (Expectation)

E[X ] =int

infin

minusinfin

xdPX le x=int

infin

minusinfin

x f (x)dx (213)

Remark 24 For a discrete random variable we can rewrite (213) as

E[X ] = sumn

xnP[X = xn] (214)

Lemma 22 Let (Xn)n=1N be the sequence of random variables Then we canchange the order of summation and the expectation

E[X1 + middot middot middot+XN ] = E[X1]+ middot middot middot+E[XN ] (215)

Proof See Exercise 26

E[X ] gives you the expected value of X but X is fluctuated around E[X ] Sowe need to measure the strength of this stochastic fluctuation The natural choicemay be X minusE[X ] Unfortunately the expectation of X minusE[X ] is always equal tozero Thus we need the variance of X which is indeed the second moment aroundE[X ]

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

24 RANDOM VARIABLES 17

Proof By Definition 22 we have

P(B | A) =P(BcapA)

P(A)=

P(B)P(A)P(A)

= P(B)

where we used A and B are independent

Example 26 (Independent two dices) Of course two dices are independent So

P(The number on the first dice is even while the one on the second is odd)= P(The number on the first dice is even)P(The number on the second dice is odd)

=12middot 1

2

Example 27 (More on two dice) Even though the two dices are independentyou can find dependent events For example

P(The first dice is bigger than second dice even while the one on the second is even) =

How about the following

P(The sum of two dice is even while the one on the second is odd ) =

See Exercise 24 for the detail

24 Random VariablesThe name random variable has a strange and stochastic history2 Although itsfragile history the invention of random variable certainly contribute a lot to theprobability theory

Definition 24 (Random Variable) The random variable X = X(ω) is a real-valued function on Ω whose value is assigned to each outcome of the experiment(event)

2J Doob quoted in Statistical Science (One of the great probabilists who established probabil-ity as a branch of mathematics) While writing my book [Stochastic Processes] I had an argumentwith Feller He asserted that everyone said ldquorandom variablerdquo and I asserted that everyone saidldquochance variablerdquo We obviously had to use the same name in our books so we decided the issueby a stochastic procedure That is we tossed for it and he won

18 CHAPTER 2 BASIC PROBABILITY THEORY

Remark 23 Note that probability and random variables is NOT same Randomvariables are function of events while the probability is a number To avoid theconfusion we usually use the capital letter to random variables

Example 28 (Lottery) A random variable X can be designed to formulate a lot-tery

bull X = 1 when we get the first prize

bull X = 2 when we get the second prize

Example 29 (Bernouilli random variable) Let X be a random variable with

X =

1 with probability p0 with probability 1minus p

(28)

for some p isin [01] The random variable X is said to be a Bernouilli randomvariable

Sometimes we use random variables to indicate the set of events For exampleinstead of saying the set that we win first prize ω isin Ω X(ω) = 1 or simplyX = 1

Definition 25 (Probability distribution) The probability distribution function F(x)is defined by

F(x) = PX le x (29)

The probability distribution function fully-determines the probability structureof a random variable X Sometimes it is convenient to consider the probabilitydensity function instead of the probability distribution

Definition 26 (probability density function) The probability density functionf (t) is defined by

f (x) =dF(x)

dx=

dPX le xdx

(210)

Sometimes we use dF(x) = dPX le x= P(X isin (xx+dx]) even when F(x)has no derivative

25 EXPECTATION VARIANCE AND STANDARD DEVIATION 19

Lemma 21 For a (good) set A

PX isin A=int

AdPX le x=

intA

f (x)dx (211)

Problem 23 Let X be an uniformly-distributed random variable on [100200]Then the distribution function is

F(x) = PX le x=xminus100

100 (212)

for x isin [100200]

bull Draw the graph of F(x)

bull Find the probability function f (x)

25 Expectation Variance and Standard DeviationLet X be a random variable Then we have some basic tools to evaluate randomvariable X First we have the most important measure the expectation or mean ofX

Definition 27 (Expectation)

E[X ] =int

infin

minusinfin

xdPX le x=int

infin

minusinfin

x f (x)dx (213)

Remark 24 For a discrete random variable we can rewrite (213) as

E[X ] = sumn

xnP[X = xn] (214)

Lemma 22 Let (Xn)n=1N be the sequence of random variables Then we canchange the order of summation and the expectation

E[X1 + middot middot middot+XN ] = E[X1]+ middot middot middot+E[XN ] (215)

Proof See Exercise 26

E[X ] gives you the expected value of X but X is fluctuated around E[X ] Sowe need to measure the strength of this stochastic fluctuation The natural choicemay be X minusE[X ] Unfortunately the expectation of X minusE[X ] is always equal tozero Thus we need the variance of X which is indeed the second moment aroundE[X ]

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

18 CHAPTER 2 BASIC PROBABILITY THEORY

Remark 23 Note that probability and random variables is NOT same Randomvariables are function of events while the probability is a number To avoid theconfusion we usually use the capital letter to random variables

Example 28 (Lottery) A random variable X can be designed to formulate a lot-tery

bull X = 1 when we get the first prize

bull X = 2 when we get the second prize

Example 29 (Bernouilli random variable) Let X be a random variable with

X =

1 with probability p0 with probability 1minus p

(28)

for some p isin [01] The random variable X is said to be a Bernouilli randomvariable

Sometimes we use random variables to indicate the set of events For exampleinstead of saying the set that we win first prize ω isin Ω X(ω) = 1 or simplyX = 1

Definition 25 (Probability distribution) The probability distribution function F(x)is defined by

F(x) = PX le x (29)

The probability distribution function fully-determines the probability structureof a random variable X Sometimes it is convenient to consider the probabilitydensity function instead of the probability distribution

Definition 26 (probability density function) The probability density functionf (t) is defined by

f (x) =dF(x)

dx=

dPX le xdx

(210)

Sometimes we use dF(x) = dPX le x= P(X isin (xx+dx]) even when F(x)has no derivative

25 EXPECTATION VARIANCE AND STANDARD DEVIATION 19

Lemma 21 For a (good) set A

PX isin A=int

AdPX le x=

intA

f (x)dx (211)

Problem 23 Let X be an uniformly-distributed random variable on [100200]Then the distribution function is

F(x) = PX le x=xminus100

100 (212)

for x isin [100200]

bull Draw the graph of F(x)

bull Find the probability function f (x)

25 Expectation Variance and Standard DeviationLet X be a random variable Then we have some basic tools to evaluate randomvariable X First we have the most important measure the expectation or mean ofX

Definition 27 (Expectation)

E[X ] =int

infin

minusinfin

xdPX le x=int

infin

minusinfin

x f (x)dx (213)

Remark 24 For a discrete random variable we can rewrite (213) as

E[X ] = sumn

xnP[X = xn] (214)

Lemma 22 Let (Xn)n=1N be the sequence of random variables Then we canchange the order of summation and the expectation

E[X1 + middot middot middot+XN ] = E[X1]+ middot middot middot+E[XN ] (215)

Proof See Exercise 26

E[X ] gives you the expected value of X but X is fluctuated around E[X ] Sowe need to measure the strength of this stochastic fluctuation The natural choicemay be X minusE[X ] Unfortunately the expectation of X minusE[X ] is always equal tozero Thus we need the variance of X which is indeed the second moment aroundE[X ]

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

25 EXPECTATION VARIANCE AND STANDARD DEVIATION 19

Lemma 21 For a (good) set A

PX isin A=int

AdPX le x=

intA

f (x)dx (211)

Problem 23 Let X be an uniformly-distributed random variable on [100200]Then the distribution function is

F(x) = PX le x=xminus100

100 (212)

for x isin [100200]

bull Draw the graph of F(x)

bull Find the probability function f (x)

25 Expectation Variance and Standard DeviationLet X be a random variable Then we have some basic tools to evaluate randomvariable X First we have the most important measure the expectation or mean ofX

Definition 27 (Expectation)

E[X ] =int

infin

minusinfin

xdPX le x=int

infin

minusinfin

x f (x)dx (213)

Remark 24 For a discrete random variable we can rewrite (213) as

E[X ] = sumn

xnP[X = xn] (214)

Lemma 22 Let (Xn)n=1N be the sequence of random variables Then we canchange the order of summation and the expectation

E[X1 + middot middot middot+XN ] = E[X1]+ middot middot middot+E[XN ] (215)

Proof See Exercise 26

E[X ] gives you the expected value of X but X is fluctuated around E[X ] Sowe need to measure the strength of this stochastic fluctuation The natural choicemay be X minusE[X ] Unfortunately the expectation of X minusE[X ] is always equal tozero Thus we need the variance of X which is indeed the second moment aroundE[X ]

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

20 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 28 (Variance)

Var[X ] = E[(X minusE[X ])2] (216)

Lemma 23 We have an alternative to calculate Var[X ]

Var[X ] = E[X2]minusE[X ]2 (217)

Proof See Exercise 26

Unfortunately the variance Var[X ] has the dimension of X2 So in some casesit is inappropriate to use the variance Thus we need the standard deviation σ [X ]which has the order of X

Definition 29 (Standard deviation)

σ [X ] = (Var[X ])12 (218)

Example 210 (Bernouilli random variable) Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1minus p Then we have

E[X ] = 1p+0(1minus p) = p (219)

Var[X ] = E[X2]minusE[X ]2 = E[X ]minusE[X ]2 = p(1minus p) (220)

where we used the fact X2 = X for Bernouille random variables

In many cases we need to deal with two or more random variables Whenthese random variables are independent we are very lucky and we can get manyuseful result Otherwise

Definition 22 We say that two random variables X and Y are independent whenthe sets X le x and Y le y are independent for all x and y In other wordswhen X and Y are independent

P(X le xY le y) = P(X le x)P(Y le y) (221)

Lemma 24 For any pair of independent random variables X and Y we have

bull E[XY ] = E[X ]E[Y ]

bull Var[X +Y ] = Var[X ]+Var[Y ]

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

26 HOW TO MAKE A RANDOM VARIABLE 21

Proof Extending the definition of the expectation we have a double integral

E[XY ] =int

xydP(X le xY le y)

Since X and Y are independent we have P(X le xY le y) = P(X le x)P(Y le y)Thus

E[XY ] =int

xydP(X le x)dP(Y le y)

=int

xdP(X le x)int

ydP(X le y)

= E[X ]E[Y ]

Using the first part it is easy to check the second part (see Exercise 29)

Example 211 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (222)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable The mean and varianceof X can be obtained easily by using Lemma 24 as

E[X ] = np (223)Var[X ] = np(1minus p) (224)

26 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x) The following theorem will help us

Theorem 22 Let U be a random variable which has a uniform distribution on[01] ie

P[U le u] = u (225)

Then the random variable X = Fminus1(U) has the distribution F(x)

Proof

P[X le x] = P[Fminus1(U)le x] = P[U le F(x)] = F(x) (226)

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

22 CHAPTER 2 BASIC PROBABILITY THEORY

27 News-vendor Problem ldquoHow many should youbuyrdquo

Suppose you are assigned to sell newspapers Every morning you buy in x newspa-pers at the price a You can sell the newspaper at the price a+b to your customersYou should decide the number x of newspapers to buy in If the number of thosewho buy newspaper is less than x you will be left with piles of unsold newspa-pers When there are more buyers than x you lost the opportunity of selling morenewspapers Thus there seems to be an optimal x to maximize your profit

Let X be the demand of newspapers which is not known when you buy innewspapers Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers If the demand X is larger than x +∆x theadditional newspapers will pay off and you get b∆x but if X is smaller than x+∆xyou will lose a∆x Thus the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ge x+∆xminusa∆xPX le x+∆x= b∆xminus (a+b)∆xPX le x+∆x

Whenever this is positive you should increase the stock thus the optimum stockshould satisfy the equilibrium equation

PX le x+∆x=b

a+b (227)

for all ∆x gt 0 Letting ∆xrarr 0 we have

PX le x=b

a+b (228)

Using the distribution function F(x) = PX le x and its inverse Fminus1 we have

x = Fminus1(

ba+b

) (229)

Using this x we can maximize the profit of news-vendors

Problem 24 Suppose you buy a newspaper at the price of 70 and sell it at 100The demand X have the following uniform distribution

PX le x=xminus100

100 (230)

for x isin [100200] Find the optimal stock for you

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

28 COVARIANCE AND CORRELATION 23

28 Covariance and CorrelationWhen we have two or more random variables it is natural to consider the relationof these random variables But how The answer is the following

Definition 210 (Covariance) Let X and Y be two (possibly not independent)random variables Define the covariance of X and Y by

Cov[X Y ] = E[(X minusE[X ])(Y minusE[Y ])] (231)

Thus the covariance measures the multiplication of the fluctuations aroundtheir mean If the fluctuations are tends to be the same direction we have largercovariance

Example 212 (The covariance of a pair of indepnedent random variables) LetX1 and X2 be the independent random variables The covariance of X1 and X2 is

Cov[X1X2] = E[X1X2]minusE[X1]E[X2] = 0

since X1 and X2 are independent Thus more generally if the two random vari-ables are independent their covariance is zero (The converse is not always trueGive some example)

Now let Y = X1 +X2 How about the covariance of X1 and Y

Cov[X1Y ] = E[X1Y ]minusE[X1]E[Y ]= E[X1(X1 +X2)]minusE[X1]E[X1 +X2]

= E[X21 ]minusE[X1]2

= Var[X1] = np(1minus p) gt 0

Thus the covariance of X1 and Y is positive as can be expected

It is easy to see that we have

Cov[X Y ] = E[XY ]minusE[X ]E[Y ] (232)

which is sometimes useful for calculation Unfortunately the covariance has theorder of XY which is not convenience to compare the strength among differentpair of random variables Donrsquot worry we have the correlation function which isnormalized by standard deviations

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

24 CHAPTER 2 BASIC PROBABILITY THEORY

Definition 211 (Correlation) Let X and Y be two (possibly not independent)random variables Define the correlation of X and Y by

ρ[X Y ] =Cov[X Y ]σ [X ]σ [Y ]

(233)

Lemma 25 For any pair of random variables we have

minus1le ρ[X Y ]le 1 (234)

Proof See Exercise 211

29 Value at RiskSuppose we have one stock with its current value x0 The value of the stockfluctuates Let X1 be the value of this stock tomorrow The rate of return R can bedefined by

R =X1minus x0

x0 (235)

The rate of return R can be positive or negative We assume R is normally dis-tributed with its mean micro and σ

Problem 25 Why did the rate of return R assume to be a normal random variableinstead of the stock price X1 itself

We need to evaluate the uncertain risk of this future stock

Definition 212 (Value at Risk) The future risk of a property can be evaluated byValue at Risk (VaR) zα the decrease of the value of the property in the worst casewhich has the probability α or

PX1minus x0 geminuszα= α (236)

or

Pzα ge x0minusX1= α (237)

In short our damage is limited to zα with the probability α

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

29 VALUE AT RISK 25

Figure 21 VaR adopted form httpwwwnomuracojptermsenglishvvarhtml

By the definition of rate of return (235) we have

P

Rgeminuszα

x0

= α (238)

or

P

Rleminuszα

x0

= 1minusα (239)

Since R is assumed to be normal random variable using the fact that

Z =Rminusmicro

σ (240)

is a standard normal random variable where micro = E[R] and σ =radic

Var[R] wehave

1minusα = P

Rleminuszα

x0

= P

Z le minuszαx0minusmicro

σ

(241)

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

26 CHAPTER 2 BASIC PROBABILITY THEORY

Since the distribution function of standard normal random variable is symmetricwe have

α = P

Z le zαx0 + micro

σ

(242)

Set xα as

α = PZ le xα (243)

or

xα = Fminus1 (α) (244)

which can be found in any standard statistics text book From (242) we have

zαx0 + micro

σ= xα (245)

or

zα = x0(Fminus1 (α)σ minusmicro) (246)

Now consider the case when we have n stocks on our portfolio Each stockshave the rate of return at one day as

(R1R2 Rn) (247)

Thus the return rate of our portfolio R is estimated by

R = c1R1 + c2R2 + middot middot middot+ cnRn (248)

where ci is the number of stocks i in our portfolioLet q0 be the value of the portfolio today and Q1 be the one for tomorrow

The value at risk (VaR) Zα of our portfolio is given by

PQ1minusq0 geminuszα= α (249)

We need to evaluate E[R] and Var[R] It is tempting to assume that R is anormal random variable with

micro = E[R] =n

sumi=1

E[Ri] (250)

σ2 = Var[R] =

n

sumi=1

Var[Ri] (251)

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

210 REFERENCES 27

This is true if R1 Rn are independent Generally there may be some correla-tion among the stocks in portfolio If we neglect it it would cause underestimateof the risk

We assume the vectore

(R1R2 Rn) (252)

is the multivariate normal random variable and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[9 p7]

Problem 26 Estimate Var[R] when we have only two different stocks ie R =R1 +R2 using ρ[R1R2] defined in (233)

Using micro and σ of the overall rate of return R we can evaluate the VaR Zα justlike (246)

210 ReferencesThere are many good books which useful to learn basic theory of probabilityThe book [7] is one of the most cost-effective book who wants to learn the basicapplied probability featuring Markov chains It has a quite good style of writingThose who want more rigorous mathematical frame work can select [3] for theirstarting point If you want directly dive into the topic like stochatic integral yourchoice is maybe [6]

211 ExercisesExercise 21 Find an example that our intuition leads to mistake in random phe-nomena

Exercise 22 Define a probability space according to the following steps

1 Take one random phenomena and describe its sample space events andprobability measure

2 Define a random variable of above phenomena

3 Derive the probability function and the probability density

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

28 CHAPTER 2 BASIC PROBABILITY THEORY

4 Give a couple of examples of set of events

Exercise 23 Explain the meaning of (23) using Example 22

Exercise 24 Check P defined in Example 24 satisfies Definition 21

Exercise 25 Calculate the both side of Example 27 Check that these events aredependent and explain why

Exercise 26 Prove Lemma 22 and 23 using Definition 27

Exercise 27 Prove Lemma 24

Exercise 28 Let X be the Bernouilli random variable with its parameter p Drawthe graph of E[X ] Var[X ] σ [X ] against p How can you evaluate X

Exercise 29 Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y

Exercise 210 (Binomial random variable) Let X be a random variable with

X =n

sumi=1

Xi (253)

where Xi are independent Bernouilli random variables with the mean p The ran-dom variable X is said to be a Binomial random variable Find the mean andvariance of X

Exercise 211 Prove for any pair of random variables we have

minus1le ρ[X Y ]le 1 (254)

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 3

Markov chain

Markov chain is one of the most basic tools to investigate probabilistic structure ofdynamics Roughly speaking Markov chain is used for any stochastic processesfor first-order approximation

Definition 31 (Rough definition of Markov Process) A stochastic process X(t)is said to be a Markov chain when the future dynamics depends probabilisticallyonly on the current state (or position) not depend on the past trajectory

Example 31 The followings are examples of Markov chains

bull Stock price

bull Brownian motion

bull Queue length at ATM

bull Stock quantity in storage

bull Genes in genome

bull Population

bull Traffic on the internet

31 Discrete-time Markov chainDefinition 31 (discrete-time Markov chain) (Xn) is said to be a discrete-timeMarkov chain if

29

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

30 CHAPTER 3 MARKOV CHAIN

bull state space is at most countable

bull state transition is only at discrete instance

and the dynamics is probabilistically depend only on its current position ie

P[Xn+1 = xn+1|Xn = xn X1 = x1] = P[Xn+1 = xn+1|Xn = xn] (31)

Definition 32 ((time-homogenous) 1-step transition probability)

pi j equiv P[Xn+1 = j | Xn = i] (32)

The probability that the next state is j assuming the current state is in i

Similarly we can define the m-step transition probability

pmi j equiv P[Xn+m = j | Xn = i] (33)

We can always calculate m-step transition probability by

pmi j = sum

kpmminus1

ik pk j (34)

32 Stationary StateThe initial distribution the probability distribution of the initial state The initialstate can be decided on the consequence of the dies

Definition 33 (Stationary distribution) The probability distribution π j is said tobe a stationary distribution when the future state probability distribution is alsoπ j if the initial distribution is π j

PX0 = j= π j =rArr PXn = j= π j (35)

In Markov chain analysis to find the stationary distribution is quite importantIf we find the stationary distribution we almost finish the analysis

Remark 31 Some Markov chain does not have the stationary distribution In or-der to have the stationary distribution we need Markov chains to be irreduciblepositive recurrent See [7 Chapter 4] In case of finite state space Markov chainhave the stationary distribution when all state can be visited with a positive prob-ability

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

33 MATRIX REPRESENTATION 31

33 Matrix RepresentationDefinition 34 (Transition (Probabitliy) Matrix)

Pequiv

p11 p12 p1mp21 p22 p2m

pm1 pm2 pmm

(36)

The matrix of the probability that a state i to another state j

Definition 35 (State probability vector at time n)

π(n)equiv (π1(n)π2(n) ) (37)

πi(n) = P[Xn = i] is the probability that the state is i at time n

Theorem 31 (Time Evolution of Markov Chain)

π(n) = π(0)Pn (38)

Given the initial distribution we can always find the probability distributionat any time in the future

Theorem 32 (Stationary Distribution of Markov Chain) When a Markov chainhas the stationary distribution π then

π = πP (39)

34 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chainThere are three possibilities (1)up (2)down and (3)hold The price fluctuationtomor- row depends on the todayfs movement Assume the following transitionprobabilities

P =

0 34 1414 0 3414 14 12

(310)

For example if todayrsquos movement is ldquouprdquo then the probability of ldquodownrdquo againtomorrow is 34

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

32 CHAPTER 3 MARKOV CHAIN

Problem 31 1 Find the steady state distribution

2 Is it good idea to hold this stock in the long run

Solutions

1

π = πP (311)

(π1π2π3) = (π1π2π3)

0 34 1414 0 3414 14 12

(312)

Using the nature of probability (π1 +π2 +π3 = 1) (312) can be solved and

(π1π2π3) = (157251325) (313)

2 Thus you can avoid holding this stock in the long term

35 Googlersquos PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages PageRank can be understood by Markov chain Let us take a lookat a simple example based on [4 Chapter 4]

Suppose we have 6 web pages on the internet2 Each web page has some linksto other pages as shown in Figure 31 For example the web page indexed by 1refers 2 and 3 and is referred back by 3 Using these link information Googledecide the importance of web pages Herersquos how

Assume you are reading a web page 3 The web page contains 3 links toother web pages You will jump to one of the other pages by pure chance That

1PageRank is actually Pagersquos rank not the rank of pages as written inhttpwwwgooglecojpintljapressfunfactshtml ldquoThe basis of Googlersquos search technol-ogy is called PageRank and assigns an rdquoimportancerdquo value to each page on the web and gives ita rank to determine how useful it is However thatrsquos not why itrsquos called PageRank Itrsquos actuallynamed after Google co-founder Larry Pagerdquo

2Actually the numbe of pages dealt by Google has reached 81 billion

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 33

Figure 31 Web pages and their links Adopted from [4 p32]

means your next page may be 2 with probability 13 You may hop the web pagesaccording to the above rule or transition probability Now your hop is governedby Markov chain Now we can the state transition probability P as

P = (pi j) =

0 12 12 0 0 00 0 0 0 0 0

13 13 0 0 13 00 0 0 0 12 120 0 0 12 0 120 0 0 1 0 0

(314)

where

pi j = PNext click is web page j|reading page i (315)

=

1

the number of links outgoing from the page i i has a link to j

0 otherwise(316)

Starting from web page 3 you hop around our web universe and eventually youmay reach the steady state The page rank of web page i is nothing but the steadystate probability that you are reading page i The web page where you stay themost likely gets the highest PageRank

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

34 CHAPTER 3 MARKOV CHAIN

Problem 32 Are there any flaws in this scheme What will happen in the long-run

When you happened to visit a web page with no outgoing link you may jumpto a web page completely unrelated to the current web page In this case your nextpage is purely randomly decided For example the web page 2 has no outgoinglink If you are in 2 then the next stop will be randomly selected

Further even though the current web page has some outgoing link you may goto a web page completely unrelated We should take into account such behaviorWith probability 110 you will jump to a random page regardless of the pagelink

Thus the transition probability is modified and when i has at least one outgo-ing link

pi j =910

1the number of links outgoing from the page i

+110

16 (317)

On the other hand when i has no outgoing link we have

pi j =16 (318)

The new transition probability matrix P is

P =160

1 28 28 1 1 110 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

(319)

Problem 33 Answer the followings

1 Verify (319)

2 Compute π(1) using

π(1) = π(0)P (320)

given that initially you are in the web page 3

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

35 GOOGLErsquoS PAGERANK AND MARKOV CHAIN 35

By Theorem 32 we can find the stationary probability π satisfying

π = πP (321)

It turns out to be

π = (003720053900415037502050286) (322)

As depicted in Figure 32 according to the stationary distribution we can say thatweb page 4 has the best PageRank

Figure 32 State Probability of Googlersquos Markov chain

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 4

Birth and Death process and PoissonProcess

41 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain The very basicof the standard queueing theory The process allows two kinds of state transitions

bull X(t) = j rarr j +1 birth

bull X(t) = j rarr jminus1 death

Moreover the process allows no twin no death at the same instance Thus forexample

Pa birth and a death at t= 0 (41)

Definition 41 (Birth and Death process) Define X(t) be a Birth and Death pro-cess with its transition rate

bull P[X(t +∆t) = j +1|X(t) = j] = λk∆t +O(∆t)

bull P[X(t +∆t) = jminus1|X(t) = j] = microk∆t +O(∆t)

42 Infinitesimal GeneratorBirth and death process is described by its infinitesimal generator Q

36

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

43 SYSTEM DYNAMICS 37

Q =

minusλ0 λ0micro1 minus(λ1 + micro1) λ1

micro2 minus(λ2 + micro2) λ2

(42)

Note that in addition to above we need to define the initail condition in orderto know its probabilistic behaviour

43 System Dynamics

Definition 42 (State probability) The state of the system at time t can be definedby the infinite-dimension vector

P(t) = (P0(t)P1(t) middot middot middot) (43)

where Pk(t) = P[X(t) = k]

The dynamics of the state is described by the differential equation of matrix

dP(t)dt

= P(t)Q (44)

Formally the differential equation can be solved by

P(t) = P(0)eQt (45)

where eQt is matrix exponential defined by

eQt =infin

sumn=0

(Qt)n

n (46)

Remark 41 It is hard to solve the system equation since it is indeed an infinite-dimension equation (If you are brave to solve it please let me know)

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

38 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

44 Poisson ProcessDefinition 43 (Poisson Process) Poisson process is a specail birth and deathprocess which has the following parameters

bull microk = 0 No death

bull λk = λ Constant birth rate

Then the corresponding system equation is

dPk(t)dt

=minusλPk(t)+λPkminus1(t) for k ge 1 internal states (47)

dP0(t)dt

=minusλP0(t) boundary state (48)

Also the initial condition

Pk(0) =

1 k = 00 otherwise

(49)

which means no population initiallyNow we can solve the equation by iteration

P0(t) = eminusλ t (410)

P1(t) = λ teminusλ t (411)middot middot middot

Pk(t) =(λ t)k

keminusλ t (412)

which is Poisson distribution

Theorem 41 The popution at time t of a constant rate pure birth process hasPoisson distribution

45 Why we use Poisson processbull IT is Poisson process

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

46 Z-TRNASFORM OF POISSON PROCESS 39

bull It is EASY to use Poisson process

Theorem 42 (The law of Poisson small number) Poisson process hArr Countingprocess of the number of independent rare events

If we have many users who use one common system but not so often then theinput to the system can be Poisson process

Let us summarize Poisson process as an input to the system

1 A(t) is the number of arrival during [0 t)

2 The probability that the number of arrival in [0 t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

keminusλ t

3 The mean number of arrival in [0 t) is given by

E[A(t)] = λ t (413)

where λ is the arrival rate

46 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes Herersquos someexamples for Poisson process

E[zA(t)] = eminusλ t+λ tz (414)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (415)

Var[A(t)] = Find it (416)

47 Independent IncrementTheorem 43 (Independent Increment)

P[A(t) = k | A(s) = m] = Pkminusm(tminus s)

The arrivals after time s is independent of the past

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

40 CHAPTER 4 BIRTH AND DEATH PROCESS AND POISSON PROCESS

48 Interearrival TimeLet T be the interarrival time of Poisson process then

P[T le t] = 1minusP0(t) = 1minus eminusλ t Exponential distribution (417)

49 Memory-less property of Poisson processTheorem 44 (Memory-less property) T exponential distribution

P[T le t + s | T gt t] = P[T le s] (418)

Proof

P[T le t + s | T gt t] =P[t lt T le t + s]

P[T gt t]= 1minus eminusλ s (419)

410 PASTA Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system This is quite importantfor performance evaluation

411 Excercise1 Find an example of Markov chains which do not have the stationary distri-

bution

2 In the setting of section 34 answer the followings

(a) Prove (313)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday where do you have to go on the following Wednesday tojoin her Describe why

(c) When you do not know when and where she starts which place do youhave to go to join her Describe why

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

411 EXCERCISE 41

3 Show E[A(t)] = λ t when A(t) is Poisson process ie

P[A(t) = k] =(λ t)k

keminusλ t

4 When A(t) is Poisson process calculate Var[A(t)] using z-transform

5 Make the graph of Poisson process and exponential distribution using Math-ematica

6 When T is exponential distribution answer the folloing

(a) What is the mean and variance of T

(b) What is the Laplace transform of T (E[eminussT ])

(c) Using the Laplace transform verify your result of the mean and vari-ance

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 5

Introduction of Queueing Systems

51 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems Thefeatures for the queueing systems

1 Public many customers share the system

2 Limitation of resources There are not enough resources if all the customerstry to use them simultaneously

3 Random Uncertainty on the Customerrsquos behaviors

Many customers use the limited amount of resources at the same time withrandom environment Thus we need to estimate the performance to balance thequality of service and the resource

Example 51 Here are some example of queueing systems

bull manufacturing system

bull Casher at a Starbucks coffee shop

bull Machines in Lab room

bull Internet

42

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

52 STARBUCKS 43

52 StarbucksSuppose you are a manager of starbucks coffee shop You need to estimate thecustomer satisfaction of your shop How can you observe the customer satisfac-tion How can you improve the performance effectively and efficiently

Problem 51 By the way most Starbucks coffee shops have a pick-up station aswell as casher Can you give some comment about the performance of the system

53 Specification of queueing systemsSystem description requirements of queueing systems

bull Arrival process(or input)

bull Service time

bull the number of server

bull service order

Let Cn be the n-th customer to the system Assume the customer Cn arrives tothe system at time Tn Let Xn = Tn+1minusTn be the n-th interarrival time Supposethe customer Cn requires to the amount of time Sn to finish its service We call Snthe service time of Cn

We assume that both Xn and Sn are random variable with distribution functionsPXn le x and PSn le x

Definition 51 Let us define some terminologies

bull E[Xn] = 1λ

the mean interarrival time

bull E[Sn] = 1micro

the mean service time

bull ρ = λ

micro the mean utilizations The ratio of the input vs the service We often

assume ρ lt 1

for the stability

Let Wn be the waiting time of the n-th customer Define the sojourn time Yn ofCn by

Yn = Wn +Sn (51)

Problem 52 What is Wn and Yn in Starbucks coffee shop

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

44 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

54 Littlersquos formulaOne of the rdquomustrdquo for performance evaluation No assumption is needed to provethis

Definition 51 Herersquos some more definitions

bull A(t) the number of arrivals in [0 t)

bull D(t) the number of departures in [0 t)

bull R(t) the sum of the time spent by customer arrived before t

bull N(t) the number of customers in the system at time t

The relation between the mean sojourn time and the mean queue length

Theorem 51 (Littlersquos formula)

E[N(t)] = λE[Y ]

Proof Seeing Figure 54 it is easy to find

A(t)

sumn=0

Yn =int t

0N(t)dt = R(t) (52)

Dividing both sides by A(t) and taking t rarr infin we have

E[Y (t)] = limtrarrinfin

1A(t)

A(t)

sumn=0

Yn = limtrarrinfin

tA(t)

1t

int t

0N(t)dt =

E[N(t)]λ

(53)

since λ = limtrarrinfin A(t)t

Example 52 (Starbucks coffee shop) Estimate the sojourn time of customers Y at the service counter

We donrsquot have to have the stopwatch to measure the arrival time and the re-ceived time of each customer In stead we can just count the number of ordersnot served and observe the number of customer waiting in front of casher

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

54 LITTLErsquoS FORMULA 45

t

A(t)D(t)

Figure 51 Littlersquos formula

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

46 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

Then we may find the average number of customer in the system

E[N(t)] = 3 (54)

Also by the count of all order served we can estimate the arrival rate ofcustomer say

λ = 100 (55)

Thus using Littlersquos formula we have the mean sojourn time of customers in Star-bucks coffee shop

E[Y ] =E[N]

λ= 003 (56)

Example 53 (Excersize room) Estimate the number of students in the room

bull E[Y ] = 1 the average time a student spent in the room (hour)

bull λ = 10 the average rate of incoming students (studentshour)

bull E[N(t)] = λE[Y ] = 10 the average number of students in the room

Example 54 (Toll gate) Estimate the time to pass the gate

bull E[N(t)] = 100 the average number of cars waiting (cars)

bull λ = 10 the average rate of incoming cars (studentshour)

bull E[Y ] = E[N]λ

= 10 the average time to pass the gate

55 Lindley equation and Loynes variableHerersquos the rdquoNewtonrdquo equation for the queueing system

Theorem 52 (Lindley Equation) For a one-server queue with First-in-first-outservice discipline the waiting time of customer can be obtained by the followingiteration

Wn+1 = max(Wn +SnminusXn0) (57)

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

56 EXERCISE 47

The Lindley equation governs the dynamics of the queueing system Althoughit is hard to believe sometimes the following alternative is much easier to handlethe waiting time

Theorem 53 (Loynersquos variable) Given that W1 = 0 the waiting time Wn can bealso expressed by

Wn = maxj=01nminus1

nminus1

sumi= j

(SiminusXi)0 (58)

Proof Use induction It is clear that W1 = 0 Assume the theorem holds for nminus1Then by the Lindley equation

Wn+1 = max(Wn +SnminusXn0)

= max

(sup

j=1nminus1

nminus1

sumi= j

(SiminusXi)0+SnminusXn0

)

= max

(sup

j=1nminus1

n

sumi= j

(SiminusXi)SnminusXn0

)

= maxj=01n

n

sumi= j

(SiminusXi)0

56 Exercise1 Restaurant Management

Your friend owns a restaurant He wants to estimate how long each cus-tomer spent in his restaurant and asking you to cooperate him (Note thatthe average sojourn time is one of the most important index for operatingrestaurants) Your friend said he knows

bull The average number of customers in his restaurant is 10

bull The average rate of incoming customers is 15 per hour

How do you answer your friend

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

48 CHAPTER 5 INTRODUCTION OF QUEUEING SYSTEMS

2 Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance Describe what is corresponding to the following terminologiesin queueing theory

bull customers

bull arrival

bull service

bull the number of customers in the system

3 Web site administration

You are responsible to operate a big WWW site A bender of PC-serverproposes you two plans which has the same cost Which plan do youchoose and describe the reason of your choice

bull Use 10 moderate-speed servers

bull Use monster machine which is 10 times faster than the moderate one

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 6

MM1 queue

The most important queueing process

bull Arrival Poisson process

bull Service Exponetial distribution

bull Server One

bull Waiting room (or buffer) Infinite

61 MM1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served)N(t) is the birth and death process with

bull λk = λ

bull microk = micro

P(Exactly one cusmter arrives in [t t +∆t]) = λ∆t (61)P(Given at least one customer in the systemexactly one service completes in [t t +∆t]) = micro∆t (62)

An MM1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate)

49

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

50 CHAPTER 6 MM1 QUEUE

Theorem 61 (Steady state probability of MM1 queue) When ρ lt 1

pk = P[N(t) = k] = (1minusρ)ρk (63)

where ρ = λmicro

Proof The balance equation

λ pkminus1 = micro pk (64)

Solving this

pk =λ

micropkminus1 =

(λ

micro

)k

p0 (65)

Then by the fact sum pk = 1 we have

pk = (1minusρ)ρk (66)

It is easy to see

E[N(t)] =ρ

(1minusρ) (67)

62 UtilizationThe quantity ρ is said to be a utilization of the system

Since p0 is the probability of the system is empty

ρ = 1minus p0 (68)

is the probability that the em is under serviceWhen ρ ge 1 the system is unstable Indeed the number of customers in the

system wil go to infin

63 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system We have two different ways to obtain the imformationof waiting time

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

63 WAITING TIME ESTIMATION 51

631 Waiting Time by Littlelsquos FormulaLet Sn be the service time of n-th customer and Wn be his waiting time Define thesojourn time Yn by

Yn = Wn +Sn (69)

Theorem 62

E[W ] =ρmicro

1minusρ (610)

Proof By Littlersquos formula we have

E[N] = λE[Y ] (611)

Thus

E[W ] = E[Y ]minusE[S] =E[N]

λminus 1

micro

=ρmicro

1minusρ

632 Waiting Time Distribution of MM1 QueuesLittlelsquos formula gives us the estimation of mean waiting time But what is thedistribution

Lemma 61 (Erlang distribution) Let Sii=1m be a sequence of independentand identical random variables which has the exponetial distribution with itsmean 1micro Let X = sum

mi=1 Si then we have

dPX le tdt

=λ (λ t)mminus1

(mminus1)eminusλ t (612)

Proof Consider a Poisson process with the rate micro Let A(t) be the number ofarrivals by time t Since t lt X le t +h= A(t) = mminus1A(t +h) = m we have

Pt lt X le t +h= PA(t) = mminus1A(t +h) = m= PA(t +h) = m | A(t) = mminus1PA(t) = mminus1

= (microh+o(h))(microt)mminus1

(mminus1)eminusmicrot

Dividing both sides by h and taking hrarr 0 we have (612)

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

52 CHAPTER 6 MM1 QUEUE

Theorem 63 Let W be the waiting time of the MM1 queue Then we have

P[W le w] = 1minusρeminusmicro(1minusρ)w (613)

Further let V be the sojourn time (waiting time plus service time) of the MM1queue then V is exponential random variable ie

P[V le v] = 1minus eminusmicro(1minusρ)v (614)

Proof By condition on the number of customers in the system N at the arrivalwe have

P[W le w] =infin

sumn=0

P[W le w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival By thelack of memory the remaining service time of the customer in service is againexponentially distibuted P[W le w | N = n] = P[S1 + + Sn le w] for n gt 0Using Lemma 61 we have

P[W le w] =infin

sumn=1

int w

0

micro(microt)nminus1

(nminus1)eminusmicrotdtP[N = n]+P[N = 0]

=int w

0

infin

sumn=1

micro(microt)nminus1

(nminus1)eminusmicrot(1minusρ)ρndt +1minusρ

= 1minusρeminusmicro(1minusρ)w (615)

Problem 61 Prove (614)

64 ExampleConsider a WWW server Users accses the server according to Poisson processwith its mean 10 accesssec Each access is processed by the sever according tothe time which has the exponential distribution with its mean 001 sec

bull λ = 10

bull 1micro = 001

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

65 EXCERCISE 53

bull ρ = λmicro = 10times001 = 01

Thus the system has the utilizaion 01 and the number of customer waiting isdistributed as

P[N(t) = n] = (1minusρ)ρn = 09times01n (616)

E[N(t)] =ρ

(1minusρ)= 19 (617)

65 Excercise1 For an MM1 queue

(a) Calculate the mean and variance of N(t)

(b) Show the graph of E[N(t)] as a function of ρ What can you say whenyou see the graph

(c) Show the graph of P[W le w] with different set of (λ micro) What canyou say when you see the graph

2 In Example 64 consider the case when the access rate is 50sec

(a) Calculate the mean waiting time

(b) Draw the graph of the distribution of N(t)

(c) Draw the graph of the distribution of W

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess how much access can you accept

3 Show the equation (615)

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 7

Reversibility

71 Output from QueueWhat is output from a queue

To consider a network of queues this is very important question since theoutput from a queue is indeed the input for the next queue

72 ReversibibilityReversibility of stochastic process is useful to consider the output for the systemIf the process is reversible we may reverse the system to get the output

721 Definition of ReversibilityDefinition 71 X(t) is said to be reversible when (X(t1)X(t2) X(tm)) has thesame distribution as (X(τminus t1) X(τminus tm)) for all τ and for all t1 tm

The reverse of the process is stochatically same as the original process

722 Local Balance for Markov ChainTransition rate from i to j

qi j = limτrarr0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (71)

qii = 0 (72)

54

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

72 REVERSIBIBILITY 55

We may have two balance equations for Markov chain

1 Global balance equation

sumj

piqi j = sumj

p jq ji (73)

2 Local balance equationpiqi j = p jq ji (74)

Theorem 71 (Local Balance and Revesibility) A stationary Markov chain isreversible if and only if there exists a sequence pi with

sumi

pi = 1 (75)

piqi j = p jq ji (76)

Proof 1 Assume the process is reverisble then we have

P[X(t) = iX(t + τ) = j] = P[X(t) = jX(t + τ) = i]

Using Bayse theorem and taking τ rarr 0 we have

piqi j = p jq ji

2 Assume there exists pi satisfying (75) and (76) Since the process isMarkov we have

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)] = pieminusqitqi jeminusq js

where qi = sum j qi j

Using the local balance we have

pieminusqitqi jeminusq js = p jeminusqitq jieminusq js

Thus

P[X(u)equiv i for u isin [0 t)X(u)equiv j for u isin [t t + s)]= P[X(u)equiv i for u isin [minust0)X(u)equiv j for u isin [minustminus sminust)]

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

56 CHAPTER 7 REVERSIBILITY

73 Output from MM1 queueTheorem 72 (Burkersquos Theorem) If ρ = λmicro lt 1 the output from an equilibiumMM1 queue is a Poisson process

Proof Let N(t) be the number of the customers in the system at time t The pointprocess of upward changes of N(t) corresponds to the arrival process of the queuewhich is Poisson Recall the transition rate of N(t) of the MM1 queue satisfies

λPnminus1 = microPn (77)

which is the local balance eqaution Thus by Theorem 71 N(t) is reversibleand N(minust) is stochastically identical to N(t) Thus the upward changes of N(t)and N(minust) are stochastically identical In fact the upward chang of N(minust) cor-responds to the departures of the MM1 queue Thus the departure process isPoisson

74 Excercise1 If you reverse an MM1 queue what is corresponding to the waiting time

of n-th customer in the original MM1 (You may make the additionalassumption regarding to the server if necessary)

2 Consider two queues whose service time is exponetially distributed withtheir mean 1micro The input to one queue is Poisson process with rate λ andits output is the input to other queue What is the average total sojourn timeof two queues

3 Can you say that MM2 is reversible If so explain why

4 Can you say that MMmm is reversible If so explain why (You shouldclearly define the output from the system)

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 8

Network of Queues

81 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network)

Specifications

bull M the number of nodes (queues) in the network

bull Source and sink One for entire network

bull i index of node

bull ri j routing probability from node i to j (Bernouilli trial)

bull Open network allow the arrival and departure from outside the network

ndash ris routing probability from node i to sink (leaving the network)

ndash rsi probability of arrival to i from outside (arrival to the network)

bull The arrival form the outside Poisson process (λ )

bull The service time of the each node exponential (micro)

82 Global Balance of Networkbull (Ncup0)M State space (M-dimensional)

bull n = (n1 nM) the vector of the number customers in each node

57

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

58 CHAPTER 8 NETWORK OF QUEUES

bull 1i = (0 0i10 0)

bull p(n) the probability of the number customers in the network is n

The global balance equation of Network

Theorem 81 (Global Balance Equation of Jackson Network) In the steady stateof a Jackson network we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

λ rsi p(nminus1i)+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

(81)

Proof The left-hand side correspond to leaving the state n with an arrival or de-parture Each terms on the right correspond to entering the state n with an ar-rival departure and routing Thus the both side should be balanced in the steadystate

83 Traffic EquationLet Λi be the average output (throughput) of a queue i

Theorem 82 (Traffic Equations)

Λi = λ rsi +M

sumj=1

r jiΛ j (82)

Proof Consider the average input and output from a queue i The output from ishould be equal to the internal and external input to i

Remark 81 The average output (Λi) can be obtained by solving the system ofequations (82)

84 Product Form SolutionTheorem 83 (The steady state probabilities of Jackson network) The solution ofthe global balance equation (81)

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

84 PRODUCT FORM SOLUTION 59

p(n) =M

prodi=1

pi(ni) (83)

andpi(ni) = (1minusρi)ρ

nii (84)

where ρi = Λimicroi

Proof First we will show pi which satisfy the rdquolocal balancerdquo-like equation

Λi p(nminus1i) = microi p(n) (85)

also satisfyies (81) Rearranging (82) we have

λ rsi = ΛiminusM

sumj=1

r jiΛ j (86)

Use this to eliminate λ rsi from (81) then we have

[λ +M

sumi=1

microi]p(n) =M

sumi=1

Λi p(nminus1i)minusM

sumi=1

M

sumj=1

r jiΛi p(nminus1i)

+M

sumi=1

microiris p(n+1i)+M

sumi=1

M

sumj=1

micro jr ji p(n+1 jminus1i)

Using (85) we have

λ =M

sumi=1

risΛi (87)

which is true because the net flow into the network is equal to the net flow outof the network Thus a solution satisfying the local balance equation will beautomatically the solution of the global balance equation By solving the localbalance equation we have

p(n) =Λi

microip(nminus1i)

= (Λimicroi)ni p(n1 0 nM)

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

60 CHAPTER 8 NETWORK OF QUEUES

Using this argument for different i we have

p(n) =M

prodi=1

(Λimicroi)ni p(0)

Taking summation over n and find p(0) to be the product of

pi(0) = 1minus Λi

microi (88)

Remark 82 The steady state probability of Jackson network is the product of thesteady state probability of MM1 queues

85 Exercises1 Consider two queues in the network

bull λ external arrival rate microi the service rate of the node i

bull rs1 = 1 rs2 = 0

bull r11 = 0 r12 = 1 r1s = 0

bull r21 = 38 r22 = 28 r2s = 38

(a) Estimate the throughputs Λi for each queue

(b) Estimate the steady state probability of the network

(c) Draw the 3-D graph of the steady state probability for λ = 1 and micro1 = 3and micro2 = 4

(d) Estimate the mean sojourn time of the network of (1c) by using LittlersquosLaw

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 9

Repairman Problem inManufacturing System

This part is based on the book John A Buzacott and J George ShanthikumarStochastic Models of Manufacturing Systems [1]

91 Repairman Problem

Suppose we have machines in our factory to make some products All machinesare equivalent and its production rate is h unit per unit time Unfortunately ourmachines are sometimes fail and needs to be repaired We have repairmen to fixthem

Each repairmen are responsible to some fixed number of machines say m As-sume we have large m then we have considerable probability that more than twomachines are failed at the same time The repairman is busy fixing one machineand the rest of the failed machines are waiting to be reparied So the productionrate per machine (we call it ldquoefficicenyrdquo) will be down On the other hand forlarge m we may have a good production efficiency but we may find our repair-man is idle quite often

Thus we have the basic tradeoff of the performance of repairman and theproductivity of each machines

Problem 91 How can you set the appropritate number of machines m assignedto one repairman satisfying the above tradeoff

61

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

62 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

92 Average AnalysisIn this section we will see what we can say about our repairman problem withoutusing stochastic argument

Let Pn be the proportional time of n machines are down By observing ourfactory we can easily estimate Pn Once we obtain the information Pn we canestimate various performance measures of our factory For example the totalproduction rate in our factory can be calculated by

H =m

sumn=0

(mminusn)Pnh (91)

Definition 91 (System efficiency) Let G = mh be the total production rate of ourfactory assuming that all machines are available Then the system efficiency ofour factory η is defined by

η =HG

= 1minusm

sumn=0

nPn

m (92)

Problem 92 Derive the right-hand side of (92)

Remark 91 Using the system efficiency η we can estimate the impact of thefailure of our machines Of course we can expect the system efficiency η will beimproved if we reduce the number of machine assigned to a repairman

Herersquos another performance measure for our factory

Definition 92 Define the operator utility ρ by

ρ =m

sumn=1

Pn = 1minusP0 (93)

which estimate the time ratio during our repairman is working

Now suppose the expected repair time of a machine is 1micro Let D(t) be thenumber of repaired machines up to time t Then in the long run (for large t) wehave

D(t) = microρt (94)

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

92 AVERAGE ANALYSIS 63

since operator is working for ρt Further let 1λ be the mean time to a failure ofa single machine (we say λ is the failure rate) Then A(t) the number of machinefailed up to time t can be estimated by

A(t) = λ

m

sumn=0

(mminusn)Pnt

= ληmt (95)

Problem 93 Derive (95)

In the long run we can expect the number of failed machine is equal to thenumber of repaired machines

Problem 94 Why

Thus for lager t we have

A(t) = D(t)ληmt = microρt

Rearranging the above we have the relation between the system efficiency η andthe operator utility ρ

η = ρmicro

mλ (96)

Further using this relation the production rate H can be represented by

H = ηG = ηmh = ρmicrohλ

(97)

Remark 92 The equation (96) clearly implies that the reduction of number ofmachines assigned to one repairman will result the better system efficiency How-ever the situation here is not so easy If we decrease m it will affect the theoperator utility ρ (typically we may see smaller ρ) We will see this effect in thefollowing section

As a factory manager you may not be satisfied with the good operator util-ity You may want to know the number of machine waiting for repair L The

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

64 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

expectation of L can be given by

E[L] =m

sumn=1

(nminus1)Pn

=m

sumn=1

nPnminusm

sumn=1

Pn

= m(1minusη)minusρ

= mminusρmicro

λminusρ

where we used the relation (96) Thus

E[L] = m(1minusη)minusρ = mminusρ(1+micro

λ) (98)

Also let T be the down time of a machine Since the total down time in [0 t)is given by

m

sumn=1

nPnt = m(1minusη)t (99)

the average down time of a machine is

E[T ] =m(1minusη)t

A(t)(910)

=1minusη

λη (911)

Let N(t) be the number of machine not operating at time t then

E[N(t)] =m

sumn=1

nPn = m(1minusη) (912)

Setting λ lowast as the average number of machine failure per unit time we have

λlowast = lim

trarrinfin

A(t)t

= ληm (913)

Combining (910) (912) and (913) we have a relation between these quantities

E[N(t)] = λlowastE[T ] (914)

which turns out to be familiar Littlersquos formula (see Theorem 51)

Problem 95 Use Theorem 51 to prove (914)

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

93 SINGLE REPAIRMAN IN MARKOV MODEL 65

93 Single Repairman in Markov ModelAssume we have m machines Machines can be failed and one repairman is re-sponsible to fix it Take one specific machine Let T be the time until this machineis failed and assume T to be an exponential random variable with its mean 1λ ie

PT le x= 1minus eminusλx (915)

Also our repairman will take the time S to fix this machine We assume S is alsoan exponential random variable with its mean 1micro ie

PSle x= 1minus eminusmicrox (916)

As we discussed before those exponential random variables have memorylessproperty Thus we make the Markov chain by modeling our m machines LetN(t) be the number of machines down at time t Then N(t) is a Markov chainwith its infinitesimal operator Q as

Q =

minusmλ mλ

micro minusmicrominus (mminus1)λ (mminus1)λ

micro λ

micro minusmicro

(917)

Note that we have m machines and one repairman Thus as the number of ma-chines down increases the rate of having one more machine failed decreaseswhile the repair rate remained same

Let Pn be the stationary probability of N(t) ie

Pn = limtrarrinfin

PN(t) = n (918)

Then the balance equations can be found as

mλ p0 = micro p1

(mminus1)λ p1 = micro p2

(mminusn+1)λ pnminus1 = micro pn

λ pmminus1 = micro pm (919)

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

66 CHAPTER 9 REPAIRMAN PROBLEM IN MANUFACTURING SYSTEM

Thus we have

Pn = (λmicro)(mminusn+1)Pnminus1

= (λmicro)2(mminusn+1)(mminusn+2)Pnminus1

= (λmicro)n(mminusn+1) middot middot middotmP0

= (λmicro)n m(mminusn)

P0

for n = 0 m Here P0 is still unknown Using the usual normalization condi-tion sum pn = 1 we have

1 = P0

m

sumn=0

(λmicro)n m(mminusn)

Thus we have the unknown P0 as

P0 =1

summn=0(λmicro)n m

(mminusn)

(920)

and for n = 0 m

Pn = (λmicro)n m(mminusn)

P0 (921)

Now what we want to know is the trade-off between the production rate Hand the operator utility ρ From (93) the operator utility ρ is calculated by

ρ = 1minusP0 (922)

Also by (96) we have

η = ρmicro

mλ= (1minusP0)

micro

mλ (923)

See Figure 91 and 92 Here we set λmicro = 100 We can see what will happenwhen we increase m the number of machines assigned to our repairman Ofcourse the utility of our repairman increases as m increases However at somepoint the utility hits 1 which means he is too busy working on repairing machineand at this point the number of machines waiting to be repaired will explode Sothe productivity of our machine decreases dramatically

Problem 96 How can you set the appropriate number of machines assigned to asingle repairman

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

93 SINGLE REPAIRMAN IN MARKOV MODEL 67

50 100 150 200 250 300 m

02

04

06

08

1

Figure 91 Operator utility ρ

50 100 150 200 250 300 m

02

04

06

08

1

Figure 92 Efficiency of machine η

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 10

Production Systems

In this chapter we will discuss how we can model and analyze two basic produc-tion systems namely produce-to-order and produce-to-stock This part is basedon the book John A Buzacott and J George Shanthikumar Stochastic Models ofManufacturing Systems [1]

101 Produce-to-Order System

Assume customers arrive according to Poisson Process with the rate λ to a singleserver system After the arrival of a customers the system starts to process the jobThe service time of the job (or production time) is assumed to be an exponentialrandom variable with its mean 1micro Upon arrival of a customer if there are anyother customers waiting the customer join the queue We assume the order of theservice is fist-come-first-serve

Then this system is indeed an MM1 queue that we discussed in Chapter 6Let N(t) be the number of customers in the system (both waiting in the queue andin service) What we want to know is the stationary distribution of N(t) ie

Pn = limtrarrinfin

N(t) (101)

As in Theorem 61 we have

Pn = (1minusρ)ρn (102)

where ρ = λmicro representing the system utility Also as in (67) we can derive

68

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

102 PRODUCE-TO-STOCK SYSTEM 69

the expectation of N(t)

E[N(t)] =ρ

(1minusρ) (103)

Let W be the waiting time in the queue and V = W + S be the sojourn time TheW corresponds to the time until the customer receives the product We can findthe sojourn time distribution as Theorem 63

P[V le x] = 1minus eminusmicro(1minusρ)x (104)

and its mean as

E[V ] =1

micro(1minusρ) (105)

102 Produce-to-Stock SystemIf you order a product in the system like produce-to-order you will find a largewaiting time One of the simplest way to reduce the waiting time and enhance thequality of service is to have stock Of course stocking products will incur storagecost So balancing the quality and the cost is our problem

Here we assume a simple scheme We allow at most z products in our storageWe attach a tag to each product Thus the number of tags is z Whenever thereare some products in our storage upon an arrival of customers the customer willtake one product and we will take off its tag The tag now become the order of anew product

Depending on the production time occasionally our products may go out ofstock on arrival of a customer In this case the customer has to wait Our problemis to set appropriate level of the number of tags z

Assume as assumed in Section 101 the arrival of customer is Poisson processwith the rate λ The service time of the job (or production time) is assumed to bean exponential random variable with its mean 1micro

Let I(t) be the number of products stocked in our storage at time t and let C(t)be the number of orders in our system (including the currently-processed item)Since we have z tags

I(t)+C(t) = z (106)

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

70 CHAPTER 10 PRODUCTION SYSTEMS

In addition let B(t) be the number of backlogged customers waiting for theirproducts Define N(t) by the sum of the backlogged customers the waiting ordersand the processing order ie

N(t) = C(t)+B(t) (107)

Then N(t) can be regarded as the number of customers in an MM1 queue sinceat the arrival of customers it will increase one and at the service completion itwill decrease one Thus we have the stationary distribution of N(t) as

Pn = limtrarrinfin

PN(t) = n= (1minusρ)ρn (108)

for ρ = λmicro lt 1 as found in Theorem 61 Since N(t) is the sum of waitingorders we have zminusN(t) products in our storage if and only if N(t) lt z Thus thenumber of products in our storage can be written by

I(t) = (zminusN(t))+ (109)

Using (108) we can obtain the probability distribution of the number of productsin storage as

PI(t) = n= P(zminusN(t))+ = n

=

PN(t) = zminusn= (1minusρ)ρzminusn for n = 12 zPN(t) gt z= ρz for n = 0

(1010)

When I(t) gt 0 the customer arrived can receive a product Thus

PNo wait= PI(t) gt 0= 1minusPI(t) = 0= 1minusρ

z (1011)

Thus as the number of tag increase we can expect that our customers do not haveto wait

Also we can evaluate the expected number of products in our storage canestimated by

E[I(t)] =z

sumn=1

nPI(t) = n

= zminus ρ(1minusρz)1minusρ

(1012)

As we can easily see that the second term is rapidly converges to ρ(1minusρ) forlarge z the storage cost is almost linear to the number of tag z

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

102 PRODUCE-TO-STOCK SYSTEM 71

Problem 101 Show (1010) and (1012)

2 4 6 8 10 z

02

04

06

08

1PNo wait

Figure 101 Probability of no wait PNo wait

2 4 6 8 10 z

2

4

6

8

EI

Figure 102 Expected number of Stocks E[I(t)]

Figure 101 and 102 shows the trade-off between quality of service and stor-age cost when ρ = 02

Problem 102 How can you set the number of tag z

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 11

Decision Making

72

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Chapter 12

Analytic Hierarchy Processes

73

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes

Bibliography

[1] John A Buzacott and J George Shanthikumar Stochastic Models of Manu-facturing Systems Prentice Hall 1992

[2] T Duffy M Hatzakis W Hsu R Labe B Liao XS Luo J Oh A Setyaand L Yang Merrill Lynch Improves Liquidity Risk Management for Re-volving Credit Lines Interfaces 35(5)353 2005

[3] Richard Durrett Probability Theory and Examples Thomson Learning1991

[4] AN Langville and CD Meyer Googlersquos PageRank and Beyond The Sci-ence of Search Engine Rankings Princeton University Press 2006

[5] Masao Mori and Tomomi Matsui Operations Research Asakura 2004

[6] Bernt Oksendal Stochastic Differential Equations An Introduction With Ap-plications Springer-Verlag 2003

[7] Sheldon M Ross Applied Probability Models With Optimization Applica-tions Dover Pubns 1992

[8] Nassim Nicholas Taleb Fooled by Randomness The Hidden Role of Chancein the Markets and in Life Random House Trade Paperbacks 2005

[9] Hiroshi Toyoizumi Applied probability and mathemati-cal finance theory Lecture note of mathematical fincancehttpwwwfwasedajptoyoizumiclassesaccountingstatisticsstatistics2mainpdf2006

74

• Introduction of Operations Research
• • A Simple Example
  • What is OR
  • Decision Making
  • History
  • Examples of OR
  • • Basic Probability Theory
    • • Why Probability
      • Probability Space
      • Conditional Probability and Independence
      • Random Variables
      • Expectation Variance and Standard Deviation
      • How to Make a Random Variable
      • News-vendor Problem ``How many should you buy
      • Covariance and Correlation
      • Value at Risk
      • References
      • Exercises
      • • Markov chain
        • • Discrete-time Markov chain
          • Stationary State
          • Matrix Representation
          • Stock Price Dynamics Evaluation Based on Markov Chain
          • Googles PageRank and Markov Chain
          • • Birth and Death process and Poisson Process
            • • Definition of Birth and Death Process
              • Infinitesimal Generator
              • System Dynamics
              • Poisson Process
              • Why we use Poisson process
              • Z-trnasform of Poisson process
              • Independent Increment
              • Interearrival Time
              • Memory-less property of Poisson process
              • PASTA Poisson Arrival See Time Average
              • Excercise
              • • Introduction of Queueing Systems
                • • Foundation of Performance Evaluation
                  • Starbucks
                  • Specification of queueing systems
                  • Littles formula
                  • Lindley equation and Loynes variable
                  • Exercise
                  • • MM1 queue
                    • • MM1 queue as a birth and death process
                      • Utilization
                      • Waiting Time Estimation
                      • • Waiting Time by Little`s Formula
                        • Waiting Time Distribution of MM1 Queues
                        • • Example
                          • Excercise
                          • • Reversibility
                            • • Output from Queue
                              • Reversibibility
                              • • Definition of Reversibility
                                • Local Balance for Markov Chain
                                • • Output from MM1 queue
                                  • Excercise
                                  • • Network of Queues
                                    • • Open Network of queues
                                      • Global Balance of Network
                                      • Traffic Equation
                                      • Product Form Solution
                                      • Exercises
                                      • • Repairman Problem in Manufacturing System
                                        • • Repairman Problem
                                          • Average Analysis
                                          • Single Repairman in Markov Model
                                          • • Production Systems
                                            • • Produce-to-Order System
                                              • Produce-to-Stock System
                                              • • Decision Making
                                                • Analytic Hierarchy Processes