STUDENTS PROBABILITY DAYWeizmann Institute of Science

March 28, 2007

STUDENTS PROBABILITY DAYWeizmann Institute of Science

March 28, 2007

Yoni Nazarathy(Supervisor: Prof. Gideon Weiss)

University of Haifa

Yoni Nazarathy(Supervisor: Prof. Gideon Weiss)

University of Haifa

0 10 20 30 40

0

5

10

15

20

Queueing Networks withInfinite Virtual Queues

An Example, An Application and a Fundamental Question

Queueing Networks withInfinite Virtual Queues

An Example, An Application and a Fundamental Question

Yoni Nazarathy, University of Haifa, 2007 2

Multi-Class Queueing Networks (Harrison 1988, Dai 1995,…)Multi-Class Queueing Networks (Harrison 1988, Dai 1995,…)

1 2

6

5 4

3

{1,..., }

{ ( ), 0}k

K

Q t t

Queues

6K

1/4

3/4

Routes

(0)kQ k

Initial Queue Levels

' ( ) , 'kk n k k

''

( )lim kk

kkn

nP

n

Servers

'

( )

( ) 1lim

k

kk

tk

S t k

S t

t m

Processing Durations

{1,..., }

{ } {0,1}I K ik ik

I

A A A

Resource Allocation (Scheduling)

( )

(0) 0 ( ) ( )

( ) ( ) 0

k

k ik k kk

k k

T t

T A T t T s t s s t

T t only when Q t

Network Dynamics

' ' ''

( ) (0) ( ( )) ( ( ( )))k k k k k k k kk

Q t Q S T t S T t

Yoni Nazarathy, University of Haifa, 2007 3

INTRODUCING: Infinite Virtual QueuesINTRODUCING: Infinite Virtual Queues

( ) (0) ( ( ))R t R S T t t

5 10 15 20 25 30

-1

-0.5

0.5

1

1.5

2

2.5

Regular Queue

( ) : {0,1,2,...}kQ t Infinite Virtual Queue

( )kQ t t

Example Realization

Relative Queue Length:

( )R t

NominalProduction

Rate

Yoni Nazarathy, University of Haifa, 2007 4

MCQN+IVQMCQN+IVQ

0

( )

(0) 0 ( ) ( )

( ) ( ) 0

k

k ik k kk

k k

T t

T A T t T s t s s t

T t only when Q t for k

{1,..., }

{ ( ), 0}k

K

Q t t

1 2

6

5 4

3

Queues

1/4

3/4

Routes

(0)kQ k

Initial Queue Levels

' 0( ) 'kk n k k

''

( )lim kk

kkn

nP

n

Servers

'

( )

( ) 1lim

k

kkt

k

S t k

S t

t m

Processing Durations

{1,..., }

{ } {0,1}I K ik ik

I

A A A

Resource Allocation (Scheduling)

Network Dynamics

0

0

{1,..., }

{ ( ), 0}

{ ( ), 0}k

k

K

Q t t k

R t t k

0(0)kQ k

' ' ' 0'

( ) (0) ( ( )) ( ( ( ))) 0( )

( ) (0) ( ( ))

k k k k k k k kk

k

k k k k k

Q t Q S T t S T t k KZ t

R t R S T t t k K

NominalProductio

nRates

Yoni Nazarathy, University of Haifa, 2007 5

An ExampleAn Example

Yoni Nazarathy, University of Haifa, 2007 6

A Push-Pull Queueing System (Weiss, Kopzon 2002,2006)A Push-Pull Queueing System (Weiss, Kopzon 2002,2006)

1 1

22

Server 1Server 2

PUSH

PULL

1

1 1 2 1

2 2 1 1

(1 )

(1 )

PULL

PUSH

11 2

21

2 1 1

21 2 1 2

( )

1 2 21

1 2 1 2

( )

1 1 1

2 2 2

1

2

Fluid Solution:

1 1 2 2,

1 1 2 2,

or

Require Full Utilization

Require Rate Stability( )

lim 0t

Z t

t

“Inherently Stable”

“Inherently Unstable”

i Proportion of time server i allocates to “Pulling”

Yoni Nazarathy, University of Haifa, 2007 7

Maximum Pressure (Dai, Lin 2005)Maximum Pressure (Dai, Lin 2005)

•Max-Pressure is a rate stable policy (even when ρ=1).

•Push-Pull acts like a ρ=1 System.

•As Proven by Dai and Lin, Max-Pressure is rate stable.

•But for the Push-Pull system Max-Pressure is not Positive Recurrent:

Queue on Server 1

Queue on Server 2

Yoni Nazarathy, University of Haifa, 2007 8

0,0 1,0 2,0 3,0 4,0 5,0

0,1

0,2

0,3

0,4

n1

n2

1 1 1 1 1 1

1 1 1 1 1 1

2

2

2

2

2 2

2

2

2

2

Positive Recurrent Policies Exist!!!Positive Recurrent Policies Exist!!!

1 1 2 2, 1 1 2 2,

0,0

1,3

2,0 3,0 4,0 5,0

0,1

0,2

0,3

0,4

n1

n2

11 1 1 1 1

1 1

1 1 1 1

2

2

2

2

2

2,1

2,2

2,3

2

2

22

2

2

3,1

3,2

3,3

2

2

2

2

2

2

4,1

4,2

4,3

2

2

2

2

2

2

5,1

5,2

5,3

2

2

22

2

2

1 1

1,0

1,4

1

1 1

2,4

0,5

2

1,5

1

1 1

2,5

2

2

2

1

1 1

4,5

2

Kopzon, Weiss 2002

Kopzon, Weiss 2006

Yoni Nazarathy, University of Haifa, 2007 9

An ApplicationAn Application

Yoni Nazarathy, University of Haifa, 2007 10

Near Optimal Control over a Finite Time HorizonNear Optimal Control over a Finite Time Horizon

Approximation Approach:1) Approximate the problem using a fluid system.2) Solve the fluid system (SCLP).3) Track the fluid solution on-line (Using MCQN+IVQs).4) Under proper scaling, the approach is asymptotically optimal.

Approximation Approach:1) Approximate the problem using a fluid system.2) Solve the fluid system (SCLP).3) Track the fluid solution on-line (Using MCQN+IVQs).4) Under proper scaling, the approach is asymptotically optimal.

Solution is intractable3

10

( )T

kk

Min Q t dt

Finite Horizon Control of MCQN

Weiss, Nazarathy 2007

Yoni Nazarathy, University of Haifa, 2007 11

Fluid formulationFluid formulation

1 2 3

0

1 1 1

0

2 2 1 2

0 0

3 3 2 3

0 0

1 31 3

22

min ( ) ( ) ( )

( ) (0) ( )

( ) (0) ( ) ( )

( ) (0) ( ) ( )

1 1( ) ( ) 1

1( ) 1

, 0

T

t

t t

t t

q t q t q t dt

q t q u s ds

q t q u s ds u s ds

q t q u s ds u s ds

u t u t

u t

u q

(0, )t T

s.t.

This is a Separated Continuous Linear Program (SCLP)

Server 1Server 2

1

23

Yoni Nazarathy, University of Haifa, 2007 12

Fluid solutionFluid solution

•SCLP – Bellman, Anderson, Pullan, Weiss.•Piecewise linear solution. •Simplex based algorithm, finds the optimal solution in a finite number of steps (Weiss).

The Optimal Solution:

•SCLP – Bellman, Anderson, Pullan, Weiss.•Piecewise linear solution. •Simplex based algorithm, finds the optimal solution in a finite number of steps (Weiss).

The Optimal Solution:

0 10 20 30 40

0

5

10

15

20

3 3

2 2

1 1

1 3

2

(0) (0) 15

(0) (0) 1

(0) (0) 8

1.0

0.25

40

Q q

Q q

Q q

T

3( )q t

2 ( )q t

1( )q t

Yoni Nazarathy, University of Haifa, 2007 13

4 Time Intervals4 Time Intervals

For each time interval, set a MCQN with Infinite Virtual Queues.

3

1

2

3

1

2

3

1

2

3

1

2

0 10 20 30 40

5

10

15

20

25

30

0 {} {} {2} {2,3}nK

31 1 10 0 1 0 14 4 4 4

{1,2,3}{1,2,3} {1,3} {1}nK

0 { | ( ) 0, }nk nk q t t

{ | ( ) 0, }nk nk q t t

Yoni Nazarathy, University of Haifa, 2007 14

Maximum Pressure (Dai, Lin) is such a policy, even when

ρ=1Maximum Pressure (Dai, Lin) is such a policy, even when

ρ=1

Now Control the MCQN+IVQ Using a Rate Stable Policy

Yoni Nazarathy, University of Haifa, 2007 15

Example realizations, N={1,10,100}Example realizations, N={1,10,100}

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

50

100

150

200

0 10 20 30 400

50

100

150

200

0 10 20 30 400

50

100

150

200

0 10 20 30 400

50

100

150

200

0 10 20 30 400

5

10

15

20

0 10 20 30 400

5

10

15

20

0 10 20 30 400

5

10

15

20

0 10 20 30 400

5

10

15

20

1N

10N

100N

seed 1 seed 2 seed 3 seed 4 seed 1 seed 2 seed 3 seed 4

Yoni Nazarathy, University of Haifa, 2007 16

A Fundamental Question

A Fundamental Question

Yoni Nazarathy, University of Haifa, 2007 17

Is there a characterization of MCQN+IVQs that allows:

• Full Utilization of all the servers that have an IVQ.

• Stability of all finite queues.• Proportional equality among

production streams.

Is there a characterization of MCQN+IVQs that allows:

• Full Utilization of all the servers that have an IVQ.

• Stability of all finite queues.• Proportional equality among

production streams.

?

Yoni Nazarathy, University of Haifa, 2007 18

ThankYou

ThankYou