using real-time scheduling principles in web service clusters to achieve predictability of execution

Using Real-Time Scheduling Principles in Web

Service Clusters to Achieve Predictability of

Service Execution

Vidura Gamini AbhayaProf. Zahir Tari and Assoc. Prof. Peter Bertok

Distributed Systems and Networking GroupSchool of Computer Science and IT

RMIT UniversityMelbourne, Australia

December 8, 2010

Intro. Prev. Work Solution Results Conclusion

Presentation Structure

1 Introduction

2 Previous Work

3 Solution

4 Empirical Results

5 Conclusion

V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings

Intro. Prev. Work Solution Results Conclusion Motivation

Motivation

Web Services enable communication between DistributedHeterogeneous Systems.



Motivation

Web Services enable communication between DistributedHeterogeneous Systems. It is the foundation for...



Motivation



Motivation

Scalability Issues...

Quality of Service ?

Performance ?

Predictability ?


Intro. Prev. Work Solution Results Conclusion Single Host

1 Introduction

2 Previous Work

3 Solution

4 Empirical Results

5 Conclusion



Our Previous Work [5,6,*]...

Achievement

Achieve Predictability and Service Differentiation in WS execution(in a single server)

* Building Web Service Middleware with Predictable Service Execution, WISE 2010, LNCS, vol. 6488 pp. 23-37.



Our Previous Work [5,6,*]...

Achievement

Achieve Predictability and Service Differentiation in WS execution(in a single server)

Highlights...

Introduced the notion of a deadline

Use Real-time scheduling principles

Select tasks with achievable deadlines, reject the rest

Selected tasks scheduled in the order of increasing deadlines

Implementation based on Apache Axis2

Tested for various request size, arrival rate combinations

* Building Web Service Middleware with Predictable Service Execution, WISE 2010, LNCS, vol. 6488 pp. 23-37.



Task Execution with a Deadline



Task Execution with a Deadline

Laxity of a Task

Laxity =Deadline

ExecutionTime



Single Host Implementation

Task selection based on a ‘Schedulability Check’

Deadline based scheduling

Implementation on a single server



Deadline based Task Schedule

EDF based task schedule

Larger laxities allow requests to be executed in staggeredmanner

Enable timely completion of requests with earlier deadlines


Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details

1 Introduction

2 Previous Work

3 Solution

4 Empirical Results

5 Conclusion



Solution Overview



Solution Overview

Sun Real-time Java Specification used asdevelopment platform

Sun Solaris 10 08/05 used as RT-OS



Solution Overview



Apache Synapse based dispatcher implementation

Apache Axis2 based executor implementation



Solution Overview



Apache Synapse based dispatcher implementation

Apache Axis2 based executor implementation

All Thread-pools replaced with custom-designedReal-time Thread pools

Introduction of Real-time scheduler component



Cluster Design

Request processing at the dispatcher

Requests are dispatched based on algorithm used

Requests are ‘mostly’ queued at dispatcher

Results are sent back to clients via dispatcher



Dispatching Algorithms

Objectives

*Ensures deadline requirement of tasks could be met*Distribute requests evenly among executors based on a condition*Avoids executors going into overload conditions

4 algorithms used

RT-RoundRobin

RT-Sequential

RT-ClassBased

RT-LaxityBased



RT-RoundRobin

Details

Requests assignment cyclesthrough all executors in RRfashion

Sched. check is done only onceper request

If sched. check fails on selectedserver → request rejected



RT-RoundRobin

Details

Requests assignment cyclesthrough all executors in RRfashion

Sched. check is done only onceper request

If sched. check fails on selectedserver → request rejected

Hightlights

POC for how a simple dispatching algorithm could be made real-time ready

Processing overhead is minimum



RT-Sequential

Details

Requests are assigned in asequential manner

Req’s sent to one executor tillsched. check fails, then assignedto the second and so on

Sched. check happens againstmultiple executors till a requestcan be assigned



RT-Sequential

Details

Requests are assigned in asequential manner

Req’s sent to one executor tillsched. check fails, then assignedto the second and so on

Sched. check happens againstmultiple executors till a requestcan be assigned

Highlights

If it’s possible to schedule a job within the cluster, it will be guaranteed



RT-ClassBased

Details

Requests are divided into classesbased on a priority scheme

Mapping of requests to executorsis pre-determined and definedoffline

Sched. check is only consideredwith the assigned executor

Reference implementation usespriorities based on task sizes



RT-ClassBased

Details

Requests are divided into classesbased on a priority scheme

Mapping of requests to executorsis pre-determined and definedoffline

Sched. check is only consideredwith the assigned executor

Reference implementation usespriorities based on task sizes

Highlights

Results in the reduction of variance of task sizes at each executor

Pre-defined mapping of request sizes to executors could be done usingpre-profiled execution times or execution time history



RT-LaxityBased

Details

Laxity = Ability of delaying arequest whilst still meeting itsdeadline

Higher the laxity the morerequests an executor could service

Sched. check is done only againstthe assigned executor



RT-LaxityBased

Details

Laxity = Ability of delaying arequest whilst still meeting itsdeadline

Higher the laxity the morerequests an executor could service

Sched. check is done only againstthe assigned executor

Highlights

Distribution of requests results in a broad range of laxities at each executor

Keeps track of the last two laxities assigned and prevents them being assignedfor the same executor consecutively


Intro. Prev. Work Solution Results Conclusion RT-RR RT-CB Others

1 Introduction

2 Previous Work

3 Solution

4 Empirical Results

5 Conclusion



RT-RoundRobin vs Round-Robin

10

20

30

40

50

60

70

80

90

100

0.05s â�� 0.1s 0.1s â�� 0.25s 0.1s - 0.5s 0.25s - 1s

Per

cent

age

of d

eadl

ines

met

Inter-arrival times

Comparison of Round-robin vs. RT-RoundRobin - Deadline Achievement Rates

RR - 2 Execs.RR - 3 Execs.RR - 4 Execs.

RT-RR - 2 Execs.RT-RR - 3 Execs.RT-RR - 4 Execs.

RT-RR accepts between 20.5% (2 Exec. - fastest) and 99.9% (4 Exec. -slowest) of the requests.



RT-RoundRobin vs Round-Robin

10

20

30

40

50

60

70

80

90

100

0.05s â�� 0.1s 0.1s â�� 0.25s 0.1s - 0.5s 0.25s - 1s

Per

cent

age

of d

eadl

ines

met

Inter-arrival times

Comparison of Round-robin vs. RT-RoundRobin - Deadline Achievement Rates

RR - 2 Execs.RR - 3 Execs.RR - 4 Execs.

RT-RR - 2 Execs.RT-RR - 3 Execs.RT-RR - 4 Execs.

0

2000

4000

6000

8000

10000

12000

0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06

Exe

cutio

n T

ime

(ms)

Task Size

Task Size distribution between 3 Executors (1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform) roundrobin 3 exec)

Executor 1Executor 2Executor 3



RT-ClassBased vs Class Based

10

20

30

40

50

60

70

80

90

100

0.05 - 0.1 0.1 - 0.25 0.1 - 0.5 0.25 - 1

Per

cent

age

of d

eadl

ines

met

Inter-arrival times

Comparison of Class-based vs. RT-CB - Deadline Achievement Rates

CB - 2 Execs.CB - 3 Execs.CB - 4 Execs.

RT-CB - 2 Execs.RT-CB - 3 Execs.RT-CB - 4 Execs.

RT-CB accepts between 28.6% (2 Exec. - fastest) and 100% (4 Exec. - slowest)of the requests.



RT-ClassBased vs Class Based

10

20

30

40

50

60

70

80

90

100

0.05 - 0.1 0.1 - 0.25 0.1 - 0.5 0.25 - 1

Per

cent

age

of d

eadl

ines

met

Inter-arrival times

Comparison of Class-based vs. RT-CB - Deadline Achievement Rates

CB - 2 Execs.CB - 3 Execs.CB - 4 Execs.

RT-CB - 2 Execs.RT-CB - 3 Execs.RT-CB - 4 Execs.

0

10

20

30

40

50

60

70

80

90

0 50 100 150 200 250 300 350 400

CP

U U

tiliz

atio

n %

Sample #

CPU utilisation at each Executor - (1 - 5000000 (Uniform); 0.25sec - 1sec (Uniform) classbased 3 exec)


0

2000

4000

6000

8000

10000

12000

14000

16000

0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06

Exe

cutio

n T

ime

(ms)

Task Size

Task Size distribution between 3 Executors (1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform) classbased 3 exec)




RT-Sequential and RT-Laxity

10

20

30

40

50

60

70

80

90

100

0.05s â�� 0.1s 0.1s â�� 0.25s 0.1s - 0.5s 0.25s - 1s

Ave

rage

d pe

rcen

tage

of d

eadl

ines

met

Inter-arrival times

Comparison Deadline Achievement Rates - Averaged by Algorithm

RT-RRRT-CB

RT-SeqRT-Lax

Highest acceptance rate for any algorithm for 2 executors with highest arrivalrate is 38.5% for RT-LaxityBased



RT-Sequential and RT-Laxity

10

20

30

40

50

60

70

80

90

100

0.05s â�� 0.1s 0.1s â�� 0.25s 0.1s - 0.5s 0.25s - 1s

Ave

rage

d pe

rcen

tage

of d

eadl

ines

met

Inter-arrival times

Comparison Deadline Achievement Rates - Averaged by Algorithm

RT-RRRT-CB

RT-SeqRT-Lax

0

10

20

30

40

50

60

0 200 400 600 800 1000 1200

CP

U U

tiliz

atio

n %

Sample #

CPU utilisation at each Executor - (1 - 5000000 (Uniform); 0.25sec - 1sec (Uniform) laxitybased 3 exec)



Intro. Prev. Work Solution Results Conclusion Future Work Recap

1 Introduction

2 Previous Work

3 Solution

4 Empirical Results

5 Conclusion



Future Work

Further analysis of cluster behaviour with mix of tasks

Advanced dispatching and re-dispatching models

Predictability in the networking layer



Conclusion

Previous work - We achieved predictability of execution on asingle server

We extend it to achieve predicability in a cluster of servers

Presented four task dispatching algorithms

Evaluated their performance empirically



Thank You !

Questions or Comments ?


using real-time scheduling principles in web service clusters to achieve predictability of execution

Documents