1

Using Hierarchical Using Hierarchical Scheduling to Support Soft Scheduling to Support Soft Real-Time Applications in Real-Time Applications in

General-Purpose General-Purpose Operating SystemsOperating Systems

John Regehr

March 20, 2001

2

OutlineOutline

Motivation and Approach Guarantees HLS Design Augmented Reservations Conclusions and Demo

3

Overview of ContributionsOverview of Contributions A general hierarchy of soft real-time

schedulers can provide guaranteed scheduling behavior

The Hierarchical Loadable Scheduler (HLS) architecture… Can be efficiently implemented in a general-

purpose OS Increases application performance

compared to case where scheduler and apps are mismatched

Augmented CPU reservations increase predictability when the OS steals CPU time from real-time applications

4

MotivationMotivation People use general-purpose OSs

(GPOSs) for many kinds of tasks e.g. Unix, Windows, MacOS variants Compatibility, commodity, convenience

Applications have diverse scheduling requirements Time-sharing Soft real-time Isolation Co-scheduling between processors or

machines

5

The ProblemThe Problem

One size does not fit all Supporting evidence: companies sell

scheduler extensions Ensim: ServerXchange Sun: Solaris Resource Manager Aurema: Active Resource Management TimeSys: Linux/RT

6

Motivating Data: Unfair CPU Motivating Data: Unfair CPU Allocation Between UsersAllocation Between Users

User 1 gets little CPU time when User 2 creates many threads

N1 %100 N2 %2

1 50.000 1 50.01 19.900 4 80.11 6.100 16 93.91 1.560 64 98.41 0.313 256 99.7

7

ApproachApproach

Allow a hierarchy of CPU schedulers to control processor allocation

Thesis Statement: It is useful and feasible to extend a

GPOS with a general, heterogeneous scheduling hierarchy.

A heterogeneous hierarchy employs different schedulers

A general hierarchy does not impose a fixed scheduling model

8

Example HierarchyExample Hierarchy

FP

VoiceRecognition

RES

J

VideoPlayer

WordProcessor

SFQ

H

LFP: Fixed Priority

RES: CPU Reservation

J: Join

SFQ: Start-Time Fair

Queuing

9

Time ScalesTime Scales Long:

System is used in a particular way Duration: usually uptime of a system or

longer Medium:

Applications start, end, and change requirements

Duration: seconds, minutes, or longer Short:

Individual scheduling decisions made by schedulers within the hierarchy

Duration: milliseconds or 10s of ms

10

What Microsoft Should DoWhat Microsoft Should Do

Put HLS into consumer Windows! By default

Support interactive, batch, and multimedia applications for a single user

However, also include Library of useful schedulers and API

for composing them API for implementing new schedulers

11

ReminderReminder

HLS is an architecture for flexibly composing schedulers

12

OutlineOutline

Motivation and Approach Guarantees HLS Design Augmented Reservations Conclusions and Demo

13

GuaranteeGuarantee Definition:

Ongoing lower (and possibly upper) bound on CPU allocation

Goals: Describe the scheduling behavior

required and provided by schedulers, and required by applications

Permit these behaviors to be reasoned about

Syntax: TYPE p1 p2 …

14

Example GuaranteesExample Guarantees 100% of a CPU:

ALL Strictly best-effort scheduling:

NULL Proportional share:

PS s PSBE s δ

CPU Reservations: RESBS x y, RESBH x y RESCS x y, RESCH x y

15

CPU Reservation GuaranteesCPU Reservation Guarantees Hard / Soft:

“Hard CPU reservation” ≠ hard real-time Soft reservations guarantee a lower bound Hard reservations also guarantee an upper

bound Basic / Continuous:

16

Guarantee Conversion Guarantee Conversion by Schedulersby Schedulers

Schedulers require and provide guarantees SFQ: PSBE → PSBE+

Rez: ALL → RESBH+

Schedulers determine if specific guarantees can be provided ALL → RESBH 5 10, RESBH 25 100

EDF-based reservation schedulerX Naïve rate monotonic reservation

scheduler

17

Selected Conversions by Selected Conversions by SchedulersSchedulers

Full table contains 23 schedulers

Scheduler ConversionsFixed Priority any → any, NULL+

SFQ PSBE → PSBE+, PS → PS+

EEVDF ALL → PSBE+

Lottery, Stride PS → PS+

Rialto, Rialto/NT ALL → RESCS+

Rez, CBS ALL → RESBH+

Linux/RT ALL → RESBS+, RESBH+

Time Sharing NULL → NULL+

18

Guarantee Conversion Guarantee Conversion by Rewrite Rulesby Rewrite Rules

Guarantees can be converted without a scheduler, using rewrite rules

Trivial examples: PSBE s δ → PS s RESBH x y → RESBS x y

Non-trivial examples: RESBS x y → RESCS x (2y-x+c) for

any c ≥ 0 RESCS x y → PSBE (x/y) x/y (y-x)

19

Partial Rewrite Rule TablePartial Rewrite Rule Table

ALL T F T F T T T T

RESBH F T T F T T T T

RESBS F T T T T T T T

RESCH F T T T T T T T

RESCS F F T F T T T T

PSBE F F T F T T T T

PS F F F F F F T T

NULL F F F F F F F T

→ ALL RESBH RESBS RESCH RESCS PSBE PS NULL

T = rewrite rule existsF = rewrite rule does not exist

20

Another View of Another View of Some Rewrite Rules Some Rewrite Rules

RESCH

RESBH RESBS PSBE

PSRESCS

21

Guarantees in ActionGuarantees in Action

FP

RES

J

VideoPlayer

VoiceRecognition

SFQ

ALL

NULL

ALL

RESBH 5 33 *RESBH 10 20*

RESBS 10 20 →

WordProcessor

PSBE 0.1 15 PSBE 0.3 35

PSBE 0.5 10

22

Related Work for Related Work for Hierarchical GuaranteesHierarchical Guarantees

Hierarchical start-time fair queuing [Goyal et al. 96]

Uniformly slower processors Open environment for real-time

applications [Deng et al. 99] BSS-I and PShED [Lipari et al. 00]

23

Guarantee ContributionsGuarantee Contributions Created a framework for reasoning

about composition of schedulers Derived rewrite rules Integrated more than 20 schedulers

Guarantees provide a model of soft real-time CPU allocation Independent of particular scheduling

algorithms Developers can program to Users can ensure that application

requirements are met

24

OutlineOutline

Motivation and Approach Guarantees HLS Design Augmented Reservations Conclusions and Demo

25

Inside a Non-Hierarchical Inside a Non-Hierarchical CPU SchedulerCPU Scheduler

Scheduling decisions are made in response to timers and thread state changes

T1 T2 T3 T4 T5

CPU

WW R W

SCHEDULER

26

Inside a Hierarchical Inside a Hierarchical CPU SchedulerCPU Scheduler

Distinguishing feature of hierarchical schedulers: revocation

VP2 VP3 VP4 VP5 VP6

VP1

WW R W

R

R

SCHEDULER

child virtualprocessors

parent virtualprocessor

27

Hierarchical Scheduler Hierarchical Scheduler Infrastructure (HSI) DesignInfrastructure (HSI) Design

Schedulers are implemented by code libraries loaded into the kernel

Scheduler instances are active entities in the hierarchy

Virtual Processors Represent potential for physical

processor allocation Used to notify schedulers

28

Scheduler NotificationsScheduler Notifications Hierarchical infrastructure notifies

a scheduler whenever: A child VP requests or releases a

processor A parent VP grants or revokes a

processor A timer expires

Threads implicitly notify schedulers when they block and unblock

Notifications are cheap: virtual function calls

29

HSI and Scheduler HSI and Scheduler ImplementationImplementation

HSI runs in Windows 2000 kernel Serialized by a spinlock Added ~3100 lines of code

Loadable schedulers: Time sharing / fixed priority, CPU

reservation, proportional share, join A representative set of schedulers,

but not a complete one Implemented Rez in about two days,

PS scheduler in a few hours

30

PerformancePerformance Test machine is a 500MHz Pentium

III Most mode change operations run in

less than 40μs Create / destroy scheduler instance, begin

/ end CPU reservation, etc. Median context switch time

Unmodified Windows 2000: 7.1μs HLS time-sharing scheduler: 11.7μs

Cost of each extra level in the scheduling hierarchy is 0.96μs

Many opportunities for optimization

31

Performance Data:Performance Data:Isolating Resource PrincipalsIsolating Resource Principals

N1 %100 N2 %2

Without Isolation (1-level)

1 50.000 1 50.01 19.900 4 80.11 6.100 16 93.91 1.560 64 98.41 0.313 256 99.7

With Isolation(2-level)

1 50.000 1 50.01 50.000 4 50.01 50.100 16 49.91 50.100 64 49.91 50.200 256 49.8

32

Scheduling a Real-Time, Scheduling a Real-Time, CPU-Bound ApplicationCPU-Bound Application

Synthetic test application Represents a virtual environment or game Must provide 1 frame per 33ms (~30 FPS)

10ms of CPU time for each frame More frames are acceptable and desirable

Machine also runs background work Goals

Application should never miss a deadline Non-real-time work should make progress

33

Performance Data: CPU Performance Data: CPU Bound Real-Time ApplicationBound Real-Time Application

30-second runs

App. Guarantee %aFPS Misses %b

NULL (high pri.) 96.70 96.70 6 3.26NULL (def. pri.) 49.90 49.90 290 50.00NULL (low pri.) 3.11 3.11 985 96.90RESBH 10 33 32.60 32.60 0 67.30RESBS 10 33 67.30 67.30 0 32.60

34

Related Work for HSIRelated Work for HSI

Scheduler activations [Anderson et al. 91]

CPU inheritance scheduling [Ford and Susarla 96]

Vassal [Candea and Jones 98]

35

ContributionsContributions

Virtual processors are a novel extension of scheduler activations [Anderson et al. 91]

Permit a wide range of schedulers to dynamically create a scheduling hierarchy in kernel of a GPOS First system to do this

Simplifies scheduling programming model by hiding OS and hardware complexities

36

OutlineOutline

Motivation and Approach Guarantees HLS Design Augmented Reservations Conclusions and Demo

37

Problem: Stolen TimeProblem: Stolen Time We have assumed until now that the

OS does not interfere with guarantees However, while processing

asynchronous data the OS may steal CPU time from applications

Time used by bottom-half mechanisms is not accounted for correctly

A solution: Augmented CPU reservations

38

Time Stolen by Network Time Stolen by Network Receive ProcessingReceive Processing

39

Augmented ReservationsAugmented Reservations

Strategy: accurately measure stolen time in order to compensate threads Rez-C: avoid “charging” threads for

stolen time Rez-FB: use a feedback loop to assign a

larger CPU reservation so requirements are met even when time is stolen

Implemented as HLS schedulers

40

Augmented Reservation Augmented Reservation PerformancePerformance

42

Related Work for Related Work for Augmented ReservationsAugmented Reservations

Scheduling bottom-half activity Mach [Rashid et al. 89] Nemesis [Leslie et al. 96]

Scheduling bottom-half activity in a GPOS FreeBSD [Jeffay et al. 98]

Feedback-based scheduling FC-EDF [Lu et al. 99]

Moving code into scheduled contexts Soft modems [Jones and Saroiu 01]

43

Augmented Reservation Augmented Reservation ContributionsContributions

Rez-C and Rez-FB 6% over-reservation to eliminate most

deadline misses vs. 24% over-reservation for plain Rez

Quantified severity of stolen time Windows 2000 + Rez and Linux/RT Network, disk, software modem, USB

44

OutlineOutline

Motivation and Approach Guarantees HLS Design Augmented Reservations Conclusions and Demo

45

ConclusionsConclusions HLS is feasible:

Composition of soft real-time schedulers can be reasoned about

Implemented and performs well HLS is useful:

Permits scheduling behavior to be tailored to specific needs

New schedulers can be implemented more easily

46

HLS DemonstrationHLS Demonstration

1. All threads scheduled by default time-sharing scheduler

2. Create a CPU reservation

3. Make proportional share scheduler the default and move time-sharing threads

4. End CPU reservation

PSTSRES

FP

LH L

TTTTTT

1. All threads scheduled by default time-sharing scheduler

TTTTTTT

1. All threads scheduled by default time-sharing scheduler

2. Create a CPU reservation

3. Make proportional share scheduler the default and move time-sharing threads

4. End CPU reservation

1. All threads scheduled by default time-sharing scheduler

2. Create a CPU reservation

3. Make proportional share scheduler the default and move time-sharing threads

4. End CPU reservation

TTTTTT TTTTTT

1. All threads scheduled by default time-sharing scheduler

2. Create a CPU reservation

3. Make proportional share scheduler the default and move time-sharing threads

4. End CPU reservation

T TTTTTT

47

The EndThe End

Papers and more info at: http://www.cs.virginia.edu/~jdr8d