scheduling in linux and web servers
DESCRIPTION
University of Virginia cs4414: Operating Systems http://rust-class.org Scheduling in Linux, 2002-2014 Energy and Scheduling OSX Mavericks Timer Coalescing Scheduling Web Servers Healthcare.gov For embedded notes, see: http://rust-class.org/class-12-scheduling-in-linux-and-web-servers.htmlTRANSCRIPT
![Page 1: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/1.jpg)
cs4414 Fall 2013David Evans
Class 12
Scheduling in Linux and Web Servers
![Page 2: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/2.jpg)
2
Plan for TodayScheduling in Linux (2002-today)Scheduling Web Services
Submitting PS3:- Schedule demo (sign up soon!)- Web submission form (11:59pm tomorrow)- Benchmark submission- Post-demo assessment (teammate evaluation)
leaderboard.html
![Page 3: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/3.jpg)
3
Schedulingin Linux
![Page 4: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/4.jpg)
4
Linux Scheduler before V2.6 (2002)Three types of processes:
#define SCHED_OTHER 0#define SCHED_FIFO 1#define SCHED_RR 2
Not (fully) pre-emptive: only user-level processes could be pre-empted
Select next process according to “goodness” function
Normal user processesNon-pre-ementable
Real-time round-robin
![Page 5: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/5.jpg)
5
/* linux/kernel/sched.c* This is the function that decides how desirable a process is.* You can weigh different processes against each other depending * on what CPU they've run on lately etc to try to handle cache * and TLB miss penalties. * * Return values: * -1000: never select this * 0: out of time, recalculate counters (but it might still be selected) * +ve: "goodness" value (the larger, the better) * +1000: realtime process, select this. */static inline int goodness(struct task_struct * p, int this_cpu, structmm_struct *this_mm){ int weight; /* * Realtime process, select the first one on the * runqueue (taking priorities within processes * into account). */ if (p->policy != SCHED_OTHER) { weight = 1000 + p->rt_priority; goto out; } /* * Give the process a first-approximation goodness value * according to the number of clock-ticks it has left. * * Don't do any other calculations if the time slice is * over.. */ weight = p->counter; if (!weight) goto out;
#ifdef __SMP__ /* Give a largish advantage to the same processor... */ /* (this is equivalent to penalizing other processors) */ if (p->processor == this_cpu) weight += PROC_CHANGE_PENALTY;#endif /* .. and a slight advantage to the current MM */ if (p->mm == this_mm) weight += 1; weight += p->priority;out: return weight;}
/* linux/kernel/sched.c* This is the function that decides how desirable a process is.* You can weigh different processes against each other depending * on what CPU they've run on lately etc to try to handle cache * and TLB miss penalties. * * Return values: * -1000: never select this * 0: out of time, recalculate counters (but it might still be selected) * +ve: "goodness" value (the larger, the better) * +1000: realtime process, select this. */static inline int goodness(struct task_struct * p, int this_cpu, struct mm_struct *this_mm){ …
![Page 6: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/6.jpg)
6
static inline int goodness(struct task_struct * p, int this_cpu, struct mm_struct *this_mm){ int weight; /* Realtime process, select the first one on the runqueue (taking priorities into account). */ if (p->policy != SCHED_OTHER) { weight = 1000 + p->rt_priority; goto out; } /* Give the process a first-approximation goodness value according to the number of clock-ticks it has left. Don't do any other calculations if the time slice is over.. */ weight = p->counter; if (!weight) goto out;#ifdef __SMP__ /* Give a largish advantage to the same processor... (equivalent to penalizing other processors) */ if (p->processor == this_cpu) weight += PROC_CHANGE_PENALTY;#endif /* .. and a slight advantage to the current MM (memory segment) */ if (p->mm == this_mm) weight += 1; weight += p->priority;out: return weight;}
This is the whole goodness function from V2.5 scheduler (only edited formatting to fit on slide).
![Page 7: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/7.jpg)
7
What is the running time of the Linux 2.2-2.5 Scheduler?
![Page 8: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/8.jpg)
8
What is the running time of
the Linux 2.2-2.5 Scheduler?
![Page 9: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/9.jpg)
9
![Page 10: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/10.jpg)
10
Linux 2.6 Scheduler (2003-2007)140 different queues (for each processor)
0-99 for “real time” processes100-139 for “normal” processes
Bit vector keeps track of which queues have ready to run processScheduler picks first process from highest priority queue with a ready process
Given time quantum that scales with priority
![Page 11: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/11.jpg)
11
Linux 2.6 Scheduler (2003-2007)
140 different queues (for each processor)
0-99 for “real time” processes100-139 for “normal” processes
Bit vector of ready-to-run
struct runqueue { struct prioarray *active; struct prioarray *expired; struct prioarray arrays[2];};struct prioarray { int nr_active; /* # Runnable */ unsigned long bitmap[5]; struct list_head queue[140];};
Scheduler picks first process from highest-priority queue with a ready process
![Page 12: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/12.jpg)
12
What is the running time of the Linux 2.6 Scheduler?
![Page 13: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/13.jpg)
13
(Sadly, O(1) scheduler has no Facebook page.)
![Page 14: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/14.jpg)
14
Linux V2.6.23+ Scheduler
![Page 15: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/15.jpg)
15
This is exactly stride scheduling (but with different terminology)!
Rotating Staircase Deadline Scheduler
![Page 16: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/16.jpg)
16
![Page 17: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/17.jpg)
17
![Page 18: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/18.jpg)
18
![Page 19: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/19.jpg)
19
Linu
x/ke
rnel
/sch
ed/f
air.c
![Page 20: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/20.jpg)
20
What is the running time of the Linux 2.6.23+ Scheduler?
Not called the (log θ N) scheduler – by Linux 2.6.23 marketingmatters: “Completely Fair Scheduler”
![Page 21: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/21.jpg)
21
(In practice) What is log2 N?
![Page 22: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/22.jpg)
22
What resources should scheduler be maximizing utility of?
![Page 23: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/23.jpg)
23
Key Resource: Energy!
Image from http://arstechnica.com/apple/2013/10/os-x-10-9/12/
![Page 24: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/24.jpg)
24
Image from http://arstechnica.com/apple/2013/10/os-x-10-9/12/
![Page 25: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/25.jpg)
25
Image from http://arstechnica.com/apple/2013/10/os-x-10-9/12/
![Page 26: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/26.jpg)
26
Image from http://arstechnica.com/apple/2013/10/os-x-10-9/12/
![Page 27: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/27.jpg)
27
Timer Coalescing
Images from http://arstechnica.com/apple/2013/06/how-os-x-mavericks-works-its-power-saving-magic/
![Page 28: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/28.jpg)
28
OS Schedulers RecapUse Resources Well
Limit unnecessary switching, Save Energy Low cost of scheduler itself
Make good decisionsLocally: pick the most important processGlobally: provide good system performance
![Page 29: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/29.jpg)
29
Scheduling Web Servers
![Page 30: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/30.jpg)
30
Web Server Overload!
healthcare.gov
Rate of incoming requests > Rate server can process requests
![Page 31: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/31.jpg)
31
Solutions
![Page 32: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/32.jpg)
32
Strategy 0:Measure
![Page 33: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/33.jpg)
33
“When the meetings ended at a CMS outpost in Herndon, Va., at about 7:00 p.m., the rescue squad already on the scene realized they had more work to do. One of the things that shocked Burt and Park’s team most—“among many jaw-dropping aspects of what we found,” as one put it—was that the people running HealthCare.gov had no “dashboard,” no quick way for engineers to measure what was going on at the website, such as how many people were using it, what the response times were for various click-throughs and where traffic was getting tied up. So late into the night of Oct. 18, Burt and the others spent about five hours coding and putting up a dashboard.”
![Page 34: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/34.jpg)
34
Developer Benchmarks• Find bottlenecks: know what to spend time
optimizing• Measure impact of changes• Predict what resources you will need to scale
service
Goal is a benchmark that represents the actual usage
![Page 35: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/35.jpg)
35
Strategy 1:Shrink and Simplify Your Content
![Page 36: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/36.jpg)
36
5 September 2001 11 September 2001
archive.org captures of New York Times (http://www.nytimes.com)
![Page 37: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/37.jpg)
37
![Page 38: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/38.jpg)
38
5 September 2001
11 September 2001
![Page 39: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/39.jpg)
39
Strategy 2:Cache to Save Effort
![Page 40: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/40.jpg)
40
Nor
vig
Num
bers
(200
1)
![Page 41: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/41.jpg)
41
“Looking over the dashboard that Park, Burt and the others had rigged up the prior Friday night, Abbott and the group discovered what they thought was the lowest-hanging fruit--a quick fix to an obvious mistake that could improve things immediately. HealthCare.gov had been constructed so that every time a user had to get information from the website's vast database, the website had to make what's called a query into that database. … The team began almost immediately to cache the data. The result was encouraging: the site's overall response time--the time it took a page to load--dropped on the evening of Oct. 22 from eight seconds to two. That was still terrible, of course, but it represented such an improvement that it cheered the engineers. They could see that HealthCare.gov could be saved instead of scrapped.”
![Page 42: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/42.jpg)
42
Strategy 3:Buy (or Rent) More Servers
![Page 43: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/43.jpg)
43
Amazon’s Elastic
Compute Cloud
(EC2)
![Page 44: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/44.jpg)
44
![Page 45: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/45.jpg)
45
![Page 46: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/46.jpg)
46
“A series of hardware upgrades had dramatically increased capacity; the system was now able to handle at least 50,000 simultaneous users and probably more. There had been more than 400 bug fixes. Uptimes had gone from an abysmal 43% at the beginning of November to 95%. And Kim and her team had knocked the error rate from 6% down to 0.5%. (By the end of January it would be below 0.5% and still dropping.)”
![Page 47: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/47.jpg)
47
Using More Servers
Dispatcher
Server 1
Server 2
Server 3
![Page 48: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/48.jpg)
48
Sharing State
Dispatcher
Server 1
Server 2
Server 3
Database
![Page 49: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/49.jpg)
49
Distributed Database
Dispatcher
Server 1
Server 2
Server 3
Database
Database
Database
Database
![Page 50: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/50.jpg)
50
Maintaining Consistency
Dispatcher
Server 1
Server 2
Server 3
Database
Database
Database
Database
![Page 51: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/51.jpg)
51
Dispatcher
Server 1
Server 2
Server 3
Database
Database
Database
Database
1. ReplicationReads are efficientWrites are complex and risky
2. Vertical PartitioningSplit database by columns
3. Horizontal Partitioning (“Sharding”)Split database by rows
4. Give up on consistency and functionality“NoSQL” (e.g., Cassandra, MongoDB, BigTable)
![Page 52: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/52.jpg)
52
Scalable Enough?
Dispatcher
Server 1
Server 2
Server 3
Database
Database
Database
Database
![Page 53: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/53.jpg)
53
Distributed Denial-of-Service
Dispatcher
Server 1
Server 2
Server 3
Database
Database
Database
DatabaseBotnetx 2000 machines
![Page 54: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/54.jpg)
54
![Page 55: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/55.jpg)
55
Example DDOS Attacks
![Page 56: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/56.jpg)
56
Strategy 4:Smarter Scheduling
![Page 57: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/57.jpg)
57
What should the server’s goal be?
![Page 58: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/58.jpg)
58
What is the bottleneck resource?
Zhtta Disk (files)
Cache
![Page 59: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/59.jpg)
59
Connecting to the Network
zhtta
Disk (files)Cache
ISP Router
![Page 60: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/60.jpg)
60
Your server250 Mbits/s$20/month
Cisco Nexus 7000 (~$100K) 48 Gb/s per slot x 10
10 Gb/s x 4 per switch
![Page 61: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/61.jpg)
61
Shortest Remaining Processing Time-first
![Page 62: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/62.jpg)
62
How close to this can you get for PS3?
![Page 63: Scheduling in Linux and Web Servers](https://reader033.vdocuments.net/reader033/viewer/2022061116/5465699eaf7959c04e8b4e58/html5/thumbnails/63.jpg)
63
ChargeMeasurement (“dashboard”) is essential for improving performance
Important to measure the right things!
Scheduling policies:Avoid wasting resourcesMake trade-offs that align with system goals
PS3 Due tomorrow (Wednesday) at 11:59pmIf you haven’t already scheduled your demo, do so now!