![Page 1: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/1.jpg)
Introduction to Multiprocessor
Synchronization
Maurice Herlihy
![Page 2: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/2.jpg)
Art of Multiprocessor Programming 2
Moore's Law
Clock speed flattening sharply
Transistor count still
rising
![Page 3: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/3.jpg)
Moore's Law (in practice)
Art of Multiprocessor Programming 3
![Page 4: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/4.jpg)
Art of Multiprocessor Programming 4
Nearly Extinct: the Uniprocesor
memory
cpu
![Page 5: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/5.jpg)
Art of Multiprocessor Programming 5
Endangered: The Shared Memory Multiprocessor
(SMP)
cache
BusBus
shared memory
cachecache
![Page 6: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/6.jpg)
Art of Multiprocessor Programming 6
The New Boss: The Multicore Processor
(CMP)
cache
BusBus
shared memory
cachecacheAll on the same chip
Oracle NiagaraChip
![Page 7: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/7.jpg)
Art of Multiprocessor Programming 8
Why is Kunle Smiling?
Niagara 1
![Page 8: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/8.jpg)
Art of Multiprocessor Programming 10
Traditional Scaling Process
User code
TraditionalUniprocessor
Speedup1.8x
7x
3.6x
Time: Moore's law
![Page 9: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/9.jpg)
Ideal Multicore Scaling Process
Art of Multiprocessor Programming 11
User code
Multicore
Speedup 1.8x
7x
3.6x
Unfortunately, not so simple…
![Page 10: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/10.jpg)
Actual Multicore Scaling Process
Art of Multiprocessor Programming 12
1.8x 2x 2.9x
User code
Multicore
Speedup
Parallelization and Synchronization require great care…
![Page 11: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/11.jpg)
Art of Multiprocessor Programming 13
Multicore Programming:Course Overview
• Fundamentals–Models, algorithms, impossibility
• Real-World programming–Architectures–Techniques
![Page 12: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/12.jpg)
Art of Multiprocessor Programming 14
Multicore Programming:Logistics
Staff: • Professor: Nir Shavit - 32-G622
[email protected]• TA: Will Hasenplaugh - 32-G770
[email protected]• Grader: Alex Matveev
![Page 13: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/13.jpg)
Art of Multiprocessor Programming 15
Multicore Programming:Logistics
Grades: • Final Exam: 65%• Assignments: 30%• Class / Recitation Participation: 5%
![Page 14: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/14.jpg)
Art of Multiprocessor Programming 16
Multicore Programming:Logistics
Homework: • 7 Assignments • Latter 5 include programming• Gradually design and test more complex
programs and more powerful primitives – such as transactional memory
• Lead up to final “firewall” design project
![Page 15: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/15.jpg)
Art of Multiprocessor Programming 17
Multicore Programming:Hardware
The beast: Dell PowerEdge R910
4 chips x 10 cores x 2 hardware threads
![Page 16: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/16.jpg)
Art of Multiprocessor Programming 18
Sequential Computation
memory
object object
thread
![Page 17: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/17.jpg)
Art of Multiprocessor Programming 19
Concurrent Computation
memory
object object
thre
ads
![Page 18: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/18.jpg)
Art of Multiprocessor Programming 20
Asynchrony
• Sudden unpredictable delays– Cache misses (short)– Page faults (long)– Scheduling quantum used up (really long)
![Page 19: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/19.jpg)
Art of Multiprocessor Programming 21
Model Summary
• Multiple threads• Single shared memory• Objects live in memory• Unpredictable asynchronous delays
![Page 20: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/20.jpg)
22
Road Map
• We are going to focus on principles first, then practice– Start with idealized models– Look at simplistic problems– Emphasize correctness over pragmatism– “Correctness may be theoretical, but
incorrectness has practical impact”
Art of Multiprocessor Programming
![Page 21: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/21.jpg)
23
Concurrency Jargon
• Hardware– Processors
• Software– Threads, processes
• Sometimes OK to confuse them, sometimes not.
Art of Multiprocessor Programming
![Page 22: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/22.jpg)
24
Parallel Primality Testing
• Challenge– Print primes from 1 to 1010
• Given– Ten-processor multiprocessor– One thread per processor
• Goal– Get ten-fold speedup (or close)
Art of Multiprocessor Programming
![Page 23: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/23.jpg)
Art of Multiprocessor Programming 25
Load Balancing
• Split the work evenly• Each thread tests range of 109
…
…109 10102·1091
P0 P1 P9
![Page 24: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/24.jpg)
26
Procedure for Thread i
void primePrint { int i = ThreadID.get(); // IDs in {0..9} for (j = i*109+1, j<(i+1)*109; j++) { if (isPrime(j)) print(j); }}
Art of Multiprocessor Programming
![Page 25: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/25.jpg)
27
Issues
• Higher ranges have fewer primes• Yet larger numbers harder to test• Thread workloads
– Uneven– Hard to predict
Art of Multiprocessor Programming
![Page 26: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/26.jpg)
Art of Multiprocessor Programming 28
Issues
• Higher ranges have fewer primes• Yet larger numbers harder to test• Thread workloads
– Uneven– Hard to predict
• Need dynamic load balancing
rejected
![Page 27: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/27.jpg)
Art of Multiprocessor Programming 29
17
18
19
Shared Counter
each thread takes a number
![Page 28: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/28.jpg)
30
Procedure for Thread i
int counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); }}
Art of Multiprocessor Programming
![Page 29: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/29.jpg)
Art of Multiprocessor Programming 31
Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); }}
Procedure for Thread i
Shared counterobject
![Page 30: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/30.jpg)
Art of Multiprocessor Programming 32
Where Things Reside
cache
BusBus
cachecache
1
shared counter
shared memory
void primePrint { int i = ThreadID.get(); // IDs in {0..9} for (j = i*109+1, j<(i+1)*109; j++) { if (isPrime(j)) print(j); }}
code
Local variables
![Page 31: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/31.jpg)
Art of Multiprocessor Programming 33
Procedure for Thread i
Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); }}
Stop when every value taken
![Page 32: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/32.jpg)
Art of Multiprocessor Programming 34
Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); }}
Procedure for Thread i
Increment & return each new value
![Page 33: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/33.jpg)
35
Counter Implementation
public class Counter { private long value;
public long getAndIncrement() { return value++; }}
Art of Multiprocessor Programming
![Page 34: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/34.jpg)
Art of Multiprocessor Programming 36
Counter Implementation
public class Counter { private long value;
public long getAndIncrement() { return value++; }} OK for single thread,
not for concurrent threads
![Page 35: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/35.jpg)
Art of Multiprocessor Programming
37
What It Means
public class Counter { private long value;
public long getAndIncrement() { return value++; }}
![Page 36: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/36.jpg)
Art of Multiprocessor Programming 38
What It Means
public class Counter { private long value;
public long getAndIncrement() { return value++; }}
temp = value; value = temp + 1; return temp;
![Page 37: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/37.jpg)
Art of Multiprocessor Programming 39
time
Not so good…
Value… 1
read 1
read 1
write 2
read 2
write 3
write 2
2 3 2
![Page 38: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/38.jpg)
Art of Multiprocessor Programming 40
Is this problem inherent?
If we could only glue reads and writes together…
read
write read
write
!! !!
![Page 39: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/39.jpg)
41
Challenge
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; }}
Art of Multiprocessor Programming
![Page 40: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/40.jpg)
Art of Multiprocessor Programming 42
Challenge
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; }}
Make these steps atomic (indivisible)
![Page 41: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/41.jpg)
Art of Multiprocessor Programming 43
Hardware Solution
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; }} ReadModifyWrite()
instruction
![Page 42: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/42.jpg)
Art of Multiprocessor Programming
44
An Aside: Java™
public class Counter { private long value;
public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; }}
![Page 43: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/43.jpg)
Art of Multiprocessor Programming 45
An Aside: Java™
public class Counter { private long value;
public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; }} Synchronized block
![Page 44: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/44.jpg)
Art of Multiprocessor Programming 46
An Aside: Java™
public class Counter { private long value;
public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; }}
Mutual Exclusion
![Page 45: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/45.jpg)
47
Mutual Exclusion,or “Alice & Bob share a pond”
A B
Art of Multiprocessor Programming
![Page 46: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/46.jpg)
48
Alice has a pet
A B
Art of Multiprocessor Programming
![Page 47: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/47.jpg)
49
Bob has a pet
A B
Art of Multiprocessor Programming
![Page 48: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/48.jpg)
50
The Problem
A B
The pets don'tget along
Art of Multiprocessor Programming
![Page 49: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/49.jpg)
51
Formalizing the Problem
• Two types of formal properties in asynchronous computation:
• Safety Properties– Nothing bad happens ever
• Liveness Properties – Something good happens eventually
Art of Multiprocessor Programming
![Page 50: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/50.jpg)
52
Formalizing our Problem
• Mutual Exclusion– Both pets never in pond simultaneously– This is a safety property
• No Deadlock– if only one wants in, it gets in– if both want in, one gets in– This is a liveness property
Art of Multiprocessor Programming
![Page 51: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/51.jpg)
53
Simple Protocol
• Idea– Just look at the pond
• Gotcha– Not atomic– Trees obscure the view
Art of Multiprocessor Programming
![Page 52: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/52.jpg)
54
Interpretation
• Threads can't “see” what other threads are doing
• Explicit communication required for coordination
Art of Multiprocessor Programming
![Page 53: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/53.jpg)
55
Cell Phone Protocol
• Idea– Bob calls Alice (or vice-versa)
• Gotcha– Bob takes shower– Alice recharges battery– Bob out shopping for pet food …
Art of Multiprocessor Programming
![Page 54: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/54.jpg)
56
Interpretation
• Message-passing doesn't work• Recipient might not be
– Listening– There at all
• Communication must be– Persistent (like writing)– Not transient (like speaking)
Art of Multiprocessor Programming
![Page 55: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/55.jpg)
57
Can Protocol
cola
cola
Art of Multiprocessor Programming
![Page 56: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/56.jpg)
58
Bob conveys a bit
A B
cola
Art of Multiprocessor Programming
![Page 57: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/57.jpg)
59
Bob conveys a bit
A B
cola
Art of Multiprocessor Programming
![Page 58: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/58.jpg)
60
Can Protocol
• Idea– Cans on Alice's windowsill– Strings lead to Bob's house– Bob pulls strings, knocks over cans
• Gotcha– Cans cannot be reused– Bob runs out of cans
Art of Multiprocessor Programming
![Page 59: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/59.jpg)
61
Interpretation
• Cannot solve mutual exclusion with interrupts– Sender sets fixed bit in receiver's space– Receiver resets bit when ready– Requires unbounded number of interrupt bits
Art of Multiprocessor Programming
![Page 60: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/60.jpg)
62
Flag Protocol
A B
Art of Multiprocessor Programming
![Page 61: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/61.jpg)
63
Alice's Protocol (sort of)
A B
Art of Multiprocessor Programming
![Page 62: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/62.jpg)
64
Bob's Protocol (sort of)
A B
Art of Multiprocessor Programming
![Page 63: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/63.jpg)
65
Alice's Protocol
• Raise flag• Wait until Bob's flag is down• Unleash pet• Lower flag when pet returns
Art of Multiprocessor Programming
![Page 64: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/64.jpg)
Art of Multiprocessor Programming 66
Bob's Protocol
• Raise flag• Wait until Alice's flag is down• Unleash pet• Lower flag when pet returns
dang
er!
![Page 65: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/65.jpg)
67
Bob's Protocol (2nd try)
• Raise flag• While Alice's flag is up
– Lower flag– Wait for Alice's flag to go down– Raise flag
• Unleash pet• Lower flag when pet returns
Art of Multiprocessor Programming
![Page 66: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/66.jpg)
Art of Multiprocessor Programming 68
Bob's Protocol
• Raise flag• While Alice's flag is up
– Lower flag– Wait for Alice's flag to go down– Raise flag
• Unleash pet• Lower flag when pet returns
Bob defers to Alice
![Page 67: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/67.jpg)
69
The Flag Principle
• Raise the flag• Look at other's flag• Flag Principle:
– If each raises and looks, then– Last to look must see both flags up
Art of Multiprocessor Programming
![Page 68: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/68.jpg)
70
Proof of Mutual Exclusion
• Assume both pets in pond– Derive a contradiction– By reasoning backwards
• Consider the last time Alice and Bob each looked before letting the pets in
• Without loss of generality assume Alice was the last to look…
Art of Multiprocessor Programming
![Page 69: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/69.jpg)
Art of Multiprocessor Programming 71
Proof
time
Alice's last look
Alice last raised her flag
Bob's last look
QED
Alice must have seen Bob's Flag. A Contradiction
Bob last raised flag
![Page 70: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/70.jpg)
72
Proof of No Deadlock
• If only one pet wants in, it gets in.
Art of Multiprocessor Programming
![Page 71: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/71.jpg)
73
Proof of No Deadlock
• If only one pet wants in, it gets in.• Deadlock requires both continually trying
to get in.
Art of Multiprocessor Programming
![Page 72: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/72.jpg)
Art of Multiprocessor Programming 74
Proof of No Deadlock
• If only one pet wants in, it gets in.• Deadlock requires both continually trying
to get in.• If Bob sees Alice's flag, he gives her
priority (a gentleman…)
QED
![Page 73: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/73.jpg)
75
Remarks
• Protocol is unfair– Bob's pet might never get in
• Protocol uses waiting– If Bob is eaten by his pet, Alice's pet might
never get in
Art of Multiprocessor Programming
![Page 74: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/74.jpg)
76
Moral of Story
• Mutual Exclusion cannot be solved by–transient communication (cell phones)– interrupts (cans)
• It can be solved by– one-bit shared variables – that can be read or written
Art of Multiprocessor Programming
![Page 75: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/75.jpg)
Art of Multiprocessor Programming 77
The Arbiter Problem (an aside)
Pick a point
Pick a point
![Page 76: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/76.jpg)
78
The Fable Continues
• Alice and Bob fall in love & marry
Art of Multiprocessor Programming
![Page 77: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/77.jpg)
79
The Fable Continues
• Alice and Bob fall in love & marry• Then they fall out of love & divorce
– She gets the pets– He has to feed them
Art of Multiprocessor Programming
![Page 78: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/78.jpg)
80
The Fable Continues
• Alice and Bob fall in love & marry• Then they fall out of love & divorce
– She gets the pets– He has to feed them
• Leading to a new coordination problem: Producer-Consumer
Art of Multiprocessor Programming
![Page 79: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/79.jpg)
81
Bob Puts Food in the Pond
A
Art of Multiprocessor Programming
![Page 80: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/80.jpg)
82
mmm…
Alice releases her pets to Feed
Bmmm…
Art of Multiprocessor Programming
![Page 81: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/81.jpg)
83
Producer/Consumer
• Alice and Bob can't meet– Each has restraining order on other– So he puts food in the pond– And later, she releases the pets
• Avoid– Releasing pets when there's no food– Putting out food if uneaten food remains
Art of Multiprocessor Programming
![Page 82: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/82.jpg)
84
Producer/Consumer
• Need a mechanism so that– Bob lets Alice know when food has been put
out– Alice lets Bob know when to put out more
food
Art of Multiprocessor Programming
![Page 83: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/83.jpg)
85
Surprise Solution
A B
cola
Art of Multiprocessor Programming
![Page 84: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/84.jpg)
86
Bob puts food in Pond
A B
cola
Art of Multiprocessor Programming
![Page 85: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/85.jpg)
87
Bob knocks over Can
A B
cola
Art of Multiprocessor Programming
![Page 86: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/86.jpg)
88
Alice Releases Pets
A B
cola
yum… Byum…
Art of Multiprocessor Programming
![Page 87: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/87.jpg)
89
Alice Resets Can when Pets are Fed
A B
cola
Art of Multiprocessor Programming
![Page 88: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/88.jpg)
Art of Multiprocessor Programming 90
Pseudocode
while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset();}
Alice's code
![Page 89: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/89.jpg)
Art of Multiprocessor Programming 91
Pseudocode
while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset();}
Alice's code
while (true) { while (can.isDown()){}; pond.stockWithFood(); can.knockOver();}
Bob's code
![Page 90: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/90.jpg)
92
Correctness• Mutual Exclusion
– Pets and Bob never together in pond
Art of Multiprocessor Programming
![Page 91: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/91.jpg)
93
Correctness• Mutual Exclusion
– Pets and Bob never together in pond• No Starvation
if Bob always willing to feed, and pets always famished, then pets eat infinitely often.
Art of Multiprocessor Programming
![Page 92: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/92.jpg)
Art of Multiprocessor Programming 94
Correctness• Mutual Exclusion
– Pets and Bob never together in pond• No Starvation
if Bob always willing to feed, and pets always famished, then pets eat infinitely often.
• Producer/ConsumerThe pets never enter pond unless there is
food, and Bob never provides food if there is unconsumed food.
safety
liveness
safety
![Page 93: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/93.jpg)
95
Could Also Solve Using Flags
A B
Art of Multiprocessor Programming
![Page 94: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/94.jpg)
96
Waiting
• Both solutions use waiting– while(mumble){}
• In some cases waiting is problematic– If one participant is delayed– So is everyone else– But delays are common & unpredictable
Art of Multiprocessor Programming
![Page 95: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/95.jpg)
97
The Fable drags on …
• Bob and Alice still have issues
Art of Multiprocessor Programming
![Page 96: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/96.jpg)
98
The Fable drags on …
• Bob and Alice still have issues• So they need to communicate
Art of Multiprocessor Programming
![Page 97: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/97.jpg)
99
The Fable drags on …
• Bob and Alice still have issues• So they need to communicate• They agree to use billboards …
Art of Multiprocessor Programming
![Page 98: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/98.jpg)
100
E1
D2C
3
Billboards are Large
B3A
1
LetterTiles
From Scrabble™ box
Art of Multiprocessor Programming
![Page 99: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/99.jpg)
101
E1
D2C
3
Write One Letter at a Time …
B3A
1
W4A
1S
1
H4
Art of Multiprocessor Programming
![Page 100: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/100.jpg)
102
To post a message
W4A
1S
1H
4A
1C
3R
1T
1H
4E
1
whew
Art of Multiprocessor Programming
![Page 101: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/101.jpg)
103
S1
Let's send another message
S1E
1L
1L
1L
1V
4
L1 A
1
M3
A1
A1
P3
Art of Multiprocessor Programming
![Page 102: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/102.jpg)
104
Uh-Oh
A1
C3
R1
T1H
4E
1S
1E
1L
1L
1
L1
OK
Art of Multiprocessor Programming
![Page 103: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/103.jpg)
105
Readers/Writers
• Devise a protocol so that– Writer writes one letter at a time– Reader reads one letter at a time– Reader sees “snapshot”
• Old message or new message• No mixed messages
Art of Multiprocessor Programming
![Page 104: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/104.jpg)
106
Readers/Writers (continued)• Easy with mutual exclusion• But mutual exclusion requires waiting
– One waits for the other– Everyone executes sequentially
• Remarkably– We can solve R/W without mutual exclusion
Art of Multiprocessor Programming
![Page 105: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/105.jpg)
Art of Multiprocessor Programming 107
Esoteric?
• Java container size() method• Single shared counter?
– incremented with each add() and– decremented with each remove()
• Threads wait to exclusively access counter
perform
ance
bottleneck
![Page 106: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/106.jpg)
108
Readers/Writers Solution
• Each thread i has size[i] counter – only it increments or decrements.
• To get object's size, a thread reads a “snapshot” of all counters without mutex
• This eliminates the bottleneck
Art of Multiprocessor Programming
4 3 0 0 1 4 2 0 0 7size
![Page 107: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/107.jpg)
109
Why do we care About Sequential Bottlenecks?
• We want as much of the code as possible to execute in parallel
• A larger sequential part implies reduced performance
• Amdahl's law: this relation is not linear…
Art of Multiprocessor Programming
Eugene Amdahl
![Page 108: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/108.jpg)
Art of Multiprocessor Programming 110
Amdahl's Law
Speedup = 1-thread execution time
N-thread execution time
![Page 109: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/109.jpg)
Art of Multiprocessor Programming 111
Amdahl's Law
Speedup =
![Page 110: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/110.jpg)
Art of Multiprocessor Programming 112
Amdahl's Law
Speedup =
Parallel fraction
![Page 111: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/111.jpg)
Art of Multiprocessor Programming 113
Amdahl's Law
Speedup =
Parallel fraction
Sequential fraction
![Page 112: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/112.jpg)
Art of Multiprocessor Programming 114
Amdahl's Law
Speedup =
Parallel fraction
Sequential fraction
Number of
threads
![Page 113: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/113.jpg)
116
Example
• Ten processors• 60% concurrent, 40% sequential• How close to 10-fold speedup?
Art of Multiprocessor Programming
![Page 114: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/114.jpg)
117
Example
• Ten processors• 60% concurrent, 40% sequential• How close to 10-fold speedup?
Speedup = 2.17 =
Art of Multiprocessor Programming
![Page 115: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/115.jpg)
118
Example
• Ten processors• 80% concurrent, 20% sequential• How close to 10-fold speedup?
Art of Multiprocessor Programming
![Page 116: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/116.jpg)
119
Example
• Ten processors• 80% concurrent, 20% sequential• How close to 10-fold speedup?
Speedup = 3.57 =
Art of Multiprocessor Programming
![Page 117: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/117.jpg)
120
Example
• Ten processors• 90% concurrent, 10% sequential• How close to 10-fold speedup?
Art of Multiprocessor Programming
![Page 118: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/118.jpg)
Art of Multiprocessor Programming 121
Example
• Ten processors• 90% concurrent, 10% sequential• How close to 10-fold speedup?
Speedup = 5.26 =
![Page 119: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/119.jpg)
122
Example
• Ten processors• 99% concurrent, 01% sequential• How close to 10-fold speedup?
Art of Multiprocessor Programming
![Page 120: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/120.jpg)
Art of Multiprocessor Programming 123
Example
• Ten processors• 99% concurrent, 01% sequential• How close to 10-fold speedup?
Speedup = 9.17 =
![Page 121: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/121.jpg)
Back to Real-World Multicore Scaling
Art of Multiprocessor Programming 124
1.8x 2x 2.9x
User code
Multicore
Speedup
Not reducing sequential % of code
![Page 122: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/122.jpg)
Shared Data Structures
75%Unshared
25%Shared
CoarseGrained
FineGrained
75%Unshared
25%Shared
![Page 123: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/123.jpg)
Shared Data Structures
75%Unshared
25%Shared
CoarseGrained
FineGrained
Why only 2.9 speedup
75%Unshared
25%Shared
Honk!Honk!
Honk!
![Page 124: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/124.jpg)
Shared Data Structures
75%Unshared
25%Shared
CoarseGrained
FineGrained
Why fine-grained parallelism maters
75%Unshared
25%Shared
Honk!Honk!
Honk!
![Page 125: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/125.jpg)
Art of Multiprocessor Programming 128
This Course
• Learn to minimize parallelization overhead of the fraction P that is easy
• Learn how to introduce parallelism into the 1-P that are hard
![Page 126: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/126.jpg)
Grading
• 11 Homeworks 55%• 2 In-class midterms 45%• Graduate credit:
– Extra problem on each homework– Term project (talk to instructor)
Art of Multiprocessor Programming
![Page 127: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/127.jpg)
Collaboration
• Permitted– talking about the homework problems with
other students; using other textbooks; using the Internet.
• Not Permitted– obtaining the answer directly from anyone
else in any form.
Art of Multiprocessor Programming
![Page 128: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/128.jpg)
Crowdsourcing!
• You can annotate the textbook online– See something interesting?– Have a question?– Answer a question?– Like or dislike a note?
• http://nb.mit.edu– Play the video
• Details to follow …
Art of Multiprocessor Programming
![Page 129: Introduction to Multiprocessor Synchronization Maurice Herlihy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d6e5503460f94a4f9ac/html5/thumbnails/129.jpg)
Art of Multiprocessor Programming
133
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.
• You are free:– to Share — to copy, distribute and transmit the work – to Remix — to adapt the work
• Under the following conditions:– Attribution. You must attribute the work to “The Art of
Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work).
– Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.
• For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to– http://creativecommons.org/licenses/by-sa/3.0/.
• Any of the above conditions can be waived if you get permission from the copyright holder.
• Nothing in this license impairs or restricts the author's moral rights.