comp 322: principles of parallel programming lecture 11 ...vs3/pdf/comp322-lec11-f09-v1.pdf · comp...
TRANSCRIPT
![Page 1: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/1.jpg)
COMP 322 Lecture 11 29 September 2009
COMP 322: Principles of Parallel Programming
Lecture 11: Parallel Programming Issues (Chapter 6)
Fall 2009
Vivek Sarkar Department of Computer Science
Rice University [email protected]
![Page 2: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/2.jpg)
COMP 322, Fall 2009 (V.Sarkar)!2
Summary of Previous Lecture!
•! Example of Pipeline Parallelism with HJ Phasers
•! Overview of POSIX Threads (Chapter 6)
•! Question for you to think about: —!Which POSIX Threads examples in the lecture or book cannot be
expressed using HJ constructs?
![Page 3: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/3.jpg)
COMP 322, Fall 2009 (V.Sarkar)!3
Acknowledgments for Today"s Lecture!•! Course text: “Principles of Parallel Programming”, Calvin Lin & Lawrence
Snyder
—!Includes resources available at http://www.pearsonhighered.com/educator/academic/product/0,3110,0321487907,00.html
•! “Phasers: a Unified Deadlock-Free Construct for Collective and Point-to-point Synchronization”, Jun Shirako, David M. Peixotto, Vivek Sarkar, William N. Scherer III
—!http://www.cs.rice.edu/~vs3/PDF/SPSS08-phasers.pdf
•! Companion slides for “The Art of Multiprocessor Programming” by Maurice Herlihy & Nir Shavit
—! http://www.cs.brown.edu/courses/cs176/ch01.ppt
![Page 4: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/4.jpg)
COMP 322, Fall 2009 (V.Sarkar)!4
Pthreads Issues: Fairness!
•! Consider three POSIX threads (0, 1, 2) that repeatedly execute the following loop: while (true) {
pthread_mutex_lock(&lock);
printf(“Hello from thread %d\n”, pthread_self());
pthread_mutex_unlock(&lock);
}
•! What is the relative number of print statements that you might expect from threads 0, 1, 2?
•! What is the minimum number of print statements that might be guaranteed from threads 0, 1, 2?
•! No guarantee of fairness in the POSIX threads standard
•! Order in which locks are acquired is not guaranteed to match the order in which the threads attempt to acquire the locks
![Page 5: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/5.jpg)
COMP 322, Fall 2009 (V.Sarkar)!5
Pthreads Issues: Serializability !
•! A concurrent execution is serializable if the execution is guaranteed to correspond to some serial execution of those threads.
Consider
/* threads compete to update global variable best_cost */
if (my_cost < best_cost) !
best_cost = my_cost; !
—!two threads
—!initial value of best_cost is 100
—!values of my_cost are 50 and 75 for threads t1 and t2
•! After execution, best_cost could be 50 or 75
•! 75 does not correspond to any serialization of the threads
! Use mutex to make the parallel program serializable
![Page 6: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/6.jpg)
COMP 322, Fall 2009 (V.Sarkar)!6
Pthreads Issues: Deadlock!
Figure 6.7 Deadlock example. Threads T1 and T2 hold
locks L1 and L2, respectively, and each thread attempts
to acquire the other lock, which cannot be granted."
![Page 7: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/7.jpg)
COMP 322, Fall 2009 (V.Sarkar)!7
Lock Hierarchies!
•! A simple way to prevent deadlocks is to prevent cycles in the resource allocation graph.
•! Impose an order on the locks, and require that all threads acquire their locks in the same order !!Lock hierarchy!
•! What if a thread doesn’t know a priori which locks it needs to acquire? 1.# On learning of a new lock, it can release all its existing locks and
then reacquire all locks in the proper order
2.# Use pthread_mutex_trylock() on the new lock. If that fails then revert to option 1. above.
![Page 8: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/8.jpg)
COMP 322, Fall 2009 (V.Sarkar)!8
Counter Implementation!
public class Counter { private long value;
public long getAndIncrement() { return value++; } }
![Page 9: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/9.jpg)
COMP 322, Fall 2009 (V.Sarkar)!9
Counter Implementation!
public class Counter { private long value;
public long getAndIncrement() { return value++; } } OK fo
r single t
hread,
not for co
ncurrent t
hreads
![Page 10: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/10.jpg)
COMP 322, Fall 2009 (V.Sarkar)!10
What It Means!
public class Counter { private long value;
public long getAndIncrement() { return value++; } }
![Page 11: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/11.jpg)
COMP 322, Fall 2009 (V.Sarkar)!11
What It Means!
public class Counter { private long value;
public long getAndIncrement() { return value++; } }
temp = value;
value = temp + 1;
return temp;
![Page 12: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/12.jpg)
COMP 322, Fall 2009 (V.Sarkar)!12
time
Not so good…!
Value… 1
read 1
read 1
write 2
read 2
write 3
write 2
2 3 2
![Page 13: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/13.jpg)
COMP 322, Fall 2009 (V.Sarkar)!13
Is this problem inherent?!
If we could only glue reads and writes together…
read
write read
write
!! !!
![Page 14: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/14.jpg)
COMP 322, Fall 2009 (V.Sarkar)!14
Challenge!
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; } }
![Page 15: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/15.jpg)
COMP 322, Fall 2009 (V.Sarkar)!15
Challenge!
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; } }
Make these steps atomic (isolated)
![Page 16: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/16.jpg)
COMP 322, Fall 2009 (V.Sarkar)!16
Hardware Solution!
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; } } ReadModifyWrite()
instruction
![Page 17: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/17.jpg)
COMP 322, Fall 2009 (V.Sarkar)!17
HJ Solution!
public class Counter { private long value;
public long getAndIncrement() { isolated { temp = value; value = temp + 1; } return temp; } }
![Page 18: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/18.jpg)
COMP 322, Fall 2009 (V.Sarkar)!18
Mutual Exclusion or “Alice & Bob share a pond”!
A B
![Page 19: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/19.jpg)
COMP 322, Fall 2009 (V.Sarkar)!19
Alice has a pet!
A B
![Page 20: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/20.jpg)
COMP 322, Fall 2009 (V.Sarkar)!20
Bob has a pet!
A B
![Page 21: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/21.jpg)
COMP 322, Fall 2009 (V.Sarkar)!21
The Problem!
A B
The pets don’t get along
![Page 22: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/22.jpg)
COMP 322, Fall 2009 (V.Sarkar)!22
Formalizing the Problem!
•! Two types of formal properties in asynchronous computation:
•! Safety Properties —!Nothing bad happens ever
•! Liveness Properties —!Something good happens eventually
![Page 23: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/23.jpg)
COMP 322, Fall 2009 (V.Sarkar)!23
Formalizing our Problem!
•! Mutual Exclusion —!Both pets never in pond simultaneously
—!This is a safety property
•! No Deadlock —!if only one wants in, it gets in
—!if both want in, one gets in.
—!This is a liveness property
![Page 24: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/24.jpg)
COMP 322, Fall 2009 (V.Sarkar)!24
Simple Protocol!
•! Idea —!Just look at the pond
•! Gotcha —!Not atomic
—!Trees obscure the view
![Page 25: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/25.jpg)
COMP 322, Fall 2009 (V.Sarkar)!25
Interpretation!
•! Threads can’t “see” what other threads are doing
•! Explicit communication required for coordination
![Page 26: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/26.jpg)
COMP 322, Fall 2009 (V.Sarkar)!26
Cell Phone Protocol!
•! Idea —!Bob calls Alice (or vice-versa)
•! Gotcha —!Bob takes shower
—!Alice recharges battery
—!Bob out shopping for pet food …
![Page 27: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/27.jpg)
COMP 322, Fall 2009 (V.Sarkar)!27
Interpretation!
•! Message-passing doesn’t work
•! Recipient might not be —!Listening
—!There at all
•! Communication must be —!Persistent (like writing)
—!Not transient (like speaking)
![Page 28: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/28.jpg)
COMP 322, Fall 2009 (V.Sarkar)!28
Can Protocol!
cola
cola
![Page 29: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/29.jpg)
COMP 322, Fall 2009 (V.Sarkar)!29
Bob conveys a bit!
A B
cola
![Page 30: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/30.jpg)
COMP 322, Fall 2009 (V.Sarkar)!30
Bob conveys a bit!
A B
cola
![Page 31: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/31.jpg)
COMP 322, Fall 2009 (V.Sarkar)!31
Can Protocol!
•! Idea —!Cans on Alice’s windowsill
—!Strings lead to Bob’s house
—!Bob pulls strings, knocks over cans
•! Gotcha —!Cans cannot be reused
—!Bob runs out of cans
![Page 32: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/32.jpg)
COMP 322, Fall 2009 (V.Sarkar)!32
Interpretation!
•! Cannot solve mutual exclusion with interrupts —!Sender sets fixed bit in receiver’s space
—!Receiver resets bit when ready
—!Requires unbounded number of interrupt bits
![Page 33: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/33.jpg)
COMP 322, Fall 2009 (V.Sarkar)!33
Flag Protocol!
A B
![Page 34: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/34.jpg)
COMP 322, Fall 2009 (V.Sarkar)!34
Alice"s Protocol (sort of)!
A B
![Page 35: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/35.jpg)
COMP 322, Fall 2009 (V.Sarkar)!35
Bob"s Protocol (sort of)!
A B
![Page 36: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/36.jpg)
COMP 322, Fall 2009 (V.Sarkar)!36 Art of Multiprocessor Programming
36
Alice"s Protocol!
•! Raise flag
•! Wait until Bob’s flag is down
•! Unleash pet
•! Lower flag when pet returns
![Page 37: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/37.jpg)
COMP 322, Fall 2009 (V.Sarkar)!37
Bob"s Protocol!
•! Raise flag
•! Wait until Alice’s flag is down
•! Unleash pet
•! Lower flag when pet returns
![Page 38: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/38.jpg)
COMP 322, Fall 2009 (V.Sarkar)!38
Bob"s Protocol (2nd try)!
•! Raise flag
•! While Alice’s flag is up —!Lower flag
—!Wait for Alice’s flag to go down
—!Raise flag
•! Unleash pet
•! Lower flag when pet returns
![Page 39: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/39.jpg)
COMP 322, Fall 2009 (V.Sarkar)!39
Remarks!
•! Protocol is unfair —!Bob’s pet might never get in
•! Protocol uses waiting —!If Bob is eaten by his pet, Alice’s pet might never get in
![Page 40: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/40.jpg)
COMP 322, Fall 2009 (V.Sarkar)!40
Moral of Story!
•!Mutual Exclusion cannot be solved by
—!transient communication (cell phones)
—!interrupts (cans)
•! It can be solved by
—! one-bit shared variables
—! that can be read or written
![Page 41: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/41.jpg)
COMP 322, Fall 2009 (V.Sarkar)!41
The Fable Continues!
•! Alice and Bob fall in love & marry
![Page 42: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/42.jpg)
COMP 322, Fall 2009 (V.Sarkar)!42
The Fable Continues!
•! Alice and Bob fall in love & marry
•! Then they fall out of love & divorce —!She gets the pets
—!He has to feed them
![Page 43: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/43.jpg)
COMP 322, Fall 2009 (V.Sarkar)!43
The Fable Continues!
•! Alice and Bob fall in love & marry
•! Then they fall out of love & divorce —!She gets the pets
—!He has to feed them
•! Leading to a new coordination problem: Producer-Consumer
![Page 44: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/44.jpg)
COMP 322, Fall 2009 (V.Sarkar)!44
Bob Puts Food in the Pond!
A
![Page 45: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/45.jpg)
COMP 322, Fall 2009 (V.Sarkar)!45
mmm…
Alice releases her pets to Feed!
B mmm…
![Page 46: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/46.jpg)
COMP 322, Fall 2009 (V.Sarkar)!46
Producer/Consumer!
•! Alice and Bob can’t meet —!Each has restraining order on other
—!So he puts food in the pond
—!And later, she releases the pets
•! Avoid —!Releasing pets when there’s no food
—!Putting out food if uneaten food remains
![Page 47: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/47.jpg)
COMP 322, Fall 2009 (V.Sarkar)!47
Producer/Consumer!
•! Need a mechanism so that —!Bob lets Alice know when food has been put out
—!Alice lets Bob know when to put out more food
![Page 48: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/48.jpg)
COMP 322, Fall 2009 (V.Sarkar)!48
Surprise Solution!
A B
cola
![Page 49: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/49.jpg)
COMP 322, Fall 2009 (V.Sarkar)!49
Bob puts food in Pond!
A B
cola
![Page 50: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/50.jpg)
COMP 322, Fall 2009 (V.Sarkar)!50
Bob knocks over Can!
A B
cola
![Page 51: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/51.jpg)
COMP 322, Fall 2009 (V.Sarkar)!51
Alice Releases Pets!
A B
cola
yum… B yum…
![Page 52: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/52.jpg)
COMP 322, Fall 2009 (V.Sarkar)!52
Alice Resets Can when Pets are Fed!
A B
cola
![Page 53: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/53.jpg)
COMP 322, Fall 2009 (V.Sarkar)!53 Art of Multiprocessor Programming
53
Pseudocode!
while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset(); }
Alice’s code
![Page 54: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/54.jpg)
COMP 322, Fall 2009 (V.Sarkar)!54
Pseudocode!
while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset(); }
Alice’s code
while (true) { while (can.isDown()){}; pond.stockWithFood(); can.knockOver(); }
Bob’s code
![Page 55: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/55.jpg)
COMP 322, Fall 2009 (V.Sarkar)!55
Correctness!
•! Mutual Exclusion —!Pets and Bob never together in pond
![Page 56: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/56.jpg)
COMP 322, Fall 2009 (V.Sarkar)!56
Correctness!•! Mutual Exclusion
—!Pets and Bob never together in pond
•! No Starvation if Bob always willing to feed, and pets always famished, then
pets eat infinitely often.
![Page 57: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/57.jpg)
COMP 322, Fall 2009 (V.Sarkar)!57
Correctness!•! Mutual Exclusion
—!Pets and Bob never together in pond
•! No Starvation if Bob always willing to feed, and pets always famished, then
pets eat infinitely often.
•! Producer/Consumer The pets never enter pond unless there is food, and Bob never
provides food if there is unconsumed food.
safety
liveness
safety
![Page 58: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/58.jpg)
COMP 322, Fall 2009 (V.Sarkar)!58
Could Also Solve Using Flags!
A B
![Page 59: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/59.jpg)
COMP 322, Fall 2009 (V.Sarkar)!59
Figure 6.1!
A bounded buffer with producers and consumers. The Put and Get cursors indicate where the producers will insert the next item and where the consumers will remove its next item.
![Page 60: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/60.jpg)
COMP 322, Fall 2009 (V.Sarkar)!60
Figure 6.3 Bounded buffer example using condition variables nonempty and nonfull.!
![Page 61: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/61.jpg)
COMP 322, Fall 2009 (V.Sarkar)!61
Bound option in phasers!•! Constructor
—!phaser(mode m, int bound_size);
•! next operation —!A task registered in SIG mode will block if it is >= bound_size
phases past the current phase
![Page 62: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/62.jpg)
COMP 322, Fall 2009 (V.Sarkar)!62
Single-Producer Single-Consumer Bounded Buffer!
finish {!
final phaser ph = new phaser(<SIG_WAIT>, bound_size);!
async phased (ph<SIG>) !
while (…) { insert(); next; } // producer!
async phased (ph<WAIT>)!
while (…) { next; remove(); } // consumer!
}!
![Page 63: COMP 322: Principles of Parallel Programming Lecture 11 ...vs3/PDF/comp322-lec11-f09-v1.pdf · COMP 322 Lecture 11 29 September 2009 COMP 322: Principles of Parallel Programming Lecture](https://reader034.vdocuments.net/reader034/viewer/2022050506/5f9764d2c950ca46747a71d9/html5/thumbnails/63.jpg)
COMP 322, Fall 2009 (V.Sarkar)!63
Summary of Today"s Lecture!
•! POSIX Threads as an illustration of parallel programming issues: —!Fairness
—!Serializability
—!Deadlock
—!Safety
—!Liveness
—!Bounded buffer example
•! Many of these issues arise in other parallel programming languages, but constructs in higher level languages such as HJ can help (async, finish, isolated, phasers)