picking patterns for parallel programs · buzzword bingo actors agents message queues tasks csp...
TRANSCRIPT
![Page 1: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/1.jpg)
Picking Patterns for Parallel Programs
Anthony Williams
Just Software Solutions Ltdhttp://www.justsoftwaresolutions.co.uk
16th April 2011
![Page 2: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/2.jpg)
Picking Patterns for Parallel Programs
Buzzword Bingo
Structural Patterns
Communication Patterns
Choosing Patterns
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 3: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/3.jpg)
Buzzword Bingo
Actors Agents Message Queues
Tasks CSP Dataflow
Map-reduce Locks Continuations
Lock-free False Sharing Fork/Join
Work-Stealing Pipelines SPMD
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 4: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/4.jpg)
Structural Patterns
![Page 5: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/5.jpg)
Loop Parallelism
![Page 6: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/6.jpg)
Loop Parallelism (I)
Apply the same operation to manyindependent data items
Great for EmbarrasinglyParallel problems
Frameworks commonly provide aparallel for each operation orequivalent
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 7: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/7.jpg)
Loop Parallelism (II)
std::vector<some_data> data;
parallel_for_each(data.begin,data.end(),process_data);
#pragma omp parallel for
for(unsigned i=0;i<data.size();++i) {
process_data(data[i]);
}
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 8: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/8.jpg)
Fork/Join
![Page 9: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/9.jpg)
Fork/Join (I)
Subdivide into parallel tasks, andthen wait for them to complete
Often used recursively
Works best when used at the toplevel
Need to watch for unevenworkloads
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 10: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/10.jpg)
Fork/Join (II)
template<typename Iter,typename Func>
void parallel_for_each(Iter first,Iter last,Func f) {
unsigned long const length=std::distance(first,last);
if(length<minimum_split_length) {
std::for_each(first,last,f);
} else {
Iter const mid_point=first+length/2;
auto top=std::async([=]{
parallel_for_each(first,mid_point,f);});
auto bottom=std::async([=]{
parallel_for_each(mid_point,last,f);});
top.wait(); bottom.wait();
}
}
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 11: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/11.jpg)
Pipelines
![Page 12: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/12.jpg)
Pipelines (I)
A set of discrete tasks that mustbe applied in sequence
The same sequence of tasks needto run over a large data set
An alternative to loop parallelism,where the order of the data isimportant
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 13: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/13.jpg)
Pipelines (II)
Item 1
Task 1 Task 2 Task 3
Item 1
Item 1
Item 2
Item 2
Item 2
Item 3
Item 3
Item 3
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 14: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/14.jpg)
Pipelines (III)
Maximum parallelism is thenumber of tasks in the pipelineCan mess with cache locality:
If each task is always run on the same corethen the data must be moved between coresIf data is kept on the same core, then thetask code must be reloaded for each step
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 15: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/15.jpg)
Actors
![Page 16: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/16.jpg)
Actors (I)
Actor 1
Actor 3
Actor 2
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 17: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/17.jpg)
Actors (II)
Each actor runs entirely isolated(no shared state)
The only communication betweenactors is via message queues
In some languages these rules areenforced. In C++ it is yourresponsibility to follow them.
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 18: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/18.jpg)
Actors (III)
Upsides:
Each actor can be analysedindependently
Data races are impossible (if youfollow the rules)
Can be easier to reason about
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 19: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/19.jpg)
Actors (IV)
Downsides:
Not good for short-lived tasks
Message passing isn’t always thebest communication mechanism
Scalability is limited to the numberof actors
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 20: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/20.jpg)
Actors (V)
Building an ATM with Actors
3 actors:
User Interface
Core ATM Logic
Communicating with the bank
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 21: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/21.jpg)
Actors (VI)
struct ping { jss::actor_ref sender; };
struct pong {};
jss::actor pp1([] {
jss::actor::receive().match<ping>(
[](ping p) {
p.sender.send(pong());
});
});
jss::actor pp2([&] {
pp1.send(ping{jss::actor::self()});
jss::actor::receive().match<pong>(
[](pong){});
});
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 22: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/22.jpg)
Speculative Execution
![Page 23: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/23.jpg)
Speculative Execution (I)
Make use of available concurrency:
Perform tasks that might beneeded
Execute multiple algorithms toobtain a result
Execute dependent tasks withpredicted data
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 24: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/24.jpg)
Speculative Execution (II)
Can reduce latency
Increases total work done
Cancelling speculative tasks can betricky
Speculative tasks must beside-effect free
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 25: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/25.jpg)
Speculative Execution (III)
template<typename Iter,typename Match>
Iter find(Iter first,Iter last,Match M,unsigned N) {
unsigned long const D=std::distance(first,last);
std::atomic<bool> done(false);
std::promise<Iter> p;
std::vector<std::future<void>> v(N);
for(unsigned i=0;i<N;++i) {
Iter const end=(i==(N-1))?last:(first+D/N);
v[i]=std::async(Searcher<Iter,Match>(first,end,M,done,p));
first=block_end;
}
for(auto&f:v) { f.wait(); }
return done?p.get_future().get():last;
}
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 26: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/26.jpg)
Speculative Execution (IV)
template<typename Iter,typename Match>
struct Searcher {
Iter first; Iter last; Match const& M;
std::atomic<bool>& done; std::promise<Iter>& p;
Searcher(...); // obvious constructor impl
void operator()() {
for(Iter it=first;it!=last && !done;++it) {
if(*it==M) {
try { p.set_value(it); }
catch(std::future_error) {}
done=true; return;
} } }
};
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 27: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/27.jpg)
Map/Reduce
−→
![Page 28: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/28.jpg)
Map/Reduce (I)
An algorithm in two halves:
Map: x[i] → f(x[i])
Reduce: Accumulate the results(e.g.
∑f(x[i]))
(Aside: Google’s MapReduceoperates on key/value pairs)
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 29: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/29.jpg)
Map/Reduce (II)
Scales well with multiple cores
Used in OpenMP and MPI
Covers many parallel algorithms
The map and reduce operationsmust be thread-safe
The overhead depends on the datagranularity
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 30: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/30.jpg)
Map/Reduce (III): Word count
typedef std::map<std::string,unsigned> map_type;
map_type count_words(word_list const& words) {
map_type counts;
std::string word;
while(words.get_next(word))
++counts[s];
return counts;
}
map_type combine_maps(
map_type counts,map_type const& other) {
for(auto const& entry: other)
counts[entry.first]+=entry.second;
return counts;
}
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 31: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/31.jpg)
Dataflow
![Page 32: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/32.jpg)
Dataflow (I)
You specify the relationships, andthe runtime handles theparallelization
Has a “Functional Programming”style feel, even in imperativelanguages
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 33: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/33.jpg)
Dataflow (II): Simple Variables
int main() {
jss::dataflow::variable<int> x,y,z;
z.task([&]{return x.get()+y.get();});
x=23;
y=19;
assert(z.get()==42);
}
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 34: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/34.jpg)
Dataflow (III): Channels
int main() {
int const count=100;
jss::dataflow::channel<int> x,y,z;
jss::dataflow::task([&]{
while(true) { z<<(x.pop()+y.pop()); }});
jss::dataflow::task([&x]{
for(int i=0;i<count;++i) { x<<rand()%20; }});
jss::dataflow::task([&y]{
for(int i=0;i<count;++i) { y<<rand()%20; }});
for(int i=0;i<count;++i) { std::cout<<z.pop()<<’\n’; }
}
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 35: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/35.jpg)
Dataflow (IV)
Dataflow concurrency is anextension of the pipeline pattern toa general DAG
The runtime provides facilities tosplit, filter and combine channels
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 36: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/36.jpg)
Communication Patterns
![Page 37: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/37.jpg)
Communication Patterns
Read Only Data
Mutexes and Locks
Futures
Queues and Channels
Other Data Structures Designedfor Concurrent Access
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 38: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/38.jpg)
Mutexes and Locks
![Page 39: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/39.jpg)
Mutexes and Locks (I)
Explicitly limit concurrency!
Low level mechanism
Good for migrating sequential code
Wrappers such assynchronized value<T> canavoid correctness issues
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 40: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/40.jpg)
Mutexes and Locks (II): synchronized value<T>
void foo(jss::synchronized_value<std::string>& s) {
std::string local=*s;
s->append("foo");
jss::update_guard<std::string> guard(s);
unsigned pos=guard->find("f");
if(pos==std::string::npos)
*guard += "bar";
else
(*guard)[pos] = ’b’;
}
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 41: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/41.jpg)
Futures
![Page 42: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/42.jpg)
Futures (I)
Synchronization is internal tolibrary
Read and write operations areexplicit
One-shot communication
Every task framework has someform of future
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 43: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/43.jpg)
Futures (II)
In C++0x, futures are used with:std::async, as in the fork/joinexamplestd::promise, as in the searchexamplestd::packaged task, a buildingblock for task queues and threadpools
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 44: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/44.jpg)
Queues and Channels
![Page 45: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/45.jpg)
Queues and Channels (I)
Allow multiple data items to betransferred
Fundamental to message-passingsystems
Myriad of choices:SP/MP, SC/MC, bounded/unbounded, etc.
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 46: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/46.jpg)
Queues and Channels (II): Bounded vs Unbounded
Bounded Queues limit thenumber of items in the queue⇒ Producer blocks if the queue is full
Unbounded Queues have no suchlimit⇒ May consume a lot of memory if producer
runs faster than consumer
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 47: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/47.jpg)
Queues and Channels (III): Bounded vs Unbounded
Unbounded queues handle“bursty” data better
Bounded queues even out theprocessing
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 48: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/48.jpg)
Queues and Channels (IV): SPSC and MPSC
Single Producer Single Consumer⇒ one-to-one channel between 2 threads, actors
or tasks
Multiple Producer SingleConsumer⇒ standard “message passing” queue, such as
an Actor’s mailbox
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 49: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/49.jpg)
Queues and Channels (V): SPMC and MPMC
Single Producer MultipleConsumer⇒ broadcast channel
Multiple Producer MultipleConsumer⇒ general purpose queue
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 50: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/50.jpg)
Queues and Channels (VI)
Special-purpose queues may haveadditional properties
Work-stealing queues
Dataflow channels
Priority queues
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 51: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/51.jpg)
Other Data Structures Designed forConcurrent Access
![Page 52: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/52.jpg)
Other Data Structures Designed for Concurrent Access (I)
More than one thread can accessthe data structure concurrently,without either one waiting for theother
There may be restrictions onwhich operations can be calledconcurrently
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 53: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/53.jpg)
Other Data Structures Designed for Concurrent Access (II)
Stacks
Hash tables
Other variations on lists, sets,maps and queues
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 54: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/54.jpg)
Concurrent Hash Tables (I)
Allow concurrent queries andmodifications
May allow concurrent removal
May allow concurrent iteration
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 55: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/55.jpg)
Concurrent Hash Tables (II)
Use for lookup tables — e.g. finduser information by ID
Cache results — e.g. DNS queries
Can be faster than map/reduce onSMP systems
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 56: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/56.jpg)
Concurrent Hash Tables (III): Word count
typedef jss::concurrent_map<
std::string,std::atomic<unsigned>> map_type;
void count_words(
map_type& counts, word_list const& words) {
std::string word;
while(words.get_next(word)) {
std::pair<map_type::iterator,bool>
value=counts.insert(s,1);
if(!value.second)
++(value.first->second);
}
}
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 57: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/57.jpg)
Other Concurrent Data Structures
Skip lists: Java providesConcurrentSkipListMap andConcurrentSkipListSet
TBB has concurrent vector
More complex data structures endup with a mutex lock
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 58: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/58.jpg)
Choosing your patterns
![Page 59: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/59.jpg)
Choosing your patterns (I)
What style of problem is it?
event-driven
recursive
embarassingly parallel
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 60: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/60.jpg)
Choosing your patterns (II)
What level of the application are weat?
top level
background processing
inner loop
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 61: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/61.jpg)
Choosing your patterns (III)
How can we best split the tasks anddata?
few/many tasks
small/large amounts of data
simple/complex interactions
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 62: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/62.jpg)
Choosing your patterns (IV)
What scale are we at?
Single multi-core processor
Multiple multi-core processors
Local cluster
Globally distributed
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 63: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/63.jpg)
Further Reading
Patterns for Parallel Programming, Timothy G. Mattson,Bervely A. Sanders, Berna L. Massingill, Addison Wesley.
Programming Scala, Dean Wampler, Alex Payne, O’ReillyMedia (Available online)
The GPars Project — Reference Documentation(Available online)
Concurrent Programming in Erlang, Joe Armstrong,Robert Virding, Claes Wikstrom, Mike Williams, PrenticeHall
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 64: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/64.jpg)
just::thread provides a complete implementation of the C++0x threadlibrary for MSVC 2005, 2008 and 2010, and g++ 4.3, 4.4 and 4.5 for
Ubuntu/Debian/Fedora. MacOSX support coming soon.For a 50% discount go to:
http://www.stdthread.co.uk/accu2011
Just::Thread Pro also coming soon, with support for many of the high levelfacilities shown in this presentation. Find out more at:
http://www.stdthread.co.uk/pro
![Page 65: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/65.jpg)
My book
C++ Concurrency in Action: PracticalMultithreading with the new C++Standard, currently available under theManning Early Access Program, anddue to be printed this summer.
http://www.stdthread.co.uk/book/
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs
![Page 66: Picking Patterns for Parallel Programs · Buzzword Bingo Actors Agents Message Queues Tasks CSP Data ow Map-reduce Locks Continuations Lock-free False Sharing Fork/Join Work-Stealing](https://reader035.vdocuments.net/reader035/viewer/2022070108/6041ffba7c30e01c5e6c48e2/html5/thumbnails/66.jpg)
Photo credits
The images used in this presentation are courtesy of:
Daniela Hartmann Florian SeroussiMary Harrsch nicolas geninEric Gilliland Barbara DodukDanie Ware PascalPaul Hughes ClayOgre
Travis subblueOrdnance Survey OpenData
Anthony Williams Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk
Picking Patterns for Parallel Programs