why parallel? why now?
DESCRIPTION
Why Parallel? Why Now?. Introduction to Parallel Programming – Part 1. Objectives. At the end of this part you should be able to: Define parallel computing Explain why parallel computing is becoming mainstream Explain why explicit parallel programming is necessary. - PowerPoint PPT PresentationTRANSCRIPT
INTEL CONFIDENTIAL
Why Parallel? Why Now?Introduction to Parallel Programming – Part 1
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
2
Objectives
At the end of this part you should be able to:Define parallel computingExplain why parallel computing is becoming
mainstreamExplain why explicit parallel programming is
necessary
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
3
What Can We Do with Faster Computers?
Solve problems fasterReduce turn-around time of big jobsIncrease responsiveness of interactive apps
Get better solutions in same amount of timeIncrease resolution of modelsMake model more sophisticated
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
4
What is Parallel Computing?
Attempt to speed solution of a particular task by1. Dividing task into sub-tasks2. Executing sub-tasks simultaneously on
multiple coresSuccessful attempts require both
1. Understanding of where parallelism can be effective
2. Knowledge of how to design and implement good solutions
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
5
Why Parallel Computing?
“The free lunch is over.” —Herb SutterWe want applications to execute fasterClock speeds no longer increasing exponentially
10 GHz1 GHz
100 MHz10 MHz1 MHz
’79 ’87 ’95 ’03 ’11
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
6
Clock Speeds Have Flattened Out
Problems caused by higher speedsExcessive power consumptionHeat dissipationCurrent leakage
Power consumption critical for mobile devicesMobile computing platforms increasingly important
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
7
Execution Optimization
Popular optimizations to increase performanceInstruction prefetchingInstruction reorderingPipelined functional unitsBranch predictionFunctional unit allocationRegister allocationHyperthreading
Added sophistication more silicon devoted to control hardware
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
8
Multi-core Architectures
Potential performance = core speed # of coresStrategy:
Limit core speed and sophisticationPut multiple cores on a single chip
Potential performancethe same
core
s
Speed
1 2 42
4
4
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
9
Old Dynamic of Parallel Computing
Parallel computersare expensive
There are not manyparallel computers
Most people do not learnparallel programming
Parallel computingnot mainstream
Parallel programmingis difficult
Parallel programmingenvironments are inadequate
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
10
Sequential Language Approach
Problem has inherent parallelismProgramming language cannot express parallelismProgrammer hides parallelism in sequential
constructsCompiler and/or hardware must find hidden
parallelismSadly, doesn’t work
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
11
Alternate Approach
Programmer and Compiler Work TogetherProblem has inherent parallelismProgrammer has way to express parallelismCompiler translates program for multiple cores
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
12
Programmer/Compiler Team
Programmers of modern CPUs must take architecture and compiler into account in order to get peak performance
“…you can actively reorganize data and algorithms to take advantage of architectural capabilities…” Introduction to Microarchitectural Optimization for Itanium® 2 Processors, p. 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
13
New Dynamic of Parallel Computing
PCs are parallel computers
Everyone has aparallel computer
More people learningparallel programming
Parallel programmingconsidered mainstream
Parallel programminggets easier
Parallel programmingenvironments improve
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
14
References
Richard H. Carver and Kuo-Chung Tai, Modern Multithreading: Implementing, Testing, and Debugging Java and C++/Pthreads/ Win32 Programs, Wiley-Interscience (2006).
Robert L. Mitchell, “Decline of the Desktop,” Computerworld (September 26, 2005).
Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill (2004).
Herb Sutter, “The Free Lunch is Over: A Fundamental Turn Toward Concurrency in Software,” Dr. Dobb’s Journal 30, 3 (March 2005).