multi-core. what is parallel programming ? classification of parallel architectures dimension of...
TRANSCRIPT
Multi-core
What is parallel programming ? Classification of parallel architectures
Dimension of instruction Dimension of data
Memory models for parallel programming Distributed memory model Share memory model
Multi-core architectures Steps in parallelization
Jaruloj Chongstitvatana 2Parallel Programming: Introduction
CONCURRENT
Two or more actions progress at the same time.
PARALLEL
Two or more actions execute simultaneously.
Jaruloj Chongstitvatana Parallel Programming: Introduction 3
How to write a program that can execute correctly on two proccesors or more.
Design considerations Correctness Efficiency Simplicity Scalability
Jaruloj Chongstitvatana Parallel Programming: Introduction 4
Single Instruction Single Data: SISD
Single Instruction Multiple Data: SIMD
Multiple Instruction Single Data: MISD
Multiple Instruction Multiple Data: MIMD
Jaruloj Chongstitvatana Parallel Programming: Introduction 5
No parallelism
Jaruloj Chongstitvatana Parallel Programming: Introduction 6
Vector machine Graphic processors
Jaruloj Chongstitvatana Parallel Programming: Introduction 7
No application (at this time)
Jaruloj Chongstitvatana Parallel Programming: Introduction 8
Multi-core architectures
Jaruloj Chongstitvatana Parallel Programming: Introduction 9
SHARED MEMORY MODEL
Multiple processors connect to one shared memory.
All processors can access same memory locations.
DISTRIBUTED MEMORY MODEL
Each processor connects to its private memory.
Jaruloj Chongstitvatana Parallel Programming: Introduction 10
SHARED MEMORY
Bottleneck for memory access
No data transfer
Easy for share data
DISTRIBUTED MEMORY
Better memory access
Data transfer from non-local memory
Easy for private data
Jaruloj Chongstitvatana Parallel Programming: Introduction 11
AMD Multicore Opteron Sun UltraSparc T1 IBM Cell Broadband Engine (CBE) Intel Core2 Duo
Jaruloj Chongstitvatana Parallel Programming: Introduction 12
Jaruloj Chongstitvatana Parallel Programming: Introduction 13
Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.
Jaruloj Chongstitvatana Parallel Programming: Introduction 14
Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.
Jaruloj Chongstitvatana Parallel Programming: Introduction 15
Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.
Jaruloj Chongstitvatana Parallel Programming: Introduction 16
Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.
Jaruloj Chongstitvatana Parallel Programming: Introduction 17
Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.
Starting from sequential code
Identify possible concurrency Design and implement Test for correctness Tuning for performance
May effect correctness If tuning is impossible, consider redesign
Jaruloj Chongstitvatana Parallel Programming: Introduction 18
Data race Deadlock
Notes: Results might be slightly different for each
run due to round-off error for different sequence of operations.
Jaruloj Chongstitvatana Parallel Programming: Introduction 19