part 3. what are the general types of parallelism that we already discussed?
TRANSCRIPT
Part 3
What are the general types of parallelism that we already discussed?
What are the general types of parallelism that we already discussed?
1. instruction-level
2. processor-level
Types:1. Array2. Vector3. Multiprocessor/multicore4. Multicomputer5. COWs (cluster of workstations)
Tanenbaum, Structured Computer Organization, Fifth Edition, (c)
2006 Pearson Education, Inc. All rights reserved. 0-13-148521-0
Type 1: array processor- ex. ILLIAC IV- 1 CPU; 64 AUs (arithmetic units)
Type 2: vector processor 1 CPU w/ (a small number, say 8) vector registers
but (typically) only one AU
example is Intel MMX, SSE1 – SSE4 SIMD instuctions SSE: 8 128-bit registers (xmm0..xmm7)
Each 128-bit register may contain: 4 32-bit fp numbers 2 64-bit fp numbers 4 32-bit integers 8 16-bit ints 16 8-bit ints
Tanenbaum, Structured Computer Organization, Fifth Edition, (c)
2006 Pearson Education, Inc. All rights reserved. 0-13-148521-0
Type 3: multiprocessor multicomputer with optional local memories and shared memory; common bus typically 64 or less CPUs OpenMP (Multi-Processing; openmp.org)
Type 4: multicomputer
like multiprocessor, but no shared memory
as many as 10,000 CPUs
different topologies (such as 2d grid, 3d grid, trees, or rings) to speed communications
Type 5: COW (Cluster Of Workstations)
COTS (Common Off The Shelf) systems communicating via COTS network hardware
MPI standard programming library Message Passing Interface http://www-unix.mcs.anl.gov/mpi/ (not to be confused with OpenMP)