Parallel ArchitecturesBased on Parallel Computing, M. J. Quinn

Ashok Srinivasanwww.cs.fsu.edu/~asriniva

Florida State University

CIS 5930-04 – Spring 2006: Parallel Computing

Outline• Interconnection Networks

– Mesh– Binary Tree– Hypercube

• Architectures– Processor Arrays– Multiprocessors

• Centralized Multiprocessors• Distributed Multiprocessors

– Multicomputers• Flynn’s Taxonomy

– SISD– SIMD– MIMD

Mesh

• 2-D Mesh– N = d2

– Diameter = 2 N0.5

– Bisection width = N0.5

– Edges/node = 4• 2-D Mesh with

wraparound (2-D torus)• 3-D Mesh• 3-D torus

Binary Tree

• Binary Tree– Diameter = 2 log N– Bisection width = 1– Edges/node = 3

• Fat Tree– Double the number

of edges at each level up the tree

Hypercube

• Hypercube– N = 2d

– Diameter = log N– Bisection width = N/2– Edges/node = log N

Processor Arrays

Centralized Multiprocessors

• Also called SMP/UMA

Cache Coherence Problem

• Solve through snooping

Distributed Multiprocessors

• Also called NUMA

Cache Coherence - Directory Based Solution

Multicomputers

• Also called a distributed memory system

Flynn’s Taxonomy

• SISD– Single Instruction, Single Data– Traditional computer

• SIMD– Single Instruction, Multiple Data

• MIMD– Multiple Instruction, Multiple Data