multiprocessors— flynn categories, large vs. small scale, cache coherency

Multiprocessors—Flynn Categories, Large vs. Small Scale,

Cache Coherency

Professor Alvin R. Lebeck

Computer Science 220

Fall 2001

2© Alvin R. Lebeck 1999

What is Parallel Computer Architecture?

• A Parallel Computer is a collection of processing elements that cooperate to solve large problems fast

– how large a collection?

– how powerful are the elements?

– how does it scale up?

– how do they cooperate and communicate?

– how is data transmitted between processors?

– what are the primitive abstractions?

– how does it all translate to performance?


Parallel Computation: Why and Why Not?

• Pros– Performance

– Cost-effectiveness (commodity parts)

– Smooth upgrade path

– Fault Tolerance

• Cons– Difficult to parallelize applications

– Requires automatic parallelization or parallel program development

– Software! AAHHHH!

CPS 220 4© Alvin R. Lebeck 1999

Flynn Categories

• SISD (Single Instruction Single Data)– Uniprocessors

• MISD (Multiple Instruction Single Data)– ???

• SIMD (Single Instruction Multiple Data)– Examples: Illiac-IV, CM-2, Intel MMX

» Simple programming model» Low overhead» Flexibility» All custom processors

• MIMD (Multiple Instruction Multiple Data)– Examples: Intel 4-way SMP, SUN ES3000, SGI Origin, Cray T3D

» Flexible» Use off-the-shelf microprocessors


Communication Models

• Shared Memory– Processors communicate with shared address space

– Easy on small-scale machines

– Advantages:

» Model of choice for uniprocessors, small-scale MPs

» Ease of programming

» Lower latency

» Easier to use hardware controlled caching

• Message passing– Processors have private memories, communicate via messages

– Advantages:

» Less hardware, easier to design

» Focuses attention on costly non-local operations

• Can support either model on either HW base


Simple Problem

for i = 1 to N

A[i] = (A[i] + B[i]) * C[i]

sum = sum + A[i]

• How do you make this loop parallel to run on many processors?


Simple Problem

• Split the loops// Independent iterations

// Run on up to N processors

for i = 1 to N

A[i] = (A[i] + B[i]) * C[i]// One last loop to run on one processor

for i = 1 to N

sum = sum + A[i]


Small Scale Shared Memory Multiprocessors

• Small number of processors connected to one shared memory

• Memory is equidistant from all processors (UMA)

• Kernel can run on any processor (symmetric MP)

• Intel dual/quad Pentium, IBM, SUN, Compaq, almost everyone

• Some are moving on-chip (e.g., IBM)

Main Memory

P

$

P

$

P

$

P

$

P

$

P

$

P

$

P

$

Cache(s)andTLB

0 N-1


Large Scale Shared Memory Multiprocessors Shared

• 100s to 1000s of nodes (processors) with single shared physical address space

• Use General Purpose Interconnection Network

– Still have cache coherence protocol

– Use messages instead of bus transactions

– No hardware broadcast

• Communication Assist• Cray T3D, T3E,

Compaq EV7, SUN ES3000

Interconnect

Cntrl/NI

Mem

P

$

Cntrl/NI

Mem

P

$

Cntrl/NI

Mem

P

$

Cntrl/NI

Mem

P

$


Message Passing Architectures

• Cannot directly access memory on another node

• IBM SP-2, Intel Paragon

• Cluster of workstations

Interconnect

CA

MemP

$CA

MemP

$

Node 0 0,N-1 Node 1 0,N-1

Node 2 0,N-1 Node 3 0,N-1

CA

MemP

$ CA

MemP

$


Important Communication Properties

• Bandwidth– Need high bandwidth in communication

– Cannot scale, but stay close

– Limits may be in network, memory, and processor

– Overhead to communicate is a problem in many machines

• Latency– Affects performance, since processor may have to wait

– Affects ease of programming, since requires more thought to overlap communication and computation

• Latency Hiding– How can a mechanism help hide latency?

– Examples: overlap message send with computation, prefetch


Small-Scale—Shared Memory

• Caches serve to:– Increase bandwidth versus bus/memory

– Reduce latency of access

– Valuable for both private data and shared data

• What about cache coherence?

Main Memory

P

$

P

$

P

$

P

$

P

$

P

$

P

$

P

$

Cache(s)andTLB

0 N-1


Cache Coherence Problem (Initial State)

P1 P2

x

Interconnection Network / Bus

Main Memory

Tim

e


Cache Coherence Problem (Step 1)

P1 P2

x


Main Memory

Tim

e

ld r2, x



P1 P2

x


Main Memory

ld r2, x

Tim

e

ld r2, x



P1 P2

x


Main Memory

ld r2, xadd r1, r2, r4st x, r1T

ime

ld r2, x


The Problem of Cache Coherence (4)

P1 P2ld r2, xadd r1, r3, r4st x, r1ld r4,y

ld r2, xadd r1, r2, r3st y, r1ld r5, x

x

Tim

e


Main Memory

y

yx


Coherence vs. Consistency

• Intuition says loads should return latest value– what is latest?

• Coherence concerns only one memory location

• Consistency concerns apparent ordering for all locations

• A Memory System is Coherent if– can serialize all operations to that location such that,

– operations performed by any processor appear in program order

» program order = order defined program text or assembly code

– value returned by a read is value written by last store to that location


Why Coherence != Consistency

/* initial A = B = flag = 0 */

P1 P2

A = 1; while (flag == 0); /* spin */

B = 1; print A;

flag = 1; print B;

Intuition says printed A = B = 1

Coherence doesn’t say anything, why?


The Sequential Consistency Memory Model

P1 P2 P3

switch randomly setafter each memory op

sequential processorsissue memory opsin program order

Memory


Sufficient Conditions for Sequential Consistency

• Every processor issues memory ops in program order

• Processor must wait for store to complete before issuing next memory operation

• After load, issuing proc waits for load to complete, and store that produced value to complete before issuing next op

• Easily implemented with shared bus.


Potential Solutions

• Snooping Solution (Snoopy Bus):– Send all requests for data to all processors

– Processors snoop to see if they have a copy and respond accordingly

– Requires broadcast, since caching information is at processors

– Works well with bus (natural broadcast medium)

– Dominates for small scale machines (most of the market)

» probably won’t scale beyond 2-4 processors

• Directory-Based Schemes– Keep track of what is being shared in one centralized place

– Distributed memory => distributed directory (avoids bottlenecks)

– Send point-to-point requests to processors

– Scales better than Snoop

– Actually existed BEFORE Snoop-based schemes


Basic Snoopy Protocols

• Write Invalidate Protocol:– Multiple readers, single writer

– Write to shared data: an invalidate is sent to all caches which snoop and invalidate any copies

– Read Miss:

» Write-through: memory is always up-to-date

» Write-back: snoop in caches to find most recent copy

• Write Broadcast (Update) Protocol:– Write to shared data: broadcast on bus, processors snoop, and update copies

– Read miss: memory is always up-to-date

• Write serialization: bus serializes requests– Bus is single point of arbitration


Snoopy Cache-Coherence Protocols

• Bus provides serialization point for consistency– but, but, what about write-buffers? Later in the semester….

• Each cache controller “snoops” all bus transactions– relevant transactions if for a block it contains

– take action to ensure coherence

» invalidate

» update

» supply value

– depends on state of the block and the protocol

• Simultaneous Operation of Independent Controllers


Snoopy Design Choices

Processorld/st

Snoop (observed bus transaction)

State Tag Data

. . .

• Controller updates state of blocks in response to processor and snoop events and generates bus xactions

• Often have duplicate cache tags

• Snoopy protocol

– set of states

– state-transition diagram

– actions

• Basic Choices

– write-through vs. write-back

– invalidate vs. update

Cache


The Simple Invalidate Snoopy Protocol

• Write-through, no-write-allocate cache

• Actions: PrRd, PrWr, BusRd, BusWr

PrRd / PrWr

Valid

BusWr

Invalid

PrWr / BusWr

PrRd / BusRd


A 3-State Write-Back Invalidation Protocol

• 2-State Protocol+ Simple hardware and protocol

– Bandwidth (every write goes on bus!)

• 3-State Protocol (MSI)– Modified

» one cache has valid/latest copy

» memory is stale

– Shared

» one or more caches have valid copy

– Invalid

• Must invalidate all other copies before entering modified state

• Requires bus transaction (order and invalidate)


MSI Processor and Bus Actions

• Processor: – PrRd

– PrWr

– Writeback on replacement of modified block

• Bus– Bus Read (BusRd) Read without intent to modify, data could come

from memory or another cache

– Bus Read-Exclusive (BusRdX) Read with intent to modify, must invalidate all other caches copies

– Writeback (BusWB) cache controller puts contents on bus and memory is updated

– Definition: cache-to-cache transfer occurs when another cache satisfies BusRd or BusRdX request

• Let’s draw it!


MSI State Diagram

PrRd

MBusRdX / BusWB

PrWr / BusRdX

S

I

PrWr

BusRd / BusWB

PrWr / BusRdX

PrRd / BusRd

BusRdX

PrRdBusRd


An example

Proc Action P1 State P2 state P3 state Bus Act Data from

1. P1 read u S -- -- BusRd Memory

2. P3 read u S -- S BusRd Memory

3. P3 write u I -- M BusRdX Memory or not

4. P1 read u S -- S BusRd P3’s cache

5. P2 read u S S S BusRd Memory

• Single writer, multiple reader protocol

• Why Modified to Shared?

• What if not in any cache?– Read, Write produces 2 bus transactions!


4-State (MESI) Invalidation Protocol

• Often called the Illinois protocol

• Modified (dirty)

• Exclusive (clean unshared) only copy, not dirty

• Shared

• Invalid

• Requires shared signal to detect if other caches have a copy of block

• Cache Flush for cache-to-cache transfers– Only one can do it though

• What does state diagram look like?


4-State Write-back Update Protocol

• Dragon (Xerox PARC)

• States– Exclusive (E): one copy, clean, memory is up-to-date

– Shared-Clean (SC): could be two or more copies, memory unknown

– Shared-Modified (SM): could be two or more copies, memory stale

– Modified (M)

• Adds Bus Update Transaction

• Adds Cache Controller Update operation

• Must obtain bus before updating local copy

• What does state diagram look like?– let’s look at the actions first


Basic Snoopy Protocols

• Write Invalidate versus Broadcast:– Invalidate requires one transaction per write-run

– Invalidate uses spatial locality: one transaction per block

– Broadcast has lower latency between write and read

– Broadcast: BW (increased) vs. latency (decreased) tradeoff

Name Protocol Type Memory-write policy Machines using

Write Once Write invalidate Write back First snoopy protocol.after first write

Synapse N+1 Write invalidate Write back 1st cache-coherent MPs

Berkeley Write invalidate Write back Berkeley SPUR

Illinois Write invalidate Write back SGI Power and Challenge

“Firefly” Write broadcast Write back private,Write through shared SPARCCenter 2000


Larger MPs

• Separate Memory per Processor

• Local or Remote access via memory controller

• Cache Coherency solution: non cached pages

• Alternative: directory/cache that tracks state of every block in every cache

– Which caches have a copies of block, dirty vs. clean, ...

• Info per memory block vs. per cache block?– In memory => simpler protocol (centralized/one location)

– In memory => directory is ƒ(memory size) vs. ƒ(cache size)

• Prevent directory as bottleneck: distribute directory entries with memory, each keeping track of which Procs have copies of their blocks


Directory Protocol

• Similar to Snoopy Protocol: 3 states– Shared: > 1 processors have data, memory up to date

– Uncached

– Exclusive: 1 processor(owner) has data; memory out of date

• In addition to cache state, must track which processors have data when in shared state

• Terms:– Local node is the node where a request originates

– Home node is the node where the memory location of an address resides

– Remote node is the node that has a copy of a cache block, whether exclusive or shared.


Example Directory Protocol• Message sent to directory causes 2 actions:

– update the directory– more messages to satifty request

• Block is in Uncached state: the copy in memory is the current value, & only possible requests for that block are:

– Read miss: requesting processor is sent back the data from memory and the requestor is the only sharing node. The state of the block is made Shared.

– Write miss: requesting processor is sent the value and becomes the Sharing node. The block is made Exclusive to indicate that the only valid copy is cached. Sharers indicates the identity of the owner.

• Block is Shared, the memory value is up-to-date:– Read miss: requesting processor is sent back the data from memory &

requesting processor is added to the sharing set.– Write miss: requesting processor is sent the value. All processors in the set

Sharers are sent invalidate messages, & Sharers set is to identity of requesting processor. The state of the block is made Exclusive.


Example Directory Protocol

• Block is Exclusive: current value of the block is held in the cache of the processor identified by the set Sharers (the owner), & 3 possible directory requests:

– Read miss: owner processor is sent a data fetch message, which causes state of block in owner’s cache to transition to Shared and causes owner to send data to directory, where it is written to memory and sent back to the requesting processor. Identity of requesting processor is added to set Sharers, which still contains the identity of the processor that was the owner (since it still has a readable copy).

– Data write-back: owner processor is replacing the block and hence must write it back. This makes the memory copy up-to-date (the home directory essentially becomes the owner), the block is now uncached, and the Sharer set is empty.

– Write miss: block has a new owner. A message is sent to old owner causing the cache to send the value of the block to the directory from which it is send to the requesting processor, which becomes the new owner. Sharers is set to identity of new owner, and state of block is made Exclusive.

multiprocessors— flynn categories, large vs. small scale, cache coherency

Documents

cache coherencyprofessor

shared memorymemory

loop parallel

shared address spaceeasy

communicationcannot

processors umakernel

parallel computation

n processorsfor