Pipelining - II

Adapted from CS 152C (UC Berkeley) lectures notes of Spring 2002

Revisiting Pipelining Lessons• Pipelining doesn’t help

latency of single task, it helps throughput of entire workload

• Pipeline rate limited by slowest pipeline stage

• Multiple tasks operating simultaneously using different resources

• Potential speedup = Number pipe stages

• Unbalanced lengths of pipe stages reduces speedup

• Time to “fill” pipeline and time to “drain” it reduces speedup

• Stall for Dependences

A

B

C

D

6 PM 7 8 9

Task

Order

Time

30 40 40 40 40 20

• Structural Hazards– Hardware design

• Control Hazard– Decision based on results

• Data Hazard– Data Dependency

Revisiting Pipelining Hazards

Control Signals for existing Datapath

The Right to Left Control can lead to hazards

MUX

PC

ADD

Registers

ADD

ADD

SignExtend

4

InstructionMemory

Address

Instruction

Read Reg1

Read Reg2

Write Reg

Write Data

Read Data1

ReadData2 M

UX

DataMemory

Address

Write Data

Read Data

MUX

IF: Instruction Fetch ID: Instruction Decode/ register file read

EX: Execute/address calculation MEM: Memory Access WB: Write back

16 32

Shift left 2

Zero

Place registers between each step

MUX

PC

ADD

Registers

ADD

ADD

SignExtend

4

InstructionMemory

Address

Instruction

Read Reg1

Read Reg2

Write Reg

Write Data

Read Data1

ReadData2 M

UX

DataMemory

Address

Write Data

Read Data

MUX

16 32

Shift left 2

Zero

IF/ID ID/EX EX/MEM MEM/WB

Example10 lw r1, r2(35)14 addI r2, r2, 320 sub r3, r4, r524 beq r6, r7, 10030 ori r8, r9, 1734 add r10, r11, r12

100 and r13, r14, 15

Start: Fetch 10

Exe

c

Reg

. Fi

le

Mem

Acc

ess

Dat

aM

em

A

B

SReg

File

IR

Inst

. Mem

D

Dec

ode

MemCtrl

WB Ctrl

M

rs rt im

10 lw r1, r2(35)14 addI r2, r2, 320 sub r3, r4, r524 beq r6, r7, 10030 ori r8, r9, 1734 add r10, r11, r12

100 and r13, r14, 15

IF

PC

Nex

t PC

10

=

n n n n

Fetch 14, Decode 10

Exe

c

Reg

. Fi

le

Mem

Acc

ess

Dat

aM

em

A

B

SReg

File

IR

Inst

. Mem

D

Dec

ode

MemCtrl

WB Ctrl

M

2 rt im

10 lw r1, r2(35)

14 addI r2, r2, 320 sub r3, r4, r524 beq r6, r7, 10030 ori r8, r9, 1734 add r10, r11, r12

100 and r13, r14, 15

lw r1

, r2(

35)

ID

IF

PC

Nex

t PC

14

=

n n n

Fetch 20, Decode 14, Exec 10

Exe

c

Reg

. Fi

le

Mem

Acc

ess

Dat

aM

em

r2

B

SReg

File

IR

Inst

. Mem

D

Dec

ode

MemCtrl

WB Ctrl

M

2 rt 35

10 lw r1, r2(35)

14 addI r2, r2, 3

20 sub r3, r4, r524 beq r6, r7, 10030 ori r8, r9, 1734 add r10, r11, r12

100 and r13, r14, 15

lw r1

addI

r2, r

2, 3

EX

PC

Nex

t PC

20

=

n n

Fetch 24, Decode 20, Exec 14, Mem 10

Exe

c

Reg

. Fi

le

Mem

Acc

ess

Dat

aM

em

r2

B

r2+3

5

Reg

File

IR

Inst

. Mem

D

Dec

ode

MemCtrl

WB Ctrl

M

4 5 3

10 lw r1, r2(35)

14 addI r2, r2, 3

20 sub r3, r4, r5

24 beq r6, r7, 10030 ori r8, r9, 1734 add r10, r11, r12

100 and r13, r14, 15

lw r1

sub

r3, r

4, r5

addI

r2, r

2, 3

ID

IF

EX

M

PC

Nex

t PC

24

=

n

Fetch 30, Dcd 24, Ex 20, Mem 14, WB 10

Exe

c

Reg

. Fi

le

Mem

Acc

ess

Dat

aM

em

r4

r5

r2+3

Reg

File

IR

Inst

. Mem

D

Dec

ode

MemCtrl

WB Ctrl

M[r2

+35]

6 7

10 lw r1, r2(35)

14 addI r2, r2, 3

20 sub r3, r4, r5

24 beq r6, r7, 100

30 ori r8, r9, 1734 add r10, r11, r12

100 and r13, r14, 15

lw r1

beq

r6, r

7 10

0

addI

r2

sub

r3

ID

IF

EX

M WB

PC

Nex

t PC

30

=

Fetch 100, Dcd 30, Ex 24, Mem 20, WB 14

Exe

c

Reg

. Fi

le

Mem

Acc

ess

Dat

aM

em

r6

r7

r2+3

Reg

File

IR

Inst

. Mem

D

Dec

ode

MemCtrl

WB Ctrl

r1=M

[r2+3

5]

9 xx

10 lw r1, r2(35)

14 addI r2, r2, 3

20 sub r3, r4, r5

24 beq r6, r7, 100

30 ori r8, r9, 17

34 add r10, r11, r12

100 and r13, r14, 15

beq

addI

r2

sub

r3r4

-r5

100

ori r

8, r9

17

ID

IF

EX

M WB

PC

Nex

t PC

100

=

Pipelining Load Instruction

• The five independent functional units in the pipeline datapath are:– Instruction Memory for the Ifetch stage– Register File’s Read ports (bus A and busB) for the

Reg/Dec stage– ALU for the Exec stage– Data Memory for the Mem stage– Register File’s Write port (bus W) for the Wr stage

Clock

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7

Ifetch Reg/Dec Exec Mem Wr1st lw

Ifetch Reg/Dec Exec Mem Wr2nd lw

Ifetch Reg/Dec Exec Mem Wr3rd lw

Pipelining the R Instruction

• Ifetch: Instruction Fetch– Fetch the instruction from the Instruction Memory

• Reg/Dec: Registers Fetch and Instruction Decode• Exec:

– ALU operates on the two register operands– Update PC

• Wr: Write the ALU output back to the register file

Cycle 1 Cycle 2 Cycle 3 Cycle 4

Ifetch Reg/Dec Exec WrR-type

Pipelining Both L and R type

• We have pipeline conflict or structural hazard:– Two instructions try to write to the register file at

the same time!– Only one write port

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9

Ifetch Reg/Dec Exec WrR-type

Ifetch Reg/Dec Exec WrR-type

Ifetch Reg/Dec Exec Mem WrLoad

Ifetch Reg/Dec Exec WrR-type

Ifetch Reg/Dec Exec WrR-type

Ops! We have a problem!

Important Observations• Each functional unit can only be used once per

instruction• Each functional unit must be used at the same

stage for all instructions:– Load uses Register File’s Write Port during its 5th

stage

– R-type uses Register File’s Write Port during its 4th stage

Ifetch Reg/Dec Exec Mem WrLoad1 2 3 4 5

Ifetch Reg/Dec Exec WrR-type1 2 3 4

Solution• Delay R-type’s register write by one cycle:

– Now R-type instructions also use Reg File’s write port at Stage 5– Mem stage is a NOOP stage: nothing is being done.

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9

Ifetch Reg/Dec Mem Wr

Ifetch Reg/Dec Mem WrR-type

Ifetch Reg/Dec Exec Mem WrLoad

Ifetch Reg/Dec Mem WrR-type

Ifetch Reg/Dec Mem WrR-type

Ifetch Reg/Dec Exec WrR-type Mem

Exec

Exec

Exec

Exec

1 2 3 4 5

Datapath (Without Pipeline)IR <- Mem[PC]; PC <– PC+4;

A <- R[rs]; B<– R[rt]

S <– A + B;

R[rd] <– S;

S <– A + SX;

M <– Mem[S]

R[rd] <– M;

S <– A or ZX;

R[rt] <– S;

S <– A + SX;

Mem[S] <- B

If CondPC < PC+SX;

Exe

c

Reg

. Fi

le

Mem

Acc

ess

Dat

aM

em

A

B

SReg

File

Equ

al

PC

Nex

t PC

IR

Inst

. Mem

D

M

Datapath (With Pipeline)IR <- Mem[PC]; PC <– PC+4;

A <- R[rs]; B<– R[rt]

S <– A + B;

R[rd] <– M;

S <– A + SX;

M <– Mem[S]

R[rd] <– M;

S <– A or ZX;

R[rt] <– M;

S <– A + SX;

Mem[S] <- B

if Cond PC < PC+SX;

M <– S

Exe

c

Reg

. Fi

le

Mem

Acc

ess

Dat

aM

em

A

B

SReg

File

Equ

al

PC

Nex

t PC

IR

Inst

. Mem

D

M

M <– S

Mem

Structural Hazard and Solution

Instr.

Order

Time (clock cycles)

Load

Instr 1

Instr 2

Instr 3

Instr 4A

LUMem Reg Mem Reg

AL

UMem Reg Mem Reg

AL

UMem Reg Mem RegA

LUReg Mem Reg

AL

UMem Reg Mem Reg

Control Hazard - #1 Stall

• Stall: wait until decision is clear• Impact: 2 lost cycles (i.e. 3 clock cycles per

branch instruction) => slow

Instr.

Order

Time (clock cycles)

Add

Beq

Load

AL

UMem Reg Mem Reg

AL

UMem Reg Mem RegA

LUReg Mem RegMem

Lostpotential

Control Hazard – #2 Predict

• Predict: guess one direction then back up if wrong• Impact: 0 lost cycles per branch instruction if right,

1 if wrong (right 50% of time)• More dynamic scheme: history of 1 branch

Instr.

Order

Time (clock cycles)

AddBeqLoad

AL

UMem Reg Mem Reg

AL

UMem Reg Mem Reg

Mem

AL

UReg Mem Reg

Control Hazard - #3 Delayed Branch

• Delayed Branch: Redefine branch behavior (takes place after next instruction)

• Impact: 0 clock cycles per branch instruction if can find instruction to put in “slot” ( 50% of time)

Instr.

Order

Time (clock cycles)

Add

Beq

Misc

AL

UMem Reg Mem Reg

AL

UMem Reg Mem Reg

Mem

AL

UReg Mem Reg

Load Mem

AL

UReg Mem Reg

Data Hazards (RAW)

• Dependencies backwards in time are hazards

Instr.

Order

Time (clock cycles)

add r1,r2,r3

sub r4,r1,r3

and r6,r1,r7

or r8,r1,r9

xor r10,r1,r11

IF

ID/RF EX MEM WBAL

UIm Reg Dm Reg

AL

UIm Reg Dm Reg

AL

UIm Reg Dm Reg

Im

AL

UReg Dm Reg

Data Hazards [contd…]• “Forward” result from one stage to another

Instr.

Order

Time (clock cycles)

add r1,r2,r3

sub r4,r1,r3

and r6,r1,r7

or r8,r1,r9

xor r10,r1,r11

IF

ID/RF

EX MEM WBAL

UIm Reg Dm Reg

AL

UIm Reg Dm RegA

LUIm Reg Dm Reg

Im

AL

UReg Dm Reg

AL

UIm Reg Dm Reg

Data Hazards [contd…]

Reg

• Dependencies backwards in time are hazards

• Can’t solve with forwarding: • Must delay/stall instruction dependent on loads

Time (clock cycles)

lw r1,0(r2)

sub r4,r1,r3

IF

ID/RF EX MEM WBAL

UIm Reg Dm

AL

UIm Reg Dm RegStall

Hazard DetectionI-Fetch DCD MemOpFetch OpFetch Exec Store

IFetch DCD ° ° °StructuralHazard

I-Fetch DCD OpFetch Jump

IFetch DCD ° ° °

Control Hazard

IF DCD EX Mem WB

IF DCD OF Ex Mem

RAW (read after write) Data Hazard

WAW Data Hazard (write after write)

IF DCD OF Ex RS WAR Data Hazard (write after read)

IF DCD EX Mem WB

IF DCD EX Mem WB

CPSC614Lec 2.28

• Read After Write (RAW) InstrJ tries to read operand before InstrI writes it

• Caused by a “Data Dependence” (in compiler nomenclature). This hazard results from an actual need for communication.

Three Generic Data Hazards

I: add r1,r2,r3J: sub r4,r1,r3

CPSC614Lec 2.29

• Write After Read (WAR) InstrJ writes operand before InstrI reads it

• Called an “anti-dependence” by compiler writers.This results from reuse of the name “r1”.

• Can’t happen in MIPS 5 stage pipeline because:– All instructions take 5 stages, and– Reads are always in stage 2, and – Writes are always in stage 5

I: sub r4,r1,r3 J: add r1,r2,r3K: mul r6,r1,r7

Three Generic Data Hazards

CPSC614Lec 2.30

Three Generic Data Hazards• Write After Write (WAW)

InstrJ writes operand before InstrI writes it.

• Called an “output dependence” by compiler writersThis also results from the reuse of name “r1”.

• Can’t happen in MIPS 5 stage pipeline because: – All instructions take 5 stages, and – Writes are always in stage 5

• Will see WAR and WAW in later more complicated pipes

I: sub r1,r4,r3 J: add r1,r2,r3K: mul r6,r1,r7

Hazard Detection• Suppose instruction i is about to be issued and a

predecessor instruction j is in the instruction pipeline.

• A RAW hazard exists on register if Rregs( i ) Wregs( j )• A WAW hazard exists on register if Wregs( i ) Wregs( j )• A WAR hazard exists on register if Wregs( i ) Rregs( j )

Window on execution:Only pending instructions cancause hazardsInst J

Inst INew Inst

InstructionMovement:

Computing CPI

2211

typetypetypetypestall

stallbase

freqSTALLfreqSTALLCPICPICPICPI

• Start with Base CPI• Add stalls

•Suppose: –CPIbase=1–Freqbranch=20%, freqload=30%–Suppose branches always cause 1 cycle stall–Loads cause a 2 cycle stall

•Then: CPI = 1 + (10.20)+(2 0.30)= 1.8

Summary

• Control Signals need to be propagated• Insert Registers between every stage to

“remember” and “propagate” values• Solutions to Control Hazard are Stall,

Predict and Delayed Branch• Solutions to Data Hazard is “Forwarding”• Effective CPI = CPIideal + CPIstall