chapter 6 pipelining(2)

8/14/2019 Chapter 6 Pipelining(2)

1/58

PIPELININGChapter 6


2/58

Basic concepts

Speed of execution of programs can be improved in twoways: Faster circuit technology to build the processor and the memory.

Arrange the hardware so that a number of operations can beperformed simultaneously. The number of operations performed persecond is increased although the elapsed time needed to performany one operation is not changed.

Pipelining is an effective way of organizing concurrent activityin a computer system to improve the speed of execution ofprograms.
http://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_G_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_L_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_E_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_P_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_P_coloringkidsboys.gif


3/58

Basic concepts (Contd.,) Processor executes a program by fetching and executing instructions one after

the other.

This is known as sequential execution.

If F i refers to the fetch step, and E i refers to the execution step of instruction I i ,then sequential execution looks like:

F 1

E 1

F 2

E 2

F 3

E 3

1 2 3

What i f the execu t ion o f one ins t ruc t ion i s over lapped w i th thefetch ing o f the nex t one?


4/58

Basic concepts (Contd.,) Computer has two separate hardware units, one for fetching instructions and one

for executing instructions.

Instruction is fetched by instruction fetch unit and deposited in an intermediatebuffer B1.

Buffer enables the instruction execution unit to execute the instruction while the fetch unit is fetching the next instruction.

Results of the execution are deposited in the destination location specified by theinstruction.

Instruction fetch unit

Ex ecution unit

Interstage buffer B1


5/58

Basic concepts (Contd.,) Comp uter i s cont ro l led by a c lock who se per iod i s such tha t the fe tch and execute

s teps of any ins t ruct ion can be completed in one c lock cycle . Firs t c lock cycle :

- Fetch un i t fe tches an ins t ruc t ion I 1 (F 1 ) an d s to res i t in B 1. Second c lock cyc le :

- Fetch uni t fe tches an ins t ruc t ion I 2 (F 2 ) , an d ex ec u tion u n i t ex ec u tes in s truc tio n I 1 (E 1 ). Third c lock cyc le :

- Fetch un i t fe tches an ins t ruc t ion I 3 (F

3 ), an d ex ec u tion u n i t ex ec u tes in s truc t io n I

2 (E

2 ).

Four th c lock cyc le :- Execut ion u ni t executes ins t ruc t ion I 3 (E 3 ).

F 1 E 1

F 2 E 2

F 3 E 3

I 1

I 2

I 3

Instruction

Clock cycle 1 2 3 4 T ime


6/58

Basic concepts (Contd.,)

Suppose the processing of an instruction is divided into four steps:F Fetch: Read the instruction from the memory.D Decode: Decode the instruction and fetch the source operands.E Execute: Perform the operation specified by the instruction.W Write: Store the result in the destination location.

There are four distinct hardware units, for each one of the steps.

Information is passed from one unit to the next through an interstage buffer. Three interstage buffers connecting four units. As an instruction progresses through the pipeline, the information needed by thedownstream units must be passed along.

F : Fetch instruction

D : Decode instruction and fetch operands

E: Ex ecute operation

W : Write results

Interstage b uf fers

B1 B2 B3


7/58


F 4 I 4

F 1

F 2

F 3

I 1

I 2

I 3

D 1

D 2

D 3

D 4

E 1

E 2

E 3

E 4

W 1

W 2

W 3

W 4

Instruction

Clock cycle 1 2 3 4 5 6 7 T ime

Clock cycle 1: F1

Clock cycle 2: D1, F2Clock cycle 3: E1, D2, F3Clock cycle 4: W1, E2, D3, F4Clock cycle 5: W2, E3, D4Clock cycle 6: W3, E3, D4Clock cycle 7: W4


8/58


Buffer B1 holds instruction I 3, which is being decoded by the instruction-decodingunit. Instruction I 3 was fetched in cycle 3. Buffer B2 holds the source and destination operands for instruction I 2 . It also holdsthe information needed for the Write step (W 2 ) of instruction I 2 . This informationwill be passed to the stage W in the following clock cycle. Buffer B1 holds the results produced by the execution unit and the destinationinformation for instruction I 1.

During clock cycle #4:

F 4 I 4

F 1

F 2

F 3

I 1

I 2

I 3

D 1

D 2

D 3

D 4

E 1

E 2

E 3

E 4

W 1

W 2

W 3

W 4

Instruction

Clock cycle 1 2 3 4 5 6 7 T ime
http://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_P_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_G_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_L_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_E_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_P_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_P_coloringkidsboys.gif


9/58


Role of Cache memory

Potential increase in performance achieved by using pipeliningis proportional to the number of pipeline stages.

This rate can be achieved only if the pipelined operation can besustained without interruption through program instruction.

If a pipelined operation cannot be sustained withoutinterruption, the pipeline is said to stall .

A condition that causes the pipeline to stall is called a hazard .


10/58


Data Hazard Execution of the instruction occurs in the E stage of the pipeline. Execution of most arithmetic and logic operations would take only one clock cycle. However, some operations such as division would take more time to complete. For example, the operation specified in instruction I2 takes three cycles to complete fromcycle 4 to cycle 6.

11 11 1

c

l c c c l 1 3 7

i ur . . n x u i n r i n in m r n n l l .

Tim


11/58

Basic concepts (Contd.,)Control or instruction hazard Pipeline may be stalled because an instruction is not available at the expected time.

For example, while fetching an instruction a cache miss may occur, and hence theinstruction may have to be fetched from the main memory.

Fetching the instruction from the main memory takes much longer than fetching theinstruction from the cache.

Thus, the fetch cycle of the instruction cannot be completed in one cycle.

For example, the fetching of instruction I 2 results in a cache miss. Thus, F 2 takes 4clock cycles instead of 1.

F 1

F 2

F 3

I 1

I 2

I 3

D 1

D 2

D 3

E 1

E 2

E 3

W 1

W 2

W 3

Instruction

1 2 3 4 5 6 7 8 9 Clock c ycle T ime


12/58


Structural hazard Two instructions require the use of a hardware resource at

the same time.

Most common case is in access to the memory: One instruction needs to access the memory as part of the Execute or Write stage. Other instruction is being fetched.

If instructions and data reside in the same cache unit, only one instruction can proceedand the other is delayed.

Many processors have separate data and instruction cachesto avoid this delay.

In general, structural hazards can be avoided by providingsufficient resources on the processor chip.


13/58


Memory address X+R1 is computed in step E 2 in cycle 4, memory access takes place

In cycle 5, operand read from the memory is written into register R2 in cycle 6.

Execution of instruction I 2 takes two clock cycles 4 and 5.

In cycle 6, both instructions I 2 and I 3 require access to register file.

Pipeline is stalled because the register file cannot handle two operations at once.

F 1

F 2

F 3

I 1

I 2 (Load X(R1),R2

I 3

E 1

M 2

D 1

D 2

D 3

W 1

W 2

Instruction

F 4 I 4

F 5 I 5 D 5

Clock c ycle 1 2 3 4 5 6 7

E 2

E 3 W 3

E 4 D 4


14/58

Data hazards Data hazard is a situation in which the pipeline is stalled because the data to beoperated on are delayed.

Consider two instructions:

I1: A = 3 + A

I2: B = 4 x A

If A = 5, and I 1 and I 2 are executed sequentially, B=32. In a pipelined processor, the execution of I 2 can begin before the execution of I 1.

The value of A used in the execution of I 2 will be the original value of 5 leading to

an incorrect result. Thus, instructions I 1 and I 2 depend on each other, because thedata used by I 2 depends on the results generated by I 1.

Results obtained using sequential execution of instructions should be the same as theresults obtained from pipelined execution.

When two instructions depend on each other, they must be performed in the correctorder.


15/58

Data hazards (contd..)

Mul instruction places the results of the multiply operation in register R4 at the endof clock cycle 4.

Register R4 is used as a source operand in the Add instruction. Hence the Decode

Unit decoding the Add instruction cannot proceed until the Write step of the firstinstruction is complete.

Data dependency arises because the destination of one instruction is used as asource in the next instruction.

F 1

F 2

F 3

I 1

I 2

I 3

D 1

D 3

E 1

E 3

E 2

W 3

Instruction 1 2 3 4 5 6 7 8 9 Clock c ycle

W 1

D 2A W 2

F 4 D 4 E 4 W 4 I 4

D 2

Mul R2, R3, R4

Add R5,R4,R6


16/58

Operand forwarding

Data hazard occurs because the destination of one instructionis used as the source in the next instruction.

Hence, instruction I 2 has to wait for the data to be written inthe register file by the Write stage at the end of step W 1.

However, these data are available at the output of the ALUonce the Execute stage completes step E 1.

Delay can be reduced or even eliminated if the result ofinstruction I1 can be forwarded directly for use in step E 2.

This is called operand forwarding .


17/58

Handling data dependency in software

Data dependency may be detected by the hardware whiledecoding the instruction: Control hardware may delay by an appropriate number of clock cycles reading

of a register till its contents become available. The pipeline stalls for that manynumber of clock cycles.

Detecting data dependencies and handling them can also beaccomplished in software. Compiler can introduce the necessary delay by introducing an appropriate

number of NOP instructions. For example, if a two-cycle delay is neededbetween two instructions then two NOP instructions can be introduced betweenthe two instructions.

I 1: Mul R2, R3, R4 NOP NOP

I 2: Add R5, R4, R6


18/58

Side effects Data dependencies are explicit easy to detect if a register specified as the

destination in one instruction is used as a source in the subsequentinstruction.

However, some instructions also modify registers that are not specified asthe destination. For example, in the autoincrement and autodecrement addressing

mode, the source register is modified as well. When a location other than the one explicitly specified in the instruction

as a destination location is affected, the instruction is said to have a side

effect . Another example of a side effect is condition code flags which implicitly

record the results of the previous instruction, and these results may beused in the subsequent instruction.


19/58

Side effects (contd..)I 1: Add R3, R4I 2 : AddWithCarry R2, R4

Instruction I 1 sets the carry flag and instruction I 2 uses the carry flag leading to animplicit dependency between the two instructions.

Instructions with side effects can lead to multiple data dependencies. Results in a significant increase in the complexity of hardware or softwareneeded to handle the dependencies. Side effects should be kept to a minimum in instruction sets designed forexecution on pipelined hardware.


20/58

Instruction hazards

Instruction fetch units fetch instructions and supply theexecution units with a steady stream of instructions.

If the stream is interrupted then the pipeline stalls.

Stream of instructions may be interrupted because of a cachemiss or a branch instruction.


21/58

Instruction hazards (contd..)

Pipeline stalls for one clock cycle.

Time lost as a result of a branch instruction is called as branch penalty.

Branch penalty is one clock cycle.

F 2 I 2 (Branch)

I 3

I k

E 2

F 3

F k E k

F k+ 1 E k+ 1 I k +1

Instruction

Ex ecution unit idle

1 2 3 4 5 Clock c ycle T ime

F 1 I 1 E 1

6

X


22/58

Instruction hazards (contd..) Branch penalty depends on the length of the pipeline, may be higher for a longer

pipeline.

For a four-stage pipeline: Branch target address is computed in stage E2.

Instructions I3 and I4 have to be discarded.

Execution unit is idle for 2 clock cycles. Branch penalty is 2 clock cycles.

X

F 1 D 1 E 1 W 1

I 2

I 1

1 2 3 4 5 6 7

F 2 D 2

F 3

F k D k E k

F k+ 1 D k+ 1

I 3

I k

I k+ 1

W k

E k+ 1

E 2

D 3

F 4 X I 4

8 T ime
http://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_G_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_N_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_L_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_E_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_P_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_I_coloringkidsboys.gifhttp://www.coloring-pages-book-for-kids-boys.com/images/free_alphabet_P_coloringkidsboys.gif


23/58

Instruction hazards (contd..) Branch penalty can be reduced by computing the branch target address earlier in

the pipeline.

Instruction fetch unit has special hardware to identify a branch instruction afterthe instruction is fetched.

Branch target address can be computed in the Decode stage (D 2 ), rather than inthe Execute stage (E

2 ).

Branch penalty is only one clock cycle.

F 1 D 1 E 1 W 1

I 2 (Branch)

I 1

1 2 3 4 5 6 7 Clock c ycle

F 2 D 2

F 3 X

F k D k E k

F k+ 1 D k+ 1

I 3

I k

I k+ 1

W k

E k+ 1

T ime


24/58


F : Fetch instruction

E : Ex ecute instruction

W : Write results

D : Dispatch/ Decode

Instruction queue

Instruction fetch unit

unit

Fetch unit fetchesinstructions beforethey are needed &stores them in a queue

Queue can hold severalinstructions

Dispatch unit takes instructions from the front of the queue anddispatches them to the Execution unit. Dispatch unit also decodesthe instruction.


25/58


X

F 1 D 1 E 1 E 1 E 1 W 1

F 4

W 3

E 3

I 5 (Branch)

I 1

F 2 D 2

1 2 3 4 5 6 7 8 9 Clock c ycle

E 2 W 2

F 3

D 3

E 4 D 4 W 4

F 5 D 5

F6

F k D k E k

F k+ 1 D k+ 1

I 2

I 3

I 4

I6

I k

I k+ 1

W k

E k+ 1

10

1 1 1 1 2 3 2 1 1 Queue length 1

I5 is a branch instruction withtarget instruction I k. Ik is fetched in cycle 7, and I 6is discarded. However, this does not stall thepipeline, since I 4 is dispatched.

I 2 , I 3, I 4 and I k are executed insuccessive clock cycles.Fetch unit computes thebranch address concurrentlywith the execution of otherinstructions. This is called asbranch folding.


26/58

Conditional branches andbranch prediction

F 1

D 1

E 1

W 1

I 2

(Branch)

I 1

1 2 3 4 5 6 7 Clock c ycle

F 2

D 2

F k D k E k

F k+ 1

D k+ 1

I 3

I k

I k+ 1

W k

E k+ 1

E 2

I 4

8

T ime


27/58

Delayed branch (contd..)

Delayed branching can minimize the penalty incurred as aresult of conditional branch instructions.

Since the instructions in the delay slots are always fetchedand partially executed, it is better to arrange for them to befully executed whether or not branch is taken. If we are able to place useful instructions in these slots, then they will always

be executed whether or not the branch is taken.

If we cannot place useful instructions in the branch delay

slots, then we can fill these slots with NOP instructions.


28/58

Delayed branch (contd..)

Add

LOOP Shift_left R1 Decrement Branch=0

R2 LOOP

NEXT

(a) Original program loop

LOOP Decrement R2 Branch=0

Shift_left

LOOP

R1 NEXT Add

R1,R3

R1,R3

(b) Reordered instructions


29/58

Branch prediction To reduce the branch penalty associated with conditionalbranches, we can predict whether the branch will be taken. Simplest form of branch prediction:

Assume that the branch will not take place. Continue to fetch instructions in sequential execution order. Until the branch condition is evaluated, instruction execution

along the predicted path must be done

Determine a priori whether a branch will be taken or notdepending on the expected program behavior.

For example, a branch instruction at the end of the loop causesa branch to the start of the loop for every pass through the loopexcept the last one. Better performance can be achieved if thisbranch is always predicted as taken.


30/58

Branch prediction (contd.,)

Branch prediction decision is the same every time aninstruction is executed. This is static branch prediction.

Branch prediction decision may change depending on theexecution history. This is dynamic branch prediction.


31/58


In dynamic branch prediction the processor hardwareassesses the likelihood of a given branch being taken bykeeping track of branch decisions every time that instructionis executed.

Simplest form of execution history used in predicting theoutcome of a given branch instruction is the result of themost recent execution of that instruction. Processor assumes that the next time the instruction is executed, the result is

likely to be the same.

For example, if the branch was taken the last time the instruction wasexecuted, then the branch is likely to be taken this time as well.


32/58

Branch prediction (contd.,) Branch prediction algorithm may be described as a two-state machine with 2

states: LT : Branch is likely to be taken

LNT: Branch is likely not to be taken

Initial state of the machine be LNT

When the branch instruction is executed, and if the branch is taken, the machine

moves to state LT. If the branch is not taken, it remains in state LNT.

When the same branch instruction is executed the next time, the branch is predicted as taken if the state of the machine is LT, else it is predicted as nottaken.

Branch taken (BT)

Branch not taken (BNT)

BT BNT LNT


33/58


BT BNT

BNT

BT

BNT

BT

BNT LNT SNT

BT

ST : Strong likely to be takenLT : Likely to be takenLNT : Likely not to be takenSNT : Strong likely not to be taken


34/58

Influence on Instruction Sets


35/58

Overview

Some instructions are much better suited to pipelineexecution than others.

Addressing modes

Conditional code flags


36/58

Addressing Modes Addressing modes include simple ones and complex ones. In choosing the addressing modes to be implemented in a

pipelined processor, we must consider the effect of eachaddressing mode on instruction flow in the pipeline:

Side effectsThe extent to which complex addressing modes cause thepipeline to stallWhether a given mode is likely to be used by compilers

F3

I

I3 3

FI

FI

3 3


37/58

Recalli u r . . ff c f L i n ru ci n n i l in i mi n .

FI

Load X(R1), R2

Load (R1), R2


38/58

Complex Addressing Mode

F

F D

D E

X + [R1] [X + [R1]] [[X + [R1]]] Load

Ne xt instruction

(a) Complex addressing mode

W

1 2 3 4 5 6 7 Clock c ycle T ime

W

F orw ard

Load (X(R1)), R2


39/58

Simple Addressing Mode

X + [R1] F D

F

F

F D

D

D

E

[X + [R1]]

[[X + [R1]]]

Add

Load

Load

Ne xt instruction

(b) Simple addressing mode

W

W

W

W

Add #X, R1, R2Load (R2), R2Load (R2), R2


40/58

Addressing Modes

In a pipelined processor, complex addressing modes donot necessarily lead to faster execution.

Advantage: reducing the number of instructions /program space

Disadvantage: cause pipeline to stall / more hardware todecode / not convenient for compiler to work with

Conclusion: complex addressing modes are not suitable

for pipelined execution.


41/58

Addressing Modes

Good addressing modes should have:

Access to an operand does not require more than one accessto the memory

Only load and store instruction access memory operandsThe addressing modes used do not have side effects

Register, register indirect, index


42/58

Conditional Codes

If an optimizing compiler attempts to reorder instruction toavoid stalling the pipeline when branches or datadependencies between successive instructions occur, it mustensure that reordering does not cause a change in the

outcome of a computation. The dependency introduced by the condition-code flags

reduces the flexibility available for the compiler to reorderinstructions.


43/58

Conditional Codes

Add Compare Branch=0

R1,R2 R3,R4 . . .

Compare

Add Branch=0

R3,R4

R1,R2 . . .

(a) A program fragment

(b) Instructions reordered

Figure 8.17. Instruction reordering.


44/58

Conditional Codes

Two conclusion:

To provide flexibility in reordering instructions, the condition-code flags should be affected by as few instruction aspossible.The compiler should be able to specify in which instructionsof a program the condition codes are affected and in whichthey are not.


45/58

Datapath and Control

Considerations


46/58

Original Design


47/58

Pipelined Design

- Separate instruction and data caches- PC is connected to IMAR- DMAR- Separate MDR- Buffers for ALU- Instruction queue- Instruction decoder output

- Reading an instruction from the instruction cache- Incrementing the PC- Decoding an instruction- Reading from or writing into the data cache- Reading the contents of up to two regs- Writing into one register in the reg file

- Performing an ALU operation


48/58


49/58

Superscalar operation (contd.,)

I 1

(F add) D 1

D 2

D 3

D 4

E 1A

E 1B

E 1C

E 2

E 3

E 3

E 3

E 4

W 1

W 2

W 3

W 4

I 2 (Add)

I 3

(Fsub)

I 4

(Sub)

F 1

F 2

F 3

F 4

1 2 3 4 5 6 Clock c ycle 7


50/58

Out-of-order execution Instructions are dispatched in the same order as they appear in the

program, however, they complete execution out-of-order. Dependencies among instructions need to be handled correctly, so that this does not

lead to any problems.

What if during the execution of an instruction an exception occursand one or more of the succeeding instructions have beenexecuted to completion? For example, the execution of instruction I1 may cause an exception after the

instruction I2 has completed execution and written the results to the destinationlocation?

If a processor permits succeeding instructions to completeexecution and write to the destination locations, before knowingwhether the prior instructions cause exceptions, it is said to allowimprecise exceptions.


51/58

Out-of-order execution(Contd.,)

To guarantee a consistent state when exceptions occur, the results of execution must bewritten to the destination locations strictly in the program order.

Step W 2 must be delayed until cycle 6, when I 1 enters the write stage.

Integer unit must retain the results of I 2 until cycle 6, and cannot accept another instruction

until then. If an exception occurs during an instruction, then all subsequent instructions that may have

been partially executed are discarded.

This is known a precise exception.

I 1 (F add) D 1

D 2

D 3

D 4

E 1A E 1B E 1C

E 2

E 3A E 3B E 3C

E 4

W 1

W 2

W 3

W 4

I 2 (Add)

I 3 (Fsub)

I 4 (Sub)

1 2 3 4 5 6 Clock c ycle

F 1

F 2

F 3

F 4

7


52/58

Execution completion

It is beneficial to allow out-of-order execution, so that theexecution unit is freed up to execute other instructions.

However, instructions must be completed in program orderto allow precise exceptions.

These requirements are conflicting.

It is possible to resolve the conflict by allowing the executionto proceed and writing the results into temporary registers.

The contents of the temporary registers are transferred topermanent registers in correct program order.


53/58

Execution completion (Contd.,) A special control unit called commitment unit is needed to ensure

in-order commitment when out-of-order execution is allowed. Commitment unit has a queue called reorder buffer to determine

which instructions should be committed next:

Instructions are entered in the queue strictly in the program order as they aredispatched for execution.

When an instruction reaches the head of the queue and itsexecution has been completed: Results are transferred from temporary registers to permanent registers.

All resources assigned to this instruction are released. The instruction is said to have retired.

Instructions are retired strictly in program order, though they maybe completed out-of-order.


54/58

Dispatch Operation

When dispatch decisions are made, dispatch unit must ensurethat all the resources needed for the execution of aninstruction are available and it reserves the resourcesneeded.

What if instructions are dispatched out of order? Deadlock occurs


55/58

Performance Considerations


56/58

Overview

The execution time T of a program that has adynamic instruction count N is given by:

where S is the average number of clock cycles it takesto fetch and execute one instruction, and R is theclock rate.

Instruction throughput is defined as the number ofinstructions executed per second.

R

S N T

S

R P

s


57/58

Overview

An n-stage pipeline has the potential to increase thethroughput by n times.

However, the only real measure of performance is thetotal execution time of a program.

Higher instruction throughput will not necessarily lead tohigher performance.

Two questions regarding pipelining

How much of this potential increase in instruction throughput can berealized in practice?What is good value of n?


58/58

chapter 6 pipelining(2)

Documents