Pipelining

• What is it?

• How does it work?

• What are the benefits?

• What could go wrong?

By Derek Closson

What is pipelining?

Pipelining is a design feature that allows individual common processor tasks to run simultaneously, such as:

FetchDecodeExecute

What is pipelining?

Or, more specifically in modern computers,

Instruction readDecodeOperand readExecuteOperand write

What is pipelining?

• No more “one instruction at a time” processing

• Processor works simultaneously on multiple instructions

• Each instruction is at a different stage (or is being delayed until its next stage is open)

How does pipelining work?

• First instruction is fetched from memory

How does pipelining work?

• First instruction is fetched from memory

• First instruction is decoded;second instruction is fetched

How does pipelining work?

• First instruction is fetched from memory

• First instruction is decoded;second instruction is fetched

• First instruction’s operands are fetched;second instruction is decoded;third instruction is fetched

How does pipelining work?

• First instruction is fetched from memory

• First instruction is decoded;second instruction is fetched

• First instruction’s operands are fetched;second instruction is decoded;third instruction is fetched

• And on, and on, and on…

How does pipelining work?

• Because each instruction demands one stage of the processor, the maximum number of simultaneous instructions is the number of stages in the processor

How does pipelining work?

A short animation

of simple processor routines

How does pipelining work?

A short(er) animation

of pipelined processor routines

What are the benefitsof pipelining?

• Uses most (if not all) of a processor’s ability at all timesEfficiency goes up

• Relatively minor hardware changes give more completed cycles per secondSpeed goes up

What can go wrong in pipelines?

• Data hazards

• Control hazards

• Structural hazards

What can go wrong…?Data hazards

• Two instructions at different stagesOne instruction needs the results ofthe other instruction’s operation

…add %r1, 5, %r2add %r2, 7, %r3…

What can go wrong…?Data hazards

• Well-designed programs can anticipate this, and spread the connected commands out with other operations (or nop in a pinch)

…add %r1, 5, %r2 (i1 fetch)sub %r4, 7, %r8 (i1 decode)mov %r5, %r6 (i1 read)add %r2, 7, %r3 (i1 execute, i2 fetch)nop (i1 write, i2 decode)…

What can go wrong…?Data hazards

• Hardware can detect some data hazards and “forward” the information to the instructions that need it, even as the data goes to the registers

• Otherwise, programs may simply have bad data - testing and good programming are the best cures!

What can go wrong…?Control hazards

• Instructions are read sequentially from memory - even though loops may return to previous statements

• Instructions following the loop are already in the pipeline!

• Or, simple if/else blocks of code. Which block do you get instructions from?

What can go wrong…?Control hazards

• At worst, the entire pipeline must be flushed: hardware can stall the influx of new instructions until the conditional is evaluated

What can go wrong…?Control hazards

• Branch prediction inserts a couple instructions of its own to guess the outcome of the conditional

• If prediction is usually correct,fewer cycles lost on average

• If prediction is wrong,flush the pipeline and undo any improper commands

What can go wrong…?Structure hazards

• Two different instructions try to use the same hardware resource

• One instruction will obviously get access first, second instruction must be stalled (maybe along with all other instructions behind it)

• Again, careful programming will prevent this