basic instruction timings - virginia techcourses.cs.vt.edu › ~cs2504 › fall2008 › notes ›...
TRANSCRIPT
Pipelining
1Basic Instruction Timings
Mak
ing som
e as
sum
ptions re
gar
din
g the
oper
atio
n tim
es for so
me
of th
e bas
ic h
ardw
are
units in
our dat
apat
h, w
e hav
e th
e fo
llow
ing tim
ings:
Instruction
class
Instruction
fetch
Register
read
ALU
operation
Data
access
Register
write
Total time
lw200 ps
100 ps
200 ps
200 ps
100 ps
800 ps
sw
200 ps
100 ps
200 ps
200 ps
700 ps
R-form
at
200 ps
100 ps
200 ps
100 ps
600 ps
beq
200 ps
100 ps
200 ps
500 ps
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
How
long w
ould
it ta
ke
to e
xec
ute
the
follow
ing seq
uen
ce o
f in
stru
ctio
ns?
lw
$1, 100($0)
lw
$2, 200($0)
lw
$3, 300($0)
But, m
aybe
ther
e’s a
way
we
can c
hea
t an
d c
om
ple
te the
sequen
ce fas
ter.
Pipelining
2Basic Idea
What
if w
e th
ink o
f th
e dat
apat
h a
s a
linea
r se
quen
ce o
f st
ages
?
Note: single-cycle
datapath
We have 5 stages,
which will m
ean
that on any given
cycle up to 5
different
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Can
we
oper
ate
the
stag
es indep
enden
tly,
usi
ng a
n e
arlier
one
to b
egin
the
nex
t in
stru
ctio
n
bef
ore
the
pre
vio
us one
has
com
ple
ted?
instructions will be
in various points of
execution.
Pipelining
3Pipelining
We’
ve
only
consi
der
ed u
nim
agin
ativ
e ex
ecution; co
nsi
der
our lo
nges
t in
stru
ctio
n:
Impro
ve
per
form
ance
by incr
easi
ng inst
ruct
ion throughput:
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Idea
l speedup
is n
um
ber
of st
ages
in the
pip
elin
e. D
o w
e ac
hie
ve
this
?
Impro
ve
per
form
ance
by incr
easi
ng inst
ruct
ion throughput:
Pipelining
4Pipelining
Det
ails
:
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
The average time
between initiating
instructions has
dropped to 200 ps.
Why do we have
idle “gaps”?
Assume:
-register file write occurs in
first half of a cycle
-register file read occurs in
second half of a cycle
Total time here is 1400 ps versus 2400 ps for the original version…
…but consider how this would look if we were had 1,000,000 more lw
instructions in our sequence…
Pipelining
5MIPS addPipeline
Her
e's how
the
pip
elin
e st
ages
would
look fro
m the
per
spec
tive
of an
add
inst
ruct
ion:
Shading indicates when
the instruction is using a
Note that the computed
result isn't written into
the register file until the
5thstage.
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
the instruction is using a
particular hardware
resource.
What
if th
e nex
t in
stru
ctio
n n
eeds th
e re
sult fro
m the add
inst
ruct
ion?
Dep
endin
g o
n w
hen
the
resu
lt is nee
ded
, w
e m
ay h
ave
to stall
the
pip
elin
e until th
e re
sult
bec
om
es a
vai
lable
.
Pipelining
6Pipelining
What
mak
es it ea
sy
-al
l in
stru
ctio
ns ar
e th
e sa
me
length
-ju
st a
few
inst
ruct
ion form
ats
-m
emory
oper
ands ap
pea
r only
in load
s an
d sto
res
What
mak
es it har
d?
-st
ruct
ura
l haz
ards:
suppose
we
had
only
one
mem
ory
-co
ntrol haz
ards:
nee
d to w
orry a
bout bra
nch
inst
ruct
ions
-dat
a haz
ards:
an
inst
ruct
ion d
epen
ds on a
pre
vio
us in
stru
ctio
n
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
We’
ll b
uild a
sim
ple
pip
elin
e an
d look a
t th
ese
issu
es
We’
ll tal
k a
bout m
oder
n p
roce
ssors
and w
hat
rea
lly m
akes
it har
d:
-ex
ception h
andling
-tryin
g to im
pro
ve
per
form
ance
with o
ut-of-
ord
er e
xec
ution, et
c.
Pipelining
7Pipeline Hazards
In som
e ca
ses, the
nex
t in
stru
ctio
n c
annot ex
ecute
in the
follow
ing c
lock
cycl
e. W
e w
ill
introduce
som
e of th
e pote
ntial
iss
ues
in the
nex
t fe
w slides
.
structural hazard
-har
dw
are
cannot su
pport the
nec
essa
ry c
om
bin
atio
n o
f oper
atio
ns at
once
-re
consi
der
the
earlie
r ex
ample
with a
sin
gle
mem
ory
unit a
nd a
fourth lw
inst
ruct
ion
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
data hazard
-dat
a th
at is nec
essa
ry to e
xec
ute
the
inst
ruct
ion is not yet
avai
lable
-co
nsi
der
:
add
$s0, $t0, $t1
sub
$t2, $s0, $s3
-load-use
haz
ard o
ccurs
when
dat
a im
ported
by a
load
inst
ruct
ion is not av
aila
ble
w
hen
it is
req
ues
ted
Pipelining
8
Her
e, the
seco
nd inst
ruct
ion n
eeds th
e final
res
ult fro
m the
firs
t in
stru
ctio
n d
uring the
regis
ter fe
tch p
ortio
n o
f th
e in
stru
ctio
n d
ecode
phas
e:
Data Hazard Example: Forwarding
Obvio
usl
y the
val
ue
will not be
avai
lable
in reg
iste
r $s0
until th
e firs
t in
stru
ctio
n h
as
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Obvio
usl
y the
val
ue
will not be
avai
lable
in reg
iste
r $s0
until th
e firs
t in
stru
ctio
n h
as
com
ple
ted.
How
ever
, th
e co
mpute
d v
alue
IS a
ctual
ly a
vai
lable
after
the
firs
t in
stru
ctio
n fin
ishes
its
third sta
ge,
just
in tim
e to
sat
isfy
the
nee
d o
f th
e A
LU
when
the
seco
nd inst
ruct
ion rea
ches
its th
ird sta
ge.
This
is in
dic
ated
above
by a
forwarding
link.
In p
rinci
ple
, th
e haz
ard h
ere
could
be
det
ecte
d a
nd h
andle
d.
But, w
hat
if th
e "f
orw
ardin
g" link a
ctual
ly w
ent bac
kw
ards?
Pipelining
9Data Hazard Example: Stalling
Her
e th
e firs
t in
stru
ctio
n is a
load
, an
d its
res
ult sim
ply
won't b
e av
aila
ble
in tim
e:
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
As in
dic
ated
, th
is c
an b
e re
solv
ed b
y stalling
the
pip
elin
e, d
elay
ing the
initia
tion o
f th
e
seco
nd inst
ruct
ion for 1 c
ycl
e.
Agai
n, if w
e ca
n d
etec
t th
is situat
ion, w
e ca
n in p
rinci
ple
im
pose
the
solu
tion show
n
above.
A p
ipel
ine
stal
l is
often
ref
erre
d to a
s a bubble
.
Pipelining
10
Control Hazards: Stall on Branch
One
appro
ach w
ould
be
to sta
ll w
hen
a b
ranch
inst
ruct
ion is dis
cover
ed, until th
e
nec
essa
ry c
om
puta
tions ar
e co
mple
ted a
nd then
fet
ch the
corr
ect in
stru
ctio
n n
ext.
A control hazard
occ
urs
when
the
inst
ruct
ion that
was
fet
ched
is not th
e one
that
is
nee
ded
.
Note
that
our pip
elin
e dis
cuss
ion so far
ass
um
es seq
uen
tial
exec
ution.
When
the
curr
ent in
stru
ctio
n is a
conditio
nal
bra
nch
, th
is m
ay b
e in
corr
ect.
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
nec
essa
ry c
om
puta
tions ar
e co
mple
ted a
nd then
fet
ch the
corr
ect in
stru
ctio
n n
ext.
Pipelining
11
Control Hazards: Branch Prediction
A sec
ond a
ppro
ach is to
predictw
het
her
the
bra
nch
will be
taken
and load
the
corr
espondin
g inst
ruct
ion into
the
pip
elin
e.
If w
e gues
s th
at the
bra
nch
will N
OT b
e ta
ken
, w
e ju
st incr
emen
t th
e PC
, fe
tch a
nd
pro
ceed
:
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
This
work
ed o
ut per
fect
ly. T
her
e w
as n
o d
elay
… h
ow
ever
, w
hat
if th
e bra
nch
HA
D b
een
taken
?
More
sophis
tica
ted v
aria
nts
act
ual
ly ret
ain info
rmat
ion (his
tory
) ab
out in
div
idual
bra
nch
inst
ruct
ions an
d u
se that
his
tory
to p
redic
t fu
ture
beh
avio
r.
Pipelining
12
Control Hazards: Delayed Branch
A third a
ppro
ach is to
delay
the
tim
e at
whic
h the
bra
nch
tak
es e
ffec
t by a
lway
s ex
ecuting
the
nex
t se
quen
tial
inst
ruct
ion follow
ing a
bra
nch
inst
ruct
ion, an
d then
mak
ing the
bra
nch
(if nec
essa
ry) im
med
iate
ly a
fter
than
one-
inst
ruct
ion d
elay
.
To d
o this
, th
e as
sem
ble
r w
ill au
tom
atic
ally
acc
om
plish
this
by p
laci
ng a
n inst
ruct
ion
imm
edia
tely
after
the
bra
nch
inst
ruct
ion that
is not af
fect
ed b
y the
bra
nch
.
This
appro
ach is use
d in the
MIP
S a
rchitec
ture
.
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
;; programmer writes:
add
$4, $5, $6
beq
$1, $2, 40
or
$7, $8, $9
;; assembler writes:
beq
$1, $2, 40
add
$4, $5, $6
or
$7, $8, $9
Of co
urs
e, it’s not al
way
s th
at sim
ple
. W
hat
would
we
do if th
e add
inst
ruct
ion h
ad
store
d its
res
ult in o
ne
of th
e re
gis
ters
use
d b
y the beq
inst
ruct
ion?
Pipelining
13
Hav
e as
sem
ble
r guar
ante
e no h
azar
ds. O
ne
appro
ach w
ould
be
to rea
rran
ge
stat
emen
ts;
anoth
er w
ould
be
to inse
rt n
o-o
p (no o
per
atio
n) st
atem
ents
, to
induce
the
nec
essa
ry sta
lls.
Wher
e do w
e in
sert the
“no-o
ps”
?
sub
$2, $1, $3
and
$12, $2, $5
or
$13, $6, $2
add
$14, $2, $2
sw
$15, 100($2)
Software Solution
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
sw
$15, 100($2)
Pro
ble
m: this
rea
lly slo
ws us dow
n!
Pipelining
14
Check Yourself
What
difficu
ltie
s ca
n y
ou iden
tify
in the
follow
ing c
ode
sequen
ces?
;; 1
lw
$t0, 0($t0)
add
$t1, $t0, $t0
;; 2
add
$t1, $t0, $t0
addi
$t2, $t0, 5
The result of the lwis needed by the add
during its second cycle; a stall is needed.
The result of the addis needed by second
addiduring its second cycle, but isn’t written
to the register file until the next cycle;
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
addi
$t2, $t0, 5
addi
$t4, $t1, 5
;; 3
addi
$t1, $t0, 1
addi
$t2, $t0, 2
addi
$t3, $t0, 3
addi
$t4, $t0, 4
addi
$t5, $t0, 5
to the register file until the next cycle;
however, we can forward the value since it’s
been computed two cycles before it’s written.
No problems here.
Pipelining
15
Basic Idea Redux
What
do w
e nee
d to a
dd/m
odify to a
ctual
ly split th
e dat
apat
h into
sta
ges
?
Inst
ruct
ions an
d d
ata
gen
eral
ly
move
from
lef
t to
rig
ht.
Tw
o e
xce
ptions:
-w
rite
-bac
k o
f dat
a to
regis
ter file
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
regis
ter file
-se
lect
ing the
nex
t val
ue
for
the
PC
(in
crem
ente
d P
C
ver
sus bra
nch
addre
ss)
The
case
s w
her
e dat
a flow
s right to
lef
t do n
ot af
fect
the
curr
ent in
stru
ctio
n, but ra
ther
they
affec
t la
terin
stru
ctio
ns.
The
firs
t ca
se c
an lea
d to a
dat
a haz
ard; th
e se
cond c
an lea
d to a
control haz
ard.
Pipelining
16
Analysis
Consi
der
a tim
e line
show
ing
over
lappin
g p
ipel
ine
logic
for a
set of in
stru
ctio
ns:
Pro
ble
m: the
origin
al c
onte
nts
of
the
IR w
ill be
lost
when
the
nex
t
inst
ruct
ion is fe
tched
, but th
ose
origin
al c
onte
nts
are
nee
ded
at a
late
r cy
cle
as w
ell. (W
hy?)
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
So, how
do w
e fix this
?
Bas
ical
ly, w
e nee
d the
ability to p
rese
rve
resu
lts gen
erat
ed in e
ach sta
ge
until th
ey a
re
actu
ally
nee
ded
.
So, w
e ca
n a
dd a
ban
k o
f st
ora
ge
loca
tions bet
wee
n e
ach p
air of stag
es.
Pipelining
17
Datapath with Pipeline Registers
Her
e's a
firs
t at
tem
pt; w
e ju
st a
dd a
n u
nsp
ecifie
d a
mount of stora
ge
bet
wee
n d
atap
ath
stag
es:
Original IR contents are
passed forward… for later use
Incremented PC value
is passed forward
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
IR is embedded here
How large must the IF/ID register storage be?
The
nex
t ord
er o
f busi
nes
s is
to e
xam
ine
the
oth
er inte
r-st
age
registe
rs.
Pipelining
18
Boundary Analysis
Let
's c
onsi
der
the
"boundar
ies"
:
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
No pipeline register is
needed after the W
B stage.
Why?
The
nex
t ord
er o
f busi
nes
s is
to e
xam
ine
the
oth
er inte
r-st
age
registe
rs.
What about the PC?
In effect it IS a pipeline register,
feeding the IF stage.
Pipelining
19
Load Instruction Analysis: IF
PC is incremented by 4.
Result is written back into
PC, but also into IF/ID
pipeline register in case it is
needed later…
… don't know what the
instruction actually is yet.
Instruction is fetched into the
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Reg
iste
r sh
adin
g indic
ates
whet
her
a w
rite
(le
ft h
alf)
or a
read
(right hal
f) is occ
urr
ing.
Instruction is fetched into the
pipeline register.
Pipelining
20
Load Instruction Analysis: ID
Values read from register
file, and extended
immediate field are stored in
the next pipeline register.
Incremented PC value is
also passed forward to next-
state pipeline register.
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Register read numbers are
supplied to register file.
16-bit immediate field is
supplied to the sign-extender.
Pipelining
21
Load Instruction Analysis: EX
Incremented PC value is NOT
carried forward… why?
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Contents of first read register and
sign-extended im
mediate are sent to
the ALU from the pipeline register.
Resulting sum is then
placed into next-stage
pipeline register.
Pipelining
22
Load Instruction Analysis: MEM
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Computed address is passed from
pipeline register to memory unit.
Retrieved data is written into
next-stage pipeline register.
Pipelining
23
Load Instruction Analysis: WB
Data is retrieved from the pipeline
register and written into the register
file.
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Pipelining
24
Load Instruction Analysis
So, w
hat
hav
e w
e le
arned
?
One
key
poin
t: a
logic
al c
om
ponen
t of th
e dat
apat
h, like
the A
LU
, m
ust
be
use
d o
nly
in a
single
pip
elin
e st
age…
oth
erw
ise,
we
hav
e a
stru
ctura
l haz
ard.
If y
ou w
ere
pay
ing v
ery c
lose
atten
tion, w
e've
unco
ver
ed a
bug in the
pro
pose
d h
andling
of a
load
inst
ruct
ion.
Tak
e an
oth
er look a
t w
hat
hap
pen
s in
the
final
sta
ge…
wher
e does
the
num
ber
of th
e
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Tak
e an
oth
er look a
t w
hat
hap
pen
s in
the
final
sta
ge…
wher
e does
the
num
ber
of th
e
write
reg
iste
r co
me
from
?
Ala
s, w
e w
ill no longer
hav
e th
e origin
al inst
ruct
ion in the
IF/ID
pip
elin
e re
gis
ter, a
nd so
we
won't h
ave
the
info
rmat
ion w
e nee
d.
Solu
tion: p
ass th
e w
rite
reg
iste
r num
ber
forw
ard to the
MEM
/WB p
ipel
ine
regis
ter, so it
is still a
vai
lable
during the
final
sta
ge.
Pipelining
25
Summary
Further
, si
milar
anal
ysi
s of oth
er inst
ruct
ions le
ads to
a c
orr
ecte
d, but in
com
ple
te, pip
elin
e
des
ign:
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
An im
portan
t ques
tion is ju
st h
ow
much
sto
rage
must
eac
h p
ipel
ine
regis
ter pro
vid
e?
That
is le
ft to the
read
er.
Pipelining
26
We
hav
e 5 sta
ges
. W
hat
nee
ds to
be
controlled
in e
ach sta
ge?
-In
stru
ctio
n F
etch
and P
C Incr
emen
t
-In
stru
ctio
n D
ecode
/ R
egis
ter Fet
ch
-Exec
ution
-M
emory
Sta
ge
-W
rite
Bac
k
How
would
control be
han
dle
d in a
n a
uto
mobile
pla
nt?
-a
fancy
control ce
nte
r te
llin
g e
ver
yone
what
to d
o?
-sh
ould
we
use
a fin
ite
stat
e m
achin
e?
Pipeline Control
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
-sh
ould
we
use
a fin
ite
stat
e m
achin
e?
Pipelining
27
Pipeline Control
Iden
tify
the
nec
essa
ry c
ontrol si
gnal
s:
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Pipelining
28
Pas
s co
ntrol si
gnal
s al
ong just
lik
e th
e dat
a:
Pipeline Control
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens
Execution/Address Calculation
stage control lines
Memory access stage
control lines
Write-back
stage control
lines
Instruction
Reg
Dst
ALU
Op1
ALU
Op0
ALU
Src
Branch
Mem
Read
Mem
Write
Reg
write
Mem to
Reg
R-form
at
11
00
00
01
0lw
00
01
01
01
1sw
X0
01
00
10
Xbeq
X0
10
10
00
X
Pipelining
29
Datapath with Control
Intro Computer Organization
Computer Science Dept Va Tech January 2006
©2006 McQuain & Ribbens