dynamic tag-check omission: a low-power instruction cache...

Dynamic Tag-Check Omission: A Low-Power Instruction Cache Architecture

Exploiting Execution Footprints

K. Inoue, V. Moshnyaga, and K. Murakami

Introduction

DEC 21164 CPU* StrongARM SA-110 CPU* Bipolar ECL CPU**

25% 43% 50%* Kamble et. al., “Analytical energy Dissipation Models for Low Power Caches”, IＳLPED’97** Joouppi et. al., “A 300-MHz 115-W 32-b Bipolar ECL Microprocessor” ,IEEE Journal

of Solid-State Circuits’93

Increase in cache size

Power consumed in on-chip caches

Breakdown of Cache Energy

# of words in a Subbank-entry(Total # of Subbanks)

8(1) 4(2) 2(4) 1(8)

This calculation is based on Kamble, et. Al., “Analytical energy Dissipation Models for Low Power Caches”, IＳLPED’97

Word: 64 bitsCache Size: 64 KBLine Size: 64 B

Data(bit-lines)

Tag(bit-lines)

Others

Energy consumed in CacheEdecode + Esram + Eio

Etag + Edata

Breakdown of Esram per access

TagMemory

Data Memory (Cache Lines)

Cache Subbanking

Subbank

History-Based Tag-Comparison (HBTC) Instruction Cache -Motivation-

Hit rate of instruction cache (I-$) is quite HIGH!Most of the tag-comparisons result in HIT

Conventional I-$Performs tag-comparison in EVERY cache access despite that some instructions obviously exist in cache.

Dissipates energyunnecessarily!

HBTC I-$OMITS tag-comparison for some cache accesses if it knows that the corresponding instructions obviously exist in cache.

Eliminate the energy consumed by the

redundant tag-checks!

Can We Know The Existence of Instructions in the I-Cache without Tag-Comparison?

We know that the target instruction exists in the cache !

Consider;• The target instruction has been referenced

before, and• No cache miss has occurred since the previous

reference of the target instruction.

1. Execute an instruction block A at time TLeave the execution footprint in the corresponding BTB-entry.$

(2. If a cache miss occurs, then erase all the footprints.)

3. Execute the instruction block A at time T+X

A $If the footprint is detected in the BTB, then omit the tag comparisons for all the instructions in the block A!

Exploit Execution Footprints Provided by the BTB!

But, How?

Branch Inst. Addr. Target Address

Target Address

EFT(Execution Footprint for Target)

EFF(EF for Fall-through)

Not-taken

ModeController Tag-check control

Branch Prediction Result

HBTC I-$ Architecture

Target Instruction Block

Branch Inst. Addr.

PBAregBranch Inst. Addr. Pred.

Result

Address for EF writing

HBTC I-$ OperationNormal Mode (NM): w/ Tag checksOmitting Mode (OM): w/o Tag checksTracing Mode (TM): w/ Tag checks

(on a BTB hit, the EF addressed by the PBAreg is set to 1)

Mode Transition

BTB HitOM

BTB Hit

HBTC I-$ Operation

Mode Transition Read EFT and EFF

Normal Mode (NM): w/ Tag checksOmitting Mode (OM): w/o Tag checksTracing Mode (TM): w/ Tag checks

BTB Hit

HBTC I-$ Operation

EF==1Mode Transition Read

EFT and EFF

BTB Hit

HBTC I-$ Operation

PC and Pred.-result

PBAreg

BTB Hit

HBTC I-$ Operation

PC and Pred.-result

PBAreg

Cache miss!

BTB Hit

HBTC I-$ Operation

PC and Pred.-result

PBAreg

Cache miss!

BTB Hit

HBTC I-$ Operation

PC and Pred.-result

PBAreg

Cache miss!

BTB Hit!Instruction

Previous BTB hit

Current BTB hit

BTB Hit

HBTC I-$ Operation

PC and Pred.-result

PBAreg

Cache miss!

BTB Hit!Instruction

Previous BTB hit

Current BTB hitValidate the EF pointed by the PBAreg!

HBTC I-$ Operation

BTB HitGOtoNM

GOtoNM

GOtoNMI-Cache miss orBTB replacement orRAS address orBranch misprediction

Mode Transition

All EFs are invalidated!

Branch to F

Branch to A

EFT EFF MODE (NM,OM,TM)

Execution Flow

State of BTB

Time(Iteration Count – Address of Branch)

Branch Target Buffer

Operation Example

PBAregAdr:T/N12 345

IterationCount

Branch to F

Branch to A

12 345

IterationCount

Performing!

Operation ExampleEFT EFF MODE (NM,OM,TM)

PBAreg--:--Branch-CInitialBranch-D

Branch to F

Branch to A

12 345

IterationCount

--:--Branch-C1-CBranch-D

Omitting-Mode

Branch to F

Branch to A

12 345

IterationCount

Omitting!

Omitting-Mode

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

C:NBranch-C2-CBranch-D

Tracing-Mode

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

Tracing-Mode

Performing!

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

Tracing-Mode

D:TBranch-C2-DBranch-D

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

Tracing-Mode

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

Tracing-Mode

Performing!

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

Tracing-Mode

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

Tracing-Mode

Omitting-Mode

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

Tracing-Mode

Omitting-ModeOmitting!

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

Tracing-Mode

Omitting-Mode

--:--Branch-C3-DBranch-D

Omitting-Mode

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

Tracing-Mode

Omitting-Mode

Omitting!

Branch to F

Branch to A

12 345

IterationCount

Omitting-Mode

Tracing-Mode

Omitting-Mode

Omitting-ModePerforming!Performing!Omitting!

Performance/Energy Overhead

PerformanceBTB access conflict (1 stall cycle)

for execution footprint writingfor execution footprint invalidation

EnergyEnergy for execution-footprint access

for reading (every BTB look-up)for writing (BTB hit in TMode)for invalidating (Cache miss or BTB replace)

Evaluation – Environment –•Cache Energy Model based on Kamble[97]

includes BTB access overhead•SimpleScalar Simulator

16 KB I-cache (partitioned into 4 subbanks), 32 B block, 2-bit bimod predictor4-way 2K-enetry BTB

•Benchmark6 from the SPECint, 4 from the Mediabench

099.go 129.compress 130.li adpcm(e) mpeg2(e)124.m88ksim 126.gcc 132.ijpeg adpcm(d) mpeg2(d)

Evaluation – Tag-Check Counts –

ITCHBTCComb.

The ITC approach works well for all programs.The HBTC approach works well for media programs.The hybrid approach makes significant reductions!

EbtbaddEbtbaddEoutputEoutputEtagEtagEdataEdata

BTB energy overhead does not have large impact.For media programs, Ecache saving is more than 15 %.

Evaluation – Cache Energy –

For media programs, the performance overhead is trivial.For others, the performance degradation might not be acceptable.

Evaluation – Execution Time –

1 2 4 8 16 32

099.go

124.m88ksim

126.gcc

129.compress

130.li

132.ijpeg

adpcm(e)

adpcm(d)

mpeg2(e)

mpeg2(d)

If the penalty is equal to or smaller than 4 cycles, the performance overhead is trivial.If the penalty is greater than 4 cycles, the performance overhead is serious.

Evaluation – Invalidation Penalty –

Invalidation Penalty [clock cycle]

Conclusions

1. Exploits execution footprints recorded in the BTB.2. Reduces tag-comparison count by 95% (adpcm(d)).3. Achieves 17 % of cache energy saving!

History-Based Tag-Comparison Instruction Cache

Future work• Analyze energy consumption based on chip design.

Buck Up Slides(History-based Tag-Comparison Cache)

Outline

1. Introduction2. History-Based Tag-Comparison Cache

• Motivation• Mechanism• Architecture• Operation

3. Evaluations4. Conclusions

Low Power Caches- Reducing both Etag and Edata -

Adding a small L0 cache

L1 Cache

L0 CacheProcessor

•Filter Cache•S-Cache•Block Buffering

Dividing cache moduleCache•MDM Cache

Multiple accessing•MRU Cache•Hash-Rehash Cache

Sequential Way-Access

Dividing cache module •Cache Sub-Banking

Accessing sequentially•Phased Cache•Pipelined Cache

Tag Line

Hit!Miss!

Replace

Low Power Caches- Reducing Edata -

Breakdown of Esram

8(1) 4(2) 2(4) 1(8)

# of words in a Subbank (Total # of Subbanks)

8(1) 4(2) 2(4) 1(8)

32-bit CPU 64-bit CPU

Esram_data_bit Esram_tag_bit

Esram_others This calculation is based on Kamble, et. Al., “Analytical energy Dissipation Models for Low Power Caches”, IＳLPED’97

CS = 32 KBL S = 32 B

CS = 64 KBLS = 64 B

CS: Cache SizeLS: Line Size

Branch to F

Branch to A

1234 567

IterationCount

EFT EFF MODE (NM,OM,TM)

Execution Flow

State of BTB

Time(Iteration Count – Address of Branch)

Branch Target Buffer

Operation Example

PBAregAdr:T/N

Branch to F

Branch to A

1234 567

IterationCount

Performing!

PBAreg--:--Branch-CInitial

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch to F

Branch to A

1234 567

IterationCount

Performing!

Branch-CC-1C:T

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch to F

Branch to A

1234 567

IterationCount

Performing!

Branch-CC-1C:T

Branch-CC-2C:T

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3--:--

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3--:--

Omitting!Omitting!

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3--:--

Branch-CC-4C:N

EFF is selected!

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3--:--

Performing!Performing! Branch-CC-4C:N

EFF is selected!

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3--:--

Branch-CC-4C:N

Branch-CD-4D:TBranch-D New Entry

Registration!

Branch to F

Branch to A

1234 567

IterationCount

--:--Branch-CInitial

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3--:--

Branch-CC-4C:N

Branch-CD-4D:TBranch-D

Performing!Performing!

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3--:--

Branch-CC-4C:N

Branch-CC-5--:--Branch-D

EFF of Branch-C

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3C:T

Branch-CC-4C:N

Omitting!Omitting!

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3C:T

Branch-CC-4C:N

Branch-CD-5--:--Branch-D

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3C:T

Branch-CC-4C:N

Omitting!Omitting!

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3C:T

Branch-CC-4C:N

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3C:T

Branch-CC-4C:N

Omitting!Omitting!

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3C:T

Branch-CC-4C:N

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3C:T

Branch-CC-4C:N

Omitting!Omitting!

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3C:T

Branch-CC-4C:N

Branch-CB-7B:TBranch-D

Branch-B

Branch to F

Branch to A

1234 567

IterationCount

Branch-CC-1C:T

Branch-CC-2C:T

Branch-CC-3C:T

Branch-CC-4C:N

Performing!Performing!

Branch-CB-7B:TBranch-D

Branch-B

Hit rate of instruction cache (I-$) is quite HIGH!Most of the tag-comparisons result in MATCH

Conventional I-$Performs tag check in EVERYcache access despite that some instructions obviously exist in the cache.

Wastes unnecessary energy!

Conventional Tag-Check Scheme

Cache-line address0 511255

16 KB Direct-Mapped Cache (32 B Lines)132.ijpeg

History-Based Tag-Check (HBTC) Scheme (1/2)• An instruction has been executed at least once.• No cache miss has occurred since the last

reference of the instruction.

We know that the instruction exists in the cache now!

How can we detect these conditions at run time?

dynamic tag-check omission: a low-power instruction cache...

Documents

cache memory - ufrgsflavio/ensino/cmp502/memoria.pdf ·...

s cache: thwarting cache attacks via cache set randomization

d’autonomie - cache-cache 1 -...

apostrophe- omission and possession

indeterminate composition by note omission,...

error analysis & omission

anonyme par omission, roman

web cache architecture and cache design

is there really an omission effect?

sins of omission ii

911cw omission report

the great omission

research on administrative omission

omission of essential terms. electricity corporation …

3次元積層プロセッサ...

problems of legislative omission in constitutional

doyle doyle sin of omission

misrepresentation by omission in settlement negotiations

mine dec omission ing

failure by omission