simultaneous multithreading processor - university … on super scalar which parts to improve?...
TRANSCRIPT
Simultaneous Multithreading Processor
James Lue
Some slides are modified from httphassanshojaniacompdfSMT_presentationpdf
Paper presented Exploiting Choice Instruction Fetch and Issue on an Implementable Simultaneous Multithreading Processor
Processors for parallelism
n Super scalar and multithreaded processor Architecture for increasing parallelism
n Super scalar Instruction level parallelism n Multithreaded processor Thread level
parallelism
Limits on super scalar Which parts to improve
Simulation Result 1IPClt15 2No general dominant cause for all applications
Simulation for unused issue cycles in pure super scalar Processor busy utilized issue slots other are wasted issue slots
I canrsquot help
Problem with super scalar and multithreaded processor
Color Thread
Vertical Waste
Horizontal Waste
Source Computer Architecture A Quantitative Approach 4ed
n Problem with the experiments results The model is too idealized (huge number of register file ports complex schedulinghellip)
Need more Realistic Model
Goal
n Minimize architectural impact on superscalar to build SMT
n Minimal performance impact on single thread applications
n High throughputs (IPC) with multiple threads
n Be able to issue the threads that use processor most efficiently in each cycle
Base Architecture
Up to 8
Choose the thread that is without I cache miss and conflict
to others
gt100 registers
32 entries
Multi-banked
8(threads)32(32-bit ISA)
Multi-banked
3
6 4can ldst
Derive from OOO super scalar
Per-thread return stack instruction retirement IQ flush trap mechanism ID (in IQBTB)
n Larger register file has some effects on the pipeline 1Latency for accessing large register file is high =gt break it into 2 cycles for register access 2 So 2 pipeline stages (RRWB) need to access register file =gt 2 more pipeline stages added in total 3 Effects of more pipeline stages Increase branch misprediction penalty one more bypass level for WB instructions stays in pipeline for longer time more instructions in the pipeline registers may be not enough for more instructions
Optimistic issue Load instruction needs 2 cycles but they still assume one cycle to load (do not wait 2 cycles for load) When load suffer cache miss or bank conflict =gt squash this load re-issue later
Increasing pipeline stages doesnrsquot increase inter-instructions latency between instructions except ldquoloadrdquo
Performance of the base architecture
Only ~4 IPC MT 3 IPC
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Processors for parallelism
n Super scalar and multithreaded processor Architecture for increasing parallelism
n Super scalar Instruction level parallelism n Multithreaded processor Thread level
parallelism
Limits on super scalar Which parts to improve
Simulation Result 1IPClt15 2No general dominant cause for all applications
Simulation for unused issue cycles in pure super scalar Processor busy utilized issue slots other are wasted issue slots
I canrsquot help
Problem with super scalar and multithreaded processor
Color Thread
Vertical Waste
Horizontal Waste
Source Computer Architecture A Quantitative Approach 4ed
n Problem with the experiments results The model is too idealized (huge number of register file ports complex schedulinghellip)
Need more Realistic Model
Goal
n Minimize architectural impact on superscalar to build SMT
n Minimal performance impact on single thread applications
n High throughputs (IPC) with multiple threads
n Be able to issue the threads that use processor most efficiently in each cycle
Base Architecture
Up to 8
Choose the thread that is without I cache miss and conflict
to others
gt100 registers
32 entries
Multi-banked
8(threads)32(32-bit ISA)
Multi-banked
3
6 4can ldst
Derive from OOO super scalar
Per-thread return stack instruction retirement IQ flush trap mechanism ID (in IQBTB)
n Larger register file has some effects on the pipeline 1Latency for accessing large register file is high =gt break it into 2 cycles for register access 2 So 2 pipeline stages (RRWB) need to access register file =gt 2 more pipeline stages added in total 3 Effects of more pipeline stages Increase branch misprediction penalty one more bypass level for WB instructions stays in pipeline for longer time more instructions in the pipeline registers may be not enough for more instructions
Optimistic issue Load instruction needs 2 cycles but they still assume one cycle to load (do not wait 2 cycles for load) When load suffer cache miss or bank conflict =gt squash this load re-issue later
Increasing pipeline stages doesnrsquot increase inter-instructions latency between instructions except ldquoloadrdquo
Performance of the base architecture
Only ~4 IPC MT 3 IPC
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Limits on super scalar Which parts to improve
Simulation Result 1IPClt15 2No general dominant cause for all applications
Simulation for unused issue cycles in pure super scalar Processor busy utilized issue slots other are wasted issue slots
I canrsquot help
Problem with super scalar and multithreaded processor
Color Thread
Vertical Waste
Horizontal Waste
Source Computer Architecture A Quantitative Approach 4ed
n Problem with the experiments results The model is too idealized (huge number of register file ports complex schedulinghellip)
Need more Realistic Model
Goal
n Minimize architectural impact on superscalar to build SMT
n Minimal performance impact on single thread applications
n High throughputs (IPC) with multiple threads
n Be able to issue the threads that use processor most efficiently in each cycle
Base Architecture
Up to 8
Choose the thread that is without I cache miss and conflict
to others
gt100 registers
32 entries
Multi-banked
8(threads)32(32-bit ISA)
Multi-banked
3
6 4can ldst
Derive from OOO super scalar
Per-thread return stack instruction retirement IQ flush trap mechanism ID (in IQBTB)
n Larger register file has some effects on the pipeline 1Latency for accessing large register file is high =gt break it into 2 cycles for register access 2 So 2 pipeline stages (RRWB) need to access register file =gt 2 more pipeline stages added in total 3 Effects of more pipeline stages Increase branch misprediction penalty one more bypass level for WB instructions stays in pipeline for longer time more instructions in the pipeline registers may be not enough for more instructions
Optimistic issue Load instruction needs 2 cycles but they still assume one cycle to load (do not wait 2 cycles for load) When load suffer cache miss or bank conflict =gt squash this load re-issue later
Increasing pipeline stages doesnrsquot increase inter-instructions latency between instructions except ldquoloadrdquo
Performance of the base architecture
Only ~4 IPC MT 3 IPC
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Problem with super scalar and multithreaded processor
Color Thread
Vertical Waste
Horizontal Waste
Source Computer Architecture A Quantitative Approach 4ed
n Problem with the experiments results The model is too idealized (huge number of register file ports complex schedulinghellip)
Need more Realistic Model
Goal
n Minimize architectural impact on superscalar to build SMT
n Minimal performance impact on single thread applications
n High throughputs (IPC) with multiple threads
n Be able to issue the threads that use processor most efficiently in each cycle
Base Architecture
Up to 8
Choose the thread that is without I cache miss and conflict
to others
gt100 registers
32 entries
Multi-banked
8(threads)32(32-bit ISA)
Multi-banked
3
6 4can ldst
Derive from OOO super scalar
Per-thread return stack instruction retirement IQ flush trap mechanism ID (in IQBTB)
n Larger register file has some effects on the pipeline 1Latency for accessing large register file is high =gt break it into 2 cycles for register access 2 So 2 pipeline stages (RRWB) need to access register file =gt 2 more pipeline stages added in total 3 Effects of more pipeline stages Increase branch misprediction penalty one more bypass level for WB instructions stays in pipeline for longer time more instructions in the pipeline registers may be not enough for more instructions
Optimistic issue Load instruction needs 2 cycles but they still assume one cycle to load (do not wait 2 cycles for load) When load suffer cache miss or bank conflict =gt squash this load re-issue later
Increasing pipeline stages doesnrsquot increase inter-instructions latency between instructions except ldquoloadrdquo
Performance of the base architecture
Only ~4 IPC MT 3 IPC
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
n Problem with the experiments results The model is too idealized (huge number of register file ports complex schedulinghellip)
Need more Realistic Model
Goal
n Minimize architectural impact on superscalar to build SMT
n Minimal performance impact on single thread applications
n High throughputs (IPC) with multiple threads
n Be able to issue the threads that use processor most efficiently in each cycle
Base Architecture
Up to 8
Choose the thread that is without I cache miss and conflict
to others
gt100 registers
32 entries
Multi-banked
8(threads)32(32-bit ISA)
Multi-banked
3
6 4can ldst
Derive from OOO super scalar
Per-thread return stack instruction retirement IQ flush trap mechanism ID (in IQBTB)
n Larger register file has some effects on the pipeline 1Latency for accessing large register file is high =gt break it into 2 cycles for register access 2 So 2 pipeline stages (RRWB) need to access register file =gt 2 more pipeline stages added in total 3 Effects of more pipeline stages Increase branch misprediction penalty one more bypass level for WB instructions stays in pipeline for longer time more instructions in the pipeline registers may be not enough for more instructions
Optimistic issue Load instruction needs 2 cycles but they still assume one cycle to load (do not wait 2 cycles for load) When load suffer cache miss or bank conflict =gt squash this load re-issue later
Increasing pipeline stages doesnrsquot increase inter-instructions latency between instructions except ldquoloadrdquo
Performance of the base architecture
Only ~4 IPC MT 3 IPC
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Goal
n Minimize architectural impact on superscalar to build SMT
n Minimal performance impact on single thread applications
n High throughputs (IPC) with multiple threads
n Be able to issue the threads that use processor most efficiently in each cycle
Base Architecture
Up to 8
Choose the thread that is without I cache miss and conflict
to others
gt100 registers
32 entries
Multi-banked
8(threads)32(32-bit ISA)
Multi-banked
3
6 4can ldst
Derive from OOO super scalar
Per-thread return stack instruction retirement IQ flush trap mechanism ID (in IQBTB)
n Larger register file has some effects on the pipeline 1Latency for accessing large register file is high =gt break it into 2 cycles for register access 2 So 2 pipeline stages (RRWB) need to access register file =gt 2 more pipeline stages added in total 3 Effects of more pipeline stages Increase branch misprediction penalty one more bypass level for WB instructions stays in pipeline for longer time more instructions in the pipeline registers may be not enough for more instructions
Optimistic issue Load instruction needs 2 cycles but they still assume one cycle to load (do not wait 2 cycles for load) When load suffer cache miss or bank conflict =gt squash this load re-issue later
Increasing pipeline stages doesnrsquot increase inter-instructions latency between instructions except ldquoloadrdquo
Performance of the base architecture
Only ~4 IPC MT 3 IPC
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Base Architecture
Up to 8
Choose the thread that is without I cache miss and conflict
to others
gt100 registers
32 entries
Multi-banked
8(threads)32(32-bit ISA)
Multi-banked
3
6 4can ldst
Derive from OOO super scalar
Per-thread return stack instruction retirement IQ flush trap mechanism ID (in IQBTB)
n Larger register file has some effects on the pipeline 1Latency for accessing large register file is high =gt break it into 2 cycles for register access 2 So 2 pipeline stages (RRWB) need to access register file =gt 2 more pipeline stages added in total 3 Effects of more pipeline stages Increase branch misprediction penalty one more bypass level for WB instructions stays in pipeline for longer time more instructions in the pipeline registers may be not enough for more instructions
Optimistic issue Load instruction needs 2 cycles but they still assume one cycle to load (do not wait 2 cycles for load) When load suffer cache miss or bank conflict =gt squash this load re-issue later
Increasing pipeline stages doesnrsquot increase inter-instructions latency between instructions except ldquoloadrdquo
Performance of the base architecture
Only ~4 IPC MT 3 IPC
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
n Larger register file has some effects on the pipeline 1Latency for accessing large register file is high =gt break it into 2 cycles for register access 2 So 2 pipeline stages (RRWB) need to access register file =gt 2 more pipeline stages added in total 3 Effects of more pipeline stages Increase branch misprediction penalty one more bypass level for WB instructions stays in pipeline for longer time more instructions in the pipeline registers may be not enough for more instructions
Optimistic issue Load instruction needs 2 cycles but they still assume one cycle to load (do not wait 2 cycles for load) When load suffer cache miss or bank conflict =gt squash this load re-issue later
Increasing pipeline stages doesnrsquot increase inter-instructions latency between instructions except ldquoloadrdquo
Performance of the base architecture
Only ~4 IPC MT 3 IPC
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Optimistic issue Load instruction needs 2 cycles but they still assume one cycle to load (do not wait 2 cycles for load) When load suffer cache miss or bank conflict =gt squash this load re-issue later
Increasing pipeline stages doesnrsquot increase inter-instructions latency between instructions except ldquoloadrdquo
Performance of the base architecture
Only ~4 IPC MT 3 IPC
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Performance of the base architecture
Only ~4 IPC MT 3 IPC
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Improve base architecture Improve fetching 1Partition the Fetch Unit fetch instructions from multiple threads 2Fetch policy fetch useful instructions 3Unblocking the Fetch Unit prevent conditions that
make fetch unit stop Improve issuing 1Issuing policy
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Partition the Fetch Unit Alg threads to fetch in a cycle Max instr per thread in a cycle
RR18 -baseline (MT) RR24 -2 cache output buses each 4 instr wide -additional circuit for multiple output buses RR42 -more expensive changes than RR24 -suffer from thread shortage (available
threads are less than required to fill fetch bandwidth)
RR28 -Best -fetch as many instr As possible for first
thread then second thread fills up to 8 -high throughput-gt cause IQ full
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Fetch Policy Factors to consider 1Donrsquot fetch threads that follow wrong paths (misprediction) 2Donrsquot fetch the threads that stay in IQ for very long time Policy BRCOUNT favor branches with fewest unresolved branches ( 1st factor) MISSCOUNT Favor threads with fewest outstanding D cache miss ( 2nd factor) ICOUNT Favor threads with fewest instruction in decode rename IQ (2nd factor) IQPOSN Donrsquot favor threads that have old instructions in IQ (2nd factor)
ICOUNT
ICOUNT ICOUNT
ICOUNT
ICOUNT
ICOUNT ICOUNT
ICOUNT
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Unblocking Fetching Unit Conditions that cause the fetch unit to be blocked 1 IQ full 2 I cache miss Scheme to solve the conditions 1 BIGQ restriction on IQ searching time
Thus double the IQ size but only search first 32 instructions in the IQ Additional space in IQ is for buffering instruction when IQ overflow
2 ITAG look up I cache tag one cycle early if itrsquos miss choose other thread and load the missing instruction into cache( fetch after the instruction is in the cache) Need more pipeline stages
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Which Instruction to issue Issue priority aim to avoid wasting issue slots Causes for issue slot waste 1Wrong path instruction 2Optimistically (issue load-dependant instructions a cycle before have D cache hit information if miss or bank conflict squash opt issued instruction waste slot)
OLDEST_FIRST deepest instructions in IQ first (default) OPT_LAST issue Optimistic instructions as late as possible SPCE_LAST issue speculative instructions as late as possible BRANCH_FIRST issue branch instructions as soon as possible
(to identify mispredicted branch quickly)
Need multiple passes to search IQ
No much difference OLDEST
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Bottlenecks 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Issue BW 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite FUs throughput increases 05 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 IQ size 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Reduced by ICOUNT increases1 with 64-entry IQ 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch BW Fetch up to 16 instr =gt 57 IPC (increase 8) 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Fetch up tp 16 instr 64-entry IQ 140 regs =gt 61 IPC 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Branch prediction 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Doubling BHTBTB size increases 2 Speculative execution Wrong path rare mechanisms to prevent speculative execution reduce throughput 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Memorycache 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 No bank conflict (infinite BW cache) increases 3 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Register file 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 Infinite excess register increase 2 10485761048577104857810485791048580104858110485821048583104858410485851048586104858710485881048589104859010485911048592104859310485941048595104859610485971048598104859910486001048601104860210486031048604104860510486061048607104860810486091048610104861110486121048613104861410486151048616104861710486181048619104862010486211048622104862310486241048625104862610486271048628104862910486301048631104863210486331048634104863510486361048637104863810486391048640104864110486421048643104864410486451048646104864710486481048649104865010486511048652104865310486541048655104865610486571048658104865910486601048661104866210486631048664104866510486661048667104866810486691048670104867110486721048673104867410486751048676104867710486781048679104868010486811048682104868310486841048685104868610486871048688104868910486901048691104869210486931048694104869510486961048697104869810486991048700104870110487021048703 ICOUNT reduces clogstrain on Reg file
Bottleneck
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Summary No heavily change to conventional superscalar No great impact on single-thread performance Achieve 54 IPC 25x compare to 8 threads super scalar These improvements are from -1Partition fetch bandwidth -2Be able to select the threads that uses resources most efficiently
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
SMT implemented processors
n Hyper-Threading in Intel
n Implemented in Atom Intel Core i3i7 Itanium Pentium 4 and Xeon
n 2 threads per core n Need OS support
and optimization
httpenwikipediaorgwikiHyper-threading
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Source Intel Technology Journal Volume 6 Issue 1 February 14 2002
Logical Processors
Physical Processor
cache execution units branch
prediction control logic buseshellip
SMT implemented processors- Hyper Threading Architecture
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
SMT implemented processors n Netburst microarchitecture( Pentium 4 and Pentium D)
Source HYPERTHREADING TECHNOLOGY IN THE NETBURST MICROARCHITECTURE
Red Partition favor faster threads like ICOUNT Green Threshold limit resources to one logical processor prevent to block other access find maximum parallelism Blue Full Share Mainly for cache share data code
Optimistic issue for load
Thank you
Thank you