benjamin c. lee - microsoft · benjamin c. lee5pt [email protected] songchun fan, andrew...
TRANSCRIPT
![Page 1: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/1.jpg)
The Computational Sprinting Game
Benjamin C. Lee
Songchun Fan, Andrew Hilton, Ziqiang Huang,Jose Joao, Alejandro Rico, Seyed Majid Zahedi
Acknowledgements: Semiconductor Research Corporation; National Science Foundation.
![Page 2: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/2.jpg)
Computational Sprints
Enhance performance briefly with extra power
3 cores @1.2GHz → 12 cores @ 2.7GHz
Nor
mal
ized
Spe
edup
0
1
2
3
4
5
6
naive
decis
ion
grad
ientsv
mlin
ear
kmea
ns als
corre
lation
page
rank cc
trian
gle
Nor
mal
ized
Pow
er
0.0
0.5
1.0
1.5
naive
decis
ion
grad
ientsv
mlin
ear
kmea
ns als
corre
lation
page
rank cc
trian
gle
Ave
rage
Tem
pera
ture
(°C
)
0
10
20
30
40
50
Non−sprinting Sprinting
naive
decis
ion
grad
ientsv
mlin
ear
kmea
ns als
corre
lation
page
rank cc
trian
gle
Fan et al. “The computational sprinting game” [ASPLOS’16]
2 / 16
![Page 3: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/3.jpg)
Sprint Management
When should processors sprint?
Which processors should sprint?
Phases and processors that benefit most
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
● ●
●
0 5 10 15 20 25 30
56
78
Epoch
Util
ity fr
om S
prin
t
●
?
3 / 16
![Page 4: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/4.jpg)
Sprint Management
Sprint consumes thermal headroom
Sprint constrains future action
Sprint requires recovery time
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
● ●
●
0 5 10 15 20 25 30
56
78
Epoch
Util
ity fr
om S
prin
t
●
××
××
××
×××
4 / 16
![Page 5: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/5.jpg)
DynaSprint
Software Run-Time
Predict workload’s sprint utility, cost
Track system’s thermal headroom
Decide whether to sprint
Hardware Support
Alloate resources (e.g., cache capacity)
Monitor performance, energy
Implement safety fallback
Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]
5 / 16
![Page 6: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/6.jpg)
DynaSprint
(1-2) Classify eventsi → phasei
(3-5) Predict phasei → phasei+1 → utilityi+1
(6-8) Control sprint if utilityi+1 > utilitythresh
Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]
6 / 16
![Page 7: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/7.jpg)
Phase Classifier
eventsi → phasei
Define signaturewith events
Cluster signatures todefine phase
Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]
7 / 16
![Page 8: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/8.jpg)
Phase Predictor
phasei → phasei+1
Track phase history
Use history topredict phase
Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]
8 / 16
![Page 9: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/9.jpg)
Utility Predictor
phasei+1 → utilityi+1
Use phase average
to predict utility
Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]
9 / 16
![Page 10: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/10.jpg)
Sprint Decision
Ui+1 > Umax ×Hconsumed/Htotal
Ui+1 Predicted utilityUmax Max utility in recent epochs
Hconsumed/Htotal Fraction of thermal headroom consumed
Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]
10 / 16
![Page 11: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/11.jpg)
Performance
Improve performance by 17% (average), 40% (max)
Outperform heuristics
Compete with oracles
Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]
11 / 16
![Page 12: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/12.jpg)
Dynamics
Sprint during high-utility epochs
Tune management policy dynamically
Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]
12 / 16
![Page 13: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/13.jpg)
Utility Distributions
Perform well for dynamic workloads
Sprint judiciously given bimodal speedups
Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]
13 / 16
![Page 14: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/14.jpg)
Prediction Accuracy
Predict utility from phase with low error
Track modest number of unique phases
Huang et al. “DynaSprint: µarchitectural sprints with dynamic utility, thermal management” [MICRO’19]
14 / 16
![Page 15: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/15.jpg)
Conclusion
Dynamic Phase Analysis
Manage sprints online
Sprint with cache capacity
Algorithmic Game Theory
Arbitrate multi-programmed sprints
Decentralize power management
Future Directions
Coordinate multi-resource sprints
Define sprints dynamically
Huang, Joao, Rico, Hilton, Lee. “DynaSprint: Microarchitectural sprints with dynamic utility and thermalmanagement.” MICRO 2019.
Fan, Zahedi, Lee. “Distributed strategies for computational sprints.” CACM 2019.
Fan, Zahedi, Lee. “The computational sprinting game.” ASPLOS 2016.
15 / 16
![Page 16: Benjamin C. Lee - Microsoft · Benjamin C. Lee5pt benjamin.c.lee@duke.edu15pt Songchun Fan, Andrew Hilton, Ziqiang Huang, Jose Joao, Alejandro Rico, Seyed Majid Zahedi15pt [width=50pt]figs/logo.png](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f05a3357e708231d413f410/html5/thumbnails/16.jpg)
The Computational Sprinting Game
Benjamin C. Lee
Songchun Fan, Andrew Hilton, Ziqiang Huang,Jose Joao, Alejandro Rico, Seyed Majid Zahedi
Acknowledgements: Semiconductor Research Corporation; National Science Foundation.