phase detection and prediction on real systems for workload-adaptive power management

Princeton UniversityElectrical Engineering

SRC Student Symposium

Cary, NC 2006

Oct 10, 2006

Phase Detection and Prediction on Real Systems

for Workload-Adaptive Power Management

Canturk ISCI

Margaret MARTONOSI

Canturk Isci - Margaret Martonosi2

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management

[ SRC Student Symp’06 ]

Program Phases Distinct and often-recurring regions of program behavior

0.20.40.60.8

11.2

105 149 193 237 281 325 369 413 457 501 545

Billions of Instructions

IPC

00.20.40.60.8

1

0 44 88 132 176 220 264 308 352 396 440

Billions of Instructions

Me

m R

efs

38

42

46

50

0 44 88 132 176 220 264 308 352 396 440Billions of Instructions

Po

we

r [W

]

How can we detect recurrent execution under real system variability?

How can we predict future phase patterns?

How can we leverage predicted phase behavior for workload-adaptive power management? Can we do better than simple,

reactive methods?




Research Overview

Dynamic Management

Power Estimation Phase AnalysisPower Estimation

Runtime Monitoring

HardwarePerformanceCounters

DynamicProgramFlow

Application

Real Measurements

Monitor application execution via specific features

Classify features into phases

Detect/Predict phase behavior

Apply dynamic power management guided by phase predictions

Validate with real measurements




Phase ClassificationPhase Prediction

Dynamic Management

Phase AnalysisPower Estimation

Runtime Monitoring


DynamicProgramFlow

Runtime Monitoring


DynamicProgramFlow

This Talk

Application

Real Measurements

Track memory accesses per instruction (Mem/Uop) via performance counters

Classify execution into phase patterns based on Mem/Uop rates

Predict future behavior with the Global Phase History Table (GPHT) predictor

Use phase predictions to guide dynamic voltage and frequency scaling (DVFS)




From Execution to Phases

Assign different Mem/Uop ranges to different phases Higher phase number more memory bound phase

Phase patterns expose available recurrence!

Simple phase definition Resilient to system variations

Invariant to dynamic power management actions

2.80E+10 2.90E+10 3.00E+10 3.10E+10 3.20E+10 3.30E+10Cycles

0

1

2

3

4

5

Ph

ases

0.000

0.005

0.010

0.015

0.020

Mem

/Uo

p R

ate

PhasesMem/Uop




Pt’’+1

Pt

Predicting Phases with the GPHT

Similar to a global history branch predictor Implemented in OS for on-the-fly phase prediction

Pt

Pt-1 Pt-2 … … Pt-N… … Pt’’ Pt’’-1 Pt’’-2 … … Pt’’-N… …

Pt’ Pt’-1 Pt’-2 … … Pt’-N… …

: : : : : :: :

: : : : : :: :

: : : : : :: :

P0 P0 P0 … … P0… …

Pt’’+1

Pt’+1

:

:

:

P0

15

20

:

:

:

-1

Last observed phase from performance counters

GPHR

PHT PHT Tags PHT Pred-n

Age / Invalid

GPHR depth

GPHR depth

PHT

entries

Predicted Phase

From GPHR(0) if no matching pattern

From the corresponding PHT Prediction entry if matching pattern in PHT

Pt-N-1 Pt’’ Pt’’-1 Pt’’-2 … … Pt’’-N… …

Pt’ Pt’-1 Pt’-2 … … Pt’-N… …

Pt




Prediction Accuracies

Compare to reactive approaches (Last Value prediction) GPHT performs significantly better for highly varying applications

Up to 6X and on average 2.4X misprediction improvement

Good performance down to 128 PHT entries Converges to last value as PHT entries 1

40

50

60

70

80

90

100

gzip_l

og

mcf

_inp

gcc_2

00

gcc_s

cila

b

wupwise_

ref

gap_r

ef

gcc_i

ntegra

te

gcc_e

xpr

amm

p_in

gcc_1

66

parse

r_re

f

apsi

_ref

bzip2_

progra

m

mgrid

_in

bzip2_

sourc

e

bzip2_

graphic

applu

_in

equak

e_in

Pre

dic

tio

n A

cc

ura

cy

(%

)

LastValue

PHT:1024, GPHR:8

PHT:128, GPHR:8

PHT:64, GPHR:8

PHT:1, GPHR:8




Phase Driven Dynamic Power Management Phase definitions Memory boundedness DVFS potential

Each predicted phase Corresponding (V,f) setting

Implementation overview:

Ap

pli

ca

tio

n

Ex

ec

uti

on

Runtime Phase Monitor:

Stop/Read performance counters

Translate counter readings to the corresponding phase

Update phase predictor states

Predict next phase

Translate predicted phase to corresponding DVFS setting

Same as current setting?

Apply new DVFS setting

Reset trigger Reinitialize/Start performance counters

No

Yes

trigger phase monitor every 100 million instructions

Exit to program execution




Complete Example GPHT can

accurately predict varying application behavior!

0

1

2

3

4

5

Ph

ases

0.000

0.004

0.008

0.012

0.016

0.020

0.024

Mem

/Uo

p

ACTUAL_PHASE PRED_PHASE (GPHT)Mem/Uop (GPHT)

0

2

4

6

8

10

12

14

Po

wer

[W

]

Power (Baseline) Power (GPHT)

0

0.3

0.6

0.9

1.2

1.5

1.8

2.1

1.5E+09 2.0E+09 2.5E+09 3.0E+09 3.5E+09 4.0E+09 4.5E+09 5.0E+09

Instructions

BIP

S

BIPS (Baseline) BIPS (GPHT)

Significant power savings compared to baseline!

Negligible performance degradation!




Improvement over Reactive Methods

7% EDP improvement over reactive methods!

Comparableor less performance degradation!

0%

5%

10%

15%

20%

bzip2_

progra

m

bzip2_

sourc

e

bzip2_

graphic

mgrid

_in

applu

_in

equak

e_in

swim

_in

mcf

_inp

aver

age

Per

f. D

egra

dat

ion Last Value GPHT

0%

10%

20%

30%

40%

50%

bzip2_

progra

m

bzip2_

sourc

e

bzip2_

graphic

mgrid

_in

applu

_in

equak

e_in

swim

_in

mcf

_inp

aver

age

ED

P I

mp

rove

men

t Last Value GPHT 63%

66%

63%

70%




Conclusions Phase characterizations help identify repetitive application behavior

under real-system variability and dynamic management actions

Runtime phase predictions with the Global Phase History Table can accurately predict future application behavior Up to 6X and on average 2.4X less mispredictions than reactive approaches

Dynamic power management guided by these phase predictions help improve system power/performance efficiency 27% EDP improvements over baseline and 7% over reactive approaches

Presented research framework and real-system experiments can guide phase-oriented characterization and dynamic adaptation applications




Thanks!




EXTRAS 1.1) Why care about phases examples

1.2) Why care about pwr phases examples

1.3) What are different features that prev studies looked at?

2) Experiment setup details




Characterizing execution regions

1.1) Why Care About Phases?

00.10.20.30.40.50.60.70.80.9

1

5 10 15 20 25Time [s]

E1 E2 E3 E4




0

0.2

0.4

0.6

0.8

1

2 7 12Time [s]

Store Refs

Load Refs

Load Misses

Store Misses

Committed Instrns

1.1) Why Care About Phases? Characterizing execution regions

Managing dynamic adaptation

OFFON




0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

3 8 13Time [s]

Load Refs

Store Misses

1.1) Why Care About Phases? Characterizing execution regions

Managing dynamic adaptation

Use current phase/behavior to predict future behavior




1.2) Why Care About Power Phases?

Useful for: Guiding power budget / temperature limit management

40

45

50

10 54 98

35

45

55

65

75

10 54 98

Slow down!

Power [W] Temp. [oC]

Time [s] Time [s]

Uncontrolled T

Enforced T

I.e. Montecito/Foxton





Useful for: Guiding power budget / temperature limit management Power/Temperature aware scheduling

Time [s]

Po

wer

[W

]

[Bellosa et al. COLP’03]





Useful for: Guiding power budget / temperature limit management Power/Temperature aware scheduling Power balancing for multiprocessor systems/activity migration

Power PowerTask1 Task2

Swap hot task

Slow down!Speed up!

Core/μP 1 Core/μP 2




Older




Dynamic Management

Power Estimation Phase AnalysisPower Estimation

This Talk Classify application execution into

phases based on HW performance counters

Predict phase behavior

Apply dynamic power management guided by phase predictions

Validate with real measurements

Runtime Monitoring


DynamicProgramFlow

Application

Real Measurements




Predicting Phases with the GPHT

Similar to a global history branch predictor Implemented in OS for on-the-fly phase prediction

Pt Pt-1 Pt-2 … … Pt-N… … Pt’’ Pt’’-1 Pt’’-2 … … Pt’’-N… …

Pt’ Pt’-1 Pt’-2 … … Pt’-N… …

: : : : : :: :

: : : : : :: :

: : : : : :: :

P0 P0 P0 … … P0… …

Pt’’+1

Pt’+1

:

:

:

P0

15

20

:

:

:

-1

Pt

Last observed phase from performance counters

GPHR

PHT PHT Tags PHT Pred-n

Age / Invalid

GPHR depth

GPHR depth

PHT

entries

Predicted Phase

From GPHR(0) if no matching pattern

From the corresponding PHT Prediction entry if matching pattern in PHT

phase detection and prediction on real systems for workload-adaptive power management

Documents