Reducing Energy Consumption of Disk Storage Using Power Aware Cache Management

Qingbo Zhu, Francis M. David, Christo F. Deveraj, Zhenmin Li, Yuanyuan Zhou

Department of Computer Science University of Illinois at Urbana-Champaign

Pei Cao**Cisco Systems Inc.

HPCA’04

02/17/2004

Data Centers: Service-based

Computing

router

Web Servers

Application Servers

Database Servers

switch

…

…

…

SAN

…

Storage Servers

Energy Problem Faced by Data Centers

Data centers High electricity bills: up to 25% TCO

$8M per year for a 30,000-square-foot data center [EERE news 2003]

Increase as much as 25% annually [Energy User News 2002]

Storage 27% of the total energy consumed

[Maximum Inc. 2002]

Disk Power Model

Disk power modes Active/idle/standby/sleep Spinup/down cost Breakeven time

Metrics Energy consumption Average response time

Disk Power Management Schemes

Oracle scheme (off-line)

Practical scheme (on-line)

access1 access2

IdleTime > BreakEvenTime

Idle for BreakEvenTime Wait time

Current Research Status

The idle periods in server workloads are too short to justify high spinup/down cost of server disks [ISCA’03][ISPASS’03] [ICS’03]

IBM Ultrastar 36Z15 -- 135J/10.9s Multi-speed disk model [ISCA’03]

RPMs: multiple intermediate power modes Smaller spinup/down costs Be able to save energy for server workloads

Most previous work assume that all requests go directly to physical disks

Observation

Many requests are filtered out by the storage cache

EMC Symmetrix storage system Up to 128GB storage cache

IBM ESS system Up to 64GB storage cache

Cache replacement and write policies affect the access sequences to physical disks Block-based

storage system

The Focus of Our Paper

Power-aware off-line and on-line cache replacement algorithms and write policies reduce the disk energy consumption

Clarification The underlying disk power management

scheme is NOT changed The storage cache is always active

Outline

Motivation Power aware cache management

Belady’s algorithm is NOT energy-optimal Off-line power-aware greedy algorithm On-line power-aware algorithm Four write policies

Simulations Conclusion Limitations and future work

Breakeven-Time for Multiple Power Modes

En

erg

y

Con

sum

pti

on

Idle Period Length

mode 0 mode 1 mode 2

mode 3

Spinup cost

Active mode

t1 t3t2 T

E(T)

Is Belady’s Algorithm Energy-Optimal?

Belady’s algorithm: performance-optimal Minimize the number of misses Evicting the block with the longest future refere

nce distance Answer: NO!

Only consider the access sequence Ignore requests’ arrival time Ignore multiple disk scenario

A Simple Example

t

A

B

BAC

Disk 0

D

An energy-optimal algorithm using dynamic programming

Belady’s algorithm

power-aware algorithm

C

Off-line Power-Aware Greedy Algorithm

Idea: evicting the block with the smallest energy penalty Observation: take advantage of the knowledge about future’s bound-to-happen

misses Cold misses Capacity misses due to previous evictions

D E F: bound-to-happen misses

A

B

BC E FAD

How to Calculate Energy Penalty of Evicting a Block

D E F: bound-to-happen misses

A

B

BC E FD A

E(DE)E(AE)E(DA)+ -Energy Penalty (A)

=

E(EF)E(BF)E(EB) +Energy Penalty (B)

= -

Re-viewEn

erg

y

Con

sum

pti

on

Idle Period Length

mode 0 mode 1 mode 2

mode 3

t1 t3t2

On-line Power Aware Algorithm

Idea: selectively keep blocks from inactive disks in the cache for a longer time

Make “inactive disks” more inactive

Idle Period Length

En

erg

y

Savin

g

mode 0

mode 1

mode 2

mode 3

Super Linear

t1

t2

t4

t3

energy saving

energy penalty

<<

How to Measure Disk Activeness?

Characteristics of inactive disks Small percentage of cold misses Large idle period lengths with high

probability

How to Keep Track of Cold Misses?

Bloom Filter: a space-efficient membership test method

A vector v of m bits k independent hash functions ranging {1..m} Given an access for block a, check the bits at

position

If any of them is 0, a is cold miss and then set all bits 1

Otherwise, it is not a cold miss though we may be wrong

1.6M blocks with v = 2M bytes and k = 7 the accuracy is 99.18%

)(),...,(),( 21 ahahah k

How to Keep Track of the Distribution of Idle Period Lengths?

Histogram-based estimation

Idle Period Length

Case Study: PA-LRU

Applies to all cache replacement algorithms LRU, 2Q, MQ etc.

PA-LRU Two LRU stacks

LRU0: blocks from active disks LRU1: blocks from inactive disks Evict blocks from LRU0 first

The evaluation of disk activeness is epoch-based Adapt to workload changes

Write Policy Write back Write through Write back with eager updates (WBEU)

Eagerly write back all the dirty blocks when the target disk becomes active due to a read miss

Write through with deferred updates (WTDU) Use a log disk which is always active Write the blocks to the log disk if the target disk is

not active Flush back all the logged blocks when the target disk

becomes active due to a read miss Retain persistent semantics

Evaluation Methodology

Experiment setup DiskSim:

IBM Ultrastar 36Z15 Enhanced by a multi-speed

disk power model Enhanced by a CacheSim

Real system traces: OLTP Cello96

Synthetic traces: Exponential distribution Pareto distribution

Energy (OLTP)

OPG: energy saving 2% - 9% over Belady’s algorithm

PA-LRU: energy saving 16% over LRU

0

0.2

0.4

0.6

0.8

1

Infinite size Belady OPG LRU PA-LRU

Practical Oracle

Average Response Time (OLTP)

OPG: 4% better than belady’s algorithm

PA-LRU: 50% better than LRU (avoid expensive spinup)

0

0.2

0.4

0.6

0.8

1Infinite size Belady OPG LRU PA-LRU

Practical

Conclusion

Power aware cache management plays an important role on disk energy consumption Belady’s algorithm is NOT energy-optimal Evict the blocks with small energy penalty Make inactive disks more inactive

Future Work and Acknowledgements

Limitations and future work Design online algorithms for a single disk as well Take prefetching into account Real system experiments

Acknowledgements Anonymous reviewers Professor Lenny Pitt (UIUC) CMU Parallel Data Lab (for DiskSim) HP Lab (for Cello Trace)

Questions?

Thanks!

Backup Slides

Write Policies (Exponential Distribution)

Write back: up to 20% saving than write through

WBEU: up to 60% saving than write through

WTDU: up to 55% saving than write through

0

0.2

0.4

0.6

0.8

1Write through Write back WBEU WTDU

Practical

Energy-optimal problem

Offline Energy-optimal Algorithm Only two power state

1: active mode 0: standby mode

Virtual time Only one disk Parameters:

b: the number of disk blocks k: the number of cache blocks n: the input size m: threshold

Cache State (C, t, i) The cache contains the blocks in set C

after the first i+1 references and the last t consecutive reference were ache hit

Offline energy optimal algorithm

Minimize energy: maximize the time the disk can spend in standby mode

A(C,t,i): the maximum time that the disk spends in the standby mode until (C,t,i) is reached

Dynamic programming: Extend to multiple

disks:

Time Breakdown

Mean Inter-arrival Time

Simulation Results: Cello96

OPG: energy saving 5% - 7% over belady’s algorithm

PA-LRU: energy saving 2% - 3%

Cello96: high cold miss ratio, larger than 65% for all disks

OPG is heuristic

D E: bound-to-happen misses

A

B

BC ED A

A Step Further…

Consider both miss ratio and energy penalty

Idea: don’t differentiate among blocks whose energy penalty is smaller than a threshold T energy penalty smaller than T: round up to T T=0: pure greedy algorithm T is large enough: belady’s algorithm

Data Centers: Service-based Computing

Internet WebServers

DatabaseServers

LocalStorage

Ethernet StorageServers

SAN