An Efficient Threshold-Based Power Management

Mechanism for HeterogeneousSoft Real-Time Clusters

Leping Wang, Ying LuUniversity of Nebraska-Lincoln, USA

April 21, 2023

2

Outline

• Motivation• Related Work• Problem Statement• Threshold-based approach• Evaluation• Conclusion

3

Motivation

• Why power management (PM) for heterogeneous clusters– The power-related costs dominate the total cost of

ownership of a cluster system

– Most PM mechanisms are applicable to homogenous systems

– Heterogeneous clusters are already widespread

4

Motivation

• Opportunities for PM in heterogeneous clusters– Turn off or hibernate idle servers

– Dynamically scale operating frequency/voltage (DVS) for underutilized servers

– Distribute more requests to power-efficient servers

5

New Challenges

• Decide not only how many but also which cluster servers should be used to process current requests, when necessary

• Identifying the optimal load distribution for a heterogeneous cluster is a non-trivial task

6

Related Work

• PM in homogeneous systems– [Bianchini et al. 2004], [Bohrer et al. 2002],

[Chase et al. 2001], [Chen et al. 2005], [Elnozahy et al. 2002],

[Rajamani et al. 2003]

• PM in heterogeneous systems– [Heath et al. PPoPP2005], [Rusu et al. RTAS2006]

7

Related Work

• Current PM approaches for heterogeneous clusters– Search-based algorithms– Extensive performance measurements– Long optimization process

high customization costs upon new installations, server failures, cluster upgrades or other changes

8

Goal and Components• Goal

– Near-optimal power consumption– QoS (average response time guarantee)– Efficient algorithm

• Three components– Vary-on/off – DVS with feedback control– Optimal workload distribution

9

System Model

1.CPU-bounded server clusters (e.g. web server cluster)

2.One front-end server

3.N heterogeneous back- end servers

10

Optimization Problem

Cast the PM to an optimization problem• Objective: Minimize the total cluster power consumption J

• QoS constraints:

• Decisions on – Which servers should be used to process the current workload

cluster , i.e., decide xi : 0 or 1

– How should the workload cluster be distributed to active back-end servers, i.e., decide λi such that

– According to i, back-end server set its CPU frequency fi

RRiˆ

N

iclusteriix

1

11

Power and Capacity Models

• Power Model

• Capacity Model

: The ith server’s throughput

: The ith server’s performance coefficient

i

i

if

iii f

N

iiiii fcxJ

1

3)(

: Total power consumption

: The ith server’s on/off state

: The ith server’s constant power consumption

: The ith server’s operating frequency

: The ith server’s dynamic power consumption

J

ix

ic

3ii f

12

Optimization Problem

• According to the M/M/1 queuing model and our server capacity model, we have

• To make , we know

iiiiii f

R

11

Rf

ii

idesiredi ˆ

1_

RRiˆ

13

Optimization Problem

• The optimization problem is formed as follows– Minimize:

Subject to:

N

i ii

iiii

RcxJ

1

3])ˆ

1([

NiR

f

Nixx

x

iii

ii

N

iclusterii

,...,2,1,ˆ1

0

,...,2,1,0)1(

,

max_

1

14

Optimization Problem

• No analytical method to get the closed-form solution on i and xi

• Time complexity of search-based algorithm

• Basic idea of our efficient PM– Use a heuristic method to decouple decisions on xi and

i, then solve them separately to obtain near-optimal solutions.

)2( Ncluster

15

Threshold-Based Approach

• An efficient PM heuristic – Efficient offline analysis:

• Divides the possible workload range into N sub-ranges

• For each sub-range, the PM decisions are derived offline

– Online execution: Periodically,

1.Front-end server: workload cluster is predicted and depending on the range cluster falls into, the corresponding PM decisions will be followed

2.Back-end server: applies DVS mechanism to decide fi

]ˆ,0( cluster

16

Offline Analysis

1. Order the heterogeneous back-end servers, i.e., generates a sequence, called ordered server list

2. Produce server activation thresholds 1, 2, … N

such that if cluster (k-1, k], it is optimal to turn on the first k servers of the ordered server list

3. Optimal workload distribution problem is solved for the N scenarios where cluster (k-1, k], k=1, 2, …, N (time complexity: (N))

17

Offline Analysis

• When cluster (k-1, k], the first k servers of the ordered server list are turned on and the optimization problem becomes– Minimize:

Subject to:

Solution: the optimal workload distribution i

k

i ii

iii

RcJ

1

3)ˆ

1(

kiR

fiii

k

iclusteri

,...,2,1,ˆ1

0

,

max_

1

18

Algorithm

• Our method, denoted as TP-CP-OP– Server Ordered List

Order all back-end servers according to their Typical Power (TP) efficiencies

– Server Activation Thresholds Consider both server Capacity constraints and Power efficiencies (CP)

– Optimal Workload Distribution (OP)

19

Dynamic Voltage Scaling

Feedforward M/M/1 Based

Controller

Feedback PI Controller

ith Back-end Server

+

Ri

R̂

i

fi

fierrori

-+

20

Evaluation

• A small cluster with 4 back-end servers– Continuous operating frequency ranged in (0, fi_max]– Discrete operating frequency levels in [fi_min , fi_max]

• A large cluster with 128 back-end servers in 8 different types

21

Evaluation

• Synthetic workload and Real Workload

• Desired average response time is set at 1s

• Evaluation metrics: average response time and power consumption

• Each simulation lasts 3000s

• Power management in every 30s

22

Evaluation

• Baseline algorithms– Threshold-based approaches: AA−AA−CA,

SP−CA−CA, EE-RT-HSC

– Optimal power management solution OPT-SOLN obtained by a search-based algorithm

23

Evaluation• Average Response Time

24

Evaluation

• Power Consumption

25

Conclusion

• A efficient power management algorithm for heterogeneous server clusters– Mathematical models based

• Minimum performance profiling

– Workload threshold based

• Low algorithm time complexity

– Balance overhead and optimal solution

• Fewer number of server on/off changes

• Near-optimal power consumption

26

Technical Report

• L. Wang and Y. Lu. Efficient power management of heterogeneous soft real-time clusters. Technical Report TR-UNL-CSE-2008-0004, University of Nebraska-Lincoln, 2008

Leping Wang, Ying Lu

Questions

or

Comments?

? Thanks!

Evaluation

• Effect of Feedback Control

Evaluation

• Effect of Feedback Control