multiple services throughput optimization in a hierarchical middleware
DESCRIPTION
CCGRID, Newport Beach, June, 2011TRANSCRIPT
Multiple Services Throughput Optimization in aHierarchical Middleware
Eddy Caron1, Benjamin Depardon2, Frédéric Desprez1
1University of Lyon. LIP Laboratory.UMR CNRS - ENS Lyon INRIA - UCBL 5668.
FRANCE2SysFera
CCGrid 2011May 24th, 2011
DIET Problem Model Planning Experiments Conclusion
Introduction
• Solve large problems• Grids: distributed and large scale environments• GridRPC approach:
• Clients submit requests to a meta-scheduler (an agent)• Agent schedules requests and find suitable servers• Examples: Ninf-G, NetSolve, GridSolve, WebCom-G, DIET
• Several middleware architectures:• Star graph• Hierarchy
• Performance depend on the middleware deployment
⇒Needs for performance modelization and algorithms to choosethe “best” hierarchy
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 2/20
DIET Problem Model Planning Experiments Conclusion
Introduction
• Solve large problems• Grids: distributed and large scale environments• GridRPC approach:
• Clients submit requests to a meta-scheduler (an agent)• Agent schedules requests and find suitable servers• Examples: Ninf-G, NetSolve, GridSolve, WebCom-G, DIET
• Several middleware architectures:• Star graph• Hierarchy
• Performance depend on the middleware deployment
⇒Needs for performance modelization and algorithms to choosethe “best” hierarchy
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 2/20
DIET Problem Model Planning Experiments Conclusion
Outline
1 The DIET middleware
2 Problem
3 Hierarchy model
4 Deployment planning
5 Experiments
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 3/20
DIET Problem Model Planning Experiments Conclusion
DIET
Distributed Interactive Engineering Toolboxhttp://graal.ens-lyon.fr/DIET
• Toolbox for Application Service Provider (ASP)• Hierarchical architecture: scalability & performance• GridRPC compliant• Testbed for theoretical results
• scheduling for heterogeneous platforms• data (re)distribution and replication• performance evaluation• algorithmic for heterogeneous and distributed platforms• . . .
• Used in production for the Decrypthon project• Now supported by a startup: SysFera
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 4/20
DIET Problem Model Planning Experiments Conclusion
AGENTS
Client
Servers
Request submission
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20
DIET Problem Model Planning Experiments Conclusion
LA1
SeD1 SeD2
LA2
SeD3 SeD4
MA
Client FindServer()
Request submission
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20
DIET Problem Model Planning Experiments Conclusion
LA1
SeD1 SeD2
LA2
SeD3 SeD4
MA
Client
FindServer()
Request submission
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20
DIET Problem Model Planning Experiments Conclusion
LA1
SeD1 SeD2
LA2
SeD3 SeD4
MA
Client
FindServer()
Request submission
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20
DIET Problem Model Planning Experiments Conclusion
LA1
SeD1 SeD2
LA2
SeD3 SeD4
MA
Client
Estimate()
Request submission
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20
DIET Problem Model Planning Experiments Conclusion
LA1
SeD1 SeD2
LA2
SeD3 SeD4
MA
Client
Aggregate()
Request submission
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20
DIET Problem Model Planning Experiments Conclusion
LA1
SeD1 SeD2
LA2
SeD3 SeD4
MA
Client
Aggregate()
Request submission
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20
DIET Problem Model Planning Experiments Conclusion
LA1
SeD1 SeD2
LA2
SeD3 SeD4
MA
Client BestServer = SeD1
Request submission
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20
DIET Problem Model Planning Experiments Conclusion
LA1
SeD1 SeD2
LA2
SeD3 SeD4
MA
Client
RunService()
SeD1
Request submission
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20
DIET Problem Model Planning Experiments Conclusion
MA LA1 LA1 ...
Goal
Underlying questions• How many agents?• How many servers for each type of service?• What is the shape of the hierarchy?
Objective
Given a platform G = (V ,E), and a set of servicesR, what is thebest attainable throughput, i.e., the number of finished requestsper time unit, in a DIET hierarchy?
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20
DIET Problem Model Planning Experiments Conclusion
SeD1 SeD1 SeD2 ...SeD2
Goal
Underlying questions• How many agents?• How many servers for each type of service?• What is the shape of the hierarchy?
Objective
Given a platform G = (V ,E), and a set of servicesR, what is thebest attainable throughput, i.e., the number of finished requestsper time unit, in a DIET hierarchy?
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20
DIET Problem Model Planning Experiments Conclusion
LA LA
SeD1 SeD1 SeD2 SeD2
MA
LA
LASeD1 SeD1
SeD2
SeD2
MALA
LA
SeD1
LA
LA
LA
LA
LA
LA
MA
Goal
Underlying questions• How many agents?• How many servers for each type of service?• What is the shape of the hierarchy?
Objective
Given a platform G = (V ,E), and a set of servicesR, what is thebest attainable throughput, i.e., the number of finished requestsper time unit, in a DIET hierarchy?
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20
DIET Problem Model Planning Experiments Conclusion
LA LA
SeD1 SeD1 SeD2 SeD2
MA
LA
LASeD1 SeD1
SeD2
SeD2
MALA
LA
SeD1
LA
LA
LA
LA
LA
LA
MA
Goal
Underlying questions• How many agents?• How many servers for each type of service?• What is the shape of the hierarchy?
Objective
Given a platform G = (V ,E), and a set of servicesR, what is thebest attainable throughput, i.e., the number of finished requestsper time unit, in a DIET hierarchy?
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20
DIET Problem Model Planning Experiments Conclusion
More formally. . .
Objective functionGiven:• a platform G = (V ,E ,W ,B),• a set of services R,• for each service i ∈ R an objective throughput ρ∗i ,
find a hierarchy such that:• ∀i , i ′ ∈ R, ρi
ρ∗i' ρi′
ρ∗i′
in steady state,
• mini∈R
{ρiρ∗i
}is maximized,
• and the hierarchy has as few agents as possible.
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 7/20
DIET Problem Model Planning Experiments Conclusion
Platform
Bc,c'
w2,B2w1,B1
w6,B6
w5,B5 w4,B4
w3,B3
• Fully connected platform• wj : computing power• Bj,j ′ : bandwidth between any two nodes
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 8/20
DIET Problem Model Planning Experiments Conclusion
Servers model
Average computation time
Si being the set of nodes allocated forservers of type i
T servercompi
=wappi + |Si | .wprei∑
j∈Siwj
Communications
T serverjrecvi
=mreqi
Bj,f ji
T serverjsendi
=mrespi
Bj,f ji
wpreiwappi
Servermreqi
mrespi
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 9/20
DIET Problem Model Planning Experiments Conclusion
Agent model
Computation
Tagentjcomp =
∑i∈R
ρservi .δ
ji .wreqi + wrespi
(∣∣∣Chld ji
∣∣∣)wj
Communications
Tagentjrecv =
∑i∈R
ρservi .
δji .mreqi
Bj,f j+
∑k∈Chld j
i
mrespi
Bj,k
Tagentjsend =
∑i∈R
ρservi .
δji .mrespi
Bj,f j+
∑k∈Chld j
i
mreqi
Bj,k
• Chld ji :
children of Aj knowing servicei
• δji :
boolean, does Aj know servicei?
wreqi
wresp
i
...
Agentmreqi
mresp
i
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 10/20
DIET Problem Model Planning Experiments Conclusion
SeD1 SeD1 SeD2 SeD2 SeD2SeD1 SeD1 SeD2
Automatic deployment: homogeneous Bj ,j ′
Bottom-up approach1. Distribute N nodes to the servers⇒ ∀i , i ′ ∈ R, ρservi
ρ∗i' ρservi′
ρ∗i′
2. With the remaining nodes, try to build an agent hierarchy• use a mixed integer linear program• build level by level, until 1 agent is enough• if not possible, reduce N and goto 1
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 11/20
DIET Problem Model Planning Experiments Conclusion
SeD1 SeD1 SeD2 SeD2 SeD2SeD1 SeD1 SeD2
LA LA LA
Automatic deployment: homogeneous Bj ,j ′
Bottom-up approach1. Distribute N nodes to the servers⇒ ∀i , i ′ ∈ R, ρservi
ρ∗i' ρservi′
ρ∗i′
2. With the remaining nodes, try to build an agent hierarchy• use a mixed integer linear program• build level by level, until 1 agent is enough• if not possible, reduce N and goto 1
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 11/20
DIET Problem Model Planning Experiments Conclusion
SeD1 SeD1 SeD2 SeD2 SeD2SeD1 SeD1 SeD2
LA LA LA
MA
Automatic deployment: homogeneous Bj ,j ′
Bottom-up approach1. Distribute N nodes to the servers⇒ ∀i , i ′ ∈ R, ρservi
ρ∗i' ρservi′
ρ∗i′
2. With the remaining nodes, try to build an agent hierarchy• use a mixed integer linear program• build level by level, until 1 agent is enough• if not possible, reduce N and goto 1
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 11/20
DIET Problem Model Planning Experiments Conclusion
wpreiwappi
Server
wreqiwrespi
...
Agent
wpreiwappi
Automatic deployment: heterogeneous Bj ,j ′
Genetic algorithm
1. Generate a random population of valid solutions
2. Evolve the population• crossover• mutation
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20
DIET Problem Model Planning Experiments Conclusion
...
1. Random initialization
...
2. Selection 3. Reproduction
+=
4. Best individual
...
Automatic deployment: heterogeneous Bj ,j ′
Genetic algorithm
1. Generate a random population of valid solutions
2. Evolve the population• crossover• mutation
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20
DIET Problem Model Planning Experiments Conclusion
3
86 9
1
5
2 4 15 4
10 7
1
58 6
11
3
9 20 3
86 9
1
5
2 4 15 4
10 7
1
58 6
11
3
9 20&
H1 H2 H'1 H'2
Automatic deployment: heterogeneous Bj ,j ′
Genetic algorithm
1. Generate a random population of valid solutions
2. Evolve the population• crossover• mutation
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20
DIET Problem Model Planning Experiments Conclusion
Type mutation Parent mutation Pruning
Automatic deployment: heterogeneous Bj ,j ′
Genetic algorithm
1. Generate a random population of valid solutions
2. Evolve the population• crossover• mutation
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20
DIET Problem Model Planning Experiments Conclusion
Experimental environment
PlatformGrid’5000• Clusters:
• Lille: Chti (3784MFlops)• Rennes: Paradent (4378MFlops)• Lyon: Sagittaire (3249MFlops) and Capricorne
(2922MFlops)
• 1Gb and 10Gb networks
Services• dgemm (Atlas library)• Fibonacci number (naive algorithm)
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 13/20
DIET Problem Model Planning Experiments Conclusion
Homogeneous comm. / Heterogeneous comp.min-first
500
1000
1500
2000
2500
1-2
2-3
5-5
10-1
0
15-1
5
20-2
0
25-2
5
500
1000
1500
2000
2500
dgemm
thro
ughput (r
equests
/ s
)
Fib
onacci th
roughput (r
equests
/ s
)
Number of nodes (Sagittaire-Capricorne)
dgemm 100 Fibonacci 30
dgemm theoreticalFibonacci theoreticaldgemm experimental
Fibonacci experimental
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 14/20
DIET Problem Model Planning Experiments Conclusion
Homogeneous comm. / Heterogeneous comp.min-first
200
400
600
800
1000
1200
1400
1600
1800
2000
22001-
22-
3
5-5
10-1
0
15-1
5
20-2
0
25-2
5
200
400
600
800
1000
1200
1400
1600
1800
2000
2200
dgemm
thro
ughp
ut (
requ
ests
/s)
Fib
onac
ci th
roug
hput
(re
ques
ts/s
)
Number of nodes (Sagittaire-Capricorne)
dgemm 100 Fibonacci 30
dgemm, min-firstFibonacci, min-first
dgemm, star graphFibonacci, star graph
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 14/20
DIET Problem Model Planning Experiments Conclusion
Homogeneous comm. / Heterogeneous comp.max-first
500
1000
1500
2000
2500
1-2
2-3
5-5
10-1
0
15-1
5
20-2
0
25-2
5
500
1000
1500
2000
2500
dgemm
thro
ughput (r
equests
/ s
)
Fib
onacci th
roughput (r
equests
/ s
)
Number of nodes (Sagittaire-Capricorne)
dgemm 100 Fibonacci 30
dgemm theoreticalFibonacci theoreticaldgemm experimental
Fibonacci experimental
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 15/20
DIET Problem Model Planning Experiments Conclusion
Homogeneous comm. / Heterogeneous comp.max-first
200
400
600
800
1000
1200
1400
1600
1800
2000
22001-
22-
3
5-5
10-1
0
15-1
5
20-2
0
25-2
5
200
400
600
800
1000
1200
1400
1600
1800
2000
2200
dgemm
thro
ughp
ut (
requ
ests
/s)
Fib
onac
ci th
roug
hput
(re
ques
ts/s
)
Number of nodes (Sagittaire-Capricorne)
dgemm 100 Fibonacci 30
dgemm, max-firstFibonacci, max-first
dgemm, star graphFibonacci, star graph
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 15/20
DIET Problem Model Planning Experiments Conclusion
Example of produced hierarchymin-first, 20-20 nodes
MA
LA LA
LA LA LA
D D D D D D F F F F F F F
LA LA LA
F F F FD D D D
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 16/20
DIET Problem Model Planning Experiments Conclusion
GA / min-first / max-first comparison
0
500
1000
1500
2000
2500
1-2
2-3
5-5
10-1
0
15-1
5
20-2
0
25-2
5
Thr
ough
put (
nb r
eque
sts/
s)
Nodes (Sagittaire-Capricorne)
dgemm
min-firstmax-first
GeneticGenetic mean
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 17/20
DIET Problem Model Planning Experiments Conclusion
GA
200
400
600
800
1000
1200
1400
1600
1800
1-1-
12-
2-1
3-4-
3
7-7-
6
10-1
0-10
13-1
4-13
17-1
7-16
200
400
600
800
1000
1200
1400
1600
1800
dgemm
thro
ughp
ut (
requ
ests
/ s)
Fib
onac
ci th
roug
hput
(re
ques
ts /
s)
Number of nodes (Sagittaire-Paradent-Chti)
dgemm 100 Fibonacci 30
dgemm theoreticalFibonacci theoreticaldgemm experimental
Fibonacci experimental
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 18/20
DIET Problem Model Planning Experiments Conclusion
Example of produced hierarchy
MA
LA LA
D D LAF F F F F
F LA
LA
F F D D D
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 19/20
DIET Problem Model Planning Experiments Conclusion
Conclusion and future work
Conclusion• Computation and communication model for hierarchical
middleware• Two kinds of heuristics:
• Bottom-up heuristic, based on linear programming• Genetic algorithm
• Experiments closely follow the predictions
Future work• Heuristics for arbitrary graphs• Dynamically adapt the hierarchy• Extend model to multiple hierarchies
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 20/20
DIET Problem Model Planning Experiments Conclusion
Conclusion and future work
Conclusion• Computation and communication model for hierarchical
middleware• Two kinds of heuristics:
• Bottom-up heuristic, based on linear programming• Genetic algorithm
• Experiments closely follow the predictions
Future work• Heuristics for arbitrary graphs• Dynamically adapt the hierarchy• Extend model to multiple hierarchies
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 20/20
DIET Problem Model Planning Experiments Conclusion
Thank you!
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 20/20