virtualized high performance computing with mellanox fdr and ro ce
DESCRIPTION
In this video from VMworld 2014, Josh Simons from VMware presents: Virtualized High Performance Computing with Mellanox FDR and RoCE on VMware ESXi 5.5. "The HPC community can realize significant benefits from adopting enterprise-capable IT solutions grounded in proven virtualization and cloud technology. And conversely, as business IT environments become increasingly compute-intensive, lessons learned by the scientists and engineers working with HPC can be transferred to their counterparts in the enterprise. It’s a win-win situation." Watch the video presentation: http://insidehpc.com/2014/09/virtualized-high-performance-computing-mellanox-fdr-roce/ Learn more in the insideHPC Guide to Virtualization, the Cloud and HPC: http://bit.ly/1w8kMfuTRANSCRIPT
© 2014 VMware Inc. All rights reserved.
Virtualized High Performance Computing with Mellanox FDR InfiniBand and RoCE on VMware ESXi 5.5 Initial Performance Results
Josh SimonsOffice of the CTOVMware, Inc.
August 26, 2014
2
High Performance Computing
“High Performance Computing refers to the practice of aggregating computing power in a way that delivers much higher performance than one could get out of a typical desktop or workstation in order to solve large problems in science, engineering, or business.” [1]
Commercial:Oil explorationPharmaceutical designFinancial and economic modelingAdvanced data visualization
HPC Applications
Science and Engineering: Atmosphere, earth, environment Bioscience, biotechnology, geneticsPhysics - applied, nuclear, particle, condensed matter;Electrical engineering, circuit design, microelectronicsMechanical engineering - from prosthetics to
spacecraft
[1] G. Sravanthi, B. Grace. A Review of High Performance Computing. IOSR Journal of Computer Engineering, vol. 16, pp36-43, 2014
VMware vCAC API
Users IT
Research Group 1 Research Group m
Public/HybridClouds
ProgrammaticControl andIntegrations
User Portals Security
NSX
Research Cluster 1 Research Cluster n
VMware vCloud Automation Center
VMwarevCenter Server
VMware vSphere VMware vSphere VMware vSphere
Blueprints
VMwarevCenter Server
VMwarevCenter Server
Secure Private Cloud for HPC
3
HPC Workloads• Scientific or technical workloads• Often floating-point intensive• Often parallel• Often storage intensive• Run on server-class systems
4
Throughput Workloads MPI Oriented Workloads
5
FDR InfiniBand Read Latency
0.5
1
2
4
8
16
32
63.9999999999999
128
256
512
1024
2048
Native
ESXi 5.5
ESXi 5.5 with SR-IOV
Message Sizes (Bytes)
Hal
f Rou
ndtr
ip L
aten
cy (
µs)
0.5
1
1.5
2
2.5
3
Four-node HP cluster DL380 G8 nodes 128 GB, 3.0 GHz FDR IB/RoCE HCAs 12-port FDR switch
ESX 5.5u1
6
RoCE Read Latency
2 8 32 128
512
2048
8191
.999
9999
9999
3276
7.99
9999
9999
1310
72
5242
87.9
9999
9999
2097
152
8388
607.
9999
9998
0.5
5
50
500
NativeESXi 5.5
Message Size (Bytes)
Hal
f R
ound
trip
late
ncy
(us)
0.5
1
1.5
2
2.5
3
7
HPC Challenge Benchmark (HPCC) with FDR IB
MPIR
andomAcc
ess
StarR
andomAcc
ess
PTRANS
StarD
GEM
M
StarS
TREAM0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
HPCC Virtual to Native Ratios (higher is better)
n4np4 n4np8
n4np16 n4np32
n4np64
8
NAMD Molecular Dynamics with FDR IB
Apoa1 f1atpase0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
NAMD Benchmarks Native to Virtual Ratios(Higher is Better)
n4np4 n4np8
n4np16 n4np32
n4np64
CONFIDENTIAL 9
LAMMPS Molecular Dynamics with FDR IB
Atomic Fluid Bulk Copper Bead-Spring Polymer0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
LAMMPS Benchmarks Native to Virtual Ratios(Higher is Better)
n4np4 n4np8
n4np16 n4np32
n4np64
To Learn More:
• Office of the CTO Expo Booth (2-6pm)
• How to Engage with Your Engineering, Science, and Research Groups About Virtualization and Cloud Computing Thursday 10:30-11:30 Moscone West 2003
Josh Simons, [email protected]