opennebulaconf2015 2.14 cloud service experience in teidehpc infrastructure - carlos ignacio...
TRANSCRIPT
OpenNebula implementation in TeideHPC infrastructure
Carlos I. González Vila
OpenNebulaConf 2015 - Barcelona
ITERFounded by the Cabildo Insular de Tenerife in 1990, the island's administrative authority. Its objectives are to promote research activities and technological development, especially those related with the use of the renewable energies.
ITER
ITER
MA
NA
GE
ME
NT
RENEWABLE ENERGIES
Departamento de Fotovoltaica Departamento de Eólica
DIS
EM
INA
TIO
N A
RE
A
MA
INT
EN
AN
CE
Photovoltaics
Sustainable architecture
Wind Energy
INFORMATION AND NEW TECHNOLOGIES
Electronics
Informatics
Groundwater resources
Environmental quality
Volcanology
ENVIRONMENT SCIENCE
TeideHPC
TeideHPC
● Weather forecasting– Wind power generation
– Volcanic smoke dispersion prediction
– Solar power generation
● Bioinformatics● Computational Fluid Dynamics
– Aerodynamics, aeroacoustics
– Industrial design
TeideHPC
● Astrophysics– Solar composition
– Cosmological Parameters Estimation
– Analysis of solar spectro-polarimetric observations
● Computer sciences● Streaming services● Render ● Cloud computing
OpenNebula
● Implementation● Study cases
– Infiniband virtualization
– Interoperability
● Use cases– Multidatastore (lbox)
– Remote visualization
– Render farm
OpenNebula Implementation
● API● Cobbler● Chef● NetApp Storage● Monitoring ● Repository
OpenNebula Implementation
High Performance Computing Cloud computing
● Ethernet network configuration● Infiniband configuration● User auth & permissions
API
OpenNebula Implementation
● API● Cobbler● Chef● NetApp Storage● Monitoring● Repository
OpenNebula Implementation
● API● Cobbler● Chef● NetApp Storage● Monitoring● Repository
OpenNebula Implementation
● API● Cobbler● Chef● NetApp Storage● Monitoring● Repository
OpenNebula Implementation
● API● Cobbler● Chef● NetApp Storage● Monitoring● Repository
Study case – Infiniband virtualization
Study case – Infiniband virtualization
● OpenNebula Ecosystem● KVM SR-IOV Driver
– Single root I/O Virtualization
Virtual functions
PCIe device
Study case – Infiniband virtualization
● OpenNebula Ecosystem● KVM SR-IOV Driver
– Single root I/O Virtualization
Virtual functions
PCIe device
# lspcib0:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]b0:00.1 Network controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]b0:00.2 Network controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
● Enable option in BIOS● Enable IO virtualization in kernel boot line● Define number of Virtual Functions in kernel module load● Burn SR-IOV capable firmware in the HCA (OEM)● Upgrade OFED version (2.3-2.0.5-rhel6.6)
Study case – Infiniband virtualization
● OpenNebula Ecosystem● KVM SR-IOV Driver● Great A'Tuin
– GPU devices support was funded by SURFsaraOpenNebula 4.14 Release notes... Support for GPU consumables, giving the ability to give exlcusive PCI passthrough access to VMs to GPU cards, for HPC computing.
Study case – Interoperability limitations
Study case – Interoperability limitations
● 2 research centers– Share resources
● Conectivity● Federation● Cloud bursting
Study case – Interoperability limitations
● 2 research centers– Share resources
● Conectivity● Federation● Cloud bursting
● End-to-end connection
– L2-L3
– Jumbo frames
● 1 lambda - 10 Gbps
● Ping < 40 ms
● VPN IPsec
● +600Mbps between 1Gbps hosts
Study case – Interoperability limitations
● 2 research centers– Share resources
● Conectivity● Federation● Cloud bursting
Tightly coupled
Study case – Interoperability limitations
● 2 research centers– Share resources
● Conectivity● Federation● Cloud bursting
Study case – Interoperability limitations
● 2 research centers– Share resources
● Conectivity● Federation● Cloud bursting
OpenNebula 4.8 docs... “The remote provider could be a commercial Cloud service, such as Amazon EC2, IBM SoftLayer or Microsoft Azure, or a partner infrastructure running a different OpenNebula instance”
Use case – Multiple datastore
Use case – Multiple datastore
● Host load balancing required
● Migrate / live migrate● Time of migration (live)
– VM RAM size
– Host bandwith (IB)
– Network
Use case – Multiple datastore
● Storage overloaded● Delete-recreate & Deploy to new
datastore● Take image snapshot (same DS)
& create template & deploy– VM HDD size
● Manual migration: – Stop VM + copy image to new DS
+ delete-recreate + deploy + replace image
Use case – Remote visualization
● Lack of rendering hardware● Low performance with ssh -X (even with
compression)● Spice
– Audio+video
– Low BW requirements
● QXL driver (variable desktop resolution)– 1024 x 768 … 2560 x 1600
Use case – Remote visualization
Use case – Remote visualization
Capture the Flag
● 404 VMs● Connected to client private
infrastructure through IPsec tunnel
● 20,8 TB HDD● 12,8 TB RAM● 12800 cores● 100 KW
References
● http://blog.scottlowe.org/2009/12/02/what-is-sr-iov/
● http://docs.opennebula.org/4.12/advanced_administration/data_center_federation/introf.html
● http://docs.opennebula.org/4.14/release_notes/release_notes/whats_new.html#opennebula-core