cern & huawei collaboration to improve openstack for running large scale scientific workloads
TRANSCRIPT
1
Huawei : Clouds at Scaleopenlab open day
21st September 2017
Tim Bell
2
Project Context
• OpenStack provides software for private and public clouds• CERN and Huawei are contributors to the open source project• Huawei is a platinum member of the OpenStack foundation
Tim Bell
3
Project Context
• Over 90% of CERN’s compute resources are delivered using OpenStack• >280,000 cores across two data centres in Geneva and Budapest
Tim Bell
4
Project Context
• With the needs of LHC computing in future years, efficient and flexible delivery of compute resources will be key• Computing needs in 2023 estimated at 60x the current capacity (HL-LHC)
Tim Bell
0
500000
1000000
1500000
2000000
2500000
3000000
3500000
4000000
4500000
5000000
2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
WLCG%CPU%Growth
Tier2
Tier1
CERN
20%%Growth
Current%state
2008<12%linear
0"
20"
40"
60"
80"
100"
120"
140"
160"
Run"1" Run"2" Run"3" Run"4"
GRID"
ATLAS"
CMS"
LHCb"
ALICE"
Compute: Growth > x60
What we think is affordable unless we do something differently
5
Project Context
• CERN has been using the T-Systems Open Telekom Cloud for public cloud evaluations• OTC is based on Huawei’s FusionSphere• Tests using 1,000 VMs demonstrated all 4 experiment workloads running successfully• Further investigations are now underway within the Helix Nebula project
Tim Bell
6
Project Description
• CERN and Huawei will work jointly on improvements to OpenStack for running large scale scientific workloads• Fellows starting in Autumn 2017
• The developments will be done within the OpenStack community following the standard open source processes
• Focus areas are• Flexible resource management• Quotas• Bare metal allocation• Compute cells
• Changes will then be included into the CERN private cloud and Huawei’s private and public cloud offerings
Tim Bell
8
Project Context
• OpenStack provides software for private and public clouds• CERN and Huawei are contributors to the open source project• Huawei is a platinum member of the OpenStack foundation
• Over 90% of CERN’s compute resources are delivered using OpenStack• >280,000 cores across two data centres in Geneva and Budapest
• With the needs of LHC computing in future years, efficient and flexible delivery of compute resources will be key• Computing needs in 2023 estimated at 60x the current capacity (HL-LHC)
• CERN has been using the T-Systems Open Telekom Cloud for public cloud evaluations• OTC is based on Huawei’s FusionSphere• Tests using 1,000 VMs demonstrated all 4 experiment workloads running successfully
Tim Bell