The Science Cloud Users:Challenges and Needs

ESA‐ESPI Workshop on “Space Data & Cloud Computing Infrastructures: Policies and Regulations”

7 July 2017Bob JonesCERN

Bob.Jones <at> cern.chHelix Nebula – The Science Cloud

Helix Nebula – The Science Cloud with Grant Agreement 687614 is a Pre‐Commercial Procurement Action funded by H2020 Framework Programme

Accelerating Science and InnovationAccelerating Science and Innovation

Data in High-Energy Physics

Based on DPHEP Study Group (2009). Data Preservation in High Energy Physics. http://arxiv.org/abs/0912.0255

Patricia Herterich

5EPFL & SDSC visit 2017-03-24

CERN Open Data Portal

• 2015• 40 TB of 2010 data

• 2016• 320 TB of 2011 data• Curation, release of

• Simulated data (MC) • Trigger information • Configuration files

http://github.com/cernopendata

The Worldwide LHC Computing Grid

Tier-1: permanent storage, re-processing, analysis

Tier-0 (CERN): data recording, reconstruction and distribution

Tier-2: Simulation,end-user analysis

2 million jobs/day

700 PB of storage

nearly 170 sites, 40+ countries

WLCG:An International collaboration to distribute and analyse LHC data

Integrates computer centres worldwide that provide computing and storage resource into a single infrastructure accessible by all LHC physicists 6

D. Giordano WLCG Workshop 9/10/2016

CERN cloud procurements 2015-2016

7

The Hybrid Cloud ModelBrings together• research organisations,• data providers,• publicly funded e‐

infrastructures,• commercial cloud service 

providers

In a hybrid cloud with procurement and governance approaches suitable for the dynamic cloud market In‐house

06/07/2017

A common approach

https://www.eiroforum.org/science‐policy/eiroforum‐directors‐meet‐european‐commissioner‐carlos‐moedas/Bob Jones (CERN)

EIROforum Directors meet the European Commission in Brussels. From left to right: ESRF DG, Francesco Sette, DG Research and Innovation, EC, Robert‐Jan Smits, ILL Director, Helmut Schober, CERN DG, Fabiola Gianotti, Chair of the European XFEL Management Board, Robert Feidenhans’l, Commissioner for Research and Innovation, Carlos Moedas, EUROfusion Programme Manager, Tony Donné, ESO DG and EIROforum Chair, Tim de Zeeuw, EMBL Director International Relations, Silke Schumacher and ESA DG, Jan Woerner. (Credit: Mark McCaughrean)

HNSciCloud Joint Pre‐Commercial ProcurementProcurers: CERN, CNRS, DESY, EMBL‐EBI, ESRF, IFAE, INFN, KIT, STFC, SURFSaraExperts: Trust‐IT & EGI.eu

The group of procurers have committed• Procurement funds• Manpower for testing/evaluation• Use‐cases with applications & data• In‐house IT resources

Resulting services will be made available to end‐users from many research communities

Co‐funded via H2020 Grant Agreement 687614

Total procurement budget >5M€

06/07/2017

What will be procuredA hybrid cloud platform for the European research community

7/6/2017 11

HNSciCloudPCP

Source: C

loud

Compu

tingfor G

ovies, DLT Solutions,

David Blankenh

orn, Van

 Ristau

and Caron Be

esley

Combining services at the IaaS level to support science workflows

The R&D services to be developed are to be integrated withResources in data centres operated by the buyers groupGEANT network

ChallengesInnovative IaaS level cloud services integrated with procurers in‐house resources and public e‐infrastructure to support a range of scientific workloads

Compute and Storagesupport a range of virtual machine and container configurations including HPC working with datasets in the petabyte range

Network Connectivity and Federated Identity Managementprovide high‐end network capacity via GEANT for the whole platform with common identity and access management

Service Payment Modelsexplore a range of purchasing options to determine those most appropriate for the scientific application workloads to be deployed

Bob Jones, CERN 12

The Pre‐Commercial Procurement process

06/07/2017

06/07/2017

Top 10 challenges for RIs and EOSC to work together 1. Scalability of services - catering for the needs of small research groups from public and

private sectors, as well as very large experiments2. Integration - services linked by a supported federated identity scheme covering more of the

research life cycle where users access data, sw, IT capacity and the expertise for performing analyses

3. Hybrid model - should not compete with but rather profit from ease of use and rates of innovation of commercial service providers

4. Provenance, citation and use of data & software that respects intellectual property rights5. Software licence models that allow flow of data across different infrastructures without

buying licences for each one6. Confidentiality of data that is still under embargo for publication or intellectual property

reasons7. Cyber security vulnerabilities must not compromise participating organisations8. GDPR compatibility for all services9. Adoption - Making end users aware of the services and encouraging them to use them10.A Governance model that ensures end‐users and procurers drive the decision making

processResult must be sustained via funding models that take a long-term view