cloud processing close to the earth data
TRANSCRIPT
Cloud Processing close to the Earth data
Herve Caumont Terradue Srl.
NextGEOSS KOM - Lisbon, January 18th 2017
Agenda
Foreword
1. A new workplace 2. When Earth data matters 3. Hubs and federations 4. For knowledge producers Final remarks
Foreword Applied research serves society, its funding is focused to solve current or future societal challenges To measure and understand the earth conditions, new and improved numerical models are leading the way, along with improved data assimilation pipelines. How will the Earth Observation community better support this goal ? “major technical challenge is the complexity of the data and analytics procedures across disciplines (rather than the size of the data per se)”
Realising the European Open Science Cloud EC Directorate-General for Research and Innovation
NextGEOSS is open
NextGEOSS builds heavily on openness, especially Open Data policies, to deliver the European GEO data hub.
Taking heritage of the GEO 2005-2015 achievements, promoting the use of Earth Observations in decision making, and innovating for GEOSS, a set of coordinated, independent EO, information and processing systems.
Looking at the next decade, NextGEOSS will engage with communities by promoting innovative GEOSS powered applications from Europe.
Cloud processing close to the data shall be a fast, cost effective and user friendly capability so that application developers can focus on creating added value products.
Terradue Cloud Platform supports NextGEOSS approach to Cloud
Terradue Cloud Platform
Integration layers: ▪ Connecting scientists to the Web and Cloud resources
empowering their research
▪ Tailored for Earth Sciences data integration & analysis
Partnership mindset: ▪ Dedicated entry points for different types of value adders
Since 2013, we’ve supported more than 40 companies and research institutes in accessing earth observation data & deploying processing services
Terradue Cloud Platform
2009-2014
2015-Forward
ESA G-POD
G-POD Cloud Experiment
GEO Supersites Virtual Archive
EO Thematic Platforms
GENESI-DR/DEC
GEOWOW
MELODIES
H2020 Projects
2006-2008
▪ Co-funded ESA and EC projects, in a common vision ▪ Validation of evolutions via user engagement on the platform,
with network effects carrying business opportunities
Terradue Cloud Platform
2009-2014
2015-Forward
ESA G-POD
G-POD Cloud Experiment
GEO Supersites Virtual Archive
EO Thematic Platforms
GENESI-DR/DEC
GEOWOW
MELODIES
H2020 Projects
2006-2008 Technological innovations, domain standards takeup
▪ Co-funded ESA and EC projects, in a common vision ▪ Validation of evolutions via user engagement on the platform,
with network effects carrying business opportunities
Terradue Cloud Platform
2009-2014
ESA G-POD
G-POD Cloud Experiment
GEO Supersites Virtual Archive
EO Thematic Platforms
GENESI-DR/DEC
GEOWOW
MELODIES
H2020 Projects
2006-2008 Technological innovations, domain standards takeup
2015-Forward
Learning from experiments, capacity building
▪ Co-funded ESA and EC projects, in a common vision ▪ Validation of evolutions via user engagement on the platform,
with network effects carrying business opportunities
Terradue Cloud Platform
▪ Co-funded ESA and EC projects, in a common vision ▪ Validation of evolutions via user engagement on the platform,
with network effects carrying business opportunities
2009-2014
2015-Forward
ESA G-POD
G-POD Cloud Experiment
GEO Supersites Virtual Archive
EO Thematic Platforms
GENESI-DR/DEC
GEOWOW
MELODIES
H2020 Projects
2006-2008
Operations level agreements, user engagement
Technological innovations, domain standards takeup
Learning from experiments, capacity building
Enabling ‘network effects’
What does a Platform mean nowadays ?
✓ A lever of Web and Cloud technologies
✓ A business model for value co-creation
✓ A framework to bring innovation to new or larger communities
Terradue Cloud Platform
A workplace & APIs to connect Data Providers, Developers, Integrators and information producers
Providers
Consumers
▶ Mission:
▶ Rapid prototyping and benchmarking of algorithms
▶ Seamless data access whatever stage of the process
▶ Automated data processing
▶ Scalable resource management
▶ Information conveyance
Long time series for earth science
Earth observation data is a unique source of global measurements over decades
Source: H. Laur, 2nd Workshop European Cloud Computing Strategic Plan Heidelberg, 26 Oct 2011
Source: C.Minet et al., 2012
When calibrated and combined with other sources, it empowers validation & interpolation models
Earth observation categories
Atmosphere
Cryosphere
Source: GMES
Example of societal benefits
Support the enactment of policies
such as the monitoring of greenhouse gas emissions
Example of societal benefits
Document both climate change
trends and new economic opportunities in the Arctic based on
sea ice monitoring
Connect existing activities to NextGEOSS
Earth observation categories
Geosphere
Hydrosphere
Source: GMES
Example of societal benefits
Estimate erosion rates sending
sediments and nutrients into waterways
Mitigate geohazards risks
impacting human settlements
Examples of societal benefits
Deliver advice to farmers based on weather conditions & soil moisture
Sustain ship routing efficiency
based on ocean conditions
Connect existing activities to NextGEOSS
Earth observation categories
Biosphere
Source: GMES
Examples of societal benefits
Document Essential Biodiversity Variables to improve ecosystem
benefits in face of increasing anthropogenic pressures
Monitor the state and trends of ecosystems in biogeographical
regions (e.g. mountain, arid, semi-arid, coastal and marine) for
internationally recognized protected aras and to define the needs of
future protected areas
Connect existing activities to NextGEOSS
NextGEOSS challenge:
Private sector
❏ overcome the fragmentation of the European landscape of existing public & private Earth Observation data infrastructures
❏ maximise their combined exploitation
Socio-economic open datasets
govdata.de
data.gov.uk
data.gouv.fr
catalog.data.gov
PublicData.eu
Private sector
▪ Address requirements for the development of the European digital economy
▪ Bring together multi-source assets having an open data policies
▪ Help more researchers & developers to compose data-driven applications
NextGEOSS foreseen impact:
Data & processors swarming
Data Staging from distributed repositories
Data packages capturing input needs Deliver new
products and analysis results towards community hubs
Appliance deployment to distributed hosts in the cloud
NextGEOSS Service Suite
Architectures & Innovation
▪ Web and Cloud services ▪ Standard App containers ▪ Mobile technologies ▪ Global Network operations ▪ Open Data Policies
▪ System of systems governance
"the architecture of a complex technical system fundamentally influences, and is influenced by, the economic system that drives the development, production, and use of the system"
Internet Architecture and Innovation MIT Press, Barbara van Schewick
Data access
Research and operational data infrastructures
Copernicus service infrastructures
Within the NextGEOSS consortium:
GEOSS registered resources
Copernicus scientific & public access
NextGEOSS Data Hub
Faceted search
User feedback Data mining
QoS
Data staging
OGC OpenSearch API
Data Buckets API
Enablers for Community hubs
NextGEOSS Data management
Data I/O workflow automation in a
distributed environment
Cloud bursting
NextGEOSS support to community hubs in
requesting, managing and accessing resources for
cloud bursting
Covering both the open science and commercial applications opportunities
Experience with community hubs
5
Climate Change
Carbon market
Climatology
Oceanography
Ship logistics
Land management
Water quality
Desertification
Biodiversity
Urban accounting
Volcanology
Seismic analysis
Disaster Risk Reduction
EC FP7 GEOWOW (2011-2013)
EC FP7 SenSyF (2013-2015)
EC FP7 MELODIES (2014-2016)
ESA Exploitation Platforms (2014 onwards)
8
14
35+
H2020 EcoPotential (2015-2018)
30+
Campaigns (2013 onwards)
NextGEOSS intended outreach through innovative
research Pilots & business opportunity Pilots
Atmosphere
Marine Land
Cold regions Biodiversity
Air pollution & Health Water management
Agriculture / Agronomy Food security
Teritorial planning Solar energy
Urban growth Smart cities
Secure societies Citizen observatories
Final Remarks
With NextGEOSS, state of the art
technology will match a proven
governance model, driven by the
public sector with possible
contributions from private entities,
and deliver to society a new
capacity to learn faster from
earth observations, and make
better informed decisions
References
WMO Global Atmoshere Watch Implementation Plan 2016-2023 http://www.wmo.int/pages/prog/arep/gaw/documents/IPFinalDraftMay11.pdf GSNL 2017-2019 GSNL Implementation plan http://www.earthobservations.org/documents/gsnl/201607_GSNL_Implementation_plan_2017_2019.pdf Realising the European Open Science Cloud, EC Directorate-General for Research and Innovation https://ec.europa.eu/research/openscience/pdf/realising_the_european_open_science_cloud_2016.pdf Internet Architecture and Innovation, MIT Press, Barbara van Schewick https://mitpress.mit.edu/books/internet-architecture-and-innovation