nansencloud – framework for synergistic remote sensing in the nansen group anton korosov, morten...

NansenCloud – framework for synergistic remote sensing in the

Nansen GroupAnton Korosov, Morten W. Hansen

Nansen Environmental and Remote Sensing Center

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Challenges with remote sensing data usage

1. «Complex data» challenge: Complicate to open and process a dataset

2. «Big data» challenge: Expensive to download, store and process

many dataset

3. «Heterogeneous data»: Difficult to find a relevant dataset

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Challenges with remote sensing data usage

1. «Complex data» challenge: Complicate to open and process a dataset

2. «Big data» challenge: Expensive to download, store and process

many dataset

3. «Heterogeneous data»: Difficult to find a relevant dataset

× 5Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Challenge #1How to open and process a dataset ?

Scientist friendly Python toolbox for processing 2D satellite earth observation data.

The main goal is to facilitate:

•easy development and testing of scientific algorithms,•easy analysis of geospatial data, and•efficient operational processing.

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

https://github.com/nansencenter/nansat

Nansat structure

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

G D A L

Re a de rs

M a p p e r sG iv e s m e a ning to the da ta by a dding us e m e ta da ta

E X P O R TAdds d is c ov e ry a nd us e m e ta da ta for d is s e m ina tion to the us e r c om m unity•Fig ure s•Ne tC D F•G e oTiff

S C IE N C ED e v e lopm e nt a nd te s t of a lg orithm s•Wind D opR IM‧•Ic e ‧C hlorophy ll •C urre nt

C O M M O N O P E R AT IO N S

•Re -proje c ti on•Tra ns e c t a na ly s is•S ubs e t ex tra c ti on …

E N V IS AT•Me ris•AS AR …

T E R R A & A Q U A

•Modis …O t h e r s o u r c e s

•NC E P•H irla m•D rifte rs …

O P e N D A P

NANSAT

Distribution of results (NORMAP)

Example of Nansat application

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Details of Nansat application

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

1. Open MODIS dataset2. Process MODIS data with a SSS algorithm3. Export SSS data as netCDF file

4. Open netCDF local file with SSS5. Access online surface currents thought OpenDAP6. Collocate SSS and U/V7. Generate a nice-looking map:

Raster and vector layersColorbarMasksContinentsLongitude/latitude notation

Ocean Salinity Science 2014, 26–28, November 2014, Exeter, UK

Original MODIS SSS and total surface current

Morphed MODIS SSS and total surface current

Ocean Salinity Science 2014, 26–28, November 2014, Exeter, UK

Ocean Salinity Science 2014, 26–28, November 2014, Exeter, UK

Morphed MODIS SSS and total surface current

Ocean Salinity Science 2014, 26–28, November 2014, Exeter, UK

Morphed MODIS SSS and total surface current

Morphed MODIS SSS and total surface current

Ocean Salinity Science 2014, 26–28, November 2014, Exeter, UK

Morphed MODIS SSS and total surface current

Ocean Salinity Science 2014, 26–28, November 2014, Exeter, UK

Morphed MODIS SSS and total surface current

Ocean Salinity Science 2014, 26–28, November 2014, Exeter, UK

Morphed MODIS SSS and total surface current

And here comes animation...

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Where Nansat is used

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

• NERSC, Bergen, Norway NORMAP Aquamar MAIRES MyOcean GLOBCURRENT COCONET OC CCI (SI CCI) OVL

• NIERSC, Saint-Petersburg, Russia MONARCH-A MTRI NtsOMZ EURUCAS SWARP

• NERSCI, Kochin, India INDOMARECLIM

• Nansen-Tutu, Johannesburg, SAR• METNO, Bergen and Oslo, Norway

OpenWind• ODL, Brest, France

OVL

Challenge #2. Big data

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

RS dataprovider files

CPU

USER

files

CPU

USERfiles

CPU

USER

Challenge #2. Big dataSolution: Store and process data in one

place

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

RS dataprovider

files

CPU

USER

USER

USER

Challenge #3: Heterogeneous data

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Files

CPU

User

Web site

Operationalprocessing

Challenge #3: User requires access

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Files

CPU

User

Web site

Operationalprocessing

Challenge #3: Web site requires access

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Files

CPU

User

Web site

Operationalprocessing

Challenge #3: Processing chain requires access

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Files

CPU

User

Web site

Operationalprocessing

Challenge #3. Solution: Unified interface

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Files

CPU

User

Web site

Operationalprocessing

Database

D J A N G O

Challenge #3: Interaction between centers

Scientific Council Meeting at the Nansen Center, 28 Nov 2014

Thank you!

Scientific Council Meeting at the Nansen Center, 28 Nov 2014