echo overview and state of the system
DESCRIPTION
ECHO Overview and State of the System. ESIP Winter Meeting 2014. Katie Baynes NASA ECHO Team [email protected]. ESIP Winter 2014. ECHO System Overview. What Metadata?. NASA ’ s Earth Observing Satellites (EOS) monitor daily events and long term environmental changes - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/1.jpg)
ECHO Overview and State of the System
ESIP Winter Meeting 2014
Katie BaynesNASA ECHO [email protected]
![Page 2: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/2.jpg)
ECHO SYSTEM OVERVIEWESIP Winter 2014
![Page 3: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/3.jpg)
What Metadata?• NASA’s Earth Observing Satellites (EOS) monitor
daily events and long term environmental changes
• EOSDIS data collection includes over 3,500 data types• Land
» Cover & Usage» Surface temperature» Soil moisture» Surface topography
• Atmosphere» Winds & Precipitation» Aerosols & Clouds» Temperature & Humidity» Solar radiation
• Ocean Dynamics » Surface temperature» Surface wind fields & Heat flux» Surface topography» Ocean color
• Cryosphere» Sea/Land Ice & Snow Cover
• Human Dimensions» Population & Land Use» Human & Environmental Health» Ecosystems
![Page 4: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/4.jpg)
Reverb• ECHO’s web front end• Novices and Experts• Searching
– Temporal– Spatial– Keywords– Platforms/Instruments/Sensors– Processing Level
• Ordering• Downloads• Services
– Sub-setting– Re-projection
http://reverb.echo.nasa.gov/reverb
![Page 5: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/5.jpg)
Metadata IngestD
at
a P
ro
vid
er
In
ge
st
ECHO Catalog
REST API
FTP Ingest
ISO 19115 Data
Provider
ECHO 10 Data
Provider
ISO 19115 Data
Provider
ECHO 10 Data
ProviderFormat X
Both Collection and Granule Level
![Page 6: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/6.jpg)
Search and Metadata Retrieval
ECHO Catalog
REST API
ReverbESIP
OpenSearch
CWICGiovanni
…
DirectClients
Cli
en
t D
at
a
Re
tr
iev
al • Multi-format Retrieval
• ECHO10• JSON• ISO 19115
• Searching• Ordering
![Page 7: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/7.jpg)
OPERATIONAL STATISTICS AND INTERESTING METRICS
ESIP Winter 2014
![Page 8: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/8.jpg)
Some ECHO Stats
• Central EOSDIS Metadata Repository– Holdings
• 3830 Collections• 142 Million Granules
– Client Applications• Reverb• ASTER Volcano Group• GHRC Data Albums Access• OpenSearch Clients (CWIC, et al)• GIBS• Non-self-identifying clients (aka ‘Other’)
![Page 9: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/9.jpg)
How Much Traffic Does ECHO See?
![Page 10: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/10.jpg)
10/13 - 10/19
10/20 - 10/27
10/27 - 11/02
11/03 - 11/09
11/10 - 11/16
11/17 - 11/23
11/24 - 11/30
12/01 - 12/07
12/08 - 12/14
12/15 - 12/21
12/22 - 12/29
12/29 - 01/05
0
100000
200000
300000
400000
500000
600000
700000
800000
131738
187327167963
148929111246 116986
Total ECHO Queries
ECHO Current
GHRC/UAH Data Albums work in full swing ~650k queries
![Page 11: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/11.jpg)
Who’s Using ECHO?
![Page 12: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/12.jpg)
Reverb Ops18% GMU
0%
GMUCWICWrapper39%
SSW1%GESDISC
0%
AVAPC7%MODAPS
0%
GHRC/UAH32%
Other2%
Client Breakdown for a Week in October 2013
![Page 13: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/13.jpg)
Reverb Ops37%
GMU0%
GMUCWICWrapper42%
SSW1%
GESDISC0%
GSFCS4PA0%
AVAPC7%
MODAPS0%
GHRC/UAH2%
Other10%
Client Breakdown for a Week in December 2013
![Page 14: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/14.jpg)
How Fast is ECHO?
![Page 15: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/15.jpg)
10/13 - 10/19
10/20 - 10/27
10/27 - 11/02
11/03 - 11/09
11/10 - 11/16
11/17 - 11/23
11/24 - 11/30
12/01 - 12/07
12/08 - 12/14
12/15 - 12/21
12/22 - 12/28
12/29 - 01/05
0.000
0.500
1.000
1.500
2.000
2.500
3.000
3.500
4.000
4.500
3.251
Average Query Time (12 Week Moving Average)
Seco
nds
![Page 16: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/16.jpg)
Adding More Interesting Metrics …
![Page 17: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/17.jpg)
Composite Reverb spatial search heat map for the last year
![Page 18: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/18.jpg)
Looking at How People are Using Reverb …
![Page 19: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/19.jpg)
ECHO Continues to work on Serving the Right People with the Right Tools
![Page 20: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/20.jpg)
WHATS BEEN GOING ON?ESIP Winter 2014
![Page 21: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/21.jpg)
High Level Overview
• ECHO Technical Interchange – Spring 2013• NRT Data• Order Workflow Re-Work• Indexing Landsat 8 Data• Revisiting User Settings• ISO Activities• More!
( R o u g h l y C h r o n o l o g i c a l )
![Page 22: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/22.jpg)
ECHO Technical Interchange
http://fc04.deviantart.net/fs43/f/2009/099/8/5/CAT_PARADE_by_dolle.png
![Page 23: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/23.jpg)
LANCE NEAR REAL TIME INDEXINGESIP Winter 2014
![Page 24: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/24.jpg)
Current NRT Holdings (with GIBS!)Spacecraft/Instrument ECHO* GIBS**
Aqua/AIRS 8 Collections24960 Granules
(2)7 Imagery LayersMay 2012 -> Present
Aura/MLS 7 Collections13152 Granules
14 Imagery LayersMay 2012 -> Present
Aura/OMI Non-Operational2 Collections364 Granules
8 Imagery LayersMay 2012 -> Present
Terra/MODIS 22 Collections48415 Granules
23 Imagery LayersMay 2012 -> Present
Aqua/MODIS 21 Collections48428 Granules
21 Imagery LayersMay 2012 -> Present
* - Granules aged off after 7-10 days** - Daily images not currently aged off
![Page 25: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/25.jpg)
Exposing ECHO’s NRT Holdings
Visual Indicators Quickly Call Out NRT Collections
In Reverb
![Page 26: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/26.jpg)
Discovering NRT in ECHO
• Extending our Data Model and API– Metadata Element
– API Query Parameter
<CollectionDataType>NEAR_REAL_TIME</CollectionDataType>
https://api.echo.nasa.gov:443/catalog-rest/echo_catalog/datasets?collection_data_type=NEAR_REAL_TIME
![Page 27: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/27.jpg)
ONGOING WORKESIP Winter 2014
![Page 28: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/28.jpg)
Revisiting Ordering Workflows
![Page 29: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/29.jpg)
Indexing New Data Collections
![Page 30: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/30.jpg)
More Visualization Options …
Pre-WRS-2
![Page 31: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/31.jpg)
Revisiting User Settings
![Page 32: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/32.jpg)
ISO19115 and ECHO
• Following activities of NASA’s MENDS Study• Incorporating updated data translations• Learning more about
community use and adoption• Working with this guy• Ingest of New Products in
ISO19115/19139
![Page 33: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/33.jpg)
Questions?
http://goo.gl/IRvY7oSample Code:
![Page 34: ECHO Overview and State of the System](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815f61550346895dce49dd/html5/thumbnails/34.jpg)