1

Maintaining the momentum of OpenSearch in Earth Science data discovery

Doug Newman (NASA ECHO) & Dr Chris Lynnes (GES DISC)12/11/13 10:50am PT

IN32A-03

2

What is OpenSearch?

3

A ‘refresher’

From www.opensearch.org ‘OpenSearch is a collection of simple formats for the

sharing of search results’

The Earth Data discovery use case:

HTTP GET Request

keyword = air temperature

bounding_box = 39.1 -96.6 39.1 -96.6

start_date = 2013-11-13T00:00:00Z

HTTP Response

ID

Spatial Extent

Temporal Extent

Metadata Link

Data Link

Search Link

ID

Spatial Extent

Temporal Extent

Metadata Link

Data Link

Search Link

OpenSearch Descriptor Document

4

Earth Data OpenSearch today

5

Is it successful?

• ESA’s ‘Next Generation User Services for Earth Observation’ will be using OpenSearch as an infrastructure standard*

• CEOS’s CWIC effort will support OpenSearch in a future iteration that will include ESA data providers**

• ESIP Federation continues to champion OpenSearch for earth science data discovery***

• NASA ECHO metrics: average number of queries per week– SOAP API (2011) – 10k (243k queries that year)– REST APIs (2013) – 87 of 115k (5 million queries so far this year)

* http://eomag.eu/articles/584/eo-user-service-next-generation-project-eo-usng** http://www.ceos.org/index.php?option=com_content&view=category&layout=blog&id=348&Itemid=482*** http://wiki.esipfed.org/index.php/Discovery_OpenSearch_Services

6

Why is it successful?

• Lightweight and simple• Standards-based• RESTful• Low entry cost

• ‘Free text + spatial + temporal’ satisfies 90% of Earth Data discovery use cases*

* Based on Reverb metrics for the last year (80,420 registered users, 700k queries so far this year (11/08/13)

7

ECHO Reverb Statistics

* Based on Reverb metrics for the last year (80,420 registered users, 700k queries so far this year (11/08/13)

Caters to 90% of Earth Data discovery use cases*

8

Maintaining success

9

Converge where possible

CEOS / CWIC

NASA / ESIP discovery cluster OGC / ESA / CNES

CNES - Centre National d'Etudes Spatiales Attribution: http://imgs.xkcd.com/comics/standards.png

10

Converge where possible (for real)

CEOS / CWIC

NASA / ESIP discovery cluster OGC / ESA / CNES

CNES - Centre National d'Etudes Spatiales

free_textbounding_box

start_dateend_date

uidplace_name

geometry

described_by

via

relation

11

Free text + spatial + temporal = success

• Pro: 90% !!!*• Con: lack of free text precision compared with

controlled vocabularies• Can free text solve this?**

• free text = ‘MODIS’ (693 hits) != instrument = ‘MODIS’ (543 hits)

• free text = ‘ozone’ (348 hits) != science keyword = ‘ozone’ (81 hits)

*Based on Reverb metrics for the last year (80,420 registered users, 700k queries so far this year (11/08/13)** Based on ECHO Catalog REST API queries and ‘fuzzy’ comparisons for ‘ozone’

12

Understanding the API

OpenSearch parameter extension* is good (once we update it)– Defining expectations of ‘free text’ search. What

does ‘air temperature’ mean?– Defining subset of ‘geometry’ capabilities

* http://www.opensearch.org/Specifications/OpenSearch/Extensions/Parameter

13

Additional functionality

• Result ordering– Described in OSDD and implemented in results

• Result ranking– As per OpenSearch ‘Relevance’ extension*– For free text search results– ECHO 10.71 (January 2013)

* http://www.opensearch.org/Specifications/OpenSearch/Extensions/Relevance/1.0

14

How do we achieve these goals?

CEOS

NASA / ESIP discovery cluster OGC / ESA / CNES

Doug NewmanChris Lynnes

Jérôme Gaspari*Yves Coene**

* CNES - Centre National d'Etudes Spatiales ** ESA

15

Questions?