sesip_0715_dn1 opensearch standards esip summer meeting 2015 doug newman (nasa earthdata –...

SESIP_0715_DN1

OpenSearch Standards

ESIP Summer Meeting 2015

Doug Newman (NASA Earthdata – Raytheon)Andrew Mitchell (NASA Earthdata)

This work was supported by NASA/GSFC under Raytheon Co. contract number

NNG10HP02C

2SESIP_0715_DN1

Why standards are important

‘Would you like to help us fight global warming?’

Charity ‘volunteer’, 06-05-15 5:30pm ET, 77th & Lexington, NYC

• Make data discovery easier!• Make data aggregation easier!• ‘Science the s**t out of it’ – Mark Watney, The Martian

• Global warming defeated!

3SESIP_0715_DN1

What is OpenSearch?

From www.opensearch.org

‘OpenSearch is a collection of simple formats for the sharing of search results’

4SESIP_0715_DN1

Why we chose it

• Lightweight and simple HTTP query in, ATOM feed out, small query palette

• Standards-based HTTP query in, ATOM feed out

• RESTful HTTP, HATEOAS

• Low entry cost Use your browser!

5SESIP_0715_DN1

Why we messed with itWe see the value of this standard and think we can apply it to our problem domain…

…If we just tweak this and add that

• Spatial• Temporal• Two step• Etc.

SESIP_0715_DN1

WHAT WE HAVE TRIED TO DO

7SESIP_0715_DN1

The ‘dream’

• All Earth data metadata repositories have an OpenSearch API

• Those APIs conform to a standard• I can apply the same search across

multiple repositories• I can compare the results (apples to

apples)• I can aggregate the results

8SESIP_0715_DN1

Steps taken to achieve this

Determine our need Determine minimal set of requirements

from that need Get it working (CWICSmart + IDN +

CWIC) Iterate on the above Combine where possible (ESIP and CWIC) Advocate with bold, decisive (and

sometimes unilateral) action

9SESIP_0715_DN1

Who is involved

• ESIP• OGC• CEOS

CEOS

ESIP OGC

SESIP_0715_DN1

WHAT WE HAVE FAILED TO DO

11SESIP_0715_DN1

The reality

• No convergence of standards• We have, somewhat ironically, added

one• We all have agendas• Sometimes they coincide, sometimes

they don’t• We don’t like to compromise• Being forceful, charismatic and British

only gets you so far in this world

12SESIP_0715_DN1

Despite our best intentions

Attribution: http://imgs.xkcd.com/comics/standards.png

OPENSEARCH

Comic sans courtesy of ‘Watchmen’ and ‘Dark Knight Returns’

13SESIP_0715_DN1

Sticking points

• Link relations• Mandatory unique identifier clause• Start index versus start page• Succinct versus verbose

14SESIP_0715_DN1

We have this

OpenSearch Specification 1.1 draft 5

OpenSearch Geo extension OpenSearch Time extension

OpenSearch Relevancy extension OpenSearch Parameter extension

CEOS Best PracticesESIP Best Practices

CEOS Developers Guide

OGC 10-032r8 – Geo and Time ext. OGC 13-026r5 EO products ext.

15SESIP_0715_DN1

OpenSearch compliance

16SESIP_0715_DN1

What we learnt (already knew)• Standards are fine if they are your

standards• Legacy implementations trump good

ideas• Basing your standard on a

specification that no one curates can bite you

• The only one following your standard is you

17SESIP_0715_DN1

How do we fix this?

• Demonstrate the value of the ‘common’ standard*

• Provide tools to enable compliance. For example, validation tools**

• Lobby – unfortunately this has a political dimension

• Evangelize – what I’m doing right now!

* Today 5pm

** Thursday 3:30pm

18SESIP_0715_DN1

GLOBAL OPENSEARCH

5pm today in the Acacia Room

YOU WILL LOVE IT

19SESIP_0715_DN1

Synopsis

• Standards are HARD• There can be an unhealthy intersect

between politics and engineering that needs to be handled

• Getting this working on a ‘federation’ level was not easy but on a ‘global’ level it is really difficult

• My solution? Keep pushing. Never give up, never surrender. But yield some ground if you can afford to occasionally

SESIP_0715_DN1

QUESTIONS

21SESIP_0715_DN1

This work was supported by NASA/GSFC under Raytheon Co. contract number NNG10HP02C