the perspective of virtual observatories supporting space weather research and operations

ESWW5 – S5, November 20, 2008.

The Perspective of Virtual Observatories Supporting Space

Weather Researchand Operations

Peter FoxTetherless World Constellation

Rensselaer Polytechnic Institute

USA


2

BackgroundScientists should be able to access a global, distributed

knowledge base of scientific data that:• appears to be integrated• appears to be locally available

But… data is obtained by multiple means (models and instruments), using various protocols, in differing vocabularies, using (sometimes unstated) assumptions, with inconsistent (or non-existent) meta-data. It may be inconsistent, incomplete, evolving, and distributed

And… there exist(ed) significant levels of semantic heterogeneity, large-scale data, complex data types, legacy systems, inflexible and unsustainable implementation technology


Diversity, Integration, Size, …

• Not just large (well organized, long-lived, well-funded) projects/ programs want to make their data available, individuals as well

• Data policies: highly variable or non-existent; affects users

• How can data be managed to solve challenging scientific, application or societal problems without the continued need for a scientist or other end user to know every detail of complex data management systems?


Diversity, Integration, Size, … • Scientific data repositories:

– Most data still created in a manner to simplify generation, not access or use

– Very diverse organization of data; files, directories, metadata, emails, etc.

– Source/origin management is driven by meta-mechanisms for integration and interoperability (but still need performance)

• Virtual Observatories• Data Grids

• Increasing realization: need management for all forms of ‘data’, I.e. virtual data products are becoming the norm

• Size matters; personal data management is as big, or a bigger problem as source data management


5

Virtual Observatories

• Conceptual examples: • In-situ: Virtual measurements

– Related measurements

• Remote sensing: Virtual, integrative measurements– Data integration

• Managing virtual data products/ sets


Virtual Observatories

Make data and tools quickly and easily accessible to a wide audience.

Operationally, virtual observatories need to find the right balance of data/model holdings, portals and client software that researchers can use without effort or interference as if all the materials were available on his/her local computer using the user’s preferred language: i.e. appear to be local and integrated


But .. Data has Lots of Audiences

From “Why EPO?”, a NASA internalreport on science education, 2005

More Strategic

Less Strategic

InformationInformation products

Scientists too!


Data is a critical component for understanding how science works and broader applications. With it, we can:

o Design and conduct scientific investigationso Make decisionso Understand the quality of data and the role of uncertainty in resultso Focus on quantitative analysis and reasoningo Explore tools for visual representation

Virtual Observatories provide new mechanisms for collecting, manipulating, and aggregating data, information and information products. They also provide the opportunity for new kinds of student and non-expert experiences.

Data is Important in non-expert situations

(e.g. Space Weather Operations)!!


What is a Non-Specialist Use Case?

E.g. Teacher accesses internet goes to An Educational Virtual Observatory and enters a search for “Aurora”.

Someone should be able to query a virtual observatory without having specialist knowledge


Teacher receives four groupings of search results:

1) Educational materials: http://www.meted.ucar.edu/topics_spacewx.php and http://www.meted.ucar.edu/hao/aurora/

2) Research, data and tools mediated for them via VOs, knows to search for brightness, or green/red line emission

3) Did you know?: Aurora is a phenomena of the upper terrestrial atmosphere (ionosphere) also known as Northern Lights

4) Did you mean?: Aurora Borealis or Aurora

Australis, etc.

What should the User Receive?


Shifting the Burden from the Userto the Provider (with the help of VxOs)

ESWW5 – S5, November 20, 2008.14


Early days of VxOs - alas there shall be more than one!

… … … …

VO1

VO2 VO3

DB2 DB3DBn

DB1

?


16

The Astronomy approach; data-types as a service

… … … …

VO App1

VO App2VO App3

DB2 DB3DBn

DB1

VOTable

Simple Image

Access Protocol

Simple Spectrum

Access Protocol

Simple Time Access

Protocol

VO layer

Limited interoperability

Lightweight semantics

Limited meaning, hard coded

Limited extensibility

Under review

Open Geospatial Consortium:

Web {Feature, Coverage, Mapping} Service

Sensor Web Enablement:

Sensor {Observation, Planning, Analysis} Service

use the same approach


… … … …

VO Portal

Web Serv.

VO API

DB2 DB3DBn

DB1

Semantic mediation layer - VSTO - low level

Semantic mediation layer - mid-upper-level

Education, clearinghouses, other services, disciplines, etc.

Metadata, schema, data

Query, access and use of data

Semantic query, hypothesis and inference

Semantic interoperability

Added value

Added value

Added value

Added value

Mediation Layer• Ontology - capturing concepts of Parameters,

Instruments, Date/Time, Data Product (and associated classes, properties) and Service Classes

• Maps queries to underlying data• Generates access requests for metadata, data• Allows queries, reasoning, analysis, new

hypothesis generation, testing, explanation, etc.


Application: Synthesizing the solar spectrum


19

Inferred plot type and return required axes data

Ability to quickly plot data to assess suitability, quality, and produce a quick copy with some customization for a preliminary study.

Graphics also require data management.

Numerous VOs in this community


• Scaling to large numbers of data providers• Building and delivering data products that are in demand

(pre-computed and on-the-fly), covering the spectrum of time latencies

• Crossing disciplines and beyond science use• Data quality (propagating and explaining)• Branding and attribution (where did this data come from and

who gets the credit, is it the correct version, is this an authoritative source?)

• Provenance/derivation (propagating key information as it passes through a variety of services, processing algorithms, …)

• Security, access to resources, policy enforcement• Interoperability at a variety of levels (~3)

Developments for Virtual Observatories


Summary/ Discussion

• The VO paradigm in is wide-spread use in Earth and Space Sciences and is increasingly able to respond to end use communities and add value to data repositories

• There is an active community; meeting, publishing, developing, implementing, i.e. they are organized and many are collaborating

• Standards and practices are being developed, and leveraged from other sources (IVoA, SPASE, …)

• Successful implementations are in production and use (some even have evaluations)

• New science and applications are being enabled and performed


http://www.voig.net/ ([email protected])

VOiG 2009, Spring, Locations TBD


25

• Who (person or program) added the comments to the science data file for the best vignetted, rectangular polarization brightness image from January, 26, 2005 1849:09UT taken by the ACOS Mark IV polarimeter?

• What was the cloud cover and atmospheric seeing conditions during the local morning of January 26, 2005 at MLSO?

• Find all good images on March 21, 2008.• Why are the quick look images from March 21,

2008, 1900UT missing?• Why does this image look bad?

Use cases

the perspective of virtual observatories supporting space weather research and operations

Documents