Imaging Node MeetingAtlas II Status and Plans

Imaging Node MeetingAtlas II Status and Plans

August 2, 2006August 2, 2006

In the Beginning…AtlasIn the Beginning…Atlas

Original Atlas was CGI-BIN based Perl Usually one mission per application

Though code became fairly multi-mission in flavor Copy and tweak

Perl::DBI interface to various databases Decision made to make completely multi-mission and augment

with OODT software Always knew this was necessary, but other priorities usurped

development MER Archive support allowed for developer focus

Original Atlas was CGI-BIN based Perl Usually one mission per application

Though code became fairly multi-mission in flavor Copy and tweak

Perl::DBI interface to various databases Decision made to make completely multi-mission and augment

with OODT software Always knew this was necessary, but other priorities usurped

development MER Archive support allowed for developer focus

Atlas II and OODTAtlas II and OODT Atlas II takes much experience learned from the original

Atlas with the following changes: Split into distinct layers and functions (at various levels of

readiness) Web Service (use of OODT here)

Performs searches across all data sets within the system. System can be distributed. As a WS, this part is intended for machine/code use.

User Interface (aka Forms) The human useable part that utilize the others

Processing Calibration, mosaic, geometry, etc…

Retrieval (use of OODT here) Data download Shopping cart style bulk ordering

Atlas II takes much experience learned from the original Atlas with the following changes: Split into distinct layers and functions (at various levels of

readiness) Web Service (use of OODT here)

Performs searches across all data sets within the system. System can be distributed. As a WS, this part is intended for machine/code use.

User Interface (aka Forms) The human useable part that utilize the others

Processing Calibration, mosaic, geometry, etc…

Retrieval (use of OODT here) Data download Shopping cart style bulk ordering

Atlas II Web Service and OODTAtlas II Web Service and OODT

The Atlas II Web Service performs the meta-data search functions of the Atlas II system. As a distributed system it utilizes (at least) two ways to access meta-data: OODT DIS query via the OODT Profile Server SQL

Both allowed so that we can talk to most repositories in the Imaging Node domain. Some facilities not running OODT, but allow SQL Some facilities don’t allow SQL, but are willing to run OODT

Profile Servers We’re kinda hosed of neither is allowed…

The Atlas II Web Service performs the meta-data search functions of the Atlas II system. As a distributed system it utilizes (at least) two ways to access meta-data: OODT DIS query via the OODT Profile Server SQL

Both allowed so that we can talk to most repositories in the Imaging Node domain. Some facilities not running OODT, but allow SQL Some facilities don’t allow SQL, but are willing to run OODT

Profile Servers We’re kinda hosed of neither is allowed…

Atlas II Web Service and OODT (2)Atlas II Web Service and OODT (2)

Imaging node currently interacts with several database and schema configurations: MySQL (IN, UofA, probably everybody else

on the planet ;-)) Sybase (e.g. MIPL, Geo) Oracle (e.g. USGS)

Imaging node currently interacts with several database and schema configurations: MySQL (IN, UofA, probably everybody else

on the planet ;-)) Sybase (e.g. MIPL, Geo) Oracle (e.g. USGS)

Atlas II Web Service and OODT (3)Atlas II Web Service and OODT (3)

Why not just use OODT Profile searches directly? OODT searches syntax, not semantics

Is longitude East positive? Is projection planetocentric? Planetodetic? What is the equivalence of the Green filter on VGR to the

wavelength of filter 3 on GLL? What is the SCET of FDS 453245?

These are the kinds of things that Atlas II worries about. Once the user-perspective query is translated, it is shipped off the the various OODT Profile servers configured for use.

Why not just use OODT Profile searches directly? OODT searches syntax, not semantics

Is longitude East positive? Is projection planetocentric? Planetodetic? What is the equivalence of the Green filter on VGR to the

wavelength of filter 3 on GLL? What is the SCET of FDS 453245?

These are the kinds of things that Atlas II worries about. Once the user-perspective query is translated, it is shipped off the the various OODT Profile servers configured for use.

OODT Profile Server configurationOODT Profile Server configuration

So, how do you know how to query a meta-data collection? Especially one that is not local? Profile Servers have an XML configuration file that Atlas II uses (and

augments with its own spicings) Mapping of schema Searchable parameters (make sure only non-pathological items are

queried). Return-ables parameters (default, required, possible)

Atlas II knows how to use these configurations to do many things within the system: Query construction (OODT DIS or SQL) User interface creation

So, how do you know how to query a meta-data collection? Especially one that is not local? Profile Servers have an XML configuration file that Atlas II uses (and

augments with its own spicings) Mapping of schema Searchable parameters (make sure only non-pathological items are

queried). Return-ables parameters (default, required, possible)

Atlas II knows how to use these configurations to do many things within the system: Query construction (OODT DIS or SQL) User interface creation

Atlas II Retrieval and OODTAtlas II Retrieval and OODT

Downloading of data via (at least) two mechanisms: OODT Product server

Understands that a set of data may be required to fulfill order for an “EDR”. (E.g., data files and labels bundled together into a ZIP archive).

‘wget’ style URL scripting straight to a web server (Atlas II to generate script for client)

In all cases, URL accesses the data and we intend for them to be immutable.

Downloading of data via (at least) two mechanisms: OODT Product server

Understands that a set of data may be required to fulfill order for an “EDR”. (E.g., data files and labels bundled together into a ZIP archive).

‘wget’ style URL scripting straight to a web server (Atlas II to generate script for client)

In all cases, URL accesses the data and we intend for them to be immutable.

Atlas II StatusAtlas II Status

Currently we are supporting both Cassini and MER operationally and have a development version for HiRISE.

Atlas II support both mission specific and multi-mission queries.

Mission specific queries: http://pdsimg.jpl.nasa.gov/search/index.jsp?mission=cassini http://pdsimg.jpl.nasa.gov/search/index.jsp?mission=mer http://atlasweb-dev.jpl.nasa.gov/search/index.jsp?mission=mro (This is

password protected)

Currently we are supporting both Cassini and MER operationally and have a development version for HiRISE.

Atlas II support both mission specific and multi-mission queries.

Mission specific queries: http://pdsimg.jpl.nasa.gov/search/index.jsp?mission=cassini http://pdsimg.jpl.nasa.gov/search/index.jsp?mission=mer http://atlasweb-dev.jpl.nasa.gov/search/index.jsp?mission=mro (This is

password protected)

Atlas II Status (cont.)Atlas II Status (cont.)

Atlas II Status (cont.)Atlas II Status (cont.)

Atlas II Status (cont.)Atlas II Status (cont.)

Multi-mission query: The same code base is used for the multi-

mission view Common parameters of each logical grouping

(data set) is dynamically found and displayed and searched across missions.

http://pdsimg.jpl.nasa.gov/search/index.jsp?MULTI_MISSION=true

Multi-mission query: The same code base is used for the multi-

mission view Common parameters of each logical grouping

(data set) is dynamically found and displayed and searched across missions.

http://pdsimg.jpl.nasa.gov/search/index.jsp?MULTI_MISSION=true

Atlas II Status (cont.)Atlas II Status (cont.)

Atlas II ConfigurationAtlas II Configuration Profile Server and/or SQL (JDBC) access to

catalog We could mirror the meta-data here, but why?

UofA dedicated to uptime preservation.

XML Profile server configuration to be shared with IN. HiRISE drives this. This encapsulates the schema so we know the mapping of the various fields to the PDS keywords.

Profile Server and/or SQL (JDBC) access to catalog

We could mirror the meta-data here, but why? UofA dedicated to uptime preservation.

XML Profile server configuration to be shared with IN. HiRISE drives this. This encapsulates the schema so we know the mapping of the various fields to the PDS keywords.

Atlas II and OODT OverviewAtlas II and OODT Overview

Data Migration PlanData Migration Plan What is required?

Sybase to MySQL to increase performance Schema optimization Configuration file to integrate with Atlas II and OODT

Order of data migration from Atlas I web-interface to Atlas II:

THEMIS, Galileo, MPF, Voyager, Viking Orbiter, MGS, ODY (GRS), MEX, Clementine, Magellan

What is required? Sybase to MySQL to increase performance Schema optimization Configuration file to integrate with Atlas II and OODT

Order of data migration from Atlas I web-interface to Atlas II:

THEMIS, Galileo, MPF, Voyager, Viking Orbiter, MGS, ODY (GRS), MEX, Clementine, Magellan

Additional FeaturesAdditional Features HiRISE interface to JPEG2000 server

Necessary to support RDR retrieval of large files.

External processing interface. This is the ability to remotely process images, i.e. stretching, multi-format downloads for reasonably sized images.

Plan to incorporate image processing via MIPL/JAI software.

HiRISE interface to JPEG2000 server Necessary to support RDR retrieval of large

files. External processing interface. This is the ability

to remotely process images, i.e. stretching, multi-format downloads for reasonably sized images.

Plan to incorporate image processing via MIPL/JAI software.