October 2010

CCSDS SM+C / INFORMATION ARCHITECTURE DISCUSSION

END-TO-END ARCHITECTURE

2

Data Acquisitionand Command Mission

OperationsInstrument /Sensor Operations

ScienceData

Archive

ScienceData

Processing

Data Analysis and

Modeling

Science Information Package

Science Team

Relay Satellite

Spacecraft / lander

Primitive Information Object

Primitive Information Object

Simple Information Object

Telemetry Information Package

Science Information Package

Instrument Planning

Information Object

Science Information Package

Science Products - Information Objects

PlanningInformation

Object

Science Information Package

Example: The Information Service Architecture-Related Needs in Science

• Generation of science data product from instrument telemetry data– Data processing which may have dependencies on external ancillary

data to create “high order” data products– Directed workflows to manage dependent processes– A science data product is a specialization of an “information object” that

contains metadata + data

• Capture and management of science data products– Registration of metadata for data products within a science data system

based on a domain information model– Storage of data products within a repository

• Access and distribution of science data products– Discovery and retrieval of science data products (including

transformation)

• Long-term preservation of the data

• Highly distributed deployment patterns

3

Definition of Information Service Architecture

• CCSDS Information Service Architecture – an architectural pattern that decouples common information services, models

and middleware from applications to improve reuse and interoperability between applications

– Prescribes:• Common information services and components • Common meta models• Common information model• Distributed middleware

– promotes several goals for the composition of information systems based on CCSDS standards including

• Constructing systems through well-defined standard interfaces• Enabling agency autonomy in their implementations• Supporting federation of space data system services across agencies• Supporting service discovery between organizations and agencies• Enabling loose coupling of services and their implementations

– coordination on several fronts • Shared messaging infrastructures• Common data definitions of the contents of messages• Common services behaviors• Common service interfaces and methods• Information services to support service registration and discovery• Explicit methods for secure access of services and exchange of information

4

Mapping to a CCSDS Reference Architecture

• With an Information Architecture, we consider three viewpoints to be critical

– Service• These are the services and their interfaces necessary to support CCSDS

activities and may be domain specific. Functions are used to implement services.

– Functional• These are the functions necessary to implement services. Within a

distributed service architecture, a set of standards functions that are cross cutting can be useful for implemented interoperable systems (registry, repository, messaging, etc)

– Information• These are the domain and meta model standards used to implement the

information architecture. Within a distributed architecture, the objects that are exchanged between services should use well defined information objects

5

Decomposed Architecture

• Functional View– a set of standard components and infrastructure services that allow for

construction of distributed systems– component: any modular element of a software system– service: an application that is constructed from a set of components

that is deployed within a distributed system• Decomposition of information and functional elements of an

information system in a distributed environment showing interoperability at each layer

6

Relationship between services, information, etc

• Services – Allows for definition of services, particularly domain-specific services– Domain-specific services built from functions (assume components=functions

are equiv for the moment)– Layering domain-specific services on top of an information service architecture

allows for moving domain services into a distributed, information-driven architecture

– Services should promote common interfaces that share common information objects

7

Information Architecture definition

• Information Architecture: a set of data models where each model defines what is needed to describe the meaning of the data and how it is organized and structured.– a set of classes, class attributes, and class relationships.

• Each domain within CCSDS maintains and publishes their own models, but they are also related to a broader CCSDS information model– Schemas developed with CCSDS WGs should be derived from a set

of information models

8

Metadata ObjectData Object

Information Object

Has a Has a

describes

Capturing domain models for CCSDS

• A domain model, which may be governed by a meta model that prescribes it structure and organization, is used to specific the information object. The domain model includes the dictionary of data elements along with their semantic relationships which is captured in a schema, or with more specificity, in a complex model such as an ontology.

• Derivation of information objects from the domain model is critical to supporting interoperability.

• Domain model can be further described by standard metamodel

9

Decomposing the architecture

• N-tiered, service-style architecture helps to decouple systems

• Services should be built from well-defined functions

• Across CCSDS, proper layering should be achieved, with reuse of standard functions as we move from domain specific and application layered standards to common infrastructure and communications

• As such, space domain functions can be mapped to use standard infrastructure services (information management, messaging, etc) in a SOA style deployment

10

Mapping Space Domain Functions

11

Decoupling of applications/space domain functions fromservices is keyto a developing SOA stylearchitecture

The messaging layer accommodates multiple patterns and approaches for tighter and looser coupling (e.g., Request-Response, Pub-Sub)

Use of common modelsis key to interoperability and reuse

The ISA doesn’t prescribe themessaging standard. It integrates withmiddleware to supportfederation.

Messaging patterns (Pub/Sub and Request/Response)

• Information Services Connected to a Messaging Infrastructure (e.g., MAL)

• Client/server (e.g., REST, SOAP, etc) distributed architecture with information services on the backend

12

Registries

• CCSDS IA has been developing a reference model for general registries– Specific registry types inherit the properties of registries

• Core registry model based on the ebXML specification that provides an information model for a registry– Extrinsic objects allow for extensions to define the specific of an object

type (documents, science data, data dictionary, services, …)

• The purpose isn’t to dictate the structure for every extrinsic object, but provide a basis for registries and to identify the core use cases and a core set of interfaces

• However, we believe core software can be developed based on the information model and the use cases that can provide building blocks for registry services

13

14

General Features Of A Registry And Repository

15

Use Cases

16

Use Cases

• PUBLISH- This use case describes the actions necessary for a user to publish an artifact in the registry

• UPDATE- This use case describes the actions necessary to update an artifact in the registry

• APPROVE - This use case describes the actions necessary to approve an artifact in the registry

• DEPRECATE - This use case describes the actions necessary to deprecate an artifact in the registry

• DELETE - This use case describes the actions necessary to delete an artifact in the registry

• VALIDATE - This use case describes the actions necessary to validate the metadata associated with an artifact

• CATALOG - This use case describes the actions necessary to catalog a new artifact

• VERSION - This use case describes the actions necessary to version a new artifact

• STORE - This use case describes the actions necessary to store the artifact

• NOTIFY - This use case describes the actions necessary to subscribe to registry events

• DISCOVER - This use case describes the actions necessary to discover registered artifacts

• RETRIEVE - This use case describes

17

Information Model

The ebXML (Electronic Business using eXtensible Markup Language) Reference Information Model (RIM) defines a unified information model for describing registry contents.

– Key Features• Registry schema is static - Implement once and use

everywhere• Every registry object is an identifiable with both a

system controlled unique immutable identifier and a user controlled logical identifier and version.

CCSDS Registry Conceptual Map

18

19

Intrinsic Objects

• An Intrinsic Object is a type of registry object that catalogues content whose type is specified and known.

• Intrinsic objects are defined in the registry schema– Service– Registry– Association– Subscription– Federation

20

Extrinsic Objects

• An Extrinsic Object is a type of registry object that catalogues content whose type is unspecified or unknown.

• Extrinsic Objects provide metadata that describes submitted content whose type is not intrinsically known to the Registry and therefore must be described by means of additional attributes. (e.g. SLOTS)

• Since the registry can contain arbitrary content without intrinsic knowledge about that content, Extrinsic Objects require special metadata attributes to provide some knowledge about the object (e.g., mime type).

21

API

• The JAXR specification defines a general-purpose API, allowing any JAXR client to access and interoperate with any registry accessible via a JAXR provider.

• Regardless of the registry provider, applications use common APIs and a common information model.

• The high-level of skill and patience required using JAXR suggests that a façade registry interface be defined that implements the registry use cases.

• The Façade interface API is the primary subject of the CCSDS registry recommendation.

22

Façade Interface

• PublishObject• VersionObject• StoreObject• CatalogObject• ValidateObject• UpdateObject• ApproveObject• DeprecateObject• DeleteObject• Notify• FindObject• FindAllAssociatedObject

• GetObject• GetObjectAssociatedObjects

• AddAssociation• AssociateObjects• DeleteAssociation• DeleteAssociationBetween

Object• PublishRegistry• CreateFederation• JoinFederation• LeaveFederation

23

Services

• The Service classes support the registration of service descriptions.

• The service information model is flexible and supports the registration of web services as well as other types of services.– Service– ServiceBinding– SpecificationLink

24

Slots

• The Slot Class provides a way to add custom attributes to any RegistryObject instance.

– The Slot instance has a Slot name which holds the attribute name and MUST be unique within the set of Slot names in that RegistryObject. The Slot instance also has a ValueList that is a collection of one or more string values.

25

Classification

• Any RegistryObject may be classified using any number of Classification instance. – A Classification instance references an instance of a

ClassificationNode as defined in the RIM. – The ClassificationNode represents a value within the

ClassificationScheme.– The ClassificationScheme represents the classification

taxonomy.

26

Association

• Any RegistryObject MAY be associated with any other RegistryObject using an Association instance where one object is the sourceObject and the other is the targetObject of the Association instance.– An Association instance MAY have an associationType

which defines the nature of the association.– There are a number of predefined Association Types

that a registry must support to be [ebRIM] compliant.

27

Subscription Events

• The Event classes support the registry Event Notification feature. These classes include AuditableEvent, Subscription, Selector and Action. They constitute the foundation of the Event Notification information model.– Auditable Event– Subscription– AdhocQuery– QueryExpression– Action

• NotifyAction– Notification

28

Federated Reference Model

• A federated registry provides services for sharing content and metadata between cooperating registries in a federated environment– Allows cooperating registries to be federated together

to appear and act as a single virtual registry/repository within the federated model.

– Federated Capabilities:• Query• Linking• Replication• Relocation• Notification

Software Development/Prototype Activities

• Information Infrastructure services are being prototyped for several earth and planetary activities– NASA Planetary Data System (PDS) is developing a generic

implementation following the Registry Reference Model.

• Several other activities are sitting on an open source information infrastructure (Object Oriented Data Technology - “OODT”) activity developed and released to Apache for use– http://incubator.apache.org/projects/oodt.html

• Several information model development activities to develop domain and infrastructure level models– Generally using Protégé as a modeling environment

29

Dan Crichton, Steve Hughes, Sean Hardman

October 2010

Planetary Data System Example

NASA PLANETARY DATA SYSTEM

31

PDS is composed of discipline and service nodes

Program Executive – Bill KnopfProgram Scientist – Ed GrayzeckProgram Manager – Tom Morgan/GSFC In 2005 NASA separated the Central node into a management office (GSFC) and an Engineering node (JPL).

Internally the Management Council – with representation from each node- deals with PDS issues and planned revisions – has monthly telecons and 3 annual meetings – chaired by Tom Morgan & Reta Beebe, who serves as Chief Scientist and Coordinator.

Mission: The mission of the Planetary Data System is to facilitate achievement of NASA’s planetary science goals by efficiently collecting, archiving, and making accessible digital data and documentation produced by or relevant to NASA’s planetary missions, research programs, and data analysis programs

PDS 2010 DEVELOPMENT

• 3-Year upgrade of the PDS

• Explicit Information Model– Information Model-based development to capture the domain– Mechanism in place to go from model to standards documents– Capture both domain and generic definition of objects (products,

services, registries)

• SOA-based Approach– Focusing on REST-based interfaces to support federation between US

and international systems

• Open Source Approach– Software distribution will be done on international scale

32

PDS DATA ARCHITECTURE & STANDARDS APPROACH

• The information model defines the things in the domain and their relationships.

• A data dictionary defines data elements.

• The report writer uses the model and data dictionary to export and translate the information model into various notations and languages.

• Updates to the ontology are reflected in the artifacts.

33

PDS DATA ARCHITECTURE

34

Tagged Data Object (Information Object)

Label Schema

Used to Create

Describes

Extracted/Specialized

InformationModel

Data Object

Data Element

Class

has

Planetary ScienceData Dictionary

Expressed As

Product

Validates

PDS INFORMATION MODEL (v4.x) DEVELOPMENT

35

Data Product Concept Map - 2009

Web & Registry View

Basic I/O View

Programmer View

User/Designers View

INFORMATION-DRIVEN ARCHITECTUREFOR PDS

36

Information Model

Derived Schemas

Information Object

Registry

Repository

Search Index

Application

Data MetaData

Registry Domain Model (Data, Services, Dictionary, …)

SearchRequest/Result

Extract

Info Object StructureExtract

Defines

MetadataRegistered

Info ObjectStored

Describes

RegistryRequests

PDS SYSTEM LAYERING

37

PDS SYSTEM CONTEXT

38

PDS INGESTION ARCHITECTURE

39

PDS SEARCH / DISTRIBUTION ARCHITECTURE

40

REGISTRY CONTEXT WITHIN PDS SYSTEM

41

DETAILED REGISTRY CONTEXT

42

REGISTRY USE CASES(Derived from CCSDS Reg/Rep Reference Model)

43

REGISTRY DEVELOPMENT

• Initially looked at related standards (UDDI and ebXML) and determined that ebXML better supports federated registries.

• Evaluated two software packages:– freebXML

• Open Source package no longer maintained• Would require further development and upgrade

– WellGEO RegRep from Wellfleet Software Corporation• COTS package developed by the main author of freebXML• Met requirements but not mature and exceeded budget constraints

• After these two evaluations, the team decided to implement the CCSDS Registry and Repository Reference Model.

• The intent of this service is to facilitate discovery, tracking, auditing, and maintenance of artifacts within PDS.– Artifacts include: catalog descriptions (e.g., investigation, instrument,

collection, etc.), products, schemas, services, etc.• The registry will maintain three types of registry entries:

– Metadata– Digital Object– Relationship

44

REGISTRY ENTRY TYPES

• Metadata Entry– Captures a description of a non-digital object.– Examples include missions, instruments, etc.

• Digital Object Entry– Captures a description of a digital object that is referenced by an URI.– Examples include products, product label schemas, etc.

• Relationship Entry– Captures an association between registered objects.– Associations are also typed (e.g., member of).

• Domain independent: Generic registry model that is specialized for the domain

45

46

REGISTRY ARCHITECTURE(Deployment Scenarios)

REGISTRY ARCHITECTURE(Interfaces)

47

• Provides a REST-based external interface for public access.– Can be secured via an

external service.

• Provides a Java API for accessing core functionality.

• Provides a metadata store interface for accessing different databases.

48

REST-BASED INTERFACE

• The REST-based interface exhibits the following characteristics:– A URL assigned to every resource– Formulate URLs in a predictable manner– Use HTTP methods for actions on a resource (GET, POST and

DELETE)– Leverage HTTP protocol headers and response codes where

applicable

• Goals include:– Keep the service simple and refrain from adding too much functionality– Allow messaging in the form of XML or JavaScript Object Notation

(JSON)– Allow for extensibility as new types of artifacts are defined

49

INTERFACE EXAMPLES

• This interface delegates all functions involving a product/artifact.http://pds.nasa.gov/services/registry/products/– GET: Retrieves a paged list of products from the registry.– POST: Publishes an product to the registry.

• This interface acts on a specific artifact:http://pds.nasa.gov/services/registry/products/{lid}/{version}/– GET: Retrieves an product from the registry.– POST: Updates the product in the registry with the given artifact.– DELETE: Removes the product from the registry.

GRAPHICAL USER INTERFACE

• Supports product listing, paging, sorting, filtering, and details including associations.

• The GUI utilizes the REST-based interface for accessing the Registry Service.

• Developed using Google Web Toolkit (GWT) a framework for building complex web applications.

• Leverages GWT for an advanced/stable interactive GUI without HTML/Stylesheet/Javascript expertise.

50

GUI INTERFACE – PRODUCT LISTING

51

Questions / Comments

52