c. huc/cnes, d. boucon/cnes-silogic, d.m. sawyer/nasa/gsfc, j.g. garrett/nasa-raytheon...
TRANSCRIPT
C. Huc/CNES, D. Boucon/CNES-SILOGIC,D.M. Sawyer/NASA/GSFC, J.G. Garrett/NASA-Raytheon
Producer-Archive Interface MethodologyProducer-Archive Interface MethodologyAbstract StandardAbstract Standard
PAIMAS
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 2
ContentsContents
Why a new standard?• Space data archiving• Needs for standardization• What standardization will bring• Abstract standard status
Proposed methodology• Context - description• The 4 phases
Specialization Aid Tool for the use of the Methodology Towards an implementation standard Conclusions
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 3
Why a new standard?Why a new standard?Space data archiving: archive typesSpace data archiving: archive types
Telemetry Operation control
Payload control
Mission exploitation
Missionarchives
Mission control system
Scientific multi-missiondata centers: long-term archives
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 4
Data Producers: • the operating systems of each mission• the mission processing and archiving centres• the scientific teams in the laboratories
Archive• specialized independent organizations (CDS)• specialized services within the agencies (NSSDC)• services common to several organizations (CDPP)
a Producer-Archive Project is a set of activities and the means used by the Information Producer as well as the Archive to ingest a given
set of information into the Archive.
Why a new standard?Why a new standard?Space data archiving: key roles and playersSpace data archiving: key roles and players
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 5
The relations between archives and data Producers are rarely simple and easy:• nonconformity of received data• unclear and imprecise definition of the data to be delivered,• failure to meet delivery schedule,• late detection of errors in archived data,• non-management of modifications
==> Can be detrimental to archived information quality and the cost of the operation.
Ever increasing diversity of the producersData complexityEach project develops its own methodology on the
basis of a process that is roughly the same from one project to another
==> Work duplicated, no generality, excessively high costs, etc.
Why a new standard?Why a new standard?Needs for standardization: problemsNeeds for standardization: problems
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 6
PAIMASGeneric abstract
standard
Communityabstract standard
PAIMASImplementation
standard
Approved by the CCSDS
Development in progress
Why a new standard?Why a new standard?Needs for standardization: multiple standardsNeeds for standardization: multiple standards
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 7
Ease some typical problems (as noted earlier) Make it easier to follow schedules and commitments Help with human resources turnover Anticipate new problems Address the legal aspects early Increase data quality of archive holdings Have weight and credibility for adoptionAnd finally, save time (and money)
Why a new standard?Why a new standard?What standardization will bringWhat standardization will bring
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 8
Today: CCSDS standard (blue book) and standardization with ISO in progress
Various communities have helped produce this document :
• space agencies (CNES, NASA, ESA, …)
• archivists (NARA, …) and
• librarians (BNF -French National Library, RLG-Research Library Group).
Why a new standard?Why a new standard?Abstract standard statusAbstract standard status
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 9
ContentsContents
Why a new standard?• Space data archiving• Needs for standardization• What standardization will bring• Abstract standard status
Proposed methodology• Context - description• The 4 phases
Specialization Aid Tool for the use of the Methodology Towards an implementation standard Conclusion
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 10
Producer
Consumer
Queries
Results sets
Orders
Submission
Information
Packages
Dissemination Information Packages
OAIS
Archival Information
Packages
MethodologyMethodologyContext: OAIS environmentContext: OAIS environment
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 11
AIP : Archival Information Package, SIP : Submission Information Package, DIP : Dissemination Information package
M A N A G E M E N T
I n g e s t
D a t aM a n a g e m e n t
S I P
A I PD I P
q u e r i e s
r e s u l t s e t sA c c e s s
PRODUCER
CONSUMER
D e s c r i p t i v eI n f o
A I P
o r d e r s
D e s c r i p t i v eI n f o
A r c h i v a lS t o r a g e
A d m i n i s t r a t i o n
P r e s e r v a t i o n P l a n n i n g
MethodologyMethodologyContext: PAIMAS focusContext: PAIMAS focus
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 12
AIP : Archival Information Package, SIP : Submission Information Package, DIP : Dissemination Information package
M A N A G E M E N T
I n g e s t
D a t aM a n a g e m e n t
S I P
A I PD I P
q u e r i e s
r e s u l t s e t sA c c e s s
PRODUCER
CONSUMER
D e s c r i p t i v eI n f o
A I P
o r d e r s
D e s c r i p t i v eI n f o
A r c h i v a lS t o r a g e
A d m i n i s t r a t i o n
P r e s e r v a t i o n P l a n n i n g
Ingest
MethodologyMethodologyContext: PAIMAS focusContext: PAIMAS focus
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 13
The archive project is broken down into 4 main phases:
• Preliminary Phase,
• Formal Definition Phase,
• Transfer Phase,
• Validation Phase.
Each phase has extensive action tables.
Specification for a community.
MethodologyMethodologyDescriptionDescription
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 14
Data ready to archive
MethodologyMethodologyThe phases: relationshipsThe phases: relationships
Preliminary Agreement
DictionaryFormal modelSubmission Agreement
Transferred object files
Validation agreement
Ph
ase
obje
ctiv
e
Preliminary Phase
Formal Definition Phase
Transfer Phase
ValidationPhase
Anomalies
Validate
the
transfe
rred
objects
• Iden
tifica
tion a
nd pre
limin
ary
agre
emen
t
Defi
ne th
e in
form
atio
n
to
be
arch
ived
•reso
urces
est
imat
ion
•Neg
otia
te th
e Su
bmiss
ion
Dev
elop
agre
emen
t (dat
a to
be
deliv
ered
, com
plem
enta
ry
ele
men
ts, s
ched
ule)
•Act
ual t
rans
fer o
f the
dat
a
Act
ual tr
ansf
er o
f the
ob
ject
s
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 15
Preliminary Phase
Preliminary Agreement
MethodologyMethodology Preliminary phase: context Preliminary phase: context
Archive
Producer
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 16
First contact
Preliminary definition,feasibility and assessment
Establishment of apreliminary agreement
MethodologyMethodologyPreliminary phase: sub-phasesPreliminary phase: sub-phases
Information to be archived, Quantification, Legal andcontractual aspects, permanent impact on the Archive,Summary of costs, etc.
Id Preliminary phase: quantification Involves
P-19 Estimate the data volume to be transmitted to the Archive Producer
P-20 Assess the permanent data volume to store Archive
P-21 Assess the storage capability need for the ingest process Archive
P-22 Assess the associated costs Archive
Action table
Description
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 17
Formal Definition Phase
Preliminary Agreement
Dictionary
Data Model
Submission Agreement
MethodologyMethodologyFormal Definition Phase: contextFormal Definition Phase: context
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 18
Organization of the FormalDefinition Phase
Formal definition
Submission Agreement
Id Formal Definition Phase: Definition of Transfer Conditions Involves
F-14 Define the communication procedures (digital network, protocols,media, etc.)
Producer and Archive
F-15 Define the Packaging Information of delivered objects (in whatform the data is delivered)
Producer and Archive
F-16 Define a transfer session (functional and time-related structure ofthe transfer of digital objects)
Producer and Archive
F-17 etc
MethodologyMethodologyFormal Definition Phase: sub-phases and action tableFormal Definition Phase: sub-phases and action table
Creation of a data dictionary, definition of transferconditions, delivery schedule, etc.
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 19
Transfer PhaseActual transfer of the objects:• carry out the transfer test• manage the transfer
Data
Model of object files to deliver
Schedule
Transferred object files
MethodologyMethodologyTransfer PhaseTransfer Phase
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 20
Validation Phase
Validate the transferred objects:• carry out the validation test• manage the validation
Transferred object files
Data ready to archive
Anomalies
MethodologyMethodologyValidation PhaseValidation Phase
Producer
Validation
acknowledgement
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 21
ContentsContents
Why a new standard?• Space data archiving• Needs for standardization• What standardization will bring• Abstract standard status
Proposed methodology• Context - description• The 4 phases
Specialization Aid Tool for the use of the Methodology Towards an implementation standard Conclusion
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 22
Adapt the generic standard to a particular community (which can range from an international organization to a simple archive service)
Steps involved to define a community standard• terminology,
• data dictionary and information model,
• standards,
• common tools.
Analyze each action of the generic standard (add actions if necessary)
SpecializationSpecialization
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 23
Example (CDPP -Plasma Physics Data Center- Archive)
F-4: Define the formats, coding rules, and standards to be applied for the objects to be delivered
Generic Action (Methodology)
Specific Action (CDPP)
Apply the ‘specification of products for delivery to the CDPP’ including
Use the Data Description Language EAST Specification defined by the CCSDS for the syntactic data description
Use the CCSDS DEDSL-Data Entity Dictionary Structure Language- —XML/DTD Syntaxfor the semantic data description
Use the CCSDS time code format for the date and time coding
Etc.
Specialization: ExampleSpecialization: Example
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 24
ContentsContents
Why a new standard?• Space data archiving• Needs for standardization• What standardization will bring• Abstract standard status
Proposed methodology• Context - description• The 4 phases
Specialization Aid Tool for the use of the Methodology Towards an implementation standard Conclusion
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 25
A methodology application and archive project aid tool would be a real advantage for both the Producer and the Archive.
First mockups produced
Aid ToolAid Tool
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 26
Aid ToolAid Tool
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 27
ContentsContents
Why a new standard?• Space data archiving• Needs for standardization• What standardization will bring• Abstract standard status
Proposed methodology• Context - description• The 4 phases
Specialization Aid Tool for the use of the Methodology Towards an implementation standard Conclusion
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 28
PAIMASGeneric abstract
standard
Implementation standard
Methodology application aid
Languages
Tools
Communitystandard
Archive project
Towards an implementation standardTowards an implementation standardContextContext
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 29
• generic data model– transfer data model (generic objects)
– transfer data dictionary (generic descriptors)
– community dictionary
• diagram of data to transfer for the
archive project
• transfer sequencing and chronology
• transfer session
Implementation standard
Languages
Tools
XML
XML Schema
UMLXMLScheme
DEDSL
DTDXMLScheme
XFDU
Towards an implementation standardTowards an implementation standardApproachApproach
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 30
Towards an implementation standardTowards an implementation standardRelationshipsRelationships
CLUS-CIS-CODIF-Jm
CLUS-CIS-CODIF-EAST
CODIF
CIS
HIA
CLUS-CIS-HIA-EAST
CLUS-CIS-HIA-Jn
CLUSTER
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 31
Delivery plan
• all the elements to be delivered
• easy to understand from the Producer side
• expression of constraints
Towards an implementation standardTowards an implementation standardSchemaSchema
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 32
ConclusionConclusion
PAIMAS identifies: • the phases in the process of transferring information,
• the objective of the phases,
• the actions that must be carried out,
• the expected results.
PAIMAS is a basis:
• for the identification of standards and implementation guides,
• for identification and development of a set of software tools.
SpaceOps, Montreal, 2004-05-15 Producer-Archive Methodology 33