archival workshop on ingest, identification, and certification standards certification (best...

Archival Workshop on Ingest, Identification, and Certification Standards

Certification (Best Practices) Checklist• Does the archive have a written plan which spells

out and justifies its assumptions and practices?• Does the archive have a mission statement?

• This defines collection parameters, and annual work plans.

• Is the plan acceptable to the archive sponsor, resource allocator, data creator, certifying board?

• Is there a plan for staging ingestion of data into archive?• This defines the at-risk components.

• Is there a concept of operations plan for the archive?• Has a legal authority been defined for the archive to justify

retention of data?• Do you have a governing board for the archive?


Certification (Best Practices) Checklist• Has the archive validated its procedures and

capabilities against this implementation standards checklist?• Have the legal authority requirements been met?• Does the archive comply with relevant “NARA / ISO” standards

that have become available?• DOD 5015.2 standard for ingesting office records• ISO PDTR 15801 draft records management standard• ISO legal imaging standard or best practices from DLM forum

• Is there a validation plan or audit process that reconfirms the archive compliance over time?

• Are updated versions of the checklist used, that include current best practices?


Certification (Best Practices) Checklist• Is the nature of the interaction of the archive staff

with data providers defined?• How does the archive help the provider understand the data

and meta-data standards to which the provider must/should conform?

• Are guidelines and requirements for preparation and submission of data provided?

• How does the archive support the providers in the data preparation?

• Is training and outreach provided?

• Are required protocols for data ingestion specified?• Is the accession process for scheduling data access defined?


Certification (Best Practices) Checklist

• Can the data ingestion process handle the expected data transfer in a manner acceptable to the provider?• Does the data ingestion process scale to handle the

expected growth in the data flow?• Does the data ingestion process support the accepted

practices of the provider?• What types of data ingestion can be supported (formats, data rates)?• Are the ingestion mechanisms appropriate for the planned data

ingestion schedule?

• Is electronic data transfer supported?• Are expectations set for the sustainable data transfer rates?



• Does the archive have approaches to insure that the accessioning process maintains the fidelity of the original data?• Are mechanisms provided to track fidelity of data?

• Can integrity of data be monitored through checksums, and are the checksum algorithms documented?

• Are data flows tracked to identify sources of error?• Are content validation mechanisms provided to check the data as

received, and after delivery to a user?



• Are adequate environmental and security controls provided for the archive?• ANSI recommended practices for environmental controls

• Temperature, humidity, dust, ...• Computer room controls (36 CFR 188)

• Security access controls• Physical security• IT security (viruses)

• Backup and disaster recovery• Physical location of archive (faults, tornadoes, hurricanes,

floods, fires)


Certification (Best Practices) Checklist• Does the archive have (and follow) processes to

migrate data across media? • Are mechanisms provided to check fidelity of information

over the life of the archive?• Are annual checks of readability done (36 CFR 1234)?

• Are certified media used (defined)?• NARA standard, National Media Lab (nta.org or nml.org)• Are mechanisms provided to monitor media viability (OAIS criteria)?

• Is automatic error checking done for media (SMART technology)?• What are the maximum allowed error levels?• Are best practices followed for archiving to tape?

• Are “stable” technologies used for media?


Certification (Best Practices) Checklist• Does the archive have (and follow) processes to

migrate data and meta-data across technologies? • Do you have a test plan to validate your migration

procedures that follows industry best practices?• Are “stable” IT technologies used in archive generation,

preservation and maintenance?• Are documented formats used for data sets and meta-

data?• Is an information model used to characterize collections?• Are access mechanisms provided that meet user “leading

or cutting edge” expectations?• Are you monitoring your user community access requirements?


Certification (Best Practices) Checklist• Does the archive make its data and services

known to its customer community?• Is outreach information made available that is appropriate

for the user community?• Does the archive effectively publicize its services?• Is the archive a member of the Society of American

Archivists, or the Association of Archivists, or the International Council on Archives, or regional archive organizations?

• Do you solicit input from your user community to assess needs and future requirements?


Certification (Best Practices) Checklist• Does the archive make its data and services effectively

available to its customer community?• Are access delivery mechanisms in synch with the expectations of the

user community?• Is cataloging information provided that is sufficient for locating data

based on descriptive information?• Does the archive provide unique identifiers for data sets that support

tracking of information?• Is cataloging information available to the customer community?• Are mechanisms provided to track usage of the archive?• Mechanisms for handling confidentiality requirements?• Do you have mechanisms to handle copyright?

• Support for proprietary data access control?


Certification (Best Practices) Checklist• Are the archive’s data correctly and

independently usable by the designated community?• Is there enough and adequate representational

information provided for the data to be usable?• Is the meta-data context correctly defined?• Can the data be used without needing expert advice?

• Are mechanisms provided to collect this information from the data providers?

• Is the level of expertise defined for access to the data sets?

• Are data sets provided that correspond to the level of expertise of the user community?



• Does the archive provide an infrastructure independent representation for the data and meta-data?• Can the collection be migrated forward in time?

• Do you have an infrastructure independent representation for the database itself?

• Mechanisms for handling proprietary formats?• Will conversion tools exist?• Do you provide (archive) original software?• Do you provide alternate representations or formats?

• Are the formats documented or published?



• Are mechanisms provided for life cycle maintenance?• Is validation of data integrity done on a regular basis?• Is the OAIS checklist followed on a regular basis?• Is there a plan for regular migration of data?• Is copyright permission format evolution tracked?

• Do you have permission to migrate to a new form?• (Need to extend fair use and rights of preservation to digital

materials.)• Pre-ingest checklist is needed


Best Practice Development

• Use ingest and identification workgroup criteria for checklist• ID meta-data• Cataloging (finding aids)• Unique identifier within local collection, within archive

community, …

• Check compliance of the certification/best practices list with the OAIS model and mandatory responsibilities section 3.


Best Practice Development• What is the appropriate data set description?

• Correspondence to OAIS terminology• Descriptive information

– Dublin Core• PDI - Reference, context, provenance, fixity• Packaging information• Representation information

• Structure• Management - audit trails• Preservation

• What is the information model for the structure of meta-data?

archival workshop on ingest, identification, and certification standards certification (best...

Documents