Seeing Connecticut Now and Then: Repository Services That Support Your Best Memories for the FutureGreg Colati and Jennifer EustisConnecticut Library Association Annual MeetingApril 2016

The Dilemma of Modern Stewardship, 1816How do we insure resources that support cultural activities that exist today will reliably exist and be discoverable in the future?

1816 2016

The Dilemma of Modern Media, 1915

The Dilemma of Modern Media, 1995

Cultural Armageddon, 2000s

Cultural Armageddon, 2011

Source: University of Southern CaliforniaCredit: Todd Lindeman and Brian Vastag/ The Washington Post

The Dilemma of Modern Stewardship, 2016How do we insure that resources that support cultural activities that exist in digital form today will reliably exist and be discoverable in the future?

?

2016 2216

The Search for A SolutionIn October 2011, UConn Libraries created the Second Generation Digital Libraries Working Group to solve the dilemma of modern stewardship.

Four requirements: Sustainabiilty, Reliability, Interoperabilty, Reusabilty

What we found: No ONE system or software can do everything

A combination of tools and services creates a preservation ecosystem that is Reliable and sustainable, and provides interoperability and reusability.

Main components: FedoraCommons and Islandora/Drupal Along with: Handle.net

By Nov. 2013, version 1.0 was in production

Why Fedora/Islandora?

• Open source: Flexible, Customizable, Auditable, Extensible

• Significant development in academic and governmental organizations

• Allows for a multi-site implementation• Ability to accommodate various metadata

standards while normalizing to one• Provides harvesting capabilities• Ability to add a large variety of file formats

from the start. Can be set up relatively quickly and scale to suit a number of different needs such as historical/cultural materials, research data sets, manuscripts and transcripts, educational videos, books, and more

What Is Fedora?Fedora: • NOT a relational database• A conceptual framework/set of abstractions about digital

information• A generic repository foundation upon which many kinds of

applications can be created.• Provides the basis for software systems to manage digital

information• Provides the basis for ensuring long-term durability• No particular administrative workflow or end-user application is

assumed.• No particular catalog or organizational scheme is assumed.• Any number of indices and views, designed for specific purposes,

both administrative and public facing, can be applied to any pattern of components of objects.

What Is Islandora?Islandora: • A set of Drupal modules that

facilitate interaction with Fedora repository

• Provides a management interface and one type of presentation interface

• A head start over build it yourself approach

Drupal:• Open source content

management system provides a framework for presentation of informational web pages as well as repository content

• Provides a management interface for user accounts

Challenges: Technical and Operational

• Relatively high barrier to entry compared to proprietary solutions

• "Out of the box" is not truly functional• Continuous and continued investment in

development required• Non-database approach can be difficult to

understand and explain• Steep learning curve both conceptually and

practically• Continued growth in storage needs and

infrastructure architecture• Variety of metadata approaches makes it

more difficult to normalize and visualize data

• How do we organize to support the needs of both large and small institutions across the state?

• How do we reconcile what is required of CTDA as a DPLA Service Hub and what participants add for descriptive information?

• Connect: Participants and users interact with the repository and each other in various ways with as little interference from the CTDA as possible.

• Preserve: Preservation of access in whatever forms and schemas users require.

• Share: Facilitates sharing of content, but does not control or determine how, where, or why.

How To Connect, Share and Preserve

Ownership vs. Custodianship

• Participants retain ownership of all content: metadata, primary content objects, and derivatives

• CTDA is the custodian of the digital assets

• Low barrier to entry: minimal standards for participation

• Participants are responsible for their own content and descriptive information about that content

Connect and Share

Collaborative Development

• The CTDA relies on collaborations between and with participants, a constant give and take about content, needs, tools & initiatives.

• One the one hand, initiatives from one help another partner and vice versa. Through collaborative effort a suite of tools can be offered as well as developed for the future.

• On the other, partners have different needs. Approaches to digital preservation or information organization vary based on the needs of each partner and their user groups.

Preserve and Share

Low Barrier

• The CTDA demands from the participants no more than preservation requires;

• The CTDA provides to participants a means to do more, should they wish.

Data Responsibility• CTDA is not the guardians of

digital content.• Non-judgmental regarding

content and metadata. Everyone and everything is welcome:

•Two requirements: that you have the right to deposit, and that the content is not malicious/destructive to the system.

• Content owners determine their quality standards

• No prescribed one way to organize, add, or manage content.

• Participants make their own decisions about organization of content and how to add and manage it.

The True Cost of Ownership is Responsibility

• Instead of imposing the program’s ideas everyone, CTDA welcomes different approaches while educating participants about technologies and services.

• Owners become stewards, considering the options of digital preservation and information organization.

• Because there are so many options, stewards can often become lost, anxious, or paralyzed.

The True Cost of Custodianship is Education

• The technical infrastructure imposes some structure.

• This does not affect how people organize or describe their content.

• CTDA provides training and suggestions on the different options available to organize and describe content.

• Strategies must meet participants’ and their users’ needs.

Participants Can• Participants can choose an a la carte

option for long-term preservation of digital assets.

• Participants may:• Deposit Content• Access Content• Add and manage content

• CTDA provides at no cost:• Repository & Preservation

services up to 500GB• Community Management

services• End User support• System management support

• CTDA provides at cost:• Additional preservation storage• Channel (or independent site)

services• Data curation services Je

rry’s

Radi

o an

d TV

Ser

vice

, 628

Par

k St

reet

Har

tford

http

://hd

l.han

dle.

net/1

1134

/400

02:9

925

Data Responsibility: Metadata & Indexing• CTDA participants are encouraged to ask this

question:If someone finds your digital asset on the web through Google or DPLA, will that user be able to determine if this is a digital asset they want to learn more about, see, or use?

• Thinking about metadata and indexing means thinking beyond a descriptive record. It's about "relationships" between objects. Records are dynamic and affect searching, browsing, and discovery.

• Metadata and indexing require constant care and re-evaluation

Management Tools• Browser-based, no software

to load or update• Load content singly or in

groups• Only you can manage your

content• Separate ingest server

provides higher throughput without impact on presentation channels

Access• Presentation Channels

• Viewers for supported file types

• Web search engine indexing• OAI-PMH harvesting• Indexing in the Digital

Public Library of America, iConn

• Inclusion in Connecticut History Illustrated (for content you specify)

• Open APIs, embed codes; for indexing, content extraction, viewer re-use

• Custom channels scoped to organizational content

*Note: Some services are fee-based

CTDA Collections • All the content, all the time,

no matter what the subject• Primary harvest site for

external aggregators• Persistent resolver for

handles for general participants

http://collections.ctdigitalarchive.org

• Basic system training • How-to documentation from

the CTDA website• Metadata consultation• Data migration and

conversion for ingest• Custom forms and tools

Metadata and Management

Indexing• Harvesting by DPLA• Harvesting by iConn• Open OAI supplier for

indexing• Search engine optimization

for discovery by Google, Bing, etc.

Data Responsibility: Metadata & Indexing• CTDA participants are encouraged to ask this

question:If someone finds your digital asset on the web through Google or DPLA, will that user be able to determine if this is a digital asset they want to learn more about, see, or use?

• Thinking about metadata and indexing means thinking beyond a descriptive record. It's about "relationships" between objects. Records are dynamic and affect searching, browsing, and discovery.

• Metadata and indexing require constant care and re-evaluation

Repository Content

• 320,000+ assets being managed

• 20+ institutions in production• 38,000+ OAI records for harvest

Florence Griswold Museum

Trinity College

Fairfield Museum

Connecticut State LibraryGroton Public Library

Connecticut Historical

Society

Find Out Morectdigitalarchive.org• General information• Service Catalog• How-to documents• News and information about

CTDA activities• Links to production channels• More!