Fedora Commons Overview and Future

Plans

Sandy Payette, Executive Director

Cornell University LibraryMetadata Working Group

June 13, 2008

A Brief History…

2001 2005 2007

MajorFunders

Mellon Mellon Moore

Mellon

Community,Partners,Donors

2012+

501(c)(3)Non-profit

Scholarly Communication

Preservation and ArchivingSemantic Knowledge Spaces

Use Cases We Support…

Data Curation and Linking

Science Humanities

R.D.W. Connor

Users of Fedora Commons open source software

http://fedora.info/wiki/index.php/Fedora_Commons_Community_Registry

• National Libraries and Archives• University Libraries and Archives• Research Institutions• Museums and Culture Organizations• Semantic and Virtual Digital Libraries• National Repository Consortia• Publishing and Professional Societies• Medical Centers and Libraries• Government Agencies

Your Applications

Storage Layer

Fedora Services and Middleware Layer

OtherFast Disk

Fedora Repository

ManageAPI

Access API

RegistrySearch

RDF Query

Access

ServiceBind

Manage

Policy

Ingest

Validate

Registry RDF IndexStore

Fedora Services

Honeycomb

Scholarly and e-Research Web 2.0

Digital Libraries Archive

Asset Management

Other

OAI

Etc.

Search

Ingest

How?

Working Together with Target Communities…

preserv

e/arch

ivee-research

data curation

open acce

ss

publicatio

n

FC – 2008CommunitySolutionCouncils

Current Collaborative Engagements

IA

BHL

Sun

PLoS JHU

UVANSDL

ORE

BNF

Oxford

Aquifer PorticoDSpace

EOLDRSG

Adore

Sakai

FIZ

Kuali

Hull

Rutgers

open access data curation

institutional repo

e-research

preservation

archiving

CUL

DOING

PLANNING

TALKING

Technology Perspective

Fedora Commons Open Source Projects

• Fedora Repository Project• The original Fedora Project

• Fedora Middleware Project• Service integration and enterprise-orientation for repositories

• Akubra Storage Project• New storage plug-in architecture; transactional file system;

• Topaz Project • Incubated project; persistence service based on semantic triplestore

• Mulgara Triplestore Project • Independent, but FC supports developer; highly scalable triplestore

Fedora Commons open source projects can be integrated together.

Fedora Repository – Key Features

• Digital Object Model– Aggregate content “datastreams” in an object… any type of content– Intermix both local content and external content – Relationships among digital objects (via RDF)– Register “content models” for known object patterns

• Repository Service– Modular– Web service interfaces (REST/SOAP)– Versioning– Dynamic service binding based on object content model types– File-centric (all essential characteristics in XML files)– RDF-based indexing (semantic triplestore index with query)– Security with pluggable authentication and XACML policies– Journaling (replay all events to create replicas of repository)

DCDC

Persistent ID

RELS-EXTRELS-EXT

Audit TrailAudit Trail

Datastream 1Datastream 1

Datastream 2Datastream 2

Datastream nDatastream n

Reserved Datastreams

Datastreams

(any type, any number)

(relationships and properties)

Basic Building Block: Fedora Digital Object Model

PolicyPolicy

Forming Collections… with relationships

PID 5

PID 3

PID 2

isMemberOfCollection

Query Query

PID1

isMemberOfCollection

isMemberOfCollection

“Smiley Stuff “Collection Object

Digital Objects... with compositional relationships

Digital Objects… information network

PID 5

PID 4

PID 1

PID 3

PID 2

providesContext

annotates hasPart

hasPart

Amazon Web Service

External commercial web content

library content

scholarly object

Fedora APIs (SOAP and REST)

Fedora Repository Service

For creating, managing, providing access to various and sundry objects

ManageAPI

Access API

RegistrySearch

RDF Query

Ingest

Validate

Manage

Policy

Access

RDF IndexStore Registry

File system(Objects)

RDBMS(Registry)

CMABind

Triplestore

Roadmap Perspective

http://www.fedora-commons.org/resources/roadmap.php

Roadmap – Active Themes

• Ease of use• Re-use and interoperability• Web orientation• Improved modularity• Pluggable storage• Workflow engine• Messaging• RDF and Semantic technologies• Content models• Solution bundles (including end-user applications)

2008 Notable Themes

• Solution bundle – out of box application for repositories– “IR2” (Institutional Repository Application “Take 2”) – Institutional repository moving towards enabling e-research– Turn-key out of box user experience– Means of achieving (2 strategies under consideration):

• FC and community developers build a brand new application• Fedora + DSpace collaboration

• “Easy fit” repositories behind the collaborative web– Motivation: enable seamless integration with e-research applications

(e.g., Zotero) and enable re-use and sharing of repository content– Object Reuse and Exchange (ORE) libraries– Lightweight interfaces on Fedora repository – Lightweight web protocol on Fedora repository (e.g., Atom, SWORD)

Fedora APIs (SOAP and REST)

Better integration with web and workflows

ManageAPI

Access API

RegistrySearch

RDF Query

Ingest

Validate

Manage

Policy

Access

RDF IndexStore Registry

File system(Objects)

RDBMS(Registry)

CMABind Additional APIs: SWORD (2008) APP (full) ? JSR 170 ? Other ?

OAI-ORE (2008)Atom (2008) new formats

Triplestore

Replication to Archival Storage• Replica repositories, each with different underlying storage

system; useful for failover, redundancy, archiving• Now: Fedora Journaling

http://fedora.info/download/2.2.1/userdocs/server/journal/index.html • Future: Journal Event Messaging via Fedora JMS

Sun Honeycomb

Leader Repository Follower Repository

Can configure multiple “followers”

JournalEventLog

API events API events

Fedora Repository ServiceGSearch

OAI

Ingest

SimpleJMS

Better Service Integration

More…

First, we are providing simple messaging (via ActiveMQ in Fedora 3.0)

repository publishes events

serviceslisten andconsumeevents or other messages

Next, lightweight integration with workflow engine(s); orchestration

DCDC

Persistent ID

RELS-EXTRELS-EXT

Audit TrailAudit Trail

ImageImage

Preserve MetadataPreserve Metadata

Custom MetadataCustom Metadata

The Model-Driven Repository:Content Models and Service Models

PolicyPolicy

DCDC

RELS-EXTRELS-EXT

Model Type Def

Model Type Def

DCDC

RELS-EXTRELS-EXT

Service DefService Def

hasServiceDef

hasModelPersistent ID

Persistent ID

Fedora Repository handles dynamic service binding at runtime

DCDC

Persistent ID

RELS-EXTRELS-EXT

Audit TrailAudit Trail

ImageImage

Preserve MetadataPreserve Metadata

Custom MetadataCustom Metadata

Runtime : dynamic services available on all objects that conform to the content model

PolicyPolicy

Get-Thumbnail-Size-ImageGet-Web-Size-ImageGet-High-Resolution-Image

File system(Objects)

Akubra Project … backend storage interoperability

ManageAPI

Access API

RegistrySearch

RDF Query

Store

Fedora Repository Service

Triplestore(Mulgara/MPTStore)

RDBMS(registry)

Akubra

Plug-in 1 Plug-in 2 Plug-in 3

Transactional File System

Sun Honeycomb

New pluggablestorage framework

Elevator Speak Takeaway…

Sloganfest 2008 – Simple Message

www.fedora-commons.org