Web Services and FedoraWeb Services and Fedora

EDUCAUSE Mid-Atlantic Regional ConferenceJanuary 14, 2003

Tim SigmonUniversity of Virginia

What Are Web Services?What Are Web Services?

• WWW mainly used for human interaction– HTML forms and HTTP transport

• Web Services are meant to support machine-to-machine or application-to-application interaction

Web Services DefinitionWeb Services Definition

A Web Service is a unit of application logic providing data and services to other applications. Applications access Web Services via ubiquitous Web protocols and data formats such as HTTP, XML, and SOAP, with no need to worry about how each Web Service is implemented.

-Microsoft

Another DefinitionAnother Definition

A Web Service is a software component that is platform and implementation independent and that can be:

• Described using a service description language

• Published to a registry of services

• Discovered through a standard mechanism

• Invoked through a declared API

• Composed with other services

- Building Web Services with Java

Web Services TechnologiesWeb Services Technologies

• XML is the common language– platform independent– easy to parse– human readable (sort of!)– extensible via namespaces and schemas

• SOAP for transport of information• WSDL for description of services• UDDI for discovery

SOAPSOAP

• Simple Object Access Protocol

• XML-based messaging framework

• Transport is usually HTTP, but can be any transport

WSDLWSDL

• Web Services Definition Language

• XML language for defining Web Services endpoints, access protocol, service interface, and implementation details

• Web Services equivalent of an IDL

• Enables automatic generation of language specific API for accessing a particular Web Service

UDDIUDDI• Universal Description, Discovery, and

Integration

• Discovery protocol for Web Services through which applications and/or Web Services can find other Web Services

• Publish, find, and bind information about how to use Web Services

• Public registries exist but UDDI can also be used internally

• UDDI has been slow to catch on

Additional Web Services Additional Web Services TechnologiesTechnologies

• WS Choreography– a language to compose and describe the

relationships between Web services (e.g., sequencing, state management, etc.)

– ability to combine simple services to produce complex results

• WS Security– many areas and standards

• WS Transactions

Web Services CharacteristicsWeb Services Characteristics• Evolutionary, not revolutionary• Builds on existing web technologies and

infrastructure• Still evolving rapidly – numerous standards

initiatives with more appearing almost monthly (W3C, OASIS, IBM, Microsoft, Sun, …)

• Tools maturing (Apache, IBM, BEA, Sun, Microsoft, many others)

• Lots of testing the water• Some highly visible Internet applications

(Amazon, Google)• Enterprise implementations are emerging• Tons of hype, including significant over-hype

Some Early AdoptersSome Early Adopters

• Google and Amazon– http://www.google.com/apis– http://www.amazon.com/apis

• US Military – logistics applications between branches of the service (e.g., provisioning cargo planes)

• According to Forrester, 52% of US companies > $1B have rolled out or are rolling out a Web Services project

This Fedora Project is not the Redhat Fedora project.

FedoraFedora

The Fedora ProjectThe Fedora Project

• Fedora Digital Object Repository System– Extensible digital object model– Repository System exposed via Web Service APIs– Scalable, persistent storage for content and metadata– Local and remote content– Associate services with objects– Content versioning

• Fedora Use cases– Content Management (CMS)– Digital Library architecture– Digital Asset Management– Institutional Repository– Scholarly publishing– Preservation

• Open source software

Fedora HistoryFedora History

• Research (1997-present) : – DARPA and NSF-funded research project at Cornell

(Carl Lagoze and Sandy Payette)– Reference implementation developed at Cornell

• First Application (1999-2001) : – University of Virginia digital library prototype

(Thornton Staples and Ross Wayland)– Scale/stress testing for 10,000,000 objects

• Open Source Software (2002-present): – Andrew W. Mellon Foundation granted Virginia and Cornell

$1 million to develop a production-quality Fedora system– Fedora 1.0 released in May 2003– www.fedora.info

Persistent ID (PID)

Disseminators

System Metadata

Datastreams

Globally unique persistent id

Public view: access methods for obtaining “disseminations” of digital object content

Internal view: metadata necessary to manage the object

Protected view: content that makes up the “basis” of the object

Digital Object Model Architectural View

Persistent ID (PID)

Default

Disseminators

Simple Image

System Metadata

Datastreams

Digital Object Model Example Disseminators

Get ProfileList ItemsGet Item

List MethodsGet DC Record

Get ThumbnailGet Medium

Get HighGet VeryHigh

Fedora and Web ServicesFedora and Web Services

• Fedora Repository is a set of Web Services– Access/Search (API-A) and Management (API-M)– Service descriptions published using WSDL– Both SOAP and REST bindings

• Back-end services– Digital object behaviors implemented as linkages to

other distributed web services– Service binding metadata (WSDL) stored in special

Fedora Behavior Mechanism objects.– Fedora acts as mediator to these services.

Fedora Repository SystemFedora Repository SystemClient and Web Service InteractionsClient and Web Service Interactions

FedoraRepository

System

ContentTransform

Service

ContentTransform

Service

user

Web

Ser

vice

Dis

patc

h

We

b S

erv

ice

Ser

vice

Ser

vice

BackendFrontend

clie

nt

app

lica

tio

n

clie

nt

app

lica

tio

nw

ebb

row

ser

user

Fedora Repository Fedora Repository Service InterfacesService Interfaces

• Management Service (API-M)– Ingest - XML-encoded object submission– Create - interactive object creation via API requests– Maintain - interactive object modification via API requests– Validate – application of integrity rules to objects– Identify - generate unique object identifiers– Security - authentication and access control– Preserve - automatic content versioning and audit trail– Export - XML-encoded object formats

• Access Service (API-A and API-A-LITE)– Search - search repository for objects– Object Reflection - what disseminations can the object provide?– Object Dissemination - request a view of the object’s content

• OAI-PMH Provider Service– OAI-DC records

Fedora Repository SystemFedora Repository System

E x ter n a lC o n ten tS o u r c e

E x ter n a lC o n ten tS o u r c e

HT

TP

E x ter n a l C o n ten tR etr iev er

X M L F ile s

Re la t io n a l D B

S e s s io n M a n a g e me n tU s e r A u th e n t ic a t io n

P o l icies

U s ers /G ro u p s

H T T P

F T P

D atas tr eam s

D ig ita l O b jec tsS to rag e S u b s ys te m

S e c u rityS u b s ys te m

W e b Se r vi c eE xpo s ur eL aye r

SO

AP

R em o teS er v ic e

L o c a lS er v ic e

M an ag e A c c e s s S e arc h O A I P ro v id e r

M an ag e m e n tS u b s ys te m

A c c e s sS u b s ys te m

HT

TP

FT

P

H T T PH T T P S O A P H T T P S O A P H T T P S O A P

C lie n tA pplica t io n

B a tchPro g ra m

S e rv e rA pplica t io n

W e bB ro ws e r

Co mp o n e n t M g mt

O b je c t M g mt

O b je c t Va lid a t io n

P ID Ge n e ra t io n

O b je c t D is s e min a t io n

O b je c t Re fle c t io n

P o lic y En fo rc e me n t

P o lic y M g mt

Co n te n t

Fedora 1.2 Software Feature SetFedora 1.2 Software Feature Set• Open Fedora APIs

– Repository as web services (REST and SOAP bindings); WSDL interface defs

• Flexible Digital Object Model– Content View: objects as bundle of items (content and metadata)– Service View: objects as a set of service methods (“behaviors”)– Extensible functionality by associating services with objects

• Repository System– Core Services: Management, Access/Search, OAI-PMH– Storage: XML object store; relational db object cache; relational db object registry– Mediation - auto-dispatching to distributed web services for content transformation– Auto-Indexing – system metadata and DC record of each object– HTTP Basic Authentication and Access Control– Built-in disseminator services: XSLT x-form, image manipulation, xml-to-PDF

• Content Versioning– Automatic version control (saves version of content/metadata when modified)– Enables date-time stamped API requests (see object as it looked at a point in time)

• Clients– Fedora Administrator: GUI client to create/maintain objects– Default Web browser interface: search; access objects via default disseminator– Command line utilities (batch load, ingest, purge, others)– Migration Utility – mass export/ingest

Fedora Fedora Software Distribution PackageSoftware Distribution Package

• Open Source (Mozilla Public License)• 100% Java (Sun Java J2SDK1.4)• Supporting Technologies

– Apache Tomcat 4.1 and Apache Axis (SOAP)– Xerces 2-2.0.2 for XML parsing and validation– Saxon 6.5 for XSLT transformation– Schematron 1.5 for validation– MySQL and Mckoi relational database– Oracle 9i support

• Deployment Platforms– Windows 2000, NT, XP– Solaris– Linux

Future Software ReleasesFuture Software Releases

• Fedora Object XML (FOXML)– Internal storage format; direct expression of Fedora object model– Better support for relationships (“kinship” metadata)– Better support for audit trail (event history)– Format identifiers for dynamic service binding

• Shibboleth authentication• Policy Enforcement

– XACML expression language– Fedora policy enforcement module

• Web interface for easy content submission• Batch object modification utility• Administrative Reporting• Object Event History (ABC/RDF disseminations)• Better support for “collections”• New ingest and export formats (METS1.3, DIDL)

December 2003 – December 2004

www.fedora.info

Questions?Questions?

DEMO: Basic Use CasesDEMO: Basic Use Cases

Image (multiple datastreams)

Image (Mr. SID)

EAD (Rita Mae Brown papers)

Text conversion (TEI to PDF)

Basic Search