1 this work is licensed under a creative commons license attribution non-commercial sharealike 2.0...

1This work is licensed under a Creative Commons License

Attribution Non-commercial ShareAlike 2.0 Germany

Towards International Repositories Infrastructures

Workshop 16/17 March, 2009Norbert Lossau,

Director Göttingen State and University Library

& Scientific Coordinator DRIVER

http://creativecommons.org/licenses/by-sa/2.0/


Attribution Non-commercial ShareAlike 2.0 Germany 2This work is licensed under a Creative Commons License


Topics

Objectives of the Workshop

Visions - use cases – infrastructure and components

International Repositories Infrastructure(s) – Where do we stand today?

Challenges

Global Data Network: a model for the International Repositories Infrastructure?

How do we proceed: our next two days (and beyond)

2







1. Identify and establish relationships with key thought leaders, major projects/activities and services, and leading practitioners from around the world

2. Suggest commonalities between infrastructures, points of possible collaboration and pathways that might take the collaboration forward

3. To come to a shared vision of an international repositories infrastructure or, at least, the infrastructure components that might best be developed internationally

4. To identify the essential components of an international repositories infrastructure







5. To review the approaches to sustainability, scalability and interoperability being taken by these components, bearing in mind the wider research infrastructure

6. To consider ways in which the progress might be coordinated and reviewed over time

7. Focus the agenda to achieve tangible outcomes






Topics




Challenges



2






High-level Vision …

Free and unrestricted access

to sciences and human knowledge representation

worldwide,

incl. cultural heritage

Berlin Declaration, October 2003






International Repositories or Knowledge Infrastructure Vision …

Discovery & Access

Management

Usage & manipulati

on

Collaboration &

Sharing

Dissemination &

Publishing

To support the… complete research cycle, working with scientific information






High-level use cases & possible infrastructure components

Preservation actionsfile format registries

validation tools

representation information registries

IngestSWORD

shared metadata services

name / factual authority services

automatic metadata creation services






High-level use cases & possible infrastructure components

Accesswidespread OAI-ORE implementation

common text-mining API?

Online Reputation and reportingeffective, real-time, automatic forward and backward citation mechanisms

factual authority (common tagging of objects with funder / grant number metadata)






Infrastructure components non-technical factors

Discovery & AccessEstablished search & browsing behaviours and pathways

Online Reputation and reportingEstablished evaluation mechanisms (impact factor)

Preservation actionsAdditional (manual) effort on the author side required?

IngestAdditional (manual) effort on the author side required?






Use cases - behind the scene“ infrastructures – essential components

A ‘component’ might be:

A service (eg sherpa-romeo, connotea, BASE, funder repository, institutional repository)

A service environment (eg, Amazon S3, Microsoft Azure, Facebook)

A technical success factor (eg consistent use of DC to point from a metadata record to the ‘full text’, use of OAI-ORE, the DRIVER Guidelines), or

a non-technical success factor (e.g. filling repositories through OA-agreements with publishers).

These components will form the focus for the workshop and the action plans that will be its principal output.









Challenges



Topics

2






International Repositories Infrastructure – Where do we stand today? „Briefings“

Author identification

Copyright and licensing

Global harvesters (other than search engines)

Harvesters – subject or discipline based

Ingest – selected issues

Institution identifiers

Peer review

Persistent identifiers

Preservation






International Repositories Infrastructure – Where do we stand today?

Prestige and profiling services

Registries

Repository software

Repository support organisations

Storage

Usage reporting and metrics

User services

Validation and certification of repositories

Versioning






Development status of components - for discussion

Advanced?

Information on the repository landscape

Global harvesters

Preservation of research papers

Repository software

Storage

Validation & certification of repositories

A brief insight into some components =>…






International Repositories Infrastructure – Where do we stand? Repositories• Informal survey carried out by SURF earlier in

2005• DRIVER Inventory Study 2007

1. Produced 7 studies in 3 publications

• Inventory study into the present type and level of OAI compliant Digital Repository activities in the EU

• A DRIVER's Guide to European Repositories• The Investigative Study of Standards for Digital

Repositories and Related Services 2. Disseminated through the DRIVER website (in Open Access) +

as 3 books (Amsterdam University Press)






International Repositories Infrastructure – Where do we stand? Repositories

• “Research Repositories in Europe: the 2008 DRIVER Inventory study”, Maurits van der Graaf (on behalf of SURF, DRIVER)

=>…






Research Repositories in Europe

TopicsGrowth, total number and situation

Contents, coverage and depositing

Technical issues and standards

Services on top of repositories

Steady increase of number of Digital Repositories

Total of 280, yearly increase by 25-30

Large part of universities in half of European countries






Conclusions content, coverage and depositing

More flexibility in access formsTrend to more OAVersion of deposited full text articles

Trend towards depositing postprint stage

Work processes for depositingNo harmonisation

Growing (partly) mandatory depositing32% in 2008, while 25% in 2006

(Still) Coverage of a third33% of Researchers delivering in repositories 35% of Research output of an institution deposited






Technical issues

Various technical issues 2008 2006persistent identifier 84% 75%

long-term availability secured 52% 73%

statistical data on access and usage 72% 70%

some form of subject indexing 86% 93%

author identifier 31% 33%

ARNO

locally developed

CDSware

Digitool

DIVA

DSpace

Fedora

GNU EPrint

iTOR

MyCoRe

OPUS

VITAL other






Technical issues

Is your repository technically prepared for Enhanced Publications?

Metadata standards on the rise: DIDL, MODS and OAI-ORE

46.1% 32.6%

YESNo, but

NO

21.3%

no, but we have plans to prepare our repository

no, no plans






International Repositories Infrastructure – Where do we stand? Repositories

• OpenDOAR – a comprehensive register of digital repositories worldwide- More than 1300 repositories listed

=>…






OpenDOAR






Repository Type=>1072 institutional and 177 disciplinary

Content Type=>815 hold journal articles, 318 Multimedia, audiovisual….69 datasets, 27 software etc.…

OpenDOAR






=>Languages:1133 English155 German96 Spanish86 French73 Japanese…3 Africaans…2 Pashto, Pushto…1 Bulgarian1 Romanian

OpenDOAR






=>Disciplines763 Multidisciplinary86 Science General…99 Health and medicine…98 History and Archaeology…75 Social Sciences General

OpenDOAR






Where do we stand? Repository platforms & their (international) communities

EPrints*, DSpace*, Fedora Commons*, OPUS (GE), DiVA (SE), CDS Invenio (CERN),






Where do we stand? Country organisa-tions and/or repository infrastructures

Australia, Belgium, Brazil, Canada, France, Germany, Hungary, Ireland, Italy, Japan, The Netherlands, Nordic countries, Portugal, Spain, UK, ???






Where do we stand? Global Harvesters

OAIster, BASE, Scientific Commons






Where do we stand? Cross-Country organisations & repository infrastructures

DRIVER

eIFL






Where do we stand? Repository Infrastructure Architectures

DRIVER






Existing Solutions: Repository Aggregation Systems (RAS)

RAS aggregate content from OAI-PMH Repositories, form an Information Space and provide community-specific functionalities via Web User Interfaces

Well known examplesBASE (DE)

DAREnet (NE)

OAIster (USA)

Others…






RAS

OAI-PMH

Institution Site

OAI-PMH

Institution Site

OAI-PMH

Institution Site

…

Aggregator

Information Space

Index

Search

Index

UI

…






Service Open Infrastructures (SOI), DRIVER

Inspired by component-oriented systemsComponents provide specific functionality in isolation

Components can be provided by different Service Providers and be shared between applications

Applications are formed by combining independent components under the control of System Managers

Service Open InfrastructureComponents are distributed services running on the network at different sites

Open to instance and types of services: instances or new functionality can be added/removed any time






Infrastructure architecture

Functionality LayerUser Interface

ServiceRecomm.Service

CommunityService

UserService

SearchService

Repositories

Data Layer

OAI-PMHService

IndexService

BrowseService

StoreService

AggregatorService

Info

rma

tion

Ser

vice

Man

ager

Ser

vice

Aut

hz&

Au

thn

Ser

vice

Res

ulS

etS

ervi

ce

UserService

ValidatorService

Text EngineService

EnablingLayer

CollectionService






ReuseFunctionality sharing

OAI-PMH

Aggregator

Index

Search

Index

UI

…

OAI-PMH

Institution Site

…OAI-PMH

Institution Site

OAI-PMH

Institution Site

…

Ena

blin

g La

yer

Mid

dlew

areUI

Search

Index

Aggregator

User Profiling

…

Others

Aggregator

UI

Search

Index

Store

Store

FunctionalityServices

Institution Site

Dynamic, distributedRun-time Infrastructure

ContentResources






Where do we stand? Repository Infrastructure Interoperability

DINI Certificate

DRAMBORA

TRAC project

DRIVER Guidelines, Maurice Vanderfeesten, SURF + DRIVER partners (some of the following slides have been presented by Maurice on the 29 August 2007, TICER Digital Libraries a-la-Carte)





- Guidelines

- Validate

- Workflow

40

Interoperability pragmatics




Guidelines

- Chapter 1: Use of OAI-PMH - Chapter 2: Use of Metadata OAI_DC - Chapter 3: Use of Best Practices for OAI_DC- Chapter 4: Use of Compound Object Wrapping - Chapter 5: Use of Vocabularies and Semantics - Chapter 6: Use of Quality labels - Chapter 7: Use of Persistent Identifiers- Chapter 8: Use of Usage Statistics Exchange- Chapter 9: Use of Intellectual Property Rights

(IPR)





DRIVER Guidelines in various languages





Attribution Non-commercial ShareAlike 2.0 GermanyThis is as optional footer

Guidelines

From the inventory study:

72.5% knows DRIVER Guidelines; 54.5% tries to follow them

Does your repository follow the DRIVER guidelines? n %

We do not know about the DRIVER guidelines 49 27.5

We know about the DRIVER guidelines, but do not follow them 32 18.0

We know about the DRIVER guidelines and (make every effort) to follow them 97 54.5





Validator

- Detects interoperability failures

- Goes deep into the metadata content

- Provides explanation about guideline principals per

interoperability feature.

- Offers recommendations on how to correctly modify

your repository to interoperable standards

- Creates a report for future reference

=>Developed at the University of Athens





Where do we stand? Technology Components – 2008 Study

Slides from the „Technology Watch“, Karen Van Godtsenhoven, University of Gent (+ DRIVER partners)






Structure of Technology Watch Report

ChaptersDRIVER-GRID interaction

Interoperability

Long Term Preservation (LTP)

+

DRIVER-CRIS interaction (added later)

Result: two main partsNew communities and technologies (GRID, CRIS, LTP)

Interoperability of EP’s (5 types)

Structure of each chapterTheory - Case studies - Outcomes for DRIVER






Interoperability Enhanced Publications

Interoperability in DRIVER context: Exchange and dissemination of EP’s as complex, compound objects, based on textual publication

Focus on five types of representing and publishing enhanced publications (relationship of files within objects)

Envelope models or packaging formatsOverlays, maps, feedsEmbedding formatsNew/Old publishing formatsWeb services

NOT focus on ingest or descriptive metadata (Russell, Vanderfeesten, Hochstenbach, Van Godtsenhoven)






Envelopes

Access to metadata, structural data, identifiers, and binary streams of publications all in one package (= envelope)

MPEG 21-DIDL in DARE context

METS

IMS – CP

ODF packages

OOXML/ Package convention

Open e-book packageComparison: table with all features in doc






Overlays, maps and feeds

SWAP, ORE and POWDER all qualify as good formats / models for the dissemination of EP’s

SWAP uptake by community is very low (high complexity)

OAI-ORE very popular in community and used in DRIVER demonstrator for EPs

POWDER: recent W3C standard, viable alternative to ORE (when the aggregations are of a very dynamic nature or cannot be simply enumerated)






New Publishing Formats

ODF (ISO 26300:2006) versus OOXML (ISO 29 500-1:2008)File format ISO standards for saving & exchanging office documents (alternative to proprietary formats e.g. doc, ppt)Open up access to structured content which can be reused by other services e.g. DRIVERGuarantee long term accessibilityControversy surrounding development of OOXML: DRIVER should adopt approach that is capable of using both ODF and OOXML Plus: many disciplinary xml types, structured and crawlable data






Where do we stand? Infrastructure Technology Components in Practice

Automating and monitoring harvesting, data processing and indexing processes






DRIVER Repository Map






DRIVER Admin (Internal) Control Panel I

Monitor repository landscape I






DRIVER Admin (Internal) Control Panel II

Monitor repository landscape II






DRIVER Admin (Internal) Control Panel III

Monitor &

Process

Repository D

ata






DRIVER Admin (Internal) Control Panel V

Check repository

index profile updates






Where do we stand? Linking publications to datasets (Enhanced Publications)






DRIVER – Enhanced Publications

Technology

The demonstrator aggregates scientific web resources via OAI-ORE v0.9 and RDF. XSLT is used to transform these into XHTML. CSS and Javascript do the rest of the presentation. A Java applet is used to dynamically display the relations between resources. Although these relations can be fed to the applet as parameters, they are not yet automatically interpreted from the RDF-serialisation



http://www.openarchives.org/ore/0.9/toc

http://www.openarchives.org/ore/0.9/rdfxml




Driver II Definition Enhanced Publication

An Enhanced Publication (EP) is:a textual publication enhanced with:

research data (evidence of the research) and/or

extra materials (to illustrate or to clarify) and/or

post-publication data (commentaries, ranking)

So: ever developing









Challenges



Topics

2






Challenges towards an International Repositories Infrastructure

Complex matrix, addresses

(min.) five main dimensions

Countries (political, finance, organisational, legal issuesetc.)

Academic disciplines

Content access & usage

Multiple content resource types

Data Models & Technology

Countries

Disciplines

Content type


Content access& usage






Essential: keep all stakeholders and their perspectives in mind

Researchers/disciplines

Research managers

Library Managers

Repository Managers (technical & content)

Computer Scientists

Publishers & further content providers

Service & Infrastructure providers

Funders






The Diversity & Wealth of academic disciplines

EC, Framework 6 Programme: 46 pages, c. 40 entries each

Countries

Disciplines

Content type


Content access& usage






Discipline Schema (Keywords): European Commission

7 main areas






Text, Manuscript

Drawing

Painting

Foto

Film

Radio, TV Broadcasts

Papyri

Cuneiform tablets

Artefacts

Buildings

Maps

Language audio recordings

Content type






Discipline Schema: Deutsche Forschungsgemeinschaft, Germany

4 main domains

HSS

Life Sciences

Natural Sciences

Engineering

14 subdomains






Disciplines and their International Repositories Infrastructures

ExamplesArXiV – Physics, Mathematics, Informatics

PubMed - Life Sciences

CLARIN, OLAC – Linguistics, language archives (datasets, international)

CESSDA – Social Sciences (datasets, international)

DARIAH – Humanities (datasets, international)

RePEc; NEEO – Economics (pre-/postprint publications, international)

METAFOR – Meteorology, Climate research (publications + datasets, international)

MACE - Architecture

IVOA - Astronomy






Issues to be addressed on the way towards an international repositories infrastructure, e.g.

Each discipline – one infrastructure?

Each information type – one infrastructure?

Same data models, technology, same services – different implementation

Same data models and syntax- different semantics

Project specific goals & funding – external liaision & collaboration

Focus on a specific community, a country, a region – cross community, cross-country initiatives






Further issues

Content! Filling repositoriesPublications: business models publishers

Research data: culture of sharing data

2









Challenges



Topics

2






Global Data Networks vs. Global Repository Infrastructure?

Data networks are „neutral carriers“ of information - Digital repositories contain the actual information

Content resources – multiple semantics and formats

Data networks are generic – knowledge infrastructures are disciplin-specific

Cultural issues for disciplines: „You share the network – but not your research data“

Financing: hardware vs. service, business cases






Further, architectural models for infrastructures?






JISC-fundedcontent providers

institutionalcontent providers

externalcontent providers

brokers aggregators catalogues indexes

institutionalportals

subjectportals

learning managementsystems

media-specificportals

end-userdesktop/browser pr

esen

tatio

n

fusion

prov

isio

n

OpenURLlink servers

shared infrastructure

authentication/authorisation (Athens)

institutional profilingservices

terminology services

service registries

identifier services

metadata schema registries

© Andy Powell (UKOLN, University of Bath), 2005

This work is licensed under a Creative Commons LicenseAttribution-ShareAlike 2.0

JISC Information Environment architecture






Ingenieur-Wissenschaft

en

1x

Weitere Wissenschaft

en

0x

Geistes- und Sozial-

Wissenschaften

2x

Natur- und Lebens-

Wissenschaften

4x

Dokum.Server

DigitalisierteSammlungen

Fernsehen/Radio

Forschungs-daten …Mail

Archive

“Open Office”

Programme Suite

(Scholarly workbench)

Publikations-/

Kommunikations-

Dienste(z.B. Wikis)

Daten-konverter

, Rohdaten-analyse

u. Referenz

Informations-

Extraktion,

Semantische

Vernetzung

Disziplin-spez.

Navigationund

Visual.

Datenaggre-

gation und Verlinkung

CAD, CAE&S,

CAM

Rapid Prototypin

gN.N.>>

>

• 3-D-Rekonstruktion von Artifakten

• Handschriften-Transkription• Analyse von

Sprachaufzeichnungen

Kataloge /Datenbanken

Multi-MediaServer

Shared Workspac

e, Kollaborationsdienst

e

2x

1x 0x 1x

Langzeit-archivierung

+ Verfügbarkei

t

1x

• Datenvisualisierung

• ... 1x

2x 6x

7x 1x 4x 3xDefinition von

Standards(Metadaten,

Formate, etc.)

1x

Bildbearbeitung

und -annotat

ion

2xSuche,

Navigation,

Visualisierung, AAR

3x

Nutzungs-statistiken

,Zitationen

1x 6x

Repositories

3x

Daten-transfer

und Workflow-integratio

n

1x

2x 0x 4x 4x0x 1x

Semantic

Social Interact

ion

1x

WissenschaftlerDisziplin-spezifische Werkzeuge und Dienste

Disziplinüber-greifende Dienste & Werkzeuge

Basisdienste

Content

1xD-Grid- und links4science-Workshop, 29. März 2007, Göttingen




wissenschaftliche Communities, Institutionen

disziplinspezifische Werkzeuge und Dienste

virtualisierte Hardwareresourcen

Persistent

Identifier Resolver

LZA-Dienst

e

RepositorySysteme

Info-Extraktion

disziplinübergreifende Werkzeuge und InfrastrukturDienste-katalog, Service Registry

......

Visuali-sierung

Ontology

Registry und

Dienste

Metadata

Registry und

Dienste

Grid-/VO-Such

e

Content





Learn from other sectors, e.g. logistics industry?

openID-center

An open platform for the integration of identification systems

Fraunhofer Institute of Material Flow and Logistics, www.openID-center.de



http://www.openid-center.de/







Challenges



Topics

2






How do we proceed? Four action plans

Organisational structures (Norbert)

Sharing citation data (Les)

‚Repository handshake‘ (Peter)

Identification Infrastructure (Andrew)

=>Aimed to stimulate discussions (drafts have been circulated beforehand)






Purpose of the action plans

The action plans are not necessarily about building infrastructure, but about whatever action needs to be taken so that the components form an infrastructure capable of supporting the use cases.






Organisational structures Suggestions…

Define a clear Statement of Intent/Code of Conduct of the Confederation in relation to Open Access and repository developments

Launch the nucleus of an International Repository Confederation, unifying diverse stakeholders from country networks, disciplinary networks, technology, research managers, funders etc.

Commission an international Inventory Study on disciplinary repository infrastructures

Start a systematic consultation process with discipline representatives, selected national research funders etc. from representative regions all over the world

Draft a roadmap for an International Repository Infrastructure






Sharing citation data Suggestions …

Revive Citebase, and expand its scope to cover a fuller range of open access material, including from OA journals and institutional repositories

Define and implement a common API for citation services such as Citebase, to enable machine query of the data

Implement the updated “CLADDIER trackback protocol” in major repository software as part of the core release

Learn lessons from above that impact on repository and journal practice, eg on metadata consistency. Act on those lessons






‚Repository handshake‘ Suggestions…

1. Establish working group incl. major interested parties

2. Define/refine priority use cases

3. Describe negotiations needed for each use case

4. Identify minimum set of tools and mechanisms

5. Identify test partners

6. Implementation






Identification Infrastructure Suggestions…

Identify relevant (inter)national activities (see briefing materials)

Define in the abstract who are the trusted sources of authority for each of the named entities (eg, a funder is trusted to assert the title of a project)

Identify relevant (inter)national naming and resolution practice (DOIs, Handles, URNs, etc)

Based on above, and relevant trends / plans, define a practical roadmap with milestones

Implement roadmap!






In addition: Organisational structures

Promote complimentary, non-technical actions to technological strands

Sharing citation data <= Engage researchers, learned societies, research managers, research funders to discuss new models for evaluation and reputation schemas

Repository handshake <= Bring together existing and future initiatives (such as the PEER project) to discuss policy and legal frameworks, business models and organisational issues

Identification infrastructures <= Explore how identifiers will be used in practice in research processes, in difderent disciplines, on a broad scale






Outlook: a global network of repository infrastructure hubs?!

[email protected]



mailto:[email protected]

http://www.driver-community.eu/

1 this work is licensed under a creative commons license attribution non-commercial sharealike 2.0...

Documents

cases infrastructure

wider research infrastructure

scientific information

essential components

workshop visions

sword shared metadata

shared vision

unrestricted access