europeana cloud - ingestion and aggregation workshop
DESCRIPTION
Europeana Cloud Kick-Off Meeting, 5 March 2013, The Hague, The Netherlands. By Chiaro LatronicoTRANSCRIPT
![Page 1: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/1.jpg)
![Page 2: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/2.jpg)
europeana cloud
Ingestion and Aggregation Workshop
Chiara LatronicoOperations Officer The European Library
Europeana Cloud Kick-Off Meeting, Den Haag, 04-05 March 2013
![Page 3: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/3.jpg)
Agenda
The European Library Datasets life-cycle and workflows Content ingestion questionnaire Ingestions tools Aggregation and delivery to Europeana Europeana Data Model (EDM) Full-text index Questions
![Page 4: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/4.jpg)
48 National Libraries ~ 40 Research and University Libraries ~ 115 Million Bibliographic Records > 16 Million Digital Objects > 25 Million Pages of Full-text
The European Library
![Page 5: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/5.jpg)
![Page 6: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/6.jpg)
![Page 7: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/7.jpg)
The European Library
Data access point for researchers Combination of bibliographic records and
metadata for digital objects
Aggregator for Europeana Cloud
![Page 8: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/8.jpg)
The European LibraryAggregation into Europeana
![Page 9: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/9.jpg)
The European LibraryIngestion Workflow Content ingestion questionnaire Scheduling of ingestion Datasets ready for harvesting Create case in CRM: case # to provider Harvesting metadata Enhance metadata (VIAF, Geonames, MACS,...) Indexing in acceptance portal E-mail to provider to accept dataset Live index = live portal Delivery to Europeana Enhancing and publishing in Europeana
![Page 10: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/10.jpg)
Content Ingestion Questionnaire Web-form Personal Information
(about the person filling the web-form) Name & surname Job title E-mail address Skype address
Information about Organization Organization name Country Website Type of institution
![Page 11: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/11.jpg)
Content Ingestion QuestionnaireHarvesting Details Which protocol will be used to transfer data?
OAI-PMH File Z39,50 FTP HTTP
Harvesting time and dates preferences
How often dataset(s) will need to be updated? Weekly Monthly Quarterly Annually On demand
![Page 12: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/12.jpg)
Content Ingestion QuestionnaireInformation about dataset(s)
Number of dataset(s) to be ingested
Number of records to be expected
Number of digital objects to be expected
Contact person(s) per dataset(s) Editorial: for collection description Technical: for collection ingestion
![Page 13: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/13.jpg)
Content Ingestion QuestionnaireInformation about Metadata
Metadata standard(s) available to describe objects Marc21 MarcXchange Unimarc ESE EDM METS MODS OAI_DC TEI
Number of formats available per dataset
![Page 14: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/14.jpg)
Content Ingestion QuestionnaireInformation about Metadata
Are the metadata ready? If yes, for which dataset(s) If not, when will they be ready?
Type of digital objects per dataset(s) TEXT IMAGE AUDIO VIDEO
![Page 15: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/15.jpg)
Content Ingestion QuestionnaireInformation about Content
Will content be delivered in addition to metadata?
If yes, for which dataset(s)? If yes, in which format(s)?
Has the content been digitized? If yes, for which dataset(s)? If not, when will the content be available?
![Page 16: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/16.jpg)
Content Ingestion QuestionnaireInformation about Authority
Will authority files be delivered? If yes, for which dataset(s)? If yes, in which format(s)?
Are controlled vocabularies utilized? If yes, which kind?
• Classification• Thesauri• Subject Headings• Other
If yes, for which dataset(s)?
Will full-text be delivered? If yes, for which dataset(s)? If yes, in which format(s)?
![Page 18: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/18.jpg)
SugarCRM aggregation tasks management
SugarCRM Customer Relation Management tool
![Page 19: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/19.jpg)
SugarCRM generation automated reports
SugarCRM Customer Relation Management tool
![Page 20: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/20.jpg)
In SugarCRM Organizations, contacts, datasets, project and more
SugarCRM is utilized for Collections control Ingestion plans Automated reports Cases per specific datasets
SugarCRM Customer Relation Management tool
![Page 21: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/21.jpg)
The European LibrarySystem architecture
![Page 22: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/22.jpg)
UIM Unique Ingestion Management
![Page 23: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/23.jpg)
Dataset in Acceptance Portal
Acceptance Portal Test environment Providers to validate data
Reports via UIM workflows Link Validation Field Validation
![Page 24: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/24.jpg)
Dataset in Acceptance Portal
![Page 25: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/25.jpg)
When Dataset in Acceptance Portal
Create an account onhttp://www.theeuropeanlibrary.org/
Use credential to log-in in acceptancehttp://www.tel.ulcc.ac.uk/acceptance/
Validate data using tabs for Default XML
![Page 26: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/26.jpg)
Dataset in Acceptance Portal
![Page 27: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/27.jpg)
Dataset(s) in Live Index and Portal
When a provider accepts dataset(s) E-mail Dataset(s) ready for live index Dataset(s) ready for Europeana
Dataset(s) indexed into the live portal It takes ~ 24 hrs for dataset(s) to be
searchable into the live portal
![Page 28: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/28.jpg)
Dataset(s) Live in Europeana
When a provider accepts dataset(s)
Dataset(s) delivered to Europeana Europeana publishes live once a month Delivery deadline ~ 21 of each month Dataset(s) searchable in Europeana by
following month
Dataset(s) published live in Europeana E-mail to provider with link to dataset(s)
into Europeana portal
![Page 29: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/29.jpg)
EDM – Europeana Data Model
Europeana Libraries project EDM for library data
Europeana Cloud Project EDM for museum and archive metadata &
content
Delivery in EDM to Europeana
![Page 30: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/30.jpg)
EDM – Europeana Data Model
![Page 31: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/31.jpg)
Europeana Preview
![Page 32: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/32.jpg)
Europeana Preview
![Page 33: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/33.jpg)
Full-Text (OCR)
Continue Full-text indexing
![Page 34: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/34.jpg)
Full-Text (OCR)
Full-text & OCR
URLs to OCR texts into metadata Extraction of Full-text Full-text indexing
![Page 35: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/35.jpg)
Full-Text (OCR)
Continue the work about Full-text Europeana Newspapers Europeana Cloud
![Page 36: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/36.jpg)
Summary
What we would like to have from you Richest possible metadata Content Full-text Authority files or ontologies
![Page 38: Europeana Cloud - Ingestion and Aggregation Workshop](https://reader030.vdocuments.net/reader030/viewer/2022013100/54c670e64a79594a538b4688/html5/thumbnails/38.jpg)
www.theeuropeanlibrary.org