metadata quality in learning repositories: issues & considerations
DESCRIPTION
TRANSCRIPT
Metadata quality in learning repositories: Issues & considerations
Nikos Palavitsinis, Nikos Manouselis, Salvador-Sanchez Alonso
ED-MEDIA 2011, 27/7-1/7, Lisbon, Portugal
Key Concepts
• Learning Repositories
• Resources
• Metadata
Quantity is NOT the problem!
• MERLOT, ARIADNE, LRE for schools, MACE, etc
• Millions of resources on various topics/areas/fields
Image taken from: http://farm1.static.flickr.com/223/475156707_d56e38f251.jpg
Quality?
• Publishers of content are under pressure to ensure that in the plethora of information that is available in today’s web, resources are highly discoverable and highly credible
Walker, 2010 Quality of metadataQuality of content
Truth or…
“Isn't it true, that only librarians like to search? Everyone else likes to find”
Roy Tennant
Images taken from: http://gigglebot.net/wp-content/uploads/wpsc/product_images/librarian.png & http://www.ala.org/img/alonline/computer%20guy.jpg
…dare?
Context of the study
about Organic.Edunet
• Completed eContentPlus project (10/2007 – 10/2010) funded with 2.5M €
• Main objective: Make learning content on Organic Agriculture and Agro-ecology available online, for a range of stakeholders.
• And: Develop educational scenarios to be used within schools & universities
Stats
• 1,5 year of operation• 53.859 visits • 208.963 page views• 41.804 unique visitors• 166 countries• 2.041 registered users• 10.967 resources
Multilingualism
• Multilingual interface (17 languages)– Text-based search– Browsing– Tag-based search– Semantic search
• Multilingual resources (10 languages)– Images, videos, articles, scenarios, etc
• Multilingual metadata (17 languages)– IEEE LOM Application Profile
How did we get there?
Started on June of 2008…
Process (1/2)
• Developed an Application Profile of IEEE LOM to be used to describe resources on Organic Agriculture & Agroecology
Organic.Edunet IEEE LOM AP
Definition of own requirements
Selection of LOM elements
Semantics Refinement
Multiplicity constraints and values
Relationships and dependencies
Required extensions
Application Profile Binding Evaluation of AP
Evaluation phase
Results’ analysis
AP modifications
Process (2/2)
• Developed a Repository Management Tool where resources would be described with IEEE LOM metadata (Confolio – http://oe.confolio.org)
May 2009
• 6.600 resources uploaded in Confolio– 1.100 fully annotated by partners– 5.500 harvested and manually enriched records
• Time to start playing with the data…
First Analysis
• Measured metadata completeness for all IEEE LOM AP elements
• Usage of metadata elements was not as high as desired, but was as low as anticipated!
Results
Improving Metadata Quality
Through a Metadata Quality Assurance Mechanism
Step 1
• Review of metadata records by experts (9/2009)– Direct feedback through e-mail &– Good & Bad metadata practices
Step 2
• Review of metadata records by subject-matter experts (12/2009)– Direct feedback through e-mail – General statistics
January 2010
Time to measure again…
Second Analysis
• Improved completeness for…– Mandatory elements: -0.3% to 82,4%
• Rights.Copyrights and Other Restrictions (82.4%)• Rights.Cost (64%)
– Recommended elements: 47,4% to 82,9%• Rights.Description (82.9%)• General.Keyword (78.1%)
– Optional elements: 13,2% to 79,9%• General.Coverage (79.9%)
What about Educational elements?
• Educational…– Intended End User Role: +66.1% (78.9%)– Typical Age Range: +57.7% (61.5%)– Context: +47.4% (57.6%)– Language: +50.9% (51.2%)– Difficulty: +36.1% (36.2%)– Semantic Density: +35.9% (36.1%)– Interactivity Level: +35.8% (36.1%)– Interactivity Type: +35.6% (35.9%)– Description: +13.2% (14.7%)
Overview
-20,00%
0,00%
20,00%
40,00%
60,00%
80,00%
100,00%
120,00%
Metadata elements
Co
mp
lete
ne
ss
After
Before
Conclusions
• Training experts on a specific field on metadata concepts & principles, improved resulting metadata completeness
• Manual effort is really important
• Further effort for optional elements needed
Limitations
Improvement coming from familiarizing with the annotation process
VSImprovement coming from the mechanisms put
into place
Is it possible to measure separately?
Future Research
• Insert more similar mechanisms throughout the learning resource/repository lifecycle
• Focus more on additional metadata quality metrics on top of completeness
• Extend this framework to other types of resources, i.e. cultural collections, etc.
Metadata quality in learning repositories: Issues & considerations
Thank [email protected]
ED-MEDIA 2011, 27/7-1/7, Lisbon, Portugal