Open Metrics for Open Repositories20x20 Pecha Kucha delivered at OR2012 on Tuesday 10th July based on the

unpublished paper available from http://opus.bath.ac.uk/30226/

Brian Kelly1, Nick Sheppard2, Jenny Delasalle3, Mark Dewey1, Owen Stephens4, Gareth J Johnson5 and Stephanie Taylor1

1 UKOLN, University of Bath, Bath, UK {B.Kelly, M.Dewey, S.Taylor}@ukoln.ac.uk2 UKCoRR/Leeds Metropolitan University, Leeds, UK {N.E.Sheppard@leedsmet.ac.uk}

3 University of Warwick, Warwick, UK {J.Delasalle@warwick.ac.uk} 4 Consultant, UK {owen@ostephens.com}

5 UKCoRR/University of Leicester, Leicester, UK {gjj6@le.ac.uk}

OA still in its infancy?

• Need for metrics• Understand how IRs are

being used• Policy decisions• Technical infrastructure• Business data• Not just research• Open Data / OER • Open landscape

• New, alternative metrics• Use in evaluation and assessment• Exploit opportunities • Usage / bibliographic data • Make openly available• Adoption of altmetrics• The BOAI at 10 – Alma Swan (video)

The Finch report

• Executive Summary• Gold vs Green• Estimated additional £50-60M• Green has failed!• What does failure look like?• What would success look like?• Evidence

Practice what you preach

• Open it up• Article level (ideal)• Software / fragmentation• Wrangle your software• Need for accurate

aggregation• Technical challenges

Julian Kleyn (2008)http://www.flickr.com/photos/juliankleyn/2604103124/

The Institutional Picture

• Total number of records• Full-text / metadata only• Raw figure / Proportion of

total • Metadata records accessed• Full-text downloaded• How records are accessed– Search/browse – Search engine referral– Referral from aggregation– Social media referral

EPrints

• Article level metrics• Open interface• www.eprints.ac.uk/cgi/irstats.cgi• Use as advocacy tool

Dspace(GA plug-in)

• More reliable than existing DS plugins

• lost stats/inflated page views from robots

• Stats available at three levels: – item, – collection – repository

http://ukcorr.org/2012/03/22/using-google-analytics-statistics-within-dspace-2/https://github.com/seesmith/Dspace-googleanalytics

The Big Picture

• OPuS - Bath (EPrints)• Leeds Metropolitan• Proportion as measure?• RepUK / aggregations• Software issues– EPrints– DSpace– Also ran

Number of Full Text• Few IRs can easily provide data• RepUK (national aggregation service)• CORE• Discrepancies• Proportion as measure?• Full text only exemplars (19)• Developing Research Management

Infrastructure• CRIS

Brunel, City University London, Cranfield, Loughborough, QMUL, RHUL, School of Advanced Study, Aberdeen, Birmingham, Cambridge, East London, Edinburgh, Exeter, Hull, St Andrews, Stirling, Surrey, Warwick, Nottingham

(Should be HTML?)

RepUK

• Technical challenges• OpenDoar• OAI-PMH• Data visualisation• Trends revealed

DepositsOai_dc node occurencesOai_dc node occurences over timeOai_dc percentabe node occurences over timeMetadata quality – validity of XML and DRIVER compliancedc:subject classification system occurrencedc:language occurrence across UK repositoriesMatches against IANA Mime Types

(Edinburgh Research Archive)

154 IRs aggregated (28/06/12)• 397790 PDF • 20578 MS Word• 44018 jpeg • 122764 html

The Researchers’ Requirements

• May have effect on deposit choice• PLoS/Arxiv display article-level metrics• Numbers aggregated (locations / versions)• PLoS show visitor numbers from PMC• May not deposit in IR to maximise numbers

at preferred location• Few publishers display article-level metrics• Opportunity for IRs to engage with authors• Measuring impact (REF, RCUK)• altmetrics

Third Party Services

• Mechanisms to harvest metadata/full-text • OAI-PMH• The search party didn’t turn up?• Slow growth• Google (Scholar)• CiteSeerX, CORE cache copies • Affects repository metrics• PIRUS2 / article level metrics

COnnecting REpositories (CORE)

• Repository Analytics• Increase the visibility of content• Provide applications to aid

content discovery• Enrich metadata

Conclusions

• Yes it’s complicated!• Stats don’t tell the

whole picture• Greatest value for

operational / strategic purposes

• Senior management• IR managers should be

proactive• Culture of openness