using feedback from data consumers to capture quality information on environmental research data
TRANSCRIPT
Session: Approaches to Improved Collection and Dissemination of Earth Science Data Quality InformationAGU 2015 | San Francisco | 14-18 December 2015
MINERAL RESOURCES FLAGSHIP
Anusuriya Devaraju and Jens Klump([email protected])
Using Feedback from Data Consumers to Capture Quality Information on Environmental Research Data
Images: Anett Moritz, bpaquality.wordpress.com
Outline• Definitions (User Feedback, Research Data, Data Quality)• Motivation• Goals & Solutions• Summary
Outline• Definitions (User Feedback, Research Data, Data Quality)• Motivation• Goals & Solutions• Summary
• Feedback refers to information about reactions to a product.• Feedback Types
User Feedback
4 |
User experience (assessment and
usage)General (comment, how-to, suggestion, dissuasion)
RatingRequirements
(feature, content)
Image by Commonwealth Fund
Research Datasets
5 |
Research data are facts, observations or experiences on which an argument, theory or test is based.1
1The University of Melbourne draft policy on the Management of Research Data and Records
Data Quality• The quality of data is often examined based several categories.*• Quality = Fitness for use (Wang & Strong, 1996)
• Appropriate for use or meets user needs • Datasets are often used for a purpose different from the intended one.• Inadequate understanding of the purpose may lead to poor quality of derived
data.
6 | * R.Y. Wang, D.M. Strong, Beyond accuracy: what data quality means to data consumers, 1996.
(Image: https://wq.io/research/quality)
Outline• Definitions (User Feedback, Research Data, Data Quality)• Motivation• Goals & Solutions• Summary
Quality Measures in PracticeData quality descriptions supplied by data providers might be incomplete or may only address specific quality aspects.
• Accessibility, e.g., persistent identification, file format• Completeness, e.g., required metadata• Compliant with community standards• Private and confidentiality concerns • Review code, e.g., check and verify replication code • Link to other research products
Scholarly and data journals may take a role in ensuring data quality, but this mechanism only applies to data sets submitted to the journals.
8 | Reference: http://www.ijdc.net/index.php/ijdc/article/view/9.1.263/358
User Feedback and Data Quality
9 | Image : http://whartonmagazine.com/blogs/women-and-leadership-moving-forward/
Data consumers may complement existing entities to assess and document the quality of published data sets.Data quality information may be gathered via a user feedback approach.
Discovered issues, data application and derived datasets
PROVIDER CONSUMER
Data creation & publication
Existing Feedback Mechanisms
10 |
Research Data Portals Feedback Mechanism
Research Data Australia (RDA) General feedback form, and user contributed tags for datadiscovery
CSIRO Data Access Portal Refer to the email of the data collector in the metadata
TERN Data Discovery Portal General contact form
Australian Ocean Data Network Portal (AODN)
General contact form and portal help forum
Atlas of Living Australia (ALA) UserVoice feedback portal
OzFlux Data Portal Email link (for all inquiries and assistance)
National Marine Mammal Data Portal General feedback form
Urban Research Infrastructure Network Email link for general inquiries, Social media buttons for distribute the link of a data set.
Examples of research data portals and their feedback mechanisms
Why Does Quality Information From Users Matter?
Feedback information from data consumers gives other users and data providers a better insight into application and assessment of published data sets.
11 |
An example of corrected groundwater chemistry data sets provided by the Geological Survey of South Australia and correction notes produced by (Gray
and Bardwell, 2015).
Data providers may use the feedback information to handle erroneous data and improve existing data collection and processing methods.
12 |
Why Does Quality Information From Users Matter?
An issue tracking component installed as part of the Terrestrial Environmental Observatories (TERENO) data portal
Outline• Definitions (User Feedback, Research Data, Data Quality)• Motivation• Goals & Solutions• Summary
GoalsDevelop a systematic and reusable approach to 1. Capture user feedback on the application and assessment of
research datasets (with identifiers)2. Link feedback information to actual data sets3. Support discovery of research data using feedback information.
14 |
Feedback Application Server
Dat
a P
orta
l with
Fe
edba
ck P
lugi
n
Linked Data & SPARQL Clients
Feedback Data Store (MySQL)
RESTFeedback Web Service
RDFSPARQLD2R Server
D2R Engine
JSON RDF
User Feedback System
15 |
Feedback from users may be gathered :• Implicit (automated
tracking of data activities)• Explicit (predefined input
templates)
1 Gather feedback
2 Store feedback
3 Publish feedback
The prototype of the user feedback system
16 |
1. Gather Feedback1
17 |
A relational data model representing key aspects of user feedback:• Feedback types and
contributors• Target data and
context• Supporting
documents
2. Store Feedback2
18 |
3. Publish Feedback3
A high level overview of the W3C PROV model
Image : http://www.w3.org/TR/2013/NOTE-prov-primer-20130430/
19 |
3. Publish Feedback
Feedback published as Linked DataEntities and agent involved in an error report
feedback activity
3
Conclusions• We developed a prototype of the user feedback system to capture
quality information (assessment and application) of research datasets from users.
• The prototype supports retrieval and publication of user feedback information by combining a number of open-source technologies.
• The feedback records are made available as Linked Data to promote integration with other sources on the Web.
• The W3C PROV model is used to represent the provenance of user feedback information.
20 |
What’s Next?• Track data application and assessment in a development
environment
21 |
Thank You…
22 |
IMPORTANT ASPECTS:VALUE, EASY, FAST..