user satisfaction of a hybrid ontology-engineering tool
Post on 24-May-2015
Embed Size (px)
DESCRIPTIONUser Satisfaction of a Hybrid Ontology-Engineering Tool C. Debruyne and I. Ciuciu LNCS 8186, p. 414 ff.
- 1. User Satisfaction of a Hybrid Ontology-engineering Tool Christophe Debruyne and Ioana Ciuciu 13-09-2013 @ META4eS, OTM 2013
2. Overview Motivation Background Usability Test Design User Satisfaction Results and Recommendations Conclusions and Future Work 2 3. Motivation Semantic interoperability and vocabularies for Linked Data initiatives Ontology engineering is far from trivial Requires appropriate methods and tools Hybrid Ontology Engineering platform in the context of the Open Semantic Cloud for Brussels (OSCB) project Evaluate usability and user satisfaction of the hybrid ontology platform 3 4. Background IBM Post-study System Usability Questionnaire (PSSUQ) 4 Information Quality System Use Interface Quality Overall 5. Background Hybrid Ontology Engineering Community promoted to first-class citizen Social interactions are formalized lead to ontology evolution supported by a glossary Concepts are described formally --> with binary fact-types called lexons e.g., informally --> with glosses 5 6. GOSPL Method and Tool 6 7. GOSPL Method and Tool 7 8. GOSPL Method and Tool With respect to the previous study System usefulness scored best Information quality needed most improvement Changes made to the system to tackle problems More (online) documentation Changes to stylesheets RSS to keep track of discussions Additional functionalities Tool support for managing annotations A reputation framework 8 9. Usability Test Design 23 MSc student volunteers part of an ontology engineering course Objective: prototype ontology related to research domain use ontology to annotate reverse engineered existing application existing research information system Evaluate formative and summative user satisfaction of the ontology engineering platform Compare with results from a previous study with similar experiment 9 10. Usability Test Design Tasks on the platform Propose changes to the hybrid ontology Discuss and vote Conclude discussions --> ontology evolution Create and manage communities Annotate information systems Participants 36 students, 9 groups for the ontology engineering experiment Background in computer science or similar 23 students participated in the survey (of which 4 complete groups) 10 11. Summative User Satisfaction Summative evaluation System usefulness and interface quality remained the same Information quality and overall satisfaction improved Formative evaluation Tracking hottest discussions Correcting mistakes Voting mechanism (!) and verbalization of constraints (!) Positive outcome given the problems encountered with application management and the reputation framework 11  Ioana Ciuciu, Christophe Debruyne:Assessing the User Satisfaction with an Ontology Engineering Tool Based on Social Processes. OTM Workshops 2012: 242-251 12. Recommendations Overview of discussions need improvement (Error-) management of application commitments Verbalization Removing voting mechanism 12 13. Conclusions and Future Work Conclusions System usefulness performs best Information quality improved a lot Overall satisfaction remained stable Given the fact that new functionalities posed problems, this can be interpreted as a considerable improvement of the tool Future work Third iteration planned in May 2014 Discussion overview + lightweight user participation 13 14. Questions?