usability assessment of a context-aware and personality-based mobile recommender system

Download Usability Assessment of a Context-Aware and Personality-Based Mobile Recommender System

Post on 28-Nov-2014

194 views

Category:

Internet

2 download

Embed Size (px)

DESCRIPTION

In this paper we present STS (South Tyrol Suggests), a context-aware mobile recommender system for places of interest (POIs) that integrates some innovative components, including: a personality questionnaire, i.e., a brief and entertaining questionnaire used by the system to learn users’ personality; an active learning module that acquires ratings-in-context for POIs that users are likely to have experienced; and a matrix factorization based recommendation module that leverages the personality information and several contextual factors in order to generate more relevant recommendations. Adopting a system oriented perspective, we describe the assessment of the combination of the implemented components. We focus on usability aspects and report the end-user assessment of STS. It was obtained from a controlled live user study as well as from the log data produced by a larger sample of users that have freely downloaded and tried STS through Google Play Store. The result of the assessment showed that the overall usability of the system falls between “good” and “excellent”, it helped us to identify potential problems and it provided valuable indications for future system improvement.

TRANSCRIPT

  • 1. Usability Assessment of a Context-Aware and Personality-Based Mobile Recommender System Matthias Braunhofer, Mehdi Elahi and Francesco Ricci ! Free University of Bozen - Bolzano Piazza Domenicani 3, 39100 Bolzano, Italy {mbraunhofer,mehdi.elahi,fricci}@unibz.it EC-Web - September 2014, Munich, Germany
  • 2. EC-Web - September 2014, Munich, Germany Outline 2 Context-Aware Recommender Systems and their Challenges Related Works STS (South Tyrol Suggests) Usability Assessment and Results Conclusions, Lessons Learned and Future Work
  • 3. EC-Web - September 2014, Munich, Germany Outline 2 Context-Aware Recommender Systems and their Challenges Related Works STS (South Tyrol Suggests) Usability Assessment and Results Conclusions, Lessons Learned and
  • 4. Context is Essential Main idea: users can experience items differently depending on the current contextual situation (e.g., season, weather, temperature, mood) Example: EC-Web - September 2014, Munich, Germany 3
  • 5. Context-Aware Recommender Systems (CARSs) CARS extend Recommender Systems (RSs) beyond users and items to the contexts in which items are experienced by users Rating prediction function is: R: Users Items Context Ratings EC-Web - September 2014, Munich, Germany 4 3 ? 4 2 5 4 ? 3 4 1 ? 1 2 5 ? 3 3 ? 5 2 5 ? 3 5 ? 5 4 5 4 ? 3 5
  • 6. Challenges for CARSs Identification of contextual factors (e.g., weather) that are worth considering when generating recommendations Acquisition of a representative set of contextually-tagged ratings Development of a predictive model for predicting the users ratings for items under various contextual situations Design and implementation of a human-computer interaction (HCI) layer on top of the predictive model EC-Web - September 2014, Munich, Germany 5
  • 7. Challenges for CARSs Identification of contextual factors (e.g., weather) that are worth considering when generating recommendations Acquisition of a representative set of contextually-tagged ratings Development of a predictive model for predicting the users ratings for items under various contextual situations Design and implementation of a human-computer interaction (HCI) layer on top of the predictive model EC-Web - September 2014, Munich, Germany 5 Focus of this research
  • 8. Context-Aware Recommender Systems and their Challenges EC-Web - September 2014, Munich, Germany Outline 6 Related Works STS (South Tyrol Suggests) Usability Assessment and Results Conclusions and Future Work
  • 9. HCI Perspective on RSs Effectiveness of a RS depends not only on the underlying prediction algorithm but also on the proper design of the human-computer interaction (Swearingen and Sinha, 2001) Users interaction with RSs: EC-Web - September 2014, Munich, Germany 7 Recommendation Algorithms Input from user (ratings) Output to user (recommendations) No. of ratings Time to register Details about item to be rated Type of rating scale No. of good recs. No. of new, unknown recs. Information about each rec. Confidence in prediction Is system logic transparent?
  • 10. Usability Assessment of RSs (1/2) Evaluation of the usability of a context-aware and group-based restaurant RS using the System Usability Scale (SUS) (Park et al., 2008) The SUS is a 10-item instrument to measure the users perceived usability of a system (Brooke, 1996) Major finding: the SUS score with 13 test users was 70.58, a rating between ok and good, and corresponding to a C grade, which is an acceptable level of usability EC-Web - September 2014, Munich, Germany 8
  • 11. Usability Assessment of RSs (2/2) Usage of eye tracking, clickstream analysis and SUS to determine the usability of a constraint-based travel advisory system called VIBE (Jannach et al., 2009) Major findings: Average SUS score was 81.5, a rating between good and excellent and corresponding to a B grade, which is a very high level of usability Identification of several usability issues: Inadequate positioning of VIBE on the online portal Too many recommendation results Too little information displayed in the recommendation results EC-Web - September 2014, Munich, Germany 9
  • 12. Context-Aware Recommender Systems and their Challenges EC-Web - September 2014, Munich, Germany Outline 10 Related Works STS (South Tyrol Suggests) Usability Assessment and Results Conclusions, Lessons Learned and F
  • 13. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Welcome screen
  • 14. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Registration screen
  • 15. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Personality questionnaire
  • 16. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Questionnaire results
  • 17. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Active learning
  • 18. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Suggestions screen
  • 19. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Context settings
  • 20. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Details screen
  • 21. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Rating dialog
  • 22. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Routing screen
  • 23. Interaction with the STS System EC-Web - September 2014, Munich, Germany 11 Bookmarked items screen
  • 24. Software Architecture and Implementation Apache Tomcat Server EC-Web - September 2014, Munich, Germany 12 Android Client Spring Dispatcher Servlet Spring Controllers Service / Application Layer JPA Entities Hibernate Objects managed by Spring IoC Container Database JSON HTTP Web Services
  • 25. Software Architecture and Implementation Apache Tomcat Server EC-Web - September 2014, Munich, Germany 12 Android Client Spring Dispatcher Servlet Spring Controllers Service / Application Layer JPA Entities Hibernate Objects managed by Spring IoC Container Database JSON HTTP Web Services
  • 26.

Recommended

View more >