DR. VASSILIS-JAVED KHAN WORKSHOP: PUTTING MOBILE SERVICES INTO CONTEXT WARSAW JUNE 2011

Download DR. VASSILIS-JAVED KHAN  WORKSHOP: PUTTING MOBILE SERVICES INTO CONTEXT WARSAW JUNE 2011

Post on 11-Jan-2016

30 views

Category:

Documents

1 download

DESCRIPTION

EVALUATING USER EXPERIENCE OF MOBILE, CONTEXT-AWARE APPLICATIONS IN AUGMENTED VIRTUALITY. DR. VASSILIS-JAVED KHAN WORKSHOP: PUTTING MOBILE SERVICES INTO CONTEXT WARSAW JUNE 2011. FINANCIAL SPONSOR: RAAK Pro scheme of:. RESEARCH PARTNERS:. - PowerPoint PPT Presentation

TRANSCRIPT

Slide 1DR. VASSILIS-JAVED KHAN WORKSHOP: PUTTING MOBILE SERVICES INTO CONTEXTWARSAW JUNE 2011EVALUATING USER EXPERIENCEOF MOBILE, CONTEXT-AWARE APPLICATIONSIN AUGMENTED VIRTUALITY# I would like to begin with introducing a scenario which frames the main idea of my presentation. Imagine a mobile phone application developer in one of his projects he wants to develop an application which enables supermarket shoppers to receive ads on their mobile phone. Design idea is to trigger ads based on the location of the shopper within the supermarket. It seems a good idea but testing this idea with actual users to find out the pros and cons of it is not a trivial task.In my talk I will present an extension to laboratory studies and an alternative to filed test studies for helping developers such as the one I mentioned to test context-aware applications. We have already conducted already 3 studies, using that setup, for evaluating the user experience of mobile, context-aware applications in augmented virtual environments1Dr. Vassilis-Javed Khan Dr. Paul Ketelaar Dr. Marnix van Gisbergen Dr. Andrs LuceroDr. Koos Nuijten Arief Ernst HhnNils DesleFINANCIAL SPONSOR: RAAK Pro scheme of:RESEARCH PARTNERS:# MESenior Lecturer, Academy of Digital Entertainment, NHTV Breda University of Applied Sciences,The NetherlandsPhilips Research, EindhovenVodafone R&D, MaastrichtMSc., CEID, UPATRAS.GRPhD., Dept. Industrial Design, Eindhoven University of Technology, TUE.NL# INDEXPosition the approach we take compared to Lab & Field studiesPresent our setupPresent 3 studies of the evaluation of a mobile, location-based app in our setupConclusionsChallenges / Future work# ULTIMATE GOAL OF CONTEXT-AWARE APPSProvide added value to userHow to evaluate users experience while taking context into account?Lab vs. Field test# LABHighly controllableUseful for usability-related studiesEcological valid?Context?# FIELD TESTContextDifficult to conductLogistic & technical challengesDifficult to observeFew participantsQualitative# Hansen et al. (2006) thought that finding a place to place their public displays would be trivial it actually proved to be a major problem. Trying to generalize their findings, the authors report no less than 19 items of a checklist to consider before actually deploying a prototype in the real setting. The authors categorized those items into three categories: hardware, software and user setting.7EVALUATION IN AUGMENTED VIRTUALITYContextEasy to manipulateLarge numbers of participantsCan include mobile phones, tablets, other interaction devicesCan observe behaviorKhan, V.J., Nuijten, K. & Desl, N. (2011). Pervasive Evaluation Application within Virtual Environments. Proceedings of International Conference on Embedded Computing and Communication Systems, PECCS 2011, 261-264# The alternative method we are exploring is to evaluate applications in virtual environments augmented with mobile phones8MIXED REALITYReal EnvironmentAugmented RealityAugmented VirtualityVirtual EnvironmentMilgram, P. & Kishino, F. (1994) A taxonomy of mixed reality visual displays. IEICE transactions on information systems, E77-D(12)VIRTUALITY CONTINUUM# that which is being augmented is not some direct representation of a real scene, but rather a virtual world, one that is generated primarily by computer9OUR SETUP: CAVE4 back-projection screens3.5 x 2.6 m.Illusion of a continuous 360o viewHead-tracking with 4 Wii-MotesUser wears a headband with infrared LEDHuman joystickV.E. created in Maya & rendered using the OGRE 3D engineTwo V.E.Supermarket,City in an islandMore to come# # # # OUR SETUPThe virtual supermarket is a generic supermarket, based on common denominators found in typical Dutch supermarketsContains: isles, freezers, checkout counters, etc., all rendered with photo realistic graphicsAudio: typical supermarket music, typical announcements~20% of the shelves are filled with productsAndroid app, Bluetooth communication with V.E.# NOVELTYImmersive User ExperienceNearly photo-realistic vs. low resolution graphicsCAVE vs. V.E. presented on monitorsResearch focused on UX, going beyond effectiveness and efficiencyi.e. perceived intrusiveness, shopping behavior, feeling of presence, etc.Capturing actual behavior vs. opinionMarketing studies used exclusively scenario-based methods# 3 STUDIESLocation-based adsStudy 1: 27 participants (pilot)Study 2: 70 participantsStudy 3: 70 participantsWanted to measure: Perceived intrusiveness of adsHas been reported:Perceived intrusiveness of ads leads to ad irritation1High values of ad irritation lead to ad avoidance11Li Hairong, S. & Lee, J., 2002. Measuring the intrusiveness of advertisements: Scale development and validation. Journal of Advertising, 31(2):3747# 16STUDY1Ad: Chewing gumPushControlled for:Product involvementAttitude towards advertising in generalMeasurement: Perceived intrusivenessResults: Location-congruent ads lead to less ad intrusiveness than location-incongruent ads (virtual location)GROUP1: SAW AD WHILE AT SOUP SHELFGROUP2: SAW AD AT COUNTER (NEXT TO CHEWING GUM)Hhn, A.E., Khan, V.J., Ketelaar, P., Nuijten, K., Gisbergen van, M., Lucero, A. (2011-to appear).The Effect of Location on Perceived Intrusiveness of Mobile Ads. Proceedings Chi-Sparks 2011# measured several control variables that have proven to affect the evaluation of advertising and location based advertising 17STUDY2Differences with study 1:More participantsOpen taskExact same location for both groupsProducts on shelf changed for groupsSoup ad for both groupsHhn, A.E., Khan, V.J., Nuijten, K., Gisbergen van, M., Lucero, A., Ketelaar, P. (2011-to appear).The Effect of Location on Perceived Intrusiveness of Mobile Ads. Proceedings ICORIA 2011Same result with study 1Location-congruent ads lead to less ad intrusiveness than location-incongruent ads (virtual location)# While standing at a counter a shopper is almost done with purchasing products and is probably on a different state while being busy in thinking, searching and choosing for products. One might hypothesize that the supermarket counter can be a location in which participants would perceive less intrusiveness compared to next to a shelf precisely because they have finished their shopping and are now in a more relaxed state. Therefore, we decided to pick the exact same location for the larger scale experiment and instead change the products that were visible on the shelf. 18STUDY3Same setup with Study 2Ad of SoupLocation-BasedMeasurement: perceived intrusivenessDifferences:Same products on shelves for both groupsOne group saw a social recommendation along with the location-based adResults: ad with social recommendation is perceived as more intrusiveGROUP1:Along with the ad they saw a social recommendationGROUP2:Saw just the adStudy conducted by: Stephan van de Kruis# CONCLUSIONSLocation does positively affect perceived intrusiveness of mobile adsSocial recommendations would not help mobile, location-based adsDemonstrated a V.E. where UX research of mobile, context-aware systems can be performedAdjustments of the V.E. and context are done easier, are more controllable and faster compared to an experiment in a real life settingReproducibilityIf there is an initial halo-effect it does not affect the actual perceptions# The halo-effect is generally defined as the influence of a global evaluation on evaluations of individual attributes of a person (Nisbett&Wilson, 1977) 20CHALLENGES / FUTURE WORKFill supermarket with a representative product set(~20K products)Head tracking with MS KinectHave a more complete phone appShop with mobile phoneInclude personalization profileAbility to interact with productsUse log and video input for analyzing behaviorSocial context: AvatarsCorroborate qualitative findings of previous research endeavors & explore new research directionsValidation studies# SUMMARY3 studies of mobile, location-based ads evaluation in augmented virtual supermarketACKNOWLEDGEMENTSPhotos & Graphics: Arief Ernst Hhn, Paula IsphordingCONTACTkhan.j@nhtv.nlvjkhan.com@v_j_khanResearch partners:Financial sponsor: RAAK Pro scheme of:#

Recommended

View more >