mobile and pervasive computing - 3 pervasive computing user studies presented by: dr. adeel akram...
TRANSCRIPT
![Page 1: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/1.jpg)
Mobile and Pervasive Computing - 3
Pervasive Computing User Studies
Presented by: Dr. Adeel AkramUniversity of Engineering and Technology,
Taxila,Pakistan
http://web.uettaxila.edu.pk/CMS/SP2014/teMPCms
A.J. Bernheim Brush, [email protected]
![Page 2: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/2.jpg)
Outline
Introduction
Types of Studies
Study Design
Example
Ten Mistakes to Avoid
Your Turn
![Page 3: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/3.jpg)
Dr. A.J. Bernheim Brush ?
Ph.D. in Computer Science
Researcher at Microsoft Research
Technology for families, workgroups (HCI/CSCW/Ubicomp)
Researching on various “studies”
http://research.microsoft.com/en-us/groups/CSCW/
![Page 4: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/4.jpg)
Why do a user study?
Bad Reasons
You don’t have anything else to do…
You think it’s a requirement to get your paper accepted
It might be fun to see how people use your stuff
Good Reasons
You’re designing stuff for people to use. Wouldn’t it be nice to know how they might use it?
There is a new domain or Scenario and you want to observe user behavior
![Page 5: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/5.jpg)
The Reality….
User Studies are a lot of work Really, more work than you ever expected
Understanding your GOAL is critical
Studying current behavior: What are people doing now?
Proof of concept: Does my novel technology work for people?
Experience using a prototype: How does using my prototype change people’s behavior or allow them to do new things
![Page 6: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/6.jpg)
Outline
Introduction
Types of Studies
Study Design
Example
Ten Mistakes to Avoid
Your Turn
![Page 7: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/7.jpg)
Many types of studiesFo
rmat
ive/
Cur
rent
Beh
avio
r
EthnographySurveyInterviewsFocus GroupsLogging
Pro
toty
ping
/
Pro
of o
f
Con
cept
“Discount Usability”Heuristic EvaluationsLab studies
Learn about a domainDesign inspiration
Typ
es
Goal
Sum
mat
ive/
Usa
ge
Expe
rien
ce
Iterative TestingFix & understand your prototype
Lab studiesField StudiesLogging
Does it work?Can people use it?How does it compare to other designs/prototypes?
![Page 8: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/8.jpg)
Many types of studiesFo
rmat
ive/
Cur
rent
Beh
avio
r
EthnographySurveyInterviewsFocus GroupsLogging
Pro
toty
ping
/
Pro
of o
f
Con
cept
“Discount Usability”Heuristic EvaluationsLab studies
Learn about a domainDesign inspiration
Typ
es
Goal
Iterative TestingFix & understand your prototype
Lab studiesField StudiesLogging
Does it work?Can people use it?How does it compare to other designs/prototypes?
Sum
mat
ive/
Usa
ge
Expe
rien
ce
![Page 9: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/9.jpg)
Surveys
Easy to get large number of people
Design guidance
Evaluation of deployed system
Surprisingly hard to do well….
Phrasing of questions
Biased responses
Pilot your survey!
![Page 10: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/10.jpg)
“Discount Usability” (Jakob Nielsen)
If you are building prototype, very useful to get feedback from users early and often
Low-cost, Quick, Iterative, Small N, Identify big problems
Lo-fi prototypes Paper version can be very helpful
People feel ok telling you to change stuff
No feedback on responsiveness etc.
Heuristic evaluation Experts review the interface based on list of heuristics
Cognitive walk-through Determine tasks, review and ask questions for each task
WARNING: Not typically a research contribution
http://www.nngroup.com/people/jakob-nielsen/
![Page 11: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/11.jpg)
Lab Studies
Bring participants into a lab
Minimize variability
Hypothesis testing
Independent variables between conditions
Interface A vs. Interface B
Control condition?
Measure dependent variable
Speed of use, … (Quantitative)
Preference, …. (Qualitative)
http://www2.sta.uwi.edu/usability/facilities.htm
![Page 12: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/12.jpg)
Field study
In-situ (“not on your turf”)
Trading “control” for realism
Think carefully if this is important
All types
current behavior,
proof-of-concept
prototype
WARNING: Often good idea to do lab study before field study
![Page 13: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/13.jpg)
How do I choose?
You might not…
What is your goal/research contribution?
LINC: An Inkable Digital Calendar
http://research.microsoft.com/pubs/69395/cscw2006lincvideo.pdf http://research.microsoft.com/en-us/um/people/ajbrush/LINCCSCW2006Video.wmv
![Page 14: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/14.jpg)
Research Question/Goal
Your Research Question is critical
There are very few right decisions, instead decisions you need to justify
Bad: How will families use SPARCS?Better: Do sharing suggestions promote sharing?
http://research.microsoft.com/apps/pubs/default.aspx?id=73932
![Page 15: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/15.jpg)
Outline
Introduction
Types of Studies
Study Design
Example
Ten Mistakes to Avoid
Your Turn
![Page 16: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/16.jpg)
Study Design
What type of study?
What will your participants do during the study?
Give them hardware? Give them tasks?
What type of participants you should recruit?
What data will you collect?
How long will the study be?
Where can you skimp during the study….
What absolutely has to work (if it’s a prototype)
![Page 17: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/17.jpg)
Human Subjects
Ethical treatment of people in the study
Respect—remember that they are doing YOU a favor
Participants can stop at any time
Consent Forms
Privacy Statements
Compensation
Your organization should have some review process
THIS IS IMPORTANT! (U.S. Department of Health & Human Services)
What will they be doing during the study
How will you report on what they did
http://www.hhs.gov/ohrp/assurances/index.html
![Page 18: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/18.jpg)
PILOT!!!!
Allow time to a pilot study,
Run through the entire methodology with volunteer participants.
Uncover system problems
Uncover experimental design problems
Uncover problems with materials
IMPORTANT IMPORTANT IMPORTANT
![Page 19: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/19.jpg)
Participant Profile
What types people do you want to participate?
All the same?
![Page 20: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/20.jpg)
Participant Profile
What types people do you want to participate?
All the same?
All different?
![Page 21: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/21.jpg)
Participant Profile
What types people do you want to participate?
All the same?
All different?
People with extreme characteristics vs. “normal” people
![Page 22: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/22.jpg)
Participant Profile
What types people do you want to participate?
All the same?
All different?
People with extreme characteristics vs. “normal” people
Consider
Age
Gender
Technology experience
….
![Page 23: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/23.jpg)
How many participants?
This can be a difficult question
What claims you are making
What is feasible
Some will drop out!
![Page 24: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/24.jpg)
Length
How long should the study be?
Another difficult question….
Novelty
What are you asking participant to do?
Long term use vs. feasibility
![Page 25: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/25.jpg)
Data Collection
How will you understand if you answered your research question?
Quantitative Data (logs, timing, # errors, …) Qualitative Data (interviews, surveys, …)
TRIANGULATE between multiple sources
![Page 26: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/26.jpg)
Logging
You must have a plan going in about how you will use the log data
Risk of forgetting to log something important
Logging too much can create an analysis nightmare
Make a list of questions you expect to answer with log data
How many times did they upload a photo?
How many days did they use your prototype?
![Page 27: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/27.jpg)
Logging
Question How/Notes1. How often was SPARCS running, when did it stop? # of logfiles, LogfileStart(0)
LogfileClosed
2. Is automatic previewing setup? When Settings:IsPreviewing
3. Is automatic publishing setup? When Settings:IsAutoPublishing
4. How many people’s feeds were they subscribed to? SPARCS_Feed ADD as well as number of unique Create View
5. When did they add or subtract feeds? SPARCS_Feed, add or remove tags
6. What type of calendar feed are they using Calendar_Type, Settings:CalendarSource tag
![Page 28: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/28.jpg)
Qualitative Data
Surveys
Pre-survey
Post condition
Post-survey
Experience Sampling Methodology
Small set of question
Event triggered, random….
Diaries
Interviews (semi-structured, structured)
Observation
![Page 29: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/29.jpg)
Analyzing Qualitative Data
Affinity Diagramming
Coding of Comments
Inter-rater reliability
![Page 30: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/30.jpg)
What if it doesn’t work?
There are many ways a study can fail
Technical problems
They don’t like it
Nobody but you cares about usability problems
Brace yourself for this
Figure out what has to work and skimp other places
Comparison between prototypes
Pilot studies
![Page 31: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/31.jpg)
Outline
Introduction
Types of Studies
Study Design
Example
Ten Mistakes to Avoid
Your Turn
![Page 32: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/32.jpg)
Example
CareNet Consolvo, Roessler,
Selton Intel Research Seattle
![Page 33: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/33.jpg)
The CareNet Display: Lessons Learned from an In
Home Evaluation of an Ambient Display
Sunny Consolvo, Peter Roessler, & Brett E. Shelton
Intel Research Seattle November ’04
![Page 34: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/34.jpg)
November '04 The CareNet Display * Intel Research Seattle 34
Meet Rita…
Rita is one of the elders we worked with 83 years old Lives alone Conditions: Mild dementia & Type 2 Diabetes
(takes insulin)
She is struggling to maintain her independence, but needs help…
![Page 35: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/35.jpg)
November '04
The CareNet Display * Intel Research Seattle 35
Rita’s Care Network
Drastic life changer:daughter Hannah
Significant contributor:son Simon
Significant contributor:son Zack
Neighbor
Daughter-in-law
Part-time professional caregiver
Son
Hannah’s boyfriend
Daughter-in-law
![Page 36: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/36.jpg)
November '04
The CareNet Display * Intel Research Seattle 36
The focus of this work
Improve the quality of life for ALL care network members, including the elder
Help members coordinate care activities
Ensure elder gets care she needs
Give time back to overburdened members
![Page 37: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/37.jpg)
November '04 The CareNet Display * Intel Research Seattle 37
The CareNet Display
Interactive digital picture frame always-on Internet access from
wireless GPRS card
Augments photo with updates meds, meals, outings, activities,
mood, falls, & shared calendar
Goal: help local care network members provide daily care
![Page 38: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/38.jpg)
November '04
The CareNet Display * Intel Research Seattle 38
Participants & Methodology
13 participants from 4 care networks In home Wizard of Oz deployments
i.e., phone calls instead of sensors
Interviews & surveys No special instructions on:
how or when to use it, or where to place it
![Page 39: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/39.jpg)
November '04
The CareNet Display * Intel Research Seattle 39
Some Results
Drastic Life Changers:
time back, less stress, “meaningful” conversations
Significant Contributors:
increased awareness of what others contribute
Peripherally Involved Members:
gave them something to talk about
Elders:
changed minds about how open they would be
![Page 40: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/40.jpg)
November '04 The CareNet Display * Intel Research Seattle 40
Challenges: Sensor data with a “human touch” Participants are afraid sensor data will be too impersonal How can we provide this “human touch” without adding to the
responsibilities of already overburdened network members?
![Page 41: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/41.jpg)
November '04
The CareNet Display * Intel Research Seattle 41
Early & in situ evaluations
Early evaluations help us learn where to concentrate our technology development
In situ evaluations provide us with valuable insight into how new tools are used and what effects they have on the intended users
![Page 42: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/42.jpg)
November '04
The CareNet Display * Intel Research Seattle 42
For more information…
Publications: S. Consolvo, P. Roessler, & B.E. Shelton, "The
CareNet Display: Lessons Learned from an In Home Evaluation of an Ambient Display," Proceedings of the 6th Int'l Conference on Ubiquitous Computing: UbiComp '04 (Sep 2004), pp.1-17.
S. Consolvo, P. Roessler, B.E. Shelton, A. LaMarca, B. Schilit, & S. Bly, "Technology for Care Networks of Elders," IEEE Pervasive Computing Mobile and Ubiquitous Systems: Successful Aging, Vol. 3, No. 2, (Apr-Jun 2004), pp.22-29.
![Page 43: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/43.jpg)
Lots of other examples…
SuperBreak Morris, Brush, Meyers
Microsoft Research
ButterflyNet Yeh, Liao, Klemmer, Guimbretière, Lee, Kakaradov, Stamberger,
Paepcke Stanford, Maryland
Wearable Jersey DisplayPage and Moere
University of Sydney
UbiFitConsolvo, McDonald, Toscos,
Chen, Froehlich, Harrison, Klasnja, LaMarca, LeGrand,
Libby, Smith, LandayIntel Research Seattle
![Page 44: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/44.jpg)
Outline
Introduction
Types of Studies
Study Design
Example
Ten Mistakes to Avoid
Your Turn
![Page 45: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/45.jpg)
#10 Not enough people involved
It takes a village
Huge time commitment, 24 hour support for field studies
Do
Include multiple people
![Page 46: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/46.jpg)
#9 Not being prepared
Don’t want to realize at the end of the study that you forgot to do something important
Do
Study design document
Research Question
Participant Profile
Methodology (within/between etc)
Timeline
Data collection
Pilot studies
WARNING: If you do not plan, you plan to fail!
![Page 47: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/47.jpg)
#8 Not enough time for logistics
Everything takes time…. (more than you think)
Recruiting
Installation (e.g. 16 people X 2 hours = )
Support
Do
Allow plenty of time for the study
Have enough people
![Page 48: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/48.jpg)
#7 Seeing what you want to see
We all want our prototypes to be popular
Do:
Think carefully about how you discuss the technology with participants
Avoid leading questions
Stay close to the data and find multiple support for conclusions
Use neutral language, “Tell me more,” “ummm”
Don’t get defensive
![Page 49: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/49.jpg)
#6 Being judgmental
Users are always “right”
They may say things that are offensive, objectionable, etc.
Do:
Leave your opinions at home
Collect feedback
Use neutral language
![Page 50: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/50.jpg)
#5 Not monitoring usage
Don’t want to find out at the end of the study that people were not using the prototype
Do:
“phone home” messages
Server logs
Have a plan about when and how you might intervene
![Page 51: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/51.jpg)
#4 Not collecting a variety of data
Hard to understand logs without interviews
Hard to know if what someone says in an interview matches their use without logs
Do
Interviews
Surveys
Logs
Support for your findings in multiple ways gives you (and reviewers) more confidence
![Page 52: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/52.jpg)
#3 Prototype is not robust enough
Technical challenges are a distraction
You are making a “product”
Do
Usability studies
Pilots with friendly folks
![Page 53: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/53.jpg)
#2 Making inappropriate claims
One of the biggest and most common mistakes
Your study is one data point
Do:
Being careful with your language
Include a “limitations” section
Tell a clear story (don’t have to tell every finding)
![Page 54: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/54.jpg)
#1 Not having a clear research question
Hard to explain the choices you made
Hard to explain your findings
May end up focusing on usability problems
Do
Have a clear research question
![Page 55: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/55.jpg)
Acknowledgments
Sunny ConsolvoIntel Research
Seattle
Beverly Harrison Intel Research
Seattle
Ed CutrellMSR India
![Page 56: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/56.jpg)
User Studies Are Fun!
Always learn something Often surprising! Inspire you
![Page 57: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/57.jpg)
Questions???
![Page 58: Mobile and Pervasive Computing - 3 Pervasive Computing User Studies Presented by: Dr. Adeel Akram University of Engineering and Technology, Taxila,Pakistan](https://reader030.vdocuments.net/reader030/viewer/2022032604/56649e5e5503460f94b575c8/html5/thumbnails/58.jpg)
References
Brush, Ubiquitous Computing Field Studies in Ubiquitous Computing Fundamentals. A.J. Brush, [email protected]
Green & Salkind Using SPSS for the Windows and Macintosh: Analyzing and Understanding Data (nth Ed.)
APA Publication Manual http://www.apastyle.org/manual/index.aspx
HCI textbooks Preece, J., Rogers, Y., and Sharp, H. Interaction design: beyond
human-computer interaction. 2002. John Wiley & Sons, Inc