QualitativeResearchinInteractionDesignandUXMattiasArvola

DepartmentofComputerandInformationScience

EthnographyinDesign

2

• http://vimeo.com/6038262

Researchfordesign

3

• Good design requires insight • Insight requires research • Empathy – The ability to take someone else's

perspective • Get an empirical benchmark that everybody agree on,

so that you after the design can show impact • Do it yourself as a designer

4

By Bill Verplank.

KindsofScientificResearch

5

• Basic research • Applied research • Contract research

ScientificResearch

6

• Systematic search for knowledge • The choice of method depends on what the research

question is • New knowledge • Replication • Systematic and critical review of previous research

ScientificResearchvs.ResearchforDesign 7

• Scientific research is: • Statement by statement,

verified by carefully described observations, or references to such

• Criticisable and verifiable • Reproducible • aimed to distinguish more

likely conclusions from less so

• As objective and without local and temporary prejudices as possible

• Cumulative • For common good

• Research for design is: • Good enough based on

resources • Check that you are on the

right track • Find a baseline to show that

you have made improvements

• Inspiration • Find out what you need to

know to move on or get unstuck

Qualityandtrustworthiness

8

• Adds new original knowledge

• Clear conditions and starting point • Purpose & questions • Methods explained

in relation to questions

• Consideration of method alternatives

• Systematic and critical analysis of carefully gathered data

• Possible sources of errors and limitations identified and discussed

• Arguments clearly articulated, and relevant for conclusions

• Not over-generalise

Ethicconsiderationsatthestartofaproject

9

• Choose the method with the least harmful consequences for people, if otherwise equal

• Expected benefits and potential scientific value of the research should should be weighed against harmful consequences

• Benefits and harm for whom? • Openness is the lodestar • Make dependencies and interests clear

Ethics

10

• Completely voluntary participation • Informed consent: verbal or written • Explain risks and possibilities of participating • Respect participants’ integrity and privacy • Confidentiality: coding and masking

• What do participants get out of it? • Let the participants read/see your results

InformedConsent

11

• Awareness of researcher’s identity and purpose • In general announcements inform about purpose,

research topic, and data gathering methods • Seek permission from gatekeepers • Make sure that there is no pressure to participate • If possible, get informed consent from every

individual

Videorecording

12

• Can intrude on private life and integrity, since people can be identified

• Use only when the same data cannot be obtained by other methods (eg. masked still images)

• Respectful and responsible recording • If minors (<15) are recorded: • Consent from both legal guardians • Informed so that the child understands

• Inform: What will be analysed in the video?

EthicsinParticipantObservation

13

• Long term immersion in everyday activities of the people being studied to record conduct under the widest range of possible settings

• Information is the result of dialogical interaction between the researcher and the informants: any interaction may constitute some form of data gathering

• Informed consent process should be dynamic and continuous

• Unfamiliar settings can be sensitive, and have clashes in ethical principles

Datacollection

14

• Take notes in a hardback notebook • Preferably two persons • Specifically if you are a novice

• Sound recording is important • Look out for tech trouble

• Take photos • Memory aid and details about the context of use

15

Inpractice Overarchingprocess

16

1. Kickoff meeting 2. Stakeholder interviews 3. Competitive analysis and literature study 4. User interviews and observations

Stakeholders

17

• People in the organisation that: • Sponsor, build, test, market, sell, and support the

product • Plus: Other people that affect the design of the

product

Kickoffmeeting

18

• Gather project members and stakeholders • Discuss • What is the product? • Who will use it? • What do the users need? • What customers and users are the most

important? • What challenges do we face?

Stakeholderinterviews

19

• General questions • What is your role in relation to this product? • What did you do before you had this role? • What should this product or service be? • Whom is it for? • When should the version we are designing for be

released? Why is it important that it is released then?

• What do you worry about in this project? What is the worst thing that could happen?

Stakeholderinterviews

20

• Cont. • What is effect should this project have one the

business or organisation? • How would you, personally, define success for this

project? • Is there anyone we should talk to that isn’t on this

list? Who are they? • How would you like to be involved in this project,

and how should we contact you?

Stakeholderinterviews

21

• Marketing: • Who are your customers and users today, and how

would you like it to be in five years? • How does this product fit into the overarching

product strategy? • Who are you biggest competitors and what is it

about them that worry you?

Stakeholderinterviews

22

• Cont. • What 3-4 qualities would you like for people to

associate with your company and your product? • What is the current status of your identity? Could

we have a copy of your graphical profile if there is one and examples of how it has been applied?

23

• Engineering questions • What technology decisions have already been

made? What are the reasons for them? • How many engineers and other tech staff are

allocated to this project, and what are their competencies?

• Can you draw a diagram and broadly explain how the existing system works?

Userinterviewsandobservations

24

• Try to understand what is done, why, what goals there are, what mental models people have, how they feel about things

• Search for similarities and differences between people, what different goals they have and if (how) they solve the same task in different ways

• Interview people in different roles • Make sure you have a protocol

Asktheuserswhattheywant?

25

Asktheuserswhattheywant?

30

• Compare to the physician who asks: • What medicin would you like to have?

• Or the architect that just make the finished drawings from the clients’ sketches?

Generalquestions31

Productorservice

Badgeneralquestions Goodgeneralquestions

Business-orientedemailsystem

Whatemailsystemdoyouuse? Weareinterestedinwhatkindofcommunicationyouhavewithotherpeopleduringatypicaldayatwork.Couldyoustartbygoingthroughyourworkdayyesterday?

Digitalcameraforhobbyuse

Whatkindofcameradoyouhave?

Whatroledoesphotographyplayinyourlife?Whatdoyouusuallytakephotosofandhowoften?

Websiteforcamerabusiness

Howdoyouuseourwebsite? 1. Tellmeaboutthelasttimeyouboughtacamera.

2. Tellmeaboutsomeoccasionwhenyouafterafterbuyingacamerafeltthatyouhaveneededextrainfoorhelp.

Serviceforhelpingstudentsfindrelevantsummerjobs

Haveyouhadajobthissummer?Whatkindofjobhaveyouhadthissummer?

?

Interviewthemes

32

• Objects • Find important nouns • Objects from the mental

model • ”I have seen you work

with three types of forms so far: travel order, travel expenses, and purchase order. What other kinds of forms do you use and why?

• Attributes of the objects

• What information does a travel order contain?

• How many fields are there in a travel order?

• Relations between objects • What is the difference

between a travel order and a travel expense form?

• Quantity • How many travel

orders do you usually handle every week?

33

• Actions • Find verbs

• So every time you get a travel order, you first take a copy for archiving before you contact the owner for to verify.

• The reason for an action • Why should the travel

order be archived? • How is an action performed

and described? • The administrator

looks at a post-it note with codes for accounts

when entering the travel order into the accounting system.

• Frequency and priority • Do you ever wait with

archiving a travel order?

• The photocopier is not warm early in the mornings, so it takes forever to take a copy for archiving, so I usually wait with all the copying until later in the day. There’s usually no hurry anyway.

34

• The role of the present product • I have to enter all

information manually into the the accounting system. since travel orders are handed in on paper.

• Relations to other people: • The cost owner • The one who hands in the

travel order • Archivist • …

• Workarounds • Do you ever not archive a

travel order?

• There is this professor who always changes his mind in the last second. I usually don’t archive his travel orders until he actually have left. I would otherwise have to retrieve them every time he changes his mind.

35

• Frustrations • Sighs, facial expressions, body language, gestures. etc.

• It’s a drag with travel orders since you have to feed them into a PDF, but you cannot save the file, just print it, so if you afterwards realise that you’ve done something wrong you have to start all over.

• Skills and experience • How long have you used this product? • You seem to know quite a bit about this technology.

What did you do before starting here? • Study how the interviewee expresses things. Choice

of terminology, etc.

36

• Goals • What characterises a

good experience at work?

• When someone recognises that you’ve done a good job.

• Why is someone doing an action? • I take photos since she

grows up so fast. Now that she is two, I can hardly remember what she was like when she was one. • Track changes

• I want to take a photo that communicates how amazing it is when I see this really striking landscape. • Capture an

experience of something

• When I design a garden, I take photos of what it looks like for future reference for the design work. When the new design is done I take photos of the result for my portfolio to show people that I can do a good work. • Support work and sales

37

EthnographicObservations

UnderstandingUXRequiresQualitativeResearch

43

FreeImages.com/kukirosen

UXResearchInvolvesInterpretation 44

FreeImages.com/JuliaFreeman-Woolpert

TheHermeneuticalCircle45

Parts

Whole

FreeImages.com/francescofattore

CharacteristicsofQualitativeResearch

46

• No sharp boundaries of study objects • Careful contextualisation of the instance • Transferability rather than generalisability • Cooperation between researcher and participants • Purposeful sampling • Alternative interpretations • Suspicion of interpretations • Going beyond what is said and done to what is

intended and motivated

MeasuresforTrustworthynessinQualitativeResearch

47

• Prolonged engagement and observation • Building trust • Learning the culture • Checking misinformation

and distortions • Finding the focus

• Triangulation • Researcher • Theory • Method

• Peer review, debriefing and external audits

• Clarifying what subjective things that have likely shaped the the interpretation • past experiences, • biases • prejudices • orientations

• Member checking • Rich and thick description • Revise working hypothesis • Negative cases with

disconfirming evidence

DataAnalysisbyAffinityDiagramming

48

A special case of thematic analysis conducted in group

1.All workshop participants read the notes, mark important passages, and notes codes in the margin 2.Every person reads their markings aloud and write

them down on a sticky note (code notes on origin and person) 3.Put notes on wall or table and move them around so

that related notes are placed together 4.Discuss what binds the groups of notes together 5.Name the groups: you now have themes/categories

AQualitativeandInterpretativeStance

52

• What is surprising here? • What did we expect to see that did not appear? • What does the data tell us that we didn’t already

know? • What in the data makes us see things differently? • Why does a pattern or anomaly arise? • What theoretical concepts can we use to make sense

of our data?

DevelopUXThemes

53

• Form Statements about: • What do people do?

• How do they do it? What do their actions lead to? • What do people know?

• What is their frame of reference? What habits of mind does it lead to?

• What do people feel? • How are things experienced? What do their experiences mean

for them? • Order them according to their affinity to form themes • What is similar across participants? • What is different? Why?

54

By Bill Verplank.

Insummary

55

• There are differences between scientific research and research for design

• Ethics including informed consent is critical • Understanding UX requires qualitative research into

the interactions, mental models, and experiences people have

• Qualitative research methods includes both observations and interviews

• Analysis and interpretation into UX themes can be made with affinitiy diagrams

www.liu.se

@mattiasarvola