uxpa 2013: effectively communicating user research findings

© 2012. ELECTRONIC INK, INC. ALL RIGHTS RESERVED. 1Effectively Communicating User Research Findings

Presented by Jim Ross

7/10/2013

Effectively Communicating User Research Findings

Principal of Research & StrategyElectronic Ink@anotheruxguy

UXPA 2013 - UX Fundamentals track

10 July 2013

© 2012. ELECTRONIC INK, INC. ALL RIGHTS RESERVED.Effectively Communicating User Research Findings 2

Why this topic?

7/10/2013

Why talk about communicating user research findings?


Why this topic?

7/10/2013

Anyone can write a report, right?


Why this topic?

7/10/2013

Anyone can create a PowerPoint presentation, right?


Why this topic?

7/10/2013

Have you ever given a presentation when it seems like everything goes right…


Why this topic?

7/10/2013

But few of your recommendations are implemented.


Why this topic?

7/10/2013

Or no one reads your long report.


Why this topic?

7/10/2013

Or your presentation puts the audience to sleep.


Why this topic?

7/10/2013

Or you don’t get the reaction you expect.

You’re wrong!

We could have done that ourselves!

We already knew that!

Source: Flickr – CT State Library

http://www.flickr.com/photos/ctarchives/4109430127/sizes/l/in/photostream/


Why this topic?

7/10/2013

Or they make changes instead of waiting for your deliverable.

Source: Flickr – Blue Oxen Associates 2009

http://www.flickr.com/photos/blueoxen/4016426157/in/photostream/


Why this topic?

7/10/2013

Or you hand off the deliverable and leave the project, and everyone interprets the research any way they wish.


Why this topic?

7/10/2013

Communicating user research findings effectively can be challenging.


Why this topic?

7/10/2013

Communicating user research findings effectively can be challenging.•Raise your hand if you’ve ever…


Why this topic?

7/10/2013


- Written a long, boring report


Why this topic?

7/10/2013


- Written a long, boring report- Given a long, boring presentation


Why this topic?

7/10/2013

To communicate user research findings effectively , you have to:• Explain

• Educate

• Entertain

• Persuade

• Guide


Why this topic?

7/10/2013

What we’ll talk about• Handling audience reactions to deliverables

• How to choose the best deliverable format

• Advantages and disadvantages of different types of research deliverables

• Tips for creating effective research deliverables

• Communicating research beyond deliverables


Why this topic?

7/10/2013

What we won’t talk about• Writing skills

• Presentation skills


Why this topic?

7/10/2013

Who am I to talk about this?• 13 years of experience as a UX researcher and designer

• Created many deliverables

• Made many mistakes

• Learned many lessons


Audience reactions to deliverables.

7/10/2013


Audience reactions to deliverables

7/10/2013

Positive reactions• “This is great, but if I presented this to our executives, it

would be like setting off a bomb in the room.” – Client



7/10/2013

Unfortunately, we don’t always get ideal reactions.



7/10/2013

Negative reactions• Yeah yeah fine, now where are the designs?•We already knew that!•We could have done that ourselves.• You’re wrong!•Why didn’t you mention this problem?• These recommendations aren’t specific enough.• But you only talked with 12 people.•Where did you get these participants?• That’s a training issue.•We can’t fix that.



7/10/2013

Yeah yeah fine, now where are the designs?



7/10/2013

Yeah yeah fine, now where are the designs?Why you get this reaction

• The time between the kickoff and the research deliverables is long.

• Uninvolved client/team members become impatient and anxious.



7/10/2013

Yeah yeah fine, now where are the designs?How to prevent this reaction

• Set expectations about the research activities at the kickoff meeting.

• Keep the client/project team involved.

• Provide frequent updates.



7/10/2013

We already knew that!



7/10/2013

We already knew that!Why you get this reaction• They usually do know much of the information.

• Research findings often sound like obvious, common sense.

• They misunderstand the purpose of research.



7/10/2013

We already knew that!How to prevent this reaction• Clarify to clients:

- You may already know a lot of this information, but it’s important for the design team to understand it.

- There’s value in gathering, validating, and publishing this information.

- You bring value through your analysis and expert, outside perspective.



7/10/2013

You’re wrong!



7/10/2013

You’re wrong!Why you get this reaction• You report the participants’ “incorrect” beliefs.

• They think you’re supposed to be a subject-matter expert.

• You’ve revealed your technology ignorance.



7/10/2013

You’re wrong!How to prevent this reaction• Clarify the need for everyone to combine their expertise:

- You’re a user experience expert.- Clients are business and subject matter experts.- Users are experts in their own needs.



7/10/2013

You’re wrong!How to prevent this reaction• Explain that

- “Incorrect” information is valuable.- Research findings are iterative, and you welcome clarifications.



7/10/2013

We could have done that ourselves!



7/10/2013

We could have done that ourselves!Why you get this reaction• Findings often seem like common sense.

• We make it seem easy – anyone can watch people and ask questions.



7/10/2013

We could have done that ourselves!How to handle this reaction• Turn it around on them: “Then why haven’t you?”

• Remind them why they haven’t done it themselves:- Lack of time- Lack of knowledge and skills- Lack of outside perspective



7/10/2013

But you only talked with 12 people.



7/10/2013

But you only talked with 12 people.Why you get this reaction• They’re more familiar with quantitative research.

• They don’t understand qualitative research.



7/10/2013

But you only talked with 12 people.How to prevent this reaction• Explain at the beginning of the project:

- The difference between qualitative and quantitative research- The value of observing behavior



7/10/2013

Where did you get these participants?



7/10/2013

Where did you get these participants?Why you get this reaction• We deliver bad news.

• It’s easier to blame the participants than to believe the bad news.



7/10/2013

Where did you get these participants?How to prevent this reaction• Involve the client and project team in recruiting by having them:

- Provide input on the types of people to recruit- Approve the screener - Approve the list of recruited participants

• Use video clips of multiple participants with the same reaction.



7/10/2013

Why didn’t you mention this problem?



7/10/2013

Why didn’t you mention this problem?Why you get this reaction• You didn’t specifically mention someone’s pet-peeve.

• They don’t understand the types of findings different research methods provide.



7/10/2013

Why didn’t you mention this problem?How to prevent this reaction• At the beginning of the project, explain the types of findings

your research will provide.- For example, usability testing versus field studies



7/10/2013

These recommendations aren’t specific enough.



7/10/2013

These recommendations aren’t specific enough.Why you get this reaction• Some expect a specific, actionable recommendation for each

problem.

• But that’s not always possible or appropriate in a research deliverable.



7/10/2013

These recommendations aren’t specific enough.How to prevent this reaction• Explain up front that

- Some recommendations will be specific and actionable.- Others will be general and may require more research or

design exploration.- It’s important not to prematurely box in the design direction.



7/10/2013

That’s a training issue.



7/10/2013

That’s a training issue.Why you get this reaction• It seems like an easy way to get out of fixing problems.

• It seems easier to change people than to change technology.



7/10/2013

That’s a training issue.How to handle this reaction• Explain that

- Usability minimizes the need for training.- Usability problems only need to be fixed once, while training is

ongoing.- Training doesn’t solve inefficiency of use.- Training doesn’t help infrequent users.- It’s easier to change technology than to change people.



7/10/2013

We can’t fix that.



7/10/2013

We can’t fix that.Why you get this reaction• The same issues that caused the problem remain in the way of

fixing it.- Technical issues- Regulatory and legal issues- Conflict with business goals- Organizational turf- Executive whims



7/10/2013

We can’t fix that.How to prevent this reaction• Consult your own technical resources.

• Present to the decision makers.

• Keep your recommendations flexible.



7/10/2013

Your experience•What kinds of negative reactions have you received?•How did you handle those?



7/10/2013

The overall lessons• Negative reactions arise from misunderstandings.

• Set the right expectations from the beginning.

• Communicate throughout the project.


Choosing a deliverable format.

7/10/2013


Choosing a deliverable format

7/10/2013

Considerations• What type of research was it?

• Who is the audience?

• Will you remain on the project or go away?

• How soon do they need the findings?



7/10/2013

What type of research was it? • Interface-related (usability testing, expert review, etc.)?



7/10/2013

What type of research was it? • Or field studies?



7/10/2013

Who is the audience?• High-level executives or designers/developers?



7/10/2013

Who is the audience?• A client or an internal project team?



7/10/2013

How soon do they need the findings?• As soon as possible?



7/10/2013

How soon do they need the findings?• Or no rush?



7/10/2013

Will you remain on the project?• The deliverable can be more high-level.



7/10/2013

Or will you deliver the findings and go away?• The deliverable has to be detailed and clear enough to stand in

for you.


Types of research deliverables.

7/10/2013

Advantages and disadvantages


Types of research deliverables

7/10/2013

• No deliverable at all

• Quick findings

• Annotated designs

• Reports

• Presentations

• Findings and Recommendations Matrix

• Other types of deliverables: personas, scenarios, workflow diagrams, etc.



7/10/2013

No deliverable at all



7/10/2013

No deliverable at allAdvantages• Saves time and money

• Encourages team participation in the research through:- Observation- Collaborative analysis and discussion



7/10/2013

No deliverable at allDisadvantages• No documentation for future use

• Knowledge only exists in people’s heads

• Nothing tangible to show for the research



7/10/2013

Quick findings



7/10/2013

Quick findingsAdvantages• Saves time and money

• Can discuss findings soon after research

• Ideal for quick iteration

• Some documentation exists for future reference



7/10/2013

Quick findingsDisadvantages• No detailed analysis

• Findings remain high-level

• Details can get lost



7/10/2013

Annotated designs



7/10/2013

Annotated designsAdvantages• Combines the best aspects of

reports and presentations- Visual

- Explains problems more clearly by pointing to them

- Appears more approachable and less dense than a report



7/10/2013

Annotated designsAdvantages• Combines the best aspects of reports and

presentations- Text

- Provides more explanation than a presentation

- Stands on its own as a deliverable and makes sense later



7/10/2013

Annotated designsDisadvantages• Only suitable for interface-related findings

• Difficult to present



7/10/2013

Reports



7/10/2013

ReportsAdvantages• Provide the most detail

• Stand alone

• Serve as an archive

• Allow your presentation to be more high-level

• Appear substantial



7/10/2013

ReportsAdvantages• Most flexible format – a container for anything:

- Text- Images- Annotated screens- Quotes- Personas- Diagrams- Visualizations- Links to video or audio clips



7/10/2013

ReportsDisadvantages• Can be too long

• Take longer to create

• People often don’t read them

• Also have to create a presentation



7/10/2013

Presentations (two types)• Presentation in addition to a report

• Presentation as the only deliverable



7/10/2013

Presentation in addition to a report



7/10/2013

Presentation in addition to a reportAdvantages• Can keep it high-level

• Don’t have to cover everything

• Can optimize for presenting – less text, more visual



7/10/2013

Presentation in addition to a reportDisadvantages• Need to create two deliverables

• Those who attend the presentation may not read the report



7/10/2013

Presentation as the only deliverable



7/10/2013

Presentation as the only deliverableAdvantages• Takes less time

• Costs less

• More people attend a presentation than read a report



7/10/2013

Presentation as the only deliverableDisadvantages• Has to cram in everything or leave out details

• Has to be detailed enough to stand alone

• Is not the best format to convey detail



7/10/2013

Presentation as the only deliverableYou either end up with: • A text-heavy presentation

• Or you have to leave out information



7/10/2013

Presentation as the only deliverableSo how do you solve this dilemma?• Don’t create a presentation as the only deliverable



7/10/2013

Presentation as the only deliverableSo how do you solve this dilemma?• Put the detail in an appendix



7/10/2013

Presentation as the only deliverableSo how do you solve this dilemma?• Create a presentation version AND a detailed, standalone

version



7/10/2013

Presentation as the only deliverableSo how do you solve this dilemma?• Record your presentation (screen and audio) so others can view

it later



7/10/2013

Presentation as the only deliverableSo how do you solve this dilemma?• Provide annotated designs instead of a presentation



7/10/2013

Presentation as the only deliverableSo how do you solve this dilemma?• Create a high-level presentation and a findings and

recommendations matrix to contain the details…



7/10/2013

Findings and recommendations matrix



7/10/2013

Findings and recommendations matrixAdvantages• Provides an easy way to see the recommendations

• Contains the detail, allowing the presentation to remain high-level

• Can take the place of a report



7/10/2013

Findings and recommendations matrixDisadvantages• Another deliverable to create, which takes additional time



7/10/2013

Other types of research deliverables• Personas

• Scenarios

• Workflow diagrams

• Task analysis diagrams

• Customer journey maps

• And more…



7/10/2013

Other types of research deliverablesAdvantages• Often communicate better than words alone

• Easier, quicker, and more interesting to read than a report

• People refer back to them more often than re-reading a report



7/10/2013

Other types of research deliverablesDisadvantages• Additional deliverables to create, which takes additional time

• Don’t stand alone – additional deliverables are necessary



7/10/2013

Your experience• Which types of deliverables have you found most or least

useful?

• Other types of deliverables?


Creating effective research deliverables.

7/10/2013


Creating effective research deliverables

7/10/2013

For an effective research deliverable• Use a good template.

• Make it look good.

• Explain the methods.

• Note the positive aspects.

• Visualize findings: using photos, quotes, audio and video clips.

• Visualize recommendations.

• Prioritize the findings and recommendations.

• Guide the audience on the next steps.

• Get it reviewed.

• Ensure that the right people attend.



7/10/2013

Use a good template.


What a research deliverable needs to do

7/10/2013

Make it look good, so that people will read it and pay attention.



7/10/2013

Explain the methods you used.



7/10/2013

Note the positive aspects.



7/10/2013

Visualize the findings.



7/10/2013




7/10/2013

Use photos to help explain your methods.



7/10/2013

Use photos to show what you found in the research.



7/10/2013

Use quotes.



7/10/2013

Use quotes, but don’t overdo it.



7/10/2013

Use quotes – longer, multiple quotes work better in reports.



7/10/2013

Use quotes sparingly in presentations.



7/10/2013

Use quotes sparingly in presentations, or create two versions.



7/10/2013

Show video clips and audio clips.



7/10/2013

Visualize the recommendations.



7/10/2013

Visualize recommendations with mockups.



7/10/2013

Visualize recommendations by showing examples.



7/10/2013

Prioritize the findings and recommendations.



7/10/2013

Guide the audience on the next steps.



7/10/2013

Get it reviewed.



7/10/2013

Ensure that the right people attend the presentation.

Source: Flickr – Charles Roffey

http://www.flickr.com/photos/charlesfred/444699189/


Beyond deliverables.

7/10/2013


Is “delivery” the best way to impart research findings?

Beyond deliverables

7/10/2013

Source: Flickr – Intangible Arts

http://www.flickr.com/photos/57382003@N00/2659211728/in/photolist-53Za15-5sSZ4s-5ucnW8-5Epkaw-5Gt1JV-5Gt1Va-5Gt21x-5GxiEm-5GxiHd-5GxiXf-5Gxj2C-5WST3v-6f6DBZ-6faHBN-6faHES-6faHJS-6i992V-6kMZJF-6qUQ74-6MYvcR-6TQBap-6Zj4Fp-71K91J-76aDmh-7bYgRf-7f3d4T-7f3d9V-7f3ddX-7f3dfH-7f76p5-7f76ub-7f76yU-7f76D5-7f76Ew-7qpb7P-7qyBop-7v6ygw-eE2hvu-9gRJgB-baN73z-8kpK8k-8Wo26j-7PNco6-8Wo26f-ai9whe-8wdRjD-9XM1MH-cpmgih-c1QF6y-8ux5rR-7EuVyh


Beyond deliverables

7/10/2013

Problems with delivery• Research is less impactful without experiencing it firsthand.

• People only take in and remember some of the information.

• Deliverables aren’t an effective substitute for you after you’re gone.


Beyond deliverables

7/10/2013

The solution: collaborative research• Involve the clients and project team in

- Planning- Observing- Analysis


Involve the clients and project team in planning.

Beyond deliverables

7/10/2013


Involve the clients and project team in observing the research.

Beyond deliverables

7/10/2013

Source: Flickr – Rosenfeld Media 2009

http://www.flickr.com/photos/rosenfeldmedia/4218842283/


Involve the clients and project team in analyzing the results.

Beyond deliverables

7/10/2013

Source: Flickr – Rosenfeld Media 2009Source: Flickr – Jason DeRuna

http://www.flickr.com/photos/rosenfeldmedia/4218842283/

http://www.flickr.com/photos/53249700@N00/2280189105/in/photolist-4tuyMg-4F1uQV-4F1uUg-4WhARc-4WhASt-5izPpD-5Tav89-5Zf5hF-6M61cR-7iyih3-9h1pNz-8DjyDM-9rLUok-dMsW8U-dMsW7W-9h4xYw-9zsav5-9zs9K7-9zpcuP-9zpbC4-9zs9TN-8BvqHG-8BsiCV-ehmXnk-ehsLvq-ehndJz-axJ816-axLNyC-axLR57-axLRqJ-axJ8nF-axLQTN-axJ8yZ-axJ9HK-axLPBL-axJ9yR-axLQbY-axLPNJ-axLNLA-axJ9dv-axLNoJ-axLPqw-axJ7fr-axJ72F-92jdxf-7Ctv4A-7Ctv2h-7Ctv6h-7CpFfr-aqvaB2


Beyond deliverables

7/10/2013

Collaborative research gives the team:• More empathy for the users

• A deeper understanding of the problems and recommended solutions

• A stronger sense of responsibility to solve the problems


But still provide a deliverable to document the research.

Beyond deliverables

7/10/2013


Beyond deliverables

7/10/2013

Remain involved after presenting the deliverable:• To answer questions and provide guidance

• To collaborate on the design

• To conduct further research


Conclusion.

7/10/2013


Conclusion

7/10/2013

To effectively communicate user research:• Set the right expectations.

• Communicate throughout the research process.

• Conduct collaborative research.

• Create a deliverable.

• Remain involved.


Questions and Discussion.

7/10/2013

uxpa 2013: effectively communicating user research findings

Technology

user research findings

handling audience reactions

ideal reactions

long report

boring presentation

boring report

powerpoint presentation

youre wrong