user experience testing in your rfp process - david rosen - highedweb 2014
Post on 02-Jul-2015
980 Views
Preview:
DESCRIPTION
TRANSCRIPT
David Rosen, User Experience Analyst
Try Before You Buy:
User Experience Testing in
Your RFP Process Can
Save You Time and Money
WHO IS THIS GUY?
David Rosen
User Experience Analyst
University of Minnesota
Twitter: @DRosen_UX
Hashtag: #heweb14
Track hashtag: #UAD8
WHAT’S AN RFP?
Requirement Points Vendor 1 Vendor 2 Vendor 3
Does the sandwich have ham
on it?
3 Yes (3) Yes (3) No (0)
What condiments are available
for the sandwich?
3 Mayo,
Mustard (2)
Mustard,
Jalapenos (1)
All condiments
(3)
Can the sandwich be toasted? 2 Yes (2) No (0) Yes (2)
How quickly will the sandwich
be made?
5 10 minutes
(3)
5 minutes (5) 15 minutes (1)
Sandwich RFP
#UAD8
EXPERIENCE CHECK
http://www.cnet.com/au/news/supercomputer-runs-longest-simulation-of-brain-activity-to-date/
#UAD8
HOW DID WE GET THIS STUFF?
Some common reasons software vendors are chosen:
• They were the least expensive
• They gave the most polished demonstration of their product
• They checked the most boxes on the RFP
• They were one of the committee member’s favorites before the process began
• We have already been using their product
#UAD8
https://creativecommons.org/licenses/by-nc/2.0/legalcode
“It’s a very sobering feeling to be up in
space and realize that one’s safety factor
was determined by the lowest bidder on a
government contract.” Alan Shepard
https://www.flickr.com/photos/barry_b/76055201/
#UAD10
REENACTMENT OF THAT MEETING
#UAD8
“No”
http://corporate.ford.com/our-company/heritage/company-milestones-news-
detail/commemorating-150-years-of-henry-ford-visionary
#UAD8
ANOTHER CHALLENGE
https://www.flickr.com/photos/stevendepolo/3797233888/
Going with the
product as-is• Customization is hard
to maintain and
expensive
#UAD8
NOT “VANILLA”
https://www.flickr.com/photos/ginnerobot/3156117675/
#UAD8
IMPLEMENTING “VANILLA”
https://www.flickr.com/photos/stevendepolo/3797233888/
“Vanilla” can be
tasty…
if you let your
users test it
#UAD8
AVOIDING NON-FAT, UNFLAVORED ICE MILK
Case Study:
Survey Tool
#UAD8
ASKING USERS IF THEY WANT FEATURES
“Who Wants a Pony?” image from http://voiceofthemonkey.com/wp-content/uploads/2011/11/Who-Wants-A-Pony.jpg
#UAD8
THE OLD TOOL LOOKED LIKE THIS
#UAD8
GETTING INCLUDED
https://www.flickr.com/photos/glynlowe/7183121469/
#UAD8
VENDOR DEMONSTRATIONS
#UAD8
NO MORE SMOKE AND MIRRORS
Because we were dictating the terms
of the demonstration, we could see
when the vendors were trying to skip
steps, or avoid less pleasant
interactions.
https://www.flickr.com/photos/xtream_i/316644980/
#UAD8
PREPARATION FOR USABILITY TESTING
“We’re reluctant to let you
try this without at least a
three-hour training
session”
#UAD8
PREPARATION FOR USABILITY TESTING
#UAD8
PREPARATION FOR USABILITY TESTING
#UAD8
SOMETIMES IT’S HARD TO WATCH
They made it
look so EASY!
#UAD8
GATHERING ISSUES
• Identifying pain
points
• Quantity of issues
is not a comparison
tool
#UAD8
NEXT STEPS
#UAD8
NEXT STEPS
#UAD8
NEXT STEPS
#UAD8
WHAT WE GAVE TO THE VENDOR
Recommendations
– A description of what we wanted to either understand better or have changed
Alert Training/Support
– Training and Help Desk teams will need to be aware of this issue
Prepare Vendor Request
– A request for a change in order for the solution to fit the need
Response Required
– We are expecting an explanation of this issue
Submit OIT Request
– This may require configuration on our part
Hold for Further Review
#UAD8
WHAT WE GAVE TO THE VENDOR
We understand that some of these tasks may or may not be standard,
but they were included as a means to evaluate the usability and
intuitiveness of the system. In this way, we are able to gain some
insight into configuration setting issues, areas that may require extra
training, and/or items that need to addressed by you, the vendor.
Attached is a workbook outlining the issues that were experienced
and that we would like you to respond to. Please review and enter
your response in the space provided. In the columns to the right, you
may indicate if any of these options are applicable.
#UAD8
WHAT WE HEARD FROM ONE OF THE VENDORS
“A response was required from us on each item, and since we were motivated to win the contract we made commitments to address the vast majority of issues. Now that all items targeted for the short term have been addressed, we know our product has become easier to use.”
#UAD8
WHAT WE HEARD FROM SOME OTHER VENDORS
http://memecrunch.com/meme/15QYJ/uhhhh-no
#UAD8
UX HELPED BUILD A BUSINESS CASE
#UAD8
LESSONS LEARNED
#UAD8
http://educationalaspirations.com/category/assessment-data/
LESSONS LEARNED
Vendors don’t understand
what we are doing by
“usability testing”
They also think that we’re
“difficult”
#UAD8
http://www.flickr.com/photos/79538062@N00/3732589905
LESSONS LEARNED
Access to the
finalists’ software
cannot be optional
#UAD8
LESSONS LEARNED
Vendor Demonstrations
need a strong moderator
from the selection team
#UAD8
https://www.flickr.com/photos/leshaines123/6909108380/
LESSONS LEARNED
Testing is best after
vendor demonstrations
are complete
#UAD8
IF YOU REMEMBER NOTHING ELSE
1. WE. SAVE. MONEY.
– We’ve had a preview of implementation
– Our training and help desk teams are
prepared ahead of time for known
issues
2. There are fewer surprises in a situation
where surprises are expensive
3. This makes vendors better vendors and
partners
#UAD8
Go do it!
1. Use the stuff you’d ask the
vendors anyway
2. You can do this on the cheap
with limited resources
#UAD8
BOOKS TO GET STARTED
#UAD8
THANK YOU!
Questions?
David Rosen
User Experience Analyst
University of Minnesota
Email: rose0414@umn.edu
Twitter: Drosen_UX
Web: z.umn.edu/usability
top related