ucd awareness research
TRANSCRIPT
The Effect of an MIT Engineering Education on User-Centered Design Skills
Li Bian, Blaine Ziegler, Simon Shum
Agenda
Introduction Motivation/Significance Methods Results Discussion Conclusion
Introduction
What is User-Centered Design (“UCD”)?– Accounts users’ needs, wants, limits– Multi-stage process with user-
experimentation
Literature Review
User-Centered System Design: New Perspectives on Human-Computer Interaction (Norman, 1986)– “stressed the need to fully explore the needs and desires of
the users and the intended uses of the product.”
User-Interface Design for Motorola Vehicle-Navigation System: A Case Study of Product Development (Marcus, 1986)– simplified navigation systems for safety purposes
Education of Interface Design – an Interdisciplinary Approach (Joshi, 2004)– In 2004 - “The HCI practice within these organizations [IT
companies] is rapidly becoming mainstream.”
– By 2009 -“The top quarter of Indian IT companies will be well integrated into HCI Design.”
Motivation/Significance
MIT Electrical Engineering & Computer Science (“Course 6”) education
– lacks courses that emphasize User-Centered Design
Help Course 6 students
– recognize shortcomings
– improve overall design philosophy
Objective
Research Question: – Do Course 6 students put less
consideration into users’ interests than do other MIT students?
Hypothesis: – Experiencing the largely tech-oriented
curriculum of a Course 6 education decreases the consideration that designers put into users’ needs
Methods
Overview:– Design scenario, open-ended
questions
– Graded subjectively on 1-5 scale
– Results compared for MIT Course 6 students vs. non-MIT Course 6 students
Methods
User-Centered towards Question 1
User-Centered towards Question 2
User-Centered towards Question 3
Average User-Centered towards all questions
Course 6
NonCourse 6
Experimental Design
Dependent
Independent
Methods
Sampling– Student center
Internal Validity– responses graded blind
– responses graded independently by 3 team members, averaged
Design Scenario
Ed Smith is a retired auto mechanic. He’s in his 70s now, and lives alone in a small, run-down apartment across the alley from the garage he used to work at. He has a son from an ex-wife, but he hasn’t spoken to either in quite some time. It’s not that he’s angry at them, he just hasn’t gotten around to it, and his friends are worried that he’s becoming too shut off from the world. They want him to get email so that he can easily communicate and share photos with other people.
He has a computer that a friend’s children set up for him, but he hasn’t even turned it on in weeks. And there’s no way he’ll ever be able to use something as complicated as Microsoft Outlook. His friends want you to design a brand new, fully-featured email client to help him stay in touch.
Questions
1. Describe how you would begin the process of designing a solution to the problem presented.
2. After coming up with your first working prototype, what steps do you take to make the second iteration even better?
3. After you have completed your design and it has been implemented, what do you look for in deciding if your design was a success?
Grading Model
1 – I would explore the various technologies relevant to the field, like network protocols and database storage, so that I could find creative ways to improve on them.
3 – I would use a commercial email client as a template, and try to trim away unnecessary features.
5 – I would interview Ed and even shadow him for a few hours if he allows it, so that I could learn more about his level of computer expertise, his budgetary concerns, and what he wants to gain from using email.
Grading Examples
Question 2: After coming up with your first working prototype, what steps do you take to make the second iteration even better?
A: Test it until it crashes, fix the bugs, run it again. Iteratively search for and correct mistakes, while checking if the spec is being met.
1!
Grading Examples
Question 1: Describe how you would begin the process of designing a solution to the problem presented.
A: Find out why he doesn’t keep in touch. Some other emotional reason? Find out how comfortable he is with technology, i.e., computers, typing, mice, etc.
5!
Grading Examples
Question 3: After you have completed your design and it has been implemented, what do you look for in deciding if your design was a success?
A: If the guy remarries his ex-wife.
5!
Grading Examples
Question 1: Describe how you would begin the process of designing a solution to the problem presented.
A: Make the application very user friendly, and incorporate features that may make using it more enjoyable for him.
1,2,3 = 2
Results
Course 6 (n = 22)
Non-Course 6 (n = 27)
t-stat p-value
Q1 (2.47, 1.18) (3.05, 1.30) 1.619 0.056
Q2 (2.47, 1.05) (3.44, 1.08) 3.182 0.00130
Q3 (2.56, 0.95) (3.23, 1.14) 2.229 0.015
Overall (2.50, 0.82) (3.24, 1.00) 2.811 0.00358
Results
H0: Course 6 students score the same as non Course 6 students
HA: Course 6 students score lower than non Course 6 students
• Reject H0 for questions 2, 3, as well as overall, at the 0.05 significance level
• Cannot reject H0 for question 1 at the 0.05 significance level
Discussion
Results suggest that the tech-oriented curriculum of a Course 6 education decreases the consideration that designers put into users’ needs
Interesting/Surprising Results: Higher variability in Non-Course 6 Students Question 1 - insignificant at the .05 level:
– Course 6 students might be good at the initial stages, but fail to recognize users’ needs in later stages
Outliers in the Course 6 student data: Experience in industry? Double major?
Discussion
Limitations & Confounds– External Validity - Results applied to this case
study only– Sampling Bias/Data Collection
Different forms of data collection Getting people to participate - People who fill
out surveys are more “people oriented”– Grading Methodology– Survey Bias– Gender representation in Course 6
males possibly less user-centered than females (higher % of males in Course 6)
Conclusion
Importance of User-Centered Design Skills – Course 6 Students place less emphasis
user’s interests than do other MIT students in a design
Further Steps & Future Research:– add more factors:
gender, school, age group– diversify case study, grading criteria, and
course distribution
Questions?
Thanks