ucd awareness research

22
The Effect of an MIT Engineering Education on User-Centered Design Skills Li Bian, Blaine Ziegler, Simon Shum

Upload: lichakuli

Post on 24-May-2015

516 views

Category:

Technology


1 download

TRANSCRIPT

Page 1: UCD Awareness Research

The Effect of an MIT Engineering Education on User-Centered Design Skills

Li Bian, Blaine Ziegler, Simon Shum

Page 2: UCD Awareness Research

Agenda

Introduction Motivation/Significance Methods Results Discussion Conclusion

Page 3: UCD Awareness Research

Introduction

What is User-Centered Design (“UCD”)?– Accounts users’ needs, wants, limits– Multi-stage process with user-

experimentation

Page 4: UCD Awareness Research

Literature Review

User-Centered System Design: New Perspectives on Human-Computer Interaction (Norman, 1986)– “stressed the need to fully explore the needs and desires of

the users and the intended uses of the product.”

User-Interface Design for Motorola Vehicle-Navigation System: A Case Study of Product Development (Marcus, 1986)– simplified navigation systems for safety purposes

Education of Interface Design – an Interdisciplinary Approach (Joshi, 2004)– In 2004 - “The HCI practice within these organizations [IT

companies] is rapidly becoming mainstream.”

– By 2009 -“The top quarter of Indian IT companies will be well integrated into HCI Design.”

Page 5: UCD Awareness Research

Motivation/Significance

MIT Electrical Engineering & Computer Science (“Course 6”) education

– lacks courses that emphasize User-Centered Design

Help Course 6 students

– recognize shortcomings

– improve overall design philosophy

Page 6: UCD Awareness Research

Objective

Research Question: – Do Course 6 students put less

consideration into users’ interests than do other MIT students?

Hypothesis: – Experiencing the largely tech-oriented

curriculum of a Course 6 education decreases the consideration that designers put into users’ needs

Page 7: UCD Awareness Research

Methods

Overview:– Design scenario, open-ended

questions

– Graded subjectively on 1-5 scale

– Results compared for MIT Course 6 students vs. non-MIT Course 6 students

Page 8: UCD Awareness Research

Methods

User-Centered towards Question 1

User-Centered towards Question 2

User-Centered towards Question 3

Average User-Centered towards all questions

Course 6

NonCourse 6

Experimental Design

Dependent

Independent

Page 9: UCD Awareness Research

Methods

Sampling– Student center

– E-mail

Internal Validity– responses graded blind

– responses graded independently by 3 team members, averaged

Page 10: UCD Awareness Research

Design Scenario

Ed Smith is a retired auto mechanic. He’s in his 70s now, and lives alone in a small, run-down apartment across the alley from the garage he used to work at. He has a son from an ex-wife, but he hasn’t spoken to either in quite some time. It’s not that he’s angry at them, he just hasn’t gotten around to it, and his friends are worried that he’s becoming too shut off from the world. They want him to get email so that he can easily communicate and share photos with other people.

He has a computer that a friend’s children set up for him, but he hasn’t even turned it on in weeks. And there’s no way he’ll ever be able to use something as complicated as Microsoft Outlook. His friends want you to design a brand new, fully-featured email client to help him stay in touch.

Page 11: UCD Awareness Research

Questions

1. Describe how you would begin the process of designing a solution to the problem presented.

2. After coming up with your first working prototype, what steps do you take to make the second iteration even better?

3. After you have completed your design and it has been implemented, what do you look for in deciding if your design was a success?

Page 12: UCD Awareness Research

Grading Model

1 – I would explore the various technologies relevant to the field, like network protocols and database storage, so that I could find creative ways to improve on them.

3 – I would use a commercial email client as a template, and try to trim away unnecessary features.

5 – I would interview Ed and even shadow him for a few hours if he allows it, so that I could learn more about his level of computer expertise, his budgetary concerns, and what he wants to gain from using email.

Page 13: UCD Awareness Research

Grading Examples

Question 2: After coming up with your first working prototype, what steps do you take to make the second iteration even better?

A: Test it until it crashes, fix the bugs, run it again. Iteratively search for and correct mistakes, while checking if the spec is being met.

1!

Page 14: UCD Awareness Research

Grading Examples

Question 1: Describe how you would begin the process of designing a solution to the problem presented.

A: Find out why he doesn’t keep in touch. Some other emotional reason? Find out how comfortable he is with technology, i.e., computers, typing, mice, etc.

5!

Page 15: UCD Awareness Research

Grading Examples

Question 3: After you have completed your design and it has been implemented, what do you look for in deciding if your design was a success?

A: If the guy remarries his ex-wife.

5!

Page 16: UCD Awareness Research

Grading Examples

Question 1: Describe how you would begin the process of designing a solution to the problem presented.

A: Make the application very user friendly, and incorporate features that may make using it more enjoyable for him.

1,2,3 = 2

Page 17: UCD Awareness Research

Results

Course 6 (n = 22)

Non-Course 6 (n = 27)

t-stat p-value

Q1 (2.47, 1.18) (3.05, 1.30) 1.619 0.056

Q2 (2.47, 1.05) (3.44, 1.08) 3.182 0.00130

Q3 (2.56, 0.95) (3.23, 1.14) 2.229 0.015

Overall (2.50, 0.82) (3.24, 1.00) 2.811 0.00358

Page 18: UCD Awareness Research

Results

H0: Course 6 students score the same as non Course 6 students

HA: Course 6 students score lower than non Course 6 students

• Reject H0 for questions 2, 3, as well as overall, at the 0.05 significance level

• Cannot reject H0 for question 1 at the 0.05 significance level

Page 19: UCD Awareness Research

Discussion

Results suggest that the tech-oriented curriculum of a Course 6 education decreases the consideration that designers put into users’ needs

Interesting/Surprising Results: Higher variability in Non-Course 6 Students Question 1 - insignificant at the .05 level:

– Course 6 students might be good at the initial stages, but fail to recognize users’ needs in later stages

Outliers in the Course 6 student data: Experience in industry? Double major?

Page 20: UCD Awareness Research

Discussion

Limitations & Confounds– External Validity - Results applied to this case

study only– Sampling Bias/Data Collection

Different forms of data collection Getting people to participate - People who fill

out surveys are more “people oriented”– Grading Methodology– Survey Bias– Gender representation in Course 6

males possibly less user-centered than females (higher % of males in Course 6)

Page 21: UCD Awareness Research

Conclusion

Importance of User-Centered Design Skills – Course 6 Students place less emphasis

user’s interests than do other MIT students in a design

Further Steps & Future Research:– add more factors:

gender, school, age group– diversify case study, grading criteria, and

course distribution

Page 22: UCD Awareness Research

Questions?

Thanks