What is trust in a recommender for software development?

Gail C. Murphy Emerson Murphy-Hill

University of British Columbia University of British Columbia

Tasktop Technologies Inc.

22/5/2010 RSSE @ ICSE 2010

recommenderB

recommenderA

recommenderC recommenderD

Catego

ry 1

Catego

ry 2

Catego

ry 3

Catego

ry 4

0

1

2

3

4

5

Series 1

Series 3

recommendern+1

RSSE @ ICSE 2010 32/5/2010

study1 study2

recommender ofdevelopment tools

interviews about howprofessionals learn

tools

3rd year studentsin multi-week case study

18 professionaldevelopers for an

hour each

RSSE @ ICSE 2010 4

study1 - Spyglass

2/5/2010

RSSE @ ICSE 2010 5

study1 - Spyglass

2/5/2010

longitudinal user study - effective in helping a developer discover previously unknown useful tools? - effective in encouraging a developer to use a previously unknown tool? - explore source code more efficiently? - notification mechanism obtrusive?

9 participants for ~ 3 weeks

RSSE @ ICSE 2010 6

study1 - Spyglass

2/5/2010

longitudinal user study - effective in helping a developer discover previously unknown useful tools? - effective in encouraging a developer to use a previously unknown tool? - explore source code more efficiently? - notification mechanism obtrusive?

9 participants for ~ 3 weeks

RSSE @ ICSE 2010 7

1. timing affects trust

2/5/2010

inefficientnavigation

Spyglass notifiesrecommendation

available

developerinvestigates

recommendation

RSSE @ ICSE 2010 8

study2 – tool discovery

2/5/2010

interviews - how do developers discover tools - focus on social discovery between peers (peer recommendation & peer observation)

18 professional developers, 1 hour interviews

RSSE @ ICSE 2010 9

2. social trust eases uptake

2/5/2010

“I’ve known them for a while and how they get things done, so I can translate that to how I get things done”

“If you can get a feel for people that you know, either because you know them online […] or in person […] you have a pretty good estimate of what your reaction will be”

RSSE @ ICSE 2010 10

3. trust trumps precision

2/5/2010

“Honestly; I bet the [recommender] would have better success rate [than a peer] at recommending things that I would like, but that doesn’t mean that I would trust the [recommender] more”

RSSE @ ICSE 2010 11

establishing trust

2/5/2010

prior interaction seems key

can trust be borrowed from a peer?

can trust be sidestepped? make it feel like discovery rather than suggestion

RSSE @ ICSE 2010 12

establishing trust

2/5/2010

prior interaction seems key

can trust be borrowed from a peer?

can trust be sidestepped? make it feel like discovery rather than suggestion

what do/would you trust?what makes you trust the source?