chase 2012 - an exploratory study of backtracking strategies used by developers
DESCRIPTION
TRANSCRIPT
An Exploratory Study ofBacktracking Strategies
Used by Developers
YoungSeok Yoon ([email protected])
Institute for Software Research (ISR)
Brad A. Myers ([email protected])
Human-Computer Interaction Institute (HCII)
School of Computer Science
Carnegie Mellon University
CHASE 2012June 2nd, 2012.
CHASE 2012 2
What is Backtracking?• When developers write source code, they often need
to step back when something goes wrong
• We define backtracking as:
going back to an earlier state ofat least some part of the code,
either by removing inserted codeor by restoring removed code
CHASE 2012 3
Examples of Backtracking
Correcting minor mistakes
Tuning parameters
Trying out methodswhile learning an unfamiliar API
Experimenting with different classes(e.g. UI layout classes)
CHASE 2012 4
Existing Support for Backtracking
CHASE 2012 5
Existing Support for Backtracking
Conventional Undo Command• Only works for the most recent changes• Have to undo all changes in order• Undone changes are thrown away after making a new edit
CHASE 2012 6
Existing Support for Backtracking
Version Control• The desired version needs to be committed in advance• Unstable code should not be committed to the repository• Inappropriate when wanted and unwanted code is mixed
CHASE 2012 7
Existing Support for Backtracking
Manual Edit / Commenting Out• Have to manually find all the relevant code• Redo the work by hand
CHASE 2012 8
Research Question
How can we better understanddevelopers' backtracking behaviors?
As a first step towardssupporting more robust backtracking tools…
CHASE 2012 9
STUDY #1.LAB STUDY
CHASE 2012 10
Study Settings• 12 participants, all MS/PhD students at CMU• 2 hrs, THINK ALOUD• Code base: Paint program [Ko 2005][Fogarty 2005]
CHASE 2012 11
Tasks• Two sets of features each done in two different ways,
▪ intended to lead participants to backtrack
Feature 1 (F1) “Thickness Control”
F1-1
F1-2
Feature 2 (F2) “X, Y coordinates indica-
tor”
F2-1
F2-2
CHASE 2012 12
Commenting Out• 7 out of 12 participants preferred to comment out
rather than to delete code
• 3 main reasons for commenting out
Reuselater
e.g. activating one alternative at a time
Goodexample
leave the code as a good example
Badexample
remind themselves that the code was
not good
CHASE 2012 13
Problems while Backtracking• Difficulties in finding relevant code fragments that
have to be reverted when:▪ there are two or more similar looking code fragments▪ relevant code fragments are scattered across multiple
locations
• Forgetting to remove temporary code▪ e.g., Debug output statements such as:
• System.out.println• JOptionPane.showMessageDialog
CHASE 2012 14
Backtracking as Restoring Code• 2 participants accidentally deleted some code
• One participant failed to restore the code from memory after spending about 10 mins.
• The other succeeded using the undo commandbecause he knew:▪ what the code looked like▪ that it was deleted very recently
CHASE 2012 15
Backtracking as Restoring Code• When trying to restore deleted code, participants often
remembered some features about the desired code▪ the original location of the code▪ the surrounding code▪ the names of some code elements (variable, method, …)▪ what the code looked like
• Participants also reproduced the same code fragments from memory repeatedly▪ e.g. complex formatted string looking like
“(X, Y) = ” + x + “, ” + y + “)”
CHASE 2012 16
STUDY #2.ONLINE SURVEY
CHASE 2012 17
Demographics
• 48 professional programmers
• 13 yrs exp. on average
• recruited from online developer communities
Male92%
Female8%
CHASE 2012 18
Flexibility of their work
The details of the im-plementation code
Which elements of the API are used
Which APIs/libraries are used
The architecture of the code itself
The results that my code achieves
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%
Highly specified be-fore I start developing
Specified, but some opportunity to nego-tiate changes
Somewhat flexible, within broad con-straints
Highly flexible
Completely unspeci-fied; I can do what-ever I want
CHASE 2012 19
How often do they have to backtrack?
Trying to find an appropriate al-gorithm
Trying out various user interface designs
Fixing code just added, because it is not working
Figuring out how to use an API correctly
Tuning parameters
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
All the time
Frequently
Sometimes
Rarely
Never
CHASE 2012 20
Backtracking Strategies• For each backtracking situation, only a few strategies
were primarily used▪ e.g. “select & overtype” when tuning parameters▪ This hints at how to detect a certain backtracking situation
• Other strategies in the open-ended responses▪ using Boolean variables to temporarily turn on/off code▪ writing a small code snippet (outside the main project)▪ moving parameters out of the code▪ writing unit tests to see how the API works
CHASE 2012 21
Future Work• Conducting an extended version of this study
▪ A field study is ongoing, using our Eclipse logging tool called FLUORITE [Yoon 2011]
▪ If you are interested in participating, please let me know!
• Designing new backtracking tools, based on the needs identified in these studies
CHASE 2012 22
Conclusion• Backtracking happens a lot while coding
• The backtracking strategies used by developers are still manual and error-prone
• Better backtracking tools would help developers write code more correctly and effectively
CHASE 2012 23
Questions?
Thanks for funding from:
Contact Info: YoungSeok YoonE-mail: [email protected]: http://www.cs.cmu.edu/~yyoon1/
FLUORITE logging tool can be found at:http://www.cs.cmu.edu/~fluorite/