issues and challenges of agile development: some perspectives from academic research
DESCRIPTION
Issues and challenges of Agile Development: Some perspectives from Academic Research. Dr. Sridhar Nerur. University of Texas at Arlington. Presentation Outline. Are Two Heads Better Than One – Pair Programming Study Pairs versus Individuals on Design Tasks: A Distributed Cognition Perspective - PowerPoint PPT PresentationTRANSCRIPT
Issues and challenges of Agile Development: Some perspectives from Academic Research
1
Dr. Sridhar Nerur
University of Texas at Arlington
Presentation OutlinePresentation Outline
Are Two Heads Better Than One – Pair Programming Study
Pairs versus Individuals on Design Tasks: A Distributed Cognition Perspective
Case study of XP (Extreme Programming) adoption
Other research projects
2
Are Two Heads Better Than One?Are Two Heads Better Than One?A Pair Programming Study A Pair Programming Study
(Balijepally, Mahapatra, Nerur & (Balijepally, Mahapatra, Nerur &
Price, 2009)Price, 2009) Methodology development driven by
practitionersEffectiveness reports were anecdotal or
experientialNo systematic effort to provide
theoretical explanation of practicesInconsistent findings on the
effectiveness of pair programming Lack of theoretical and methodological
rigor (e.g., Nosek 1998, Williams 2000)
3
Effectiveness of Pair Effectiveness of Pair ProgrammingProgramming
Research Objectives◦To examine the efficacy of pairs vis-
à-vis individuals in a programming task
◦To study the effect of pairing on a programmer’s task satisfaction and confidence in performance
◦To understand how the above are affected by programming task complexity
4
Theoretical BackgroundTheoretical BackgroundTask Typology
◦Based on Steiner’s typology (1972) programming tasks are unitary (not divisible among group members), optimizing (quality is more important than speed), and disjunctive (group must arrive at a single solution)
◦Tasks may be intellective or judgmental ( Laughlin and Ellis 1986)
◦Programming tasks are relatively intellective with high solution demonstrability
5
Theoretical BackgroundTheoretical BackgroundNominal Pair (Laughlin et al.
2002)◦A nominal pair is created by
randomly pairing individuals who worked alone
◦Comparing the performance of a collaborating pair with the best and the second best members of a nominal pair provides better insight into group performance
6
Theoretical BackgroundTheoretical BackgroundGroup Performance
◦Group members have access to a larger pool of resources (Flor and Hutchins 1991; Hinsz et al. 1997)
◦Group processes may suffer from process losses due to social loafing (Karau and Williams 1982)
◦Performance is contingent on task type◦With intellective tasks, groups outperform
an average individual but rarely perform better than the best individual (Hill 1982, Laughlin et al. 2003)
◦Quest for assembly bonus effect (when the group outperforms the best individual) still continues (Collins and Guetzkow 1964)
7
Research ModelResearch Model
8
H1
H3
H2
H4
Perceptual Outcomes
Task
Complexity
Programming Setting
Individuals or Pair*
Satisfaction
Software quality
Confidence in Performance*1 – Best in a nominal pair
2 – Second-best in a nominal pair
3 – Collaborating pair
Research Method and Data Research Method and Data AnalysisAnalysis
Lab Experiment2 (Individual vs. pair) X 2(Simple Vs.
Complex task) factorial design120 student subjects (30 pairs and 60
individuals)Pilot done to check experimental protocol
and to determine time needed for the experimental task
15 minutes for the warm up task and 2 hours for the experimental task
MANCOVA followed by ANCOVA for each dependent measure
Programming ability used as a covariate9
ResultsResults
10
Hypothesis Result ExplanationSoftware Quality
1 Programming performance of a collaborating pair measured in terms of software quality will be higher than the performance of the second-best member of a nominal pair
Supported (p < 0.01)
A collaborating pair outperforms the second-best member of an independently working nominal pair.
Satisfaction2 A. The mean satisfaction with
the programming task of a collaborating pair will exceed the level reported by the best member of a nominal pair.
Supported(p=0.04)
Collaborating pairs are more satisfied than each member of an independently working nominal pair.
B. The mean satisfaction with the programming task of a collaborating pair will exceed the level reported by the second-best member of a nominal pair.
Supported (p = 0.02)
ResultsResults
11
Confidence in Performance
A. The mean confidence in performance of a collaborating programming pair will exceed the level reported by the best member of a nominal pair.
Not Supported(p = 0.15)
Collaborating pairs are only as confident of their performance as the best member of an independently working nominal pair, but more confident than its second-best member.
B. The mean confidence in performance of a collaborating programming pair will exceed the level reported by the second-best member of a nominal pair.
Supported (p < 0.01)
Moderating Effect of Task ComplexityTask complexity affects the performance, in terms of software quality, of the best member of a nominal pair more adversely than that of a collaborating pair
Not Supported (p = 0.18)
Task complexity affects the performance of collaborating pairs and nominal pairs in similar ways.
Hypothesis Result Explanation
ContributionsContributionsOne of the earliest theoretically
grounded empirical research studies on pair programming.
Offers insight into performance of programming pairs
Examines the contingent effect of task complexity on programmer’s performance
Creates a new benchmark to evaluate pair programming performance through the use of nominal pairs
12
Pairs vs Individuals on Pairs vs Individuals on Design Tasks (Mangalaraj, Design Tasks (Mangalaraj,
Nerur, Mahapatra, & Price – Nerur, Mahapatra, & Price – under review)under review)
Motivation◦Research on pair designing is sparse
◦Design tasks are cognitively more challenging
◦Very limited empirical research on design patterns in tasks that are design-centric
13
Theoretical FoundationTheoretical FoundationAccording to the theory of
distributed cognition, cognition is distributed across problem-solvers (i.e., social interactions), artifacts (i.e., embodied cognition), and time
Two aspects of distributed cognition that are relevant to our research: social and structural (or embodied)
We use pairs (social) and design patterns (cognitive artifacts) in our study
14
Research ObjectivesResearch ObjectivesTo study the role of design
patterns in facilitating the solution of a software design problem
To assess the effectiveness of pairs vs. individuals working on a software design task
15
Theoretical BackgroundTheoretical BackgroundDesign problems are ill structured,
ambiguous, and have no pre specified solution path (Simon 1973)
Key activities in solving a design problem are◦ Problem structuring◦ Searching through the solution space
Problem structuring is facilitated by availability of partial solutions and external representation media (Guindon 1990, Zhang and Norman 1994)
Expert developers use their design experience (stored as design schemas) to constrain the search space (Guindon et al. 1987)
16
Research ModelResearch Model
17
Individual/Pair
With design patterns/without design patterns
Outcome Variables
Solution quality
Task completion time
Task satisfaction
H4, H5a, H5b, H6
H1, H2, H3
Research MethodResearch Method
18
Research Method & Data Research Method & Data AnalysisAnalysisLab Experiment2 (Individual vs. pair) X 2 (Design Pattern
Vs. No Design Pattern) factorial design92 professionals with no prior pattern
experiencePilot done to check experimental protocol
and to determine time needed for the experimental task
Participants were provided a free seminar on patterns
ANOVA/ANCOVA used for analysisObject-oriented programming experience
was used as a covariate
19
ResultsResults
20
Hypothesis ResultH1: Quality of the design solution will be higher when design patterns are used during software design.
Supported
H2: Time taken to complete a design task will be shorter when design patterns are used.
Supported
H3: Task satisfaction will be higher when design patterns are used in a software design task.
Supported
H4: Solution quality of collaborating pairs will be higher than that of the second best member of a nominal pair in a software design task.
Supported
H5a: Time taken by collaborating pairs to complete a software design task will be longer than that of the best member of a nominal pair.
H5b: Time taken by collaborating pairs to complete a software design task will be longer than that of the second-best member of a nominal pair.
Not Supported
Supported
H6: Task satisfaction will be higher among collaborating pairs when compared with the average level of task satisfaction of nominal pairs.
Supported
ContributionsContributionsOne of the earliest theoretically
grounded empirical research on the impact of design patterns on design performance
Demonstrates the benefits of using design patterns ◦ Improves design quality◦ Reduces task completion time◦ Enhances developer satisfaction
Consistent with group literature, pairs were found to be better than second-best individuals of nominal pairs
21
SynthesisSynthesisTheoretically grounded research provides
a deeper understanding of the performance of pairs in software development
Studies demonstrate that pairs outperform second-best individuals of nominal pairs regardless of task type and experience level.
This confirms the long-standing research in group theory, but is counter to the dominant belief in the agile community
The use of nominal groups in this stream of research is a novelty. It permits a more fine-grained analysis that is more meaningful to practitioners
22
ImplicationsImplicationsPairing should be used judiciously
◦Performance gains are realized when low-performing rather than best developers are paired
The use of design patterns is beneficial
Our studies have established a benchmark for theoretically grounded research in software engineering
23
Table 1: Core XP practices adopted by ABC Corp.XP Practice Definition* Relative
Importance
Release Planning
Part of the planning game (original XP practice) in which the customer specifies the requirements for the project and creates the release plan based on the developer’s cost/time estimates.
0.139
Customer Team
Customer is part of the development team. He/she determines requirements and performs acceptance tests.
0.126
Collective Ownership
Members of the project team collectively own the code and any one can change any of the code at any time.
0.122
Pair Programming
Pair programming involves two programmers working side by side at one computer, collaborating on the design, coding and testing on a continuous basis.
0.109
Iteration Planning
A part of the planning game that involves planning for the two-week iterations.
0.100
Sustainable Pace
Teams work at a sustainable pace, usually 40 hour per week. Team members work overtime only when needed.
0.098
Case Study – Problems with Adoption ofXP practices at ABC Corp.
Refactoring Ongoing redesign of software to improve its responsiveness to change.
0.049
Simple Design Simplest design is done for the functionality that has been defined and not in anticipation of future requirements.
0.046
Unit Tests Involves the development of the test module prior to writing the code.
0.040
Small Releases
Software is released in small increments that address the most valuable business requirements.
0.040
Acceptance Tests
Customer writes the acceptance tests. Testing is largely automated.
0.037
Continuous Integration
Builds are done every few hours to lessen code integration problems.
0.028
Metaphor It is the common vision of how the system works. The metaphor is intended to provide a broad view of the project’s goal.
0.023
Open Workspace
Developers work in an open lab-like environment with no cubicles.
0.022
Coding Standards
Rules that everyone in the team must follow in writing code. 0.020
* These definitions are based on the following sources: (Highsmith, 2002; Beck, 2000; Jeffries, 2001)
Table 2: Comparison of the two projectsCharacteristics Project X Project YApplication type New project Maintenance and enhancementLanguage/Technologies
Java C, C++, Java, Motif
Project size Approximately 30,000 lines of code.
Several hundred thousand lines, with more than 10 year old code base.
Tools used IntelliJ and CruiseControl
C++ editor, SlickEdit, and ClearCase
Team Size 6 members 7 membersXP Score (Maximum 4.0)
2.64 1.58
Table 3: Comparison of core XP practices across the two projects.XP Practice Project X Project YRelease and iteration planning
Release planning is done periodically.
Iteration planning involves all members of the team. Developers have the freedom to choose stories they wish to work on.
Release planning is done periodically.
Though iteration planning involves all members of the team, stories are assigned based on skill specialization.
Customer team
Proxy customer writes user stories and collaborates in acceptance testing.
No dedicated customer/proxy customer is involved.
Collective ownership
Team members are collectively responsible for code development and maintenance.
Limited collective code ownership due to skill specialization.
Pair-programming
It is done to a greater extent.
In general, team members seemed to enjoy pair-programming and they hold positive attitude towards it.
Pair programming was seldom done.
Examples of differences between the two projects
Individual Factors•Attitude towards XP•Knowledge of XP
Team Factors•Task management•Team leadership
Technology Factors•Compatibility•Tools support
Task Factors•Type of application•Project size
Environmental Factors•Budget constraints•Time constraints
Acceptance of Extreme Programming
Figure 3. Extreme Programming Acceptance Framework
Other research studiesOther research studiesEfficacy of pairs vs individuals with
and without Test-Driven Development
Impact of problem representation on solution quality
Understanding cognitive processes – mental models, cognitive ability
Agile Project Management challenges
29
30
Questions??