review: agile software testing in large-scale project talha majeed comp 587 spring 2011
Post on 19-Dec-2015
215 views
TRANSCRIPT
Review:Agile Software Testing in Large-Scale ProjectTalha MajeedCOMP 587Spring 2011
2
Agenda Definitions Introduction Adopting XP and diverging from XP Guidelines in four Development areas
Test design and activity execution Working with professional testers Planning Defect management
Summery Conclusion
3
Definitions Agile testing is a software testing practice that
follows the principles of the overall agile manifesto
In agile testing, testing is not a phase. It is integrated into Development & Testing
QA is not the last line of defense but involves at the beginning of the project.
Agile testing focuses on testing iteratively, as often as stable code base is available, until quality is achieved from end customer’s perspective.
4
Definitions Extreme Programming (XP) is a software
development methodology which is intended to improve software quality and responsiveness to changing customer requirements. Frequent "releases" in short development cycles which is
intended to improve productivity . XP Approach is that if a little testing can eliminate a few
flaws, a lot of testing can eliminate many more flaws. Unit tests determine whether a given feature works as
intended. A programmer writes as many automated tests as they can think of that might "break" the code; if all tests run successfully, then the coding is complete.
Acceptance tests verify that the requirements as understood by the programmers satisfy the customer's actual requirements.
5
Introduction Israel Air Force (IAF) Enterprise information
system Critical to daily operations & information security Highly complex and extreme quality
Project was risky size Scope (XP-based method)
Risk management Metrics were developed Collected & analyzed data
6
Project overview Data collection
Four releases (eight months) Quantitative data
Project team used for project tracking & decision making
Qualitative data Recording of formal reflection meeting &
debriefings
7
Confirming to XP Team implemented standard XP
practices Short release
Working software two week iteration Customer feedback Goal to achieve measureable testing
benefits Short average time for defect fixes Short average defect longevity
8
Confirming to XP Planning game practice
To plan project iterations and releases Entire teem met and customer described and
prioritized stories Goal was to achieve simple design by
refactoring activities XP’s whole-team and site-together practices
Two measureable results Defect management overhead was lower Fewer false defects
9
Diverging from XP The project differed from classic XP in three
aspects First aspect
Team was required to produce detailed specification in addition to code and test due to organizational policy.
Second aspect Project’s accepting testing( user testing) was far too
complex for customer to fully specify Third aspect
Modified test-driven development to focus on the continuous creation of automated acceptance tests rather than unit tests. Never consider a feature “complete before all its tests
were written and passed.
10
Test design and execution Each team’s member was responsible for
testing In a traditional project, "everyone is responsible
for quality,” but in agile project, everyone actually writes testes.
Advantages First, It eliminated the team’s reliance's on single
tester Second, Developers test-awareness increased Finally, team members wrote specification and
code to write test cases for their work.
11
Test design and execution Product size = test size Progress was measured by defining the
project’s product size metrics as the number of regression test steps ran at each iteration
The above approach was chosen for two reasons
First, test size is more correlated with complexity than are lines of code or specification
12
Test design and execution Second, only features with full
regression testing at each iteration are counted as delivered product size.
Figure 1 plots the product size metric period of time
13
Test design and execution Untested work = no work (major shifting to agile
development) Untested work equals to no work in all channels
and all under conditions. Written Questionnaire response after two releases 90% team members considered acceptance and
system tests “highly important” 10% ranked as “important” 33% were interested in taking a leading role in
acceptance testing 60% were “interested” in leading system testing.
14
Working with professional testers Why professional testers?
Add value through more testing Developers code more features
Professional tester two key challenges for organization Testing bottlenecks Coordinating the testing work among
testers and programmers
15
Working with professional testers
Easing bottlenecks Developer s simply code less, test more
Figure 2. illustrates by comparing the product size metric points attained by tests that the entire team
ran to the team’s tester alone
16
Working with professional testers Encourage interaction over isolation Initially testers worked alone due to two
reasons Testers must be independent of
programmers to prevent implementation details from affecting their test designs.
Specification were up-to-date so testers write test according to specification
Testers were integrated to project team
17
Activity Planning Planning for quality activities involve
Time allocation Challenges in feature testing Regression testing Repair defects
18
Activity PlanningFigure 3. Testing and defect repair
(a) Net hours devoted to running (b) Net hours devoted to fixing defects and maintaining regression tests.
19
Activity planning Integrate feature testing and coding
Testing and coding time are usually equal Regression testing: divide and conquer Alocate bug-fix time globally
To estimate the time to fix bug is difficult Reasonably predict the time required to
fixing defects in forthcoming iterations
20
Defect management Defect management involves two major
challenges Managing workflow Selecting and scheduling the defects to fix
Use a team-centered defect-management approach Anyone can open defect Anyone can close a defect, after fixing it Anyone who finds a defect also assigns it to
someone to be fixed.
21
Defect management Fix defects as soon as possible
Bug fix time was slightly over an hour Bug fix requires more time ( a day or more) should be
suspended to prevent bottleneck and to ensure deadlines are met.
Advantages Defects require far less time to fix Working on clean highly stable code base makes new
development faster Avoids unpleasant customer negotiations over which
defects to solve.
22
Defect management
Figure 3. Average net time to fix a defect per iteration
23
Summery Presented definitions of Agile testing, XP
, Unit test, Acceptance tests etc. How Project adopts/ diverges from XP Identified How agile software testing
works on large-scale project Showed the graphs of real data from
Israel Air Force enterprise information system.
Described the Guidelines in four Development areas
24
Conclusion Project's data and analysis are useful both
to practitioners wishing to establish new agile teams and to research work
Even on large-scale project, the team achieved full regression testing at each iteration and developer testing
There is a need of quality data on large-scale project so, it is highly encouraged for further investigation on long-term agile projects.