nosqaa requirements inspection
TRANSCRIPT
Requirements Inspection
Leland Smith
2
What is an Inspection
Inspections are a disciplined engineering practice for detecting and correcting defects in software artifacts, and preventing their leakage into field operations.
Focused review to identify issues Defined roles and steps Requires training and preparation Adheres to time contracts Manages and reports metrics Formal documented process
3
Why Inspections?
By far, THE most effective way to reduce defect detection cost reduce rework improve product quality reduce cycle time enhance competency
Cumulative industry evidence is beyond reasonable dispute
4
Michael Fagan initiated inspections at IBM in1976 - pioneered their early adoption and later evolution
… finds 60-90% of all defects and provides feedback to programmers that enables them to avoid injecting defects in future design and coding work - Fagan
IEEE has standardized inspections A Key Process Area (kpa) of the CMM (Capability
Maturity Model) The hardest single part of building a software
system is deciding precisely what to build. – Fred Brooks
The single most common reason software engineers inject defects has been and is still poorly written requirements. - Steve Rakitin
Industry Experience
5
Requirements – PNC Bank began conducting requirement inspections in February 2006
Architecture / Design- work in progress Code Test Plans / Test Cases – PNC Bank began
conducting test inspections on a limited scale in November 2007
Inspection Types
6
FIND defects
in Requirements not Fix defects (or to provide brilliant insights into
how the author should have done it!) essential to efficiency and effectiveness
Requirements Inspection Goal
7
Impact without Requirements Inspection
Poor Requirements
Incomplete
Unclear
Incorrect
Application Design and Programming
Errors
Rework Due to Poor
Requirements
Test Case / Script Design and
Programming Errors
Rework Due to Poor
Requirements
Rework Due to Poor
Requirements
Impact to Customer Due to
Poor Requirements
Rework Due to Poor
Requirements
Impact to Bank $$$ due to Poor Requirements
Rework Due to Poor
Requirements
Analysis Phase Design / Build Phase
Test PhaseDeploy / Run
Phase
8
Target of Requirements Inspection
Functional Requirements Use Case (High Priority)
DescriptionPre and post conditions, actorsHappy path and alternate pathsInclusions and exclusions
Non-functional Requirements Business rules modifying functional requirements Constraints imposed upon the system
9
Requirements Inspection Overview
Requirements Inspection
Remediated Requirements
ChecklistsInspector Training
Defect Log
Use Cases and
Supplementary Reqs.
10
Requirements Inspection Process
1. Planning2. Individual Review3. Team Review4. Defect and
Investigation Resolution
5. Validate & Exit
11
Requirements Inspection Team Roles
Moderator
Leads the team through the process AND keeps team focused; manages inspection time contract; ensures work product entrance criteria is met; records defects and items to investigate; distributes inspection materials; tracks metrics
Author
Creator/maintainer of the work products being inspected; has a vested interest in ensuring all defects are found
12
Requirements Inspection Team Roles
ReaderRepresentative from the development team; during team meeting will paraphrase each statement in their own words to demonstrate sufficient understanding to carry out next stage of design.
Tester May be one or more persons responsible for test planning and/or testing; will consider how they would test the product - which paths, conditions, data, states, expected results, etc - and express as questions in team review.
13
Requirements Inspection Steps
1. Planning - Moderator conducts value-based assessment to engage; Moderator, Project Lead, and Author select persons for Reader and Tester roles; schedule training; schedule inspection
2. Individual Review - Author provides Moderator requirements and supporting documentation prior to team review. Moderator distributes needs and features, glossary, requirements, and diagrams. Inspectors independently review requirements and identify anything not perfectly clear, anything which seems wrong, or anything they disagree with (substance, not style)
3. Team Review - Walk through requirements; Moderator records defects, keeps track of time spent for individual and team review
14
Requirements Inspection Steps
4. Defect & Investigation Resolution - Author performs requirement remediation based upon defects recorded in defect log and updates log with appropriate responses (accept, reject, or defer). At completion of remediation, Author sends copy of defect log to Moderator with total remediation hours spent.
5. Validate & Exit - Author reviews defect log with Moderator and Project Lead; verifies no secondary defects exist; discusses rejected defects; determines if further inspections are needed.
15
Management’s Role
Do not participate in the Inspection Participate in planning and exit meetings Kept apprised of progress by Moderator Encouraged to attend Inspection training
16
Requirements Inspection Performance Indicators
Minimum of 4 Inspectors Requirements coverage runs 60 requirements per
hour (average) Maximum 2 hours for Inspection Team Review
National City experience has shown that effectiveness declines sharply after 2 hours
Typically, you hold two meetings to cover 240 requirements2 mtgs x (2 hr/mtg x 60 req/hr)
Preparation – if participants are not prepared, postpone Inspection and inform management
Effective inspections are HARD WORK!
17
Requirement Defects Found
0
50
100
150
200
250
300
350
400
450
500
Fre
qu
en
cy o
f D
efe
cts
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
120.0%
Cu
mu
lati
ve P
erc
en
tag
e
18
Author’s Response
0
200
400
600
800
1000
1200
Fre
qu
en
cy o
f A
cti
on
Taken
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
120.00%
Cu
mu
lati
ve %
19
Competency Improvement
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
Pe
rce
nt
of
Re
qu
ire
me
nts
Incomplete
Unclear
Unverifiable
Incorrect
Irrelevant
Investigations
Inconsistent
Infeasible
Defect type as percent of requirements
20
Requirements Inspection Process Assets
Inspector training deck Inspection checklist Moderator training deck Database – collect/manage and report metrics Inspection group mailbox Template letters for
Engagement request Denial Various meetings
Periodic meetings with Moderators
21
Summary
Process implementation High return on investment Low risk Short wait for payback of initial investment
Defects cost more the longer they exist
22