telerik software academy software quality assurance
DESCRIPTION
Incident Management – Main Concepts Incident Reporting Defect Lifecycle Metrics and Incident Management Some Golden Rules for Incident Reporting Incident Management Tools 3TRANSCRIPT
Incident Management
Telerik Software Academyhttp://academy.telerik.com
Software Quality Assurance
The Lectors Snejina Lazarova
Product ManagerTalent Management System
Dimo MitevQA ArchitectBackend Services Team
2
Table of Contents Incident Management – Main Concepts
Incident Reporting Defect Lifecycle Metrics and Incident Management Some Golden Rules for Incident Reporting
Incident Management Tools
3
Incident ManagementMain Concepts
What Are Incidents? Testing often leads to observing deviations from expected results Different names are used for that:
Incidents Bugs Defects Problems Issues
5
Incident vs. Bug – A Matter of Semantics
Sometimes a distinction between incidents and bugs (defects) is made Incident
Any situation where the system exhibits questionable behavior
Bug An incident is referred to as a bug
(defect) when the root cause is some problem in the item we're testing
6
What Else Could Cause an Incident?
Other causes of incidents include: Misconfiguration or failure of the
test environment Corrupted test data Bad tests Invalid expected results Tester mistakes
According to the test policy – any type of incident can be logged for tracking 7
The Earlier – The Cheaper
Incident logging or defect reporting are not necessarily happening during testing Incidents can be logged, reported,
tracked, and managed during development and reviews
8
What Do We Report Defects Against?
Defects can be reported against: The code or the system itself Requirements Design specifications User and operator guides and tests
9
Glossary Defect (bug)
A flaw in a component or system that can cause the component or system to fail
Error A human action that produces an
incorrect result Failure
Deviation of the component or system from its expected delivery, service, or result 10
Glossary (2) Incident
Any event occurring that requires investigation
Occurs anytime the actual results of a test and the expected results of that test differ
Incident logging Recording the details of any
incident that occurred (e.g., during testing)
Root cause analysis An analysis technique aimed at
identifying the root causes of defects
11
Incident Reporting
Managing Defects Defects found can reach count that is hard to manage A process for handling defects from
discovery to final resolution is needed
Should include reporting, classifying, assigning and managing defects
13
Central Database A central database for each project should be established All incidents and failures discovered
during testing are registered and administered
Developers, QAs and stakeholders have access
14
What Goes in an Incident Report?
An incident report usually includes: Summary Steps to reproduce
Including inputs given and outputs observed
Isolation steps tried Impact of the problem Expected and actual behavior
15
What Goes in an Incident Report? (2)
An incident report usually includes: Date and time of the failure Phase of the project Test case that produced the
incident Name of the tester Test environment
16
What Goes in an Incident Report? (3)
References to external sources Specification documents Various work items
Attachments Videos and screenshots
Any additional information about the configuration
17
What Goes in an Incident Report? (4)
Root cause of the defect Usually set by the programmer,
when fixing the defect Status and history information Comments Final conclusions and recommendations
18
What Goes in an Incident Report? (5)
Severity and priority of the defect Sometimes classified by testers Sometimes a bug triage committee
is responsible for that Determines also the risks, costs,
opportunities and benefits associated with fixing or not fixing the defect
19
Defect Severity What is a defect "severity"?
The degree of impact on the operation of the system
Possible severity classification could be: 1 – Blocking 2 – Critical 3 – High 4 – Medium 5 – Low 20
Defect Severity Levels Blocking
Stops the user from using the feature as it is meant to be used
No reasonable workaround Critical
Data corruption Easily and repeatably throws an
exception No reasonable workaround Feature does not work as expected
21
Defect Severity Levels (2)
High Throws an exception when not
following the happy path Confusing UI Has a reasonable workaround
Medium Feature works off the happy path
with minor issues Small UI issues One or more reasonable
workarounds22
Defect Severity Levels (3)
Low Cosmetic issues Many workarounds Low visibility to users
23
Defect Priority What is a defect "priority"?
Indicates how quickly the particular problem should be corrected
Possible priority classification could be: 1 – Immediate 2 – Next Release 3 – On Occasion 4 – Open (not planned for now)
24
Defect Priority(2) Covey's Quadrants
Defects are categorized by four quadrants: QI - Important and Urgent QII - Important but Not Urgent QIII - Not Important but Urgent QIV - Not Important and Not Urgent
25
Defect Priority(3) The ABC Method
A = vital B = important C = nice Then these categories are
subdivided into A1, A2, A3, ..., B1, B2, ... and so forth
The Payoff versus Time Method Weight each defect by the payoff
expected from it versus the time it takes to be done
26
Defect Priority(4) Paired Comparison
Uses a simple scoring system for comparing activities
1 = slightly prefer 2 = moderately prefer 3 = greatly prefer
27
Option A: B: C: D:A: A,1 C,2 A,1B: C,2 D,2C: C,2D:
A=1+1=2B=0C=2+2+2=6D=2
The option with highest result has the highest priority
Defect Lifecycle
Defect Lifecycle Defect lifecycles are usually shown as state transition diagrams
Different defect-tracking systems may use different defect lifecycles
29
Defect Lifecycle Graph Simple defect lifecycle graph
30
Defect Lifecycle States New
The bug is posted for the first time The bug is not yet approved
Open The test lead approves that the bug
is genuine Changes the state as “OPEN”.
Assign The bug is assigned to
corresponding developer or developer team
31
Defect Lifecycle States (2)
Test The bug has been fixed and is
released to testing team Rejected
If the developer feels that the bug is not genuine, he rejects the bug
Duplicate The bug is repeated twice or the
two bugs mention the same concept of the bug
32
Defect Lifecycle States (3)
Deferred The bug is expected to be fixed in
next releases Reasons for changing the bug to
this status may have many factors: Bug may be low Lack of time for the release the bug may not have major effect on
the software
33
Defect Lifecycle States (4)
Verified Once the bug is fixed and the status
is changed to “TEST”, the tester tests the bug
If the bug is not present in the software, he approves that the bug is fixed
34
Defect Lifecycle States (5)
Reopened The bug still exists even after the
bug is fixed by the developer The bug traverses the life cycle
once again Closed
The bug is fixed, tested and approved
35
Metrics and Incident Management
Defect Management Metrics
Various metrics can be used for defect management during a project Helps managing defect trends Helps determining readiness for
release
37
Defect Management Metrics (2)
Total number of bugs Number of open (active) bugs/tasks
Number of resolved bugs/tasks38
Defect Management Metrics (3)
Bugs per category Bug cluster analysis Defect density analysis Number of defects discovered on a time unit E.g., week, testing iteration, etc.
39
Defect Management Metrics (4)
Mean-time to fix a defect The time between reporting and
fixing/closing the bug Time estimates versus actual time spent comparison
Gives confidence in the estimates given by the team
40
Bug Convergence Bug Convergence
Also called open/closed charts The point at which the rate of fixed
bugs exceeds the rate of found bugs
A visible indication that the team is making progress against the active bug count
A sign that the project end is within reach
41
Defect Detection Percentage
Gives a measure of testing effectiveness
Some defects are found prior to release while others - after deployment of the system
The defect detection percentage (DDP) compares field defects with test defects, also called escaped defects
42
defects (testers) defects (testers) +
defects (field)
DDP
=
Some Golden Rules for Incident Reporting
Golden Rules for Bug Reporting
Watch your tests Run your tests with care and
attention You never know when you're going
to find a problem Reporting intermittent or sporadic symptoms Some defects cannot be reproduced
always Report how many times you tried to
reproduce it and how many times it did in fact occur
44
Golden Rules for Bug Reporting (2)
Isolate the defect Make carefully chosen changes to
the steps used to reproduce it Move from boundary values to more
generalized conditions Provide information on the defect's impact Makes setting priority and severity
easier and more accurate
45
Golden Rules for Bug Reporting (3)
Mind your language Choose the right words in your
report Be clear and unambiguous, neutral,
fact-focused and impartial Be concise – avoid useless detailes
Make reviews of bug reports Make an experienced tester take a
look a your report46
Incident Management Tools
Telerik TeamPulse TeamPulse is an agile project management solution Requirements Management Bug Management Planning and Scheduling Time Tracking Ideas and Feedback Management Filtering Reporting
48
TeamPulse Demo Login Setup a new Project Enter a new work item (Story/Task, Bug, Issue, Risk, Feedback)
Manage work items Resolve and Close Search, Reports, Email notifications, etc.
49
JIRA What is JIRA?
A proprietary issue tracking product,
Developed by Atlassian Used for
Bug tracking Issue tracking Project management
http://www.atlassian.com/software/jira/ 50
JIRA - Demo
Login Manage Dashboard Enter a new Project Enter a new
Component Enter a Defect Manage Defect Resolve and Close Search, Reports,
Email, etc. 51
Bugzilla What is Bugzilla?
Web-based bugtracker Originally developed and used by
the Mozilla project http://www.bugzilla.org/
52
BugzillaDemo
Team Foundation Server
What is TFS? Microsoft product offering
Source control Data collection Reporting Project tracking
TFSDemo
Other Bug-tracking Tools
Some other bug-tracking tools: MantisBT TRAC GNATS
56
Incident Management
Questions? ?
?? ? ??
?? ?
?
Free Trainings @ Telerik Academy
C# Programming @ Telerik Academy csharpfundamentals.telerik.com
Telerik Software Academy academy.telerik.com
Telerik Academy @ Facebook facebook.com/TelerikAcademy
Telerik Software Academy Forums forums.academy.telerik.com