1EMC CONFIDENTIAL—INTERNAL USE ONLY

Bugs – From Finding to Preventing

Jacky Guo

2EMC CONFIDENTIAL—INTERNAL USE ONLY

About this presentation

•Overview

– Testing vs. Quality Engineering

– Start preventing bugs

3EMC CONFIDENTIAL—INTERNAL USE ONLY

Objectives

• After this presentation, you'll be able to:

– Identify a phased approach for preventing bugs

– Identify two actions for tomorrow to prevent bugs

4EMC CONFIDENTIAL—INTERNAL USE ONLY

5EMC CONFIDENTIAL—INTERNAL USE ONLY

Test vs. Quality Engineering (SQE)

"Test: to try something out… in order to find out… how well it works…"

"Engineering: the application of science in the design planning, construction and maintenance of…

manufactured things"

6EMC CONFIDENTIAL—INTERNAL USE ONLY

Test vs. Quality Engineering (SQE) (continued)

Test

• Late in the timeline

• Reactive

• Find defects

• Singular in affect

Quality Engineering

• First in the timeline

• Proactive and reactive

• Understand defects

• Multiplicative in affect

7EMC CONFIDENTIAL—INTERNAL USE ONLY

Phased approach

• 3 elements– Clearly written requirements/specs– "Look" at code– "Test" early

8EMC CONFIDENTIAL—INTERNAL USE ONLY

Phase 1: how do I start?

• Written requirements– Clarify user& user scenario– Have your design principle

• Simple– One Step instead of two or three– Avoid repeated things – Batch command

• Bug the documents– "Does everyone see the same picture?"– "Will" versus "might, should, desirable"

• Testability/ Sustainability• SMART

9EMC CONFIDENTIAL—INTERNAL USE ONLY

10EMC CONFIDENTIAL—INTERNAL USE ONLY

11EMC CONFIDENTIAL—INTERNAL USE ONLY

12EMC CONFIDENTIAL—INTERNAL USE ONLY

13EMC CONFIDENTIAL—INTERNAL USE ONLY

14EMC CONFIDENTIAL—INTERNAL USE ONLY

Phase 1: how do I start? (continued)

"Look" at Dev Design• Understand dev design

AssumptionDependencyWork flowData flowError/Exception handling/Log system

• Cover different user scenario• Testability• Risk

15EMC CONFIDENTIAL—INTERNAL USE ONLY

16EMC CONFIDENTIAL—INTERNAL USE ONLY

Phase 1: how do I start? (continued)

"Look" at code• Code inspections or reviews– Reviews are light-weight– Target code subsets– Inspect only the most critical/difficult sections.

• Static analysis– Built into VS2005– Find Bugs

• Compiler warnings• Debugging skill

17EMC CONFIDENTIAL—INTERNAL USE ONLY

18EMC CONFIDENTIAL—INTERNAL USE ONLY

19EMC CONFIDENTIAL—INTERNAL USE ONLY

Phase 1: how do I start? (continued)

Set the example• Test requirements• Test design– Archive/version control(Testlink)– Reviewed by program manager and developers

• Test code– Small code reviews

20EMC CONFIDENTIAL—INTERNAL USE ONLY

Phase 1: done

Congratulations• You have just grabbed low-hanging fruit• You have prevented countless bugs• You have risen above at least 50% of the Industry

Now what?

21EMC CONFIDENTIAL—INTERNAL USE ONLY

Phase 2: change the process

Learn about your bugs• Find the cause– Unexpected customer requirement?– Vague spec?– Rushed design?– Domain knowledge?– Code error?

• Be gentle, accurate and honest

• Make a change

22EMC CONFIDENTIAL—INTERNAL USE ONLY

Phase 2: change the process (continued)

Assess the value/impact of that change• Hard Metrics– Defects per 1K LOC– Test cycles to ship criteria

• Soft Metrics– Less panic at the end– Fewer pizzas ordered– People are happier

23EMC CONFIDENTIAL—INTERNAL USE ONLY

Call to action

• Start small, get a win– What are people most upset about?– What is the low-hanging fruit?– What does the RCA data say?

• Be prepared to set the example

24EMC CONFIDENTIAL—INTERNAL USE ONLY

Call to action (continued)

What can you do tomorrow (pick 1)?• Get a best practice on the schedule• Document one requirement or design– Get it reviewed

• Code-review one critical section• RCA 10 bugs• Implement 10 unit tests

25EMC CONFIDENTIAL—INTERNAL USE ONLY

SUMMARYReview process

•Requirement spec review•Dev Design review•Test Plan review(involve Dev)•Code Review(including bug fix)•Release spec review

26EMC CONFIDENTIAL—INTERNAL USE ONLY

SUMMARYTest case Management

• TC from Bugs by customer/external users• TC from review dev design• TC from user scenario• TC from Bugs review in case of regression• Regression Test strategy("diff" VS "full")• BVT runs before Dev Check in

27EMC CONFIDENTIAL—INTERNAL USE ONLY

SUMMARYBug Management

•Map to TC•Bug quality•Root cause(process may needs change)•Bug fix review to evaluate the risk•Control the bar to fix bug(potential risk)

28EMC CONFIDENTIAL—INTERNAL USE ONLY