chicago april 18th april 22th• test traceability matrix • defect reports • release notes...

Renaissance Hotel 1 West Wacker Drive Chicago IL 60601

Speaker(s): Clareice & Clyneice Chaney When: April 21, 2016

Company: MITRE & Quality squared Time: 3:30 pm - 4:30 pm

CHICAGO April 18th — April 22th

Testing Deliverables:

Define, Align, Document and Approve

Testing DeliverablesDefine, Align, Document, and Approve

QAI QUEST Chicago, IL

Clareice Chaney, MITRE Corporation, CPCM, PMP

Clyneice Chaney, MITRE Corporation, CQMgr, PMP

April 21, 2016

Approved for Public Release; Distribution Unlimited. Case Number 16-0366

The author's affiliation with The MITRE Corporation is provided for identification purposes only, and is not intended to convey or imply MITRE's concurrence with, or support for, the positions, opinions or viewpoints expressed by the author.

About the Speakers

Clareice Chaney has over 30 years of experience in commercial and government contracting with an emphasis in information technology contracting and specific expertise in performance-based contracting techniques.

Clyneice Chaney brings over 25 years of testing, quality assurance, and process improvement experience. Clyneice holds certifications from the American Society for Quality as a Certified Quality Manager, QAI Global Institute's Certified Quality Analyst, and Project Management Institute's Professional Project Manager.

QAI Quest 2016 April 18-22 Chicago, IL

Deliverables … What We Know...

PRODUCTS

OUTCOMES

DOCUMENTS

RESULTS

OUTPUTS

ARTIFACTS

CONTRACTUAL

SW

Deliverables come in different shapes, sizes, and attributes

RELEASES

Deliverables differ from project milestones

Deliverables are often contractual in nature


Deliverables don’t show up magically

like a rabbit in a hat

Where do they come from? How do you ensure they fit

the assigned work? What can you do ?

Why do they

matter?


SW Methodology Impact

Deliverables

SIMILARITIES DIFFERENCES/EXPECTATIONS

WATERFALL • Testing Strategy• Test Plans• Test Cases

Documents• Test Scripts• Test Data• Test Results/

Reports

• Install/Configurations Guides• Defect Report• Release notes• Test Traceability Matrix• Defect Reports• Release Notes

AGILE/RAPID • Testing Strategy• Test Plans• Test Cases

Documents• Test Scripts• Test Data• Test

Results/Reports

• Test Automation• Release Planning (Test Plan) • Participate in story sizing• Sprint QA Activities• Infrastructure Considerations

(Test environment/SW)• Pair tests (Dev + QA)• Reviews (stories, requirements,

test cases with customer, programmer)QAI Quest 2016 April 18-22 Chicago, IL

Quality of Service Quality of Product

Release Quality

Defect Density

Meeting Product “Doneness”

Number of Defects

Schedule

Release Quality

Test Planning Effectiveness

Defect Detection

Adapted from James Bach, Rapid Software Testing.

Project and Test Team Goals Impact Deliverables


Testing Approaches Impact Deliverables


Project Schedules Impact Deliverables


Regulations & Standards Impact Deliverables

• 508 Compliance

• FISMA/Security Requirements

• ECLC


Documents: Legal, Contractual, and Planning Impact Deliverables


Considerations for Identifying Deliverables

• What is the purpose and goal of the project? oWhat are the test objectives?

• What does the client or stakeholders want at project end?

• Is the project goal tangible or intangible?

• Are there various elements that must be approved within the project?

• Will the project require many levels of work prior to completion

Adapted from: Understanding Project Management Deliverables – Tips on Managing and Monitoring PM Deliverableshttp://www.brighthubpm.com/risk-management/69108-defining-project-management-deliverables/


http://www.brighthubpm.com/risk-management/69108-defining-project-management-deliverables/

Sources Checked for Potential Deliverables?Now What?

Potential Deliverables…. Do they mean the same to everyone?


DEFINING DELIVERABLES…Filling in the Expectations.. Reality GAP

Client Needs

RFP/SOW/Task

Project Objectives

EXPECTATIONS

REALITY


Deliverables have standards and specifications for completion

Deliverables have stakeholders

Deliverables have quality components

Deliverables may be associated with phases

Deliverables may have dependencies

Deliverables may be defined differently by different stakeholders

Considerations for Defining Test Deliverables


Testing Deliverables Are Changing

Traditional Test Deliverables

Alternative Test Deliverables


Align to: Business Requirements

Align to:High Level Scope & Vision Statements

Align to:Product Requirements

Align to:Process Requirements

CONTRACT

Testing objectives for the project and each release or testing effort should be clearly stated and understood by the testing team

Align to:Stakeholder Needs

ALIGN…Filling in the Expectations.. Reality GAP


DOCUMENT:…Filling in the Expectations.. Reality GAP

Work Packages

Tasks

Deliverables: Formal/ Informal

SW

Tool-based

EXPECTATIONS

REALITY


Document Deliverable Expectations

• Deliverable Description - high-level overview of the deliverable objective and scope, the methodology for creating the deliverable, applicable standards, deliverable requirements, and deliverable format

• Deliverable Acceptance Criteria - high-level overview of the deliverable acceptance criteria

• Deliverable Schedule - timeframes for client reviews and acceptance, frequency of submissions, and submission dates

• Resources Required – who is involved and how much time for deliverable preparation and reviews


Document Deliverables?Format and Content Considerations

• Document Name• Assigned• Contributors• Prerequisites/Dependencies• Purpose and Audience• Recommended Accomplishments

• List of deliverables content, related activities, and decisions that should be accomplished before completion

• Assumptions specific to each testing phase• Risks• Expected Results (behavior expected after a test case step that

requires verification or validation)• Schedule


APPROVE…Filling in the Expectations.. Reality GAP

Reviews

QA Activities

Sign-Offs

Informal or Formal

Consensus

"Done"

EXPECTATIONS

REALITY


References

1. Formulating a Test Strategy for an Agile Software Project http://www.slideshare.net/tharindakasun/agile-testing-strategy

2. Agile Testing: A Beginners Guide http://www.guru99.com/agile-testing-a-beginner-s-guide.html

3. Understanding Project Management Deliverables – Tips on Managing and Monitoring PM Deliverables http://www.brighthubpm.com/risk-management/69108-defining-project-management-deliverables/

4. Software Testing Deliverables: From Test Plans to Status Reports David W. Johnson, http://searchsoftwarequality.techtarget.com/tip/Software-testing-deliverables-From-test-plans-to-status-reports

5. James, Bach, Rapid Software Testing


http://www.slideshare.net/tharindakasun/agile-testing-strategy

http://www.guru99.com/agile-testing-a-beginner-s-guide.html

http://www.brighthubpm.com/risk-management/69108-defining-project-management-deliverables/

http://searchsoftwarequality.techtarget.com/tip/Software-testing-deliverables-From-test-plans-to-status-reports