software tester’s certification exam prep€¦ · software tester’s certification exam prep...

Software Tester’sCertification Exam Prep

Course Materials & Appendicies

Copyright © 2011 byQuality Assurance Institute2101 Park Center Drive,Suite 205Orlando, FL 32835-7614Phone 407-363-1111Fax 407-363-1112Web site: www.qaiworldwide.org

Software Tester Certification Exam Prep Course

© QAI Worldwide ID_CSTE-EPC-3927

Revision and Copyright

Copyright

Copyright © Quality Assurance Institute 2011 All Rights ReservedNo part of this publication, or translations of it, may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or any other media embodiments now known or hereafter to become known, without the prior written permission of the Quality Assurance Institute.

DESCRIPTION BY DATE

Table of ContentsPage

ID_CSTE-EPC-3927 © QAI Worldwide Page iii

Software Tester’s Certification Exam Prep - - - - - - - - - - - - - - - - - - - - - -1Course Objectives - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -2Introductions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -3Why Become Certified? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -4The Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -5About the Exam - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -6Course Table of Contents - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -7

Skill Category 1 - - - - - - - - - - - - - - - - - - - - - - - - - - - - 8Testing Vocabulary - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -9Quality Assurance - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 10Quality Control - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 11Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 12Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 13Questions? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 14The Cost of Quality - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 15Cost of Quality - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 16Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 18Questions? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 19Software Quality Factors - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 20How Quality Is Defined - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 23The Views of Quality - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 24The Two Quality Gaps - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 26Questions? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 27What Is Testing? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 28Why Do We Test Software? - - - - - - - - - - - - - - - - - - - - - - - - - - - - 29Why Are Defects Hard to Find? - - - - - - - - - - - - - - - - - - - - - - - - - - - 30Test Your Sensitivity - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 31What Were Your Results? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 32The Answer! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 33What Is a Defect? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 34What Is a Process Defect? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 35Process Control - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 36Common Causes of Variation - - - - - - - - - - - - - - - - - - - - - - - - - - - - 37Special Causes of Variation - - - - - - - - - - - - - - - - - - - - - - - - - - - - 38How Are Processes Brought Under Control? - - - - - - - - - - - - - - - - - - - - 39What Is a Product Defect? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 40Defects Typically Found in Software Systems - - - - - - - - - - - - - - - - - - - - 41Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 42Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 43SEI’s Five Levels of Process Maturity - - - - - - - - - - - - - - - - - - - - - - - - 44Maturity Defined - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 45Factors Affecting Software Testing - - - - - - - - - - - - - - - - - - - - - - - - - 46Factors Affecting Software Testing (cont) - - - - - - - - - - - - - - - - - - - - - - 48Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 49Factors Affecting Software Testing - - - - - - - - - - - - - - - - - - - - - - - - - 50Testing Cost Curve - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 53Questions? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 54Phases of the SDLC Testing Process - - - - - - - - - - - - - - - - - - - - - - - - 55

Software Quality Analyst Certification Exam Prep Course

Table of Contents (Continued) Page

Page iv © QAI Worldwide ID_CSTE-EPC-3927

Test Matrices - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 56Independent Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 57Questions? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 59What Is a Process? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 60The PDCA Cycle for Building and Improving Software Testing Processes - - - - - - 61Workbench - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 62Verification and Validation Techniques - - - - - - - - - - - - - - - - - - - - - - - 63Levels of Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 64Static Versus Dynamic Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - 65A Life Cycle Quality Approach - - - - - - - - - - - - - - - - - - - - - - - - - - - 66The “V” Testing Concept - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 67Basic Test Stages - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 68Unit Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 69Integration Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 70System Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 71User Acceptance Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 72Test Techniques/Stages - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 73Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 74Questions? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 76Structural Test Techniques - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 77Functional Test Techniques - - - - - - - - - - - - - - - - - - - - - - - - - - - - 78Special Test Types - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 79White-Box Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 80White-Box Techniques - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 81Black-Box Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 82Black-Box Techniques - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 83Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 86Incremental Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 87Thread Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 88Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 89Questions? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 92Walkthroughs, Reviews, Inspections - - - - - - - - - - - - - - - - - - - - - - - - 93What Are Reviews? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 94Inspection Roles and Responsibilities - - - - - - - - - - - - - - - - - - - - - - - 95Sample Inspection Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - 96Inspection Meeting Rules - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 97Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 98Questions? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 99Special Testing Techniques - - - - - - - - - - - - - - - - - - - - - - - - - - - - 100Combining Test Techniques - - - - - - - - - - - - - - - - - - - - - - - - - - - - 101Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 102Skill Category 1 - Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 103

Skill Category 2 - - - - - - - - - - - - - - - - - - - - - - - - - - 104What Is the Test Environment? - - - - - - - - - - - - - - - - - - - - - - - - - - - 105What’s Included in the Test Environment? - - - - - - - - - - - - - - - - - - - - - 106What Is the Key Component to an Effective Test Environment? - - - - - - - - - - - 107What Is a Brief Description of Management Support? - - - - - - - - - - - - - - - - 108How Does Management Communicate and Demonstrate Its Tone? - - - - - - - - - 109Integrity and Ethical Values - - - - - - - - - - - - - - - - - - - - - - - - - - - - 110Commitment to Competency - - - - - - - - - - - - - - - - - - - - - - - - - - - - 111Philosophy and Operating Style - - - - - - - - - - - - - - - - - - - - - - - - - - 112



ID_CSTE-EPC-3927 © QAI Worldwide Page v

Organizational Structure - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 113Much of the test environment is communicated through the test work processes - - 114Process Engineering - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 115Importance of Work Processes - - - - - - - - - - - - - - - - - - - - - - - - - - 116Developing Work Processes - - - - - - - - - - - - - - - - - - - - - - - - - - - 117Attributes of a Standard for a Standard - - - - - - - - - - - - - - - - - - - - - - 118Tester’s Workbench - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 119How Is the Test Function Managed by Process? - - - - - - - - - - - - - - - - - 120Responsibility for Building Work Processes - - - - - - - - - - - - - - - - - - - - 121Key Concepts for Process Engineering - - - - - - - - - - - - - - - - - - - - - - 122Test Process Selection - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 123Professional Test Standards - - - - - - - - - - - - - - - - - - - - - - - - - - - 124Analysis and Improvement of the Test Process - - - - - - - - - - - - - - - - - - 125Reasons for Test Process Analysis - - - - - - - - - - - - - - - - - - - - - - - - 126Effectiveness and Efficiency of the Test Process - - - - - - - - - - - - - - - - - 127What Makes Process Improvement Work? - - - - - - - - - - - - - - - - - - - - 128Test Process Alignment - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 129Software Development Methodology Impacts Test Processes - - - - - - - - - - - 130Adapting Testing Processes to Different Software Development Processes - - - - 131Test Tools - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 132Selecting Test Tools - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 133Tool Usage - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 134Test Tool Guidelines - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 135Benefits of Using Test Tools - - - - - - - - - - - - - - - - - - - - - - - - - - - 136Effective Testing Necessitates Testers to Be Competent in Software Testing - - - 137Skill Category 2 - Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 138

Skill Category 3 - - - - - - - - - - - - - - - - - - - - - - - - - - - 139Test Administration - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 140Budgeting and Scheduling - - - - - - - - - - - - - - - - - - - - - - - - - - - - 141What Factors Influence Estimation? - - - - - - - - - - - - - - - - - - - - - - - - 142What Techniques Are Used to Estimate Budget? - - - - - - - - - - - - - - - - - 143Scheduling - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 144Staffing Approaches for Software Testing Projects - - - - - - - - - - - - - - - - 145Software Testing Staffing Matrix - - - - - - - - - - - - - - - - - - - - - - - - - 146What Causes the Need for the Test Processes to Be Customized? - - - - - - - - 147Software Development Constraints - - - - - - - - - - - - - - - - - - - - - - - - 148Questions? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 149Test Supervision - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 150Communication Skills - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 151Written and Oral Communication - - - - - - - - - - - - - - - - - - - - - - - - - 1523 Step Listening Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1533 Step Listening Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1543 Step Listening Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1553 Step Listening Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 156Interviewing Skills - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 157Analyzing Skills and the Analysis Process - - - - - - - - - - - - - - - - - - - - - 158Negotiations - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 159Negotiation Resolution Methods - - - - - - - - - - - - - - - - - - - - - - - - - 160Resolving Complaints - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 161Judgment - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 162Constructive Criticism - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 163



Page vi © QAI Worldwide ID_CSTE-EPC-3927

Project Relationships - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 164Motivation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 166Mentoring and Recognition - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 167Testing Leadership - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 168Team Building - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 169Team Development - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 170Team Ethics - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 171Quality Management Organization Structure - - - - - - - - - - - - - - - - - - - - 172Traditional vs Quality Management Philosophy - - - - - - - - - - - - - - - - - - - 173Why a Code of Ethics Is Important - - - - - - - - - - - - - - - - - - - - - - - - - 174Managing Change - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 175Test Manager Must Manage Change - - - - - - - - - - - - - - - - - - - - - - - - 176Software Configuration Management - - - - - - - - - - - - - - - - - - - - - - - - 177Software Change Management - - - - - - - - - - - - - - - - - - - - - - - - - - 178Software Version Control - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 179Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 180Skill Category 3 - Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 181

Skill Category 4 - - - - - - - - - - - - - - - - - - - - - - - - - - 182Concepts and Vocabulary - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 183Risks Associated with Software Testing - - - - - - - - - - - - - - - - - - - - - - 186Risks Associated with Software Development - - - - - - - - - - - - - - - - - - - 187Risks Associated with Software Testing - - - - - - - - - - - - - - - - - - - - - - 189Risks Associated with Premature Release - - - - - - - - - - - - - - - - - - - - - 190Risk Analysis - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 191Risk Analysis Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 192Form The Risk Analysis Team - - - - - - - - - - - - - - - - - - - - - - - - - - - 193Identify Risks - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 194Eliminate the Magnitude of Risk - - - - - - - - - - - - - - - - - - - - - - - - - - 195Select Testing Priorities - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 196Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 197Risk Management - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 199Risk Reduction Methods - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 200Contingency Planning - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 201Test Planning - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 202Prerequisites to Test Planning - - - - - - - - - - - - - - - - - - - - - - - - - - - 203Test Plan - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 204Attributes of a Test Plan - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 206Understanding the Characteristics of the Software Being Developed - - - - - - - - 207Build the Test Plan - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 208Set Test Objectives - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 209Develop the Test Matrix - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 210Sample Test Matrix - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 211Define Test Administration - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 212Guidelines to Writing the Test Plan - - - - - - - - - - - - - - - - - - - - - - - - - 213Test Plan Standards - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 214Skill Category 4 - Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 215

Skill Category 5 - - - - - - - - - - - - - - - - - - - - - - - - - - 216Test Case Design - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 217Types of Functional Test Cases - - - - - - - - - - - - - - - - - - - - - - - - - - 218



ID_CSTE-EPC-3927 © QAI Worldwide Page vii

Types of Structural Test Cases - - - - - - - - - - - - - - - - - - - - - - - - - - 219Types of Erroneous Test Cases - - - - - - - - - - - - - - - - - - - - - - - - - 220Types of Stress Test Cases - - - - - - - - - - - - - - - - - - - - - - - - - - - 221Test Scripts - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 222Process to Build a USE CASE - - - - - - - - - - - - - - - - - - - - - - - - - - 223System Boundary Diagram - - - - - - - - - - - - - - - - - - - - - - - - - - - - 224Define a USE CASE - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 225Information Needed to Define a USE CASE - - - - - - - - - - - - - - - - - - - - 226Develop Test Cases - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 227Correlation of USE CASES to Test Cases - - - - - - - - - - - - - - - - - - - - - 228Building Test Cases - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 229Process for Building Test Cases - - - - - - - - - - - - - - - - - - - - - - - - - 230Example Payroll Application Test Cases - - - - - - - - - - - - - - - - - - - - - 231Questions? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 232Test Scripts - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 233Test Coverage - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 234Performing Tests - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 235Performing Unit Tests - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 236Performing Integration Tests - - - - - - - - - - - - - - - - - - - - - - - - - - - 237Performing System Tests - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 238The System Test Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - 239Example of a Tester’s Dashboard - - - - - - - - - - - - - - - - - - - - - - - - - 240When Is Testing Complete? - - - - - - - - - - - - - - - - - - - - - - - - - - - 241Addressing General Testing Concerns - - - - - - - - - - - - - - - - - - - - - - 242What Is a Defect? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 243How Do We Describe a Defect? - - - - - - - - - - - - - - - - - - - - - - - - - 244Test Problem Documentation Example - - - - - - - - - - - - - - - - - - - - - - 245Problem Effect - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 246Problem Cause - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 247Defect Management and What to Report - - - - - - - - - - - - - - - - - - - - - 248The Defect Management Process - - - - - - - - - - - - - - - - - - - - - - - - - 249Success Factors for Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - 250Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 251Skill Category 5 - Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 252

Skill Category 6 - - - - - - - - - - - - - - - - - - - - - - - - - - - 253Prerequisites to Test Reporting - - - - - - - - - - - - - - - - - - - - - - - - - - 254Measures and Metrics - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 255Objective and Subjective Measures - - - - - - - - - - - - - - - - - - - - - - - - 256How Do You Know a Metric Is Good? - - - - - - - - - - - - - - - - - - - - - - - 257Productivity Versus Quality - - - - - - - - - - - - - - - - - - - - - - - - - - - - 258Test Metrics Categories - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 259Test Charts and Diagrams Used to Build Test Reports - - - - - - - - - - - - - - 260Code Coverage Analysis - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 268Test Tools Used to Enhance Test Reporting - - - - - - - - - - - - - - - - - - - 269Reporting Test Results - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 270Test Reports - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 271Guidelines for Report Writing - - - - - - - - - - - - - - - - - - - - - - - - - - - 272Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 273Skill Category 6 - Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 274



Page viii © QAI Worldwide ID_CSTE-EPC-3927

Skill Category 7 - - - - - - - - - - - - - - - - - - - - - - - - - - 275Software Acceptance Testing Concepts - - - - - - - - - - - - - - - - - - - - - - 276Four Components - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 277Roles and Responsibilities - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 278Acceptance Test Planning - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 282Acceptance Criteria - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 283Acceptance Criteria Categories - - - - - - - - - - - - - - - - - - - - - - - - - - 284Safety and Security Criteria - - - - - - - - - - - - - - - - - - - - - - - - - - - - 285Acceptance Test Plan - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 286USE CASE Test Data - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 287Acceptance Test Execution - - - - - - - - - - - - - - - - - - - - - - - - - - - - 288Possible Acceptance Decisions - - - - - - - - - - - - - - - - - - - - - - - - - - 289Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 290Skill Category 7 - Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 291

Skill Category 8 - - - - - - - - - - - - - - - - - - - - - - - - - - 292Challenges in Testing Acquired Software - - - - - - - - - - - - - - - - - - - - - - 293Software Tester’s Responsibility for Software Developed by an Outside Organization 296COTS Specific Testing Challenges - - - - - - - - - - - - - - - - - - - - - - - - - 297COTS Testing Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 298Contracted Software Specific Testing Challenges - - - - - - - - - - - - - - - - - - 306Contracted Software Test Process - - - - - - - - - - - - - - - - - - - - - - - - - 307Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 318Skill Category 8 - Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 319

Skill Category 9 - - - - - - - - - - - - - - - - - - - - - - - - - - 320Internal Control Defined - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 321Internal Control Terms - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 322Internal Auditor’s Responsibilities - - - - - - - - - - - - - - - - - - - - - - - - - 324Risk Versus Control - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 325Levels of Organizational Control - - - - - - - - - - - - - - - - - - - - - - - - - - 326Internal Control Models - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 329Testing Internal Controls - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 332Testing Security Controls - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 334Skill Category 9 - Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 340

Skill Category 10 - - - - - - - - - - - - - - - - - - - - - - - - - - 341Risks Associated with New Technology - - - - - - - - - - - - - - - - - - - - - - 342Risks Associated with Web-based Applications - - - - - - - - - - - - - - - - - - - 344Risks Associated with Distributed Application Architecture - - - - - - - - - - - - - 345Risks Associated with Wireless Technology - - - - - - - - - - - - - - - - - - - - 346Risks Associated with New Business Models - - - - - - - - - - - - - - - - - - - - 347Risks Associated with New Communication Methods - - - - - - - - - - - - - - - - 348Risks Associated with New Test Tools - - - - - - - - - - - - - - - - - - - - - - - 349Testing the Effectiveness of Integrating New Technology - - - - - - - - - - - - - - 350Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 351Skill Category 10 - Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 352

Test Taking Tips - - - - - - - - - - - - - - - - - - - - - - - - - - 353On the Day of the Exam - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 354



ID_CSTE-EPC-3927 © QAI Worldwide Page ix

About the Exam - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 355After the Exam - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 356First Thing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 357Tip #1 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 358The How-to’s - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 359Sample Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 360More How-to’s - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 361Second Round - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 362Check Back - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 363Don’t Change Anything - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 364Change the Chances - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 365Fluff and Stuff - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 366The Distracters - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 367Test Taking Tip Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 368The End - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 369

Appendix A How to Take the CSTE Examination - - - - - - - - - A-1A.1 CSTE Examination Overview - - - - - - - - - - - - - - - - - - - - - - - - -A-1A.2 Guidelines to Answer Questions - - - - - - - - - - - - - - - - - - - - - - - -A-2A.3 Sample CSTE Examination - - - - - - - - - - - - - - - - - - - - - - - - - -A-4



Page x © QAI Worldwide ID_CSTE-EPC-3927

This page intentionally left blank.

Software Tester Certtification Exam Prep Course

ID_CSTE-EPC-3927 © QAI Worldwide Page 1

KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSIn this section we’ll review basic quality principles, learn to distinguish quality assurance from quality control activities, and examine the current challenges test professionals are facing.

NOTES:


Page 2 © QAI Worldwide ID_CSTE-EPC-3927

Course ObjectivesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS

• Reinforce current knowledge• Introduce new principles and concepts • Provide you with a background of the IT testing fundamentals• Help the testing professional study and prepare for the CAST examination.

- Due to time limitations of this course, it is not intended to cover all the components of the common body of knowledge (CBOK) in great depth.

NOTES:



IntroductionsKEY CONCEPT PRESENTATION GUIDE


• Who are you and your company• Why and when are you taking the exam• Have you taken an exam before – where/when• Time you spend studying now• Study group experience

NOTES:



Why Become Certified?KEY CONCEPT PRESENTATION GUIDE


• The IT and software testing industry is competitive• Recognition by peers• Proof of your profession and skills in the IT field• Increased confidence in personal capabilities • Indicates a professional level of competence • Potentially more rapid career advancement • Greater acceptance in the role as advisor to management

NOTES:



The ProcessKEY CONCEPT PRESENTATION GUIDE


• Satisfy all of the prerequisites required prior to applying for candidacy- One of the following must be met (Effective June 2007):

• 4 + 2 = Bachelor’s degree in an accredited college-level institution and 2 years experience or…

• 2 + 4 = Associate’s degree and higher and 4 years experience or…• 0 + 6 = No degree and 6 years experience in the IT service area

• Subscribe to the Code of Ethics • Submit a completed Certification Candidacy Application

NOTES:



About the ExamKEY CONCEPT PRESENTATION GUIDE


Four parts to the exam Part Questions Type 1 50 Objective or multiple choice questions.

2 10 Essay or Scenario based questions. 3 50 Objective or multiple choice question. 4 10 Essay or Scenario based questions.

Minimum passing = overall average of 75%

NOTES:



Course Table of ContentsKEY CONCEPT PRESENTATION GUIDE


• Software Testing Principles and Concepts• Building the Test Environment• Managing the Test Project• Test Planning• Executing the Test Plan• Test Status, Analysis and Reporting• User Acceptance Testing• Testing Software Developed by Outside Originations• Testing Software Controls and the Adequacy of Security Procedures• Testing New Technologies

NOTES:



Skill Category 1KEY CONCEPT PRESENTATION GUIDESoftware Testing Principles and Concepts


The following topics will be discussed in this Skill Category:• Vocabulary• Why Do We Test Software?• Factors Affecting Software Testing• Life Cycle Testing• Test Matrices• Independent Testing• Tester’s Workbench• Levels of Testing• Testing Techniques

NOTES:



Testing VocabularyKEY CONCEPT PRESENTATION GUIDE


Understand the technical terms used to describe various testing techniques, tools, principles, concepts and activities

NOTES:



Quality AssuranceKEY CONCEPT PRESENTATION GUIDE


The Quality Assurance function is charged with implementing the quality policy established by executive management. QAI recommends that the QA director position be a strong leadership position, and emphasizes the strong interpersonal skills necessary to successfully execute activities involved in making improvement occur.• Activities that modify the development process to prevent the introduction of flaws

- Staff function- Implements management’s quality policies- Responsible for continuous improvement of the software development process- Proactive approach focused on defect prevention

• Examples:- Defining change control procedures- Facilitating post-project reviews- Analyzing metrics to identify opportunities for process improvement

NOTES:



Quality ControlKEY CONCEPT PRESENTATION GUIDE


The Quality Control function is charged with ensuring the product quality is compared with the product standard, reports any non-conformance to those standards and verify the action taken when non-conformance is detected corrects the defect.

• Activities within the development process to detect the introduction of flaws- Test planning and execution- Quality control measures a product against the existence of an attribute - Determines whether the product conforms to a standard or procedure (also known as

compliance checking).• Proactive approach focused on defect detection• Examples:

- Writing and executing test cases and scripts- Participating in verification and validation activities- Reporting defects to identify opportunities for process improvement

NOTES:



ExerciseKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSQuality Assurance or Quality Control?

Inspection of source code.

Unit testing to validate that the program works.

Analysis of defects to determine the stage of origin.

Analyze metrics collected that measure the effectiveness of system and unit Testing

NOTES:

QA QCQA QC





The responsibilities of this job include facilitation, process configuration, measurement, and risk analysis.a. Quality Controlb. Quality Assurance

The responsibilities of this job include conducting inspections, reviews, testing and focusing on the product.a. Quality Controlb. Quality Assurance

NOTES:



Questions?KEY CONCEPT PRESENTATION GUIDE


NOTES:



The Cost of QualityKEY CONCEPT PRESENTATION GUIDE


• Money spent beyond the cost to build the product right the first time• Frequently referred to today as Cost of Non Quality or Cost of Poor Performance

NOTES:



Cost of QualityKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe total production costs associated with the delivery of software applications include both the actual production costs and the cost of quality. The first component - production costs - consists of the costs associated with producing the product “right the first time,” or RTF costs. These costs include labor, materials, and equipment (hardware, software, tools) associated with the actual development of the application. The second component includes the additional costs associated with assuring that the product delivered meets the quality goals established for the product. This cost component is called the Cost of Quality, and includes all costs associated with the prevention, identification, and correction of product defects (includes repair and damage costs.)The cost of quality will vary from one organization to the next. The majority of costs associated with the Cost of Quality are associated with the identification and correction of defects• Preventive Costs - Costs associated with preventing errors

- Training- Establishing methods and procedures- Tool acquisition

• Appraisal Costs - Costs associated with the detection of errors- Inspections- Testing

• Failure Costs - Costs associated with defective products delivered to customers- Repairs- Staffing Help Desk- Damage caused by defect

NOTES:



Cost of Quality (cont)KEY CONCEPT PRESENTATION GUIDE


Mark “P” for Prevention; “A” for Appraisal; “F” for Failure:• Conducting Reviews and Inspections• Implementing Standards• Conducting Rework• Attending Training/Education• Planning for Quality• Code Inspection• Cancelled Projects• Failure & Recovery of Data• Defect Reporting Standards• Quality Improvement• Testing• Conducting a Post Project Review• Lost Benefits

NOTES:





Which of the following is NOT a category of the Cost of Quality?a. Failure Costb. Appraisal Costc. Build Costd. Preventive CostIn defining the cost of quality, appraisal costs are BEST described as:a. Costs incurred to review completed products against requirementsb. Costs which can not be recoupedc. All costs associated with defective productsd. None of the above

NOTES:





NOTES:



Software Quality FactorsKEY CONCEPT PRESENTATION GUIDE


NOTES:

Correctness ReliabilityEfficiency Integrity

Usability

MaintainabilityFlexibilityTestability

PortabilityReusability

Interoperability

Correctness ReliabilityEfficiency Integrity

Usability

MaintainabilityFlexibilityTestability

PortabilityReusability

Interoperability



Software Quality Factors (cont)KEY CONCEPT PRESENTATION GUIDE


NOTES:



Software Quality Factors (cont)KEY CONCEPT PRESENTATION GUIDE


• Attributes of software• Needed for trade off decisions• Should be included in software requirements• Should be included in the test plan• Lack of the needed factors cause customer dissatisfaction

NOTES:



How Quality Is DefinedKEY CONCEPT PRESENTATION GUIDE


NOTES:

QUALITY IN PERCEPTIONQUALITY IN FACT

Treating every customer with integrity, courtesy, and respect.

Doing it on time.

Meeting the customer’s expectations.Doing it right the first time.

Satisfying our customer’s needs.Doing it the right way.

Delivering the right product.Doing the right thing.

QUALITY IN PERCEPTIONQUALITY IN FACT

Treating every customer with integrity, courtesy, and respect.

Doing it on time.

Meeting the customer’s expectations.Doing it right the first time.

Satisfying our customer’s needs.Doing it the right way.

Delivering the right product.Doing the right thing.



The Views of QualityKEY CONCEPT PRESENTATION GUIDE


An organization’s quality policy must define and view quality from their customer's perspectives. If there are conflicts, they must be resolved.Producer’s view:Conformance to requirements• Doing the right thing• Doing it the right way• Doing it right the first time• Doing it on time without exceeding cost

NOTES:



The Views of Quality (cont)KEY CONCEPT PRESENTATION GUIDE


Customer’s view:Fit for use• Receiving the right product for use• Being satisfied that needs have been met• Expectations have been met• Treated with integrity, courtesy, and respect

NOTES:



The Two Quality GapsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSQuality: A product is a quality product if it is defect free. To the producer, a product is a quality product if it meets or conforms to the statement of requirements that defines the product. This statement is usually shortened to: quality means it meets requirements. From a customer’s perspective, quality means “fit for use.”Which is better view? Two Working Definitions:• Producer’s View - Meets requirements• Consumer’s View - Fit for use

NOTES:

Start

ProducerGap

CustomerGap

Quality as delivered

Producer view of quality as specified

Customer view of quality as expected

Start

ProducerGap

CustomerGap

Quality as delivered

Producer view of quality as specified

Customer view of quality as expected

Producer’s View of Quality, as Specified

Customer’s View of Quality, as Expected





NOTES:



What Is Testing?KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSGlenford J. MyersDifferences in the definition of testing are important because an incorrect understanding of the work often results in a failure to test a system well. Defining testing as the process of finding defects suggests a destructive attitude with which many people are uncomfortable -- particularly developers. Many project team members do not like testing, either because it consumes valuable time, or because defects are thought to symbolize a failure to perform well. Project and test managers should make it clear that a successful test is one that finds defects, rather that one that is defect-free. Testing must be an integral part of the software development life cycle. By verifying and validating artifacts throughout the lifecycle, teams can reduce rework and deliver higher quality applications. If testing is viewed a a final exam, then defects aren’t uncovered until the end of the project where they are much more costly to correct.Debugging is not testing. Testing identifies defects, debugging corrects them.Testing IS - The process of evaluating a deliverable with the intent of finding errors. • Testing is not a stage/phase of the project • Testing is not just finding broken code• Testing is not a final exam • Testing is not “debugging”

NOTES:



Why Do We Test Software?KEY CONCEPT PRESENTATION GUIDE


• To find defects• To reduce risk inherent in computer systems • To satisfy the customer• To prove that a program is good or no good• Developers are unable to build defect-free software• The user / customer does not know what they want• The development process is defective• To detect variations from specification/expectation• To establish confidence that a program does what it is supposed to do

NOTES:



Why Are Defects Hard to Find?KEY CONCEPT PRESENTATION GUIDE


Finding defects is not an easy process. If it were, testing would not consume approximately ½ of the total development effort. This slide lists some of the reasons why defects are hard to find.

• Not looking Tests often are not performed because particular test condition was unknown• Looking but not seeing• Like losing your keys only to discover they were in plain sight the entire time• The size and complexity of applications often makes it impossible to test everything

NOTES:



Test Your SensitivityKEY CONCEPT PRESENTATION GUIDE


• First read the sentence enclosed in the box that next appears.• Then, count the F’s in the sentence.• Count them only once.• Do not go back and count them again!

NOTES:

FINISHED FILES ARE THE RESULT OF YEARS OF SCIENTIFIC STUDY

COMBINED WITH THE EXPERIENCE OF MANY YEARS.





What Were Your Results?KEY CONCEPT PRESENTATION GUIDE


Number of F’s __________

NOTES:



The Answer!KEY CONCEPT PRESENTATION GUIDE


There are 6 F’s...

NOTES:







What Is a Defect?KEY CONCEPT PRESENTATION GUIDE


A defect is defined as the lack of a desirable state. Two Types of Defects:• Process Defects• Product Defects

NOTES:



What Is a Process Defect?KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSA process is considered defective when the output of that process is outside the accepted control limits.A defective process is a process that will produce defects because the process is flawed. For example, in cutting wood, you cannot get a clean cut from a saw that has broken teeth. Likewise, if a programmer is not properly trained in how to save information on a PC under certain circumstances, that information will be lost, and the programmer will have to repeat the steps to recreate the information.Sigma is a statistical term dealing with standard deviation. The ‘Sigma’ concept states that if one can measure the amount of “defects” in a process, one can systematically determine how to eliminate them, getting as close to zero defects (i.e., perfection) as possible. Motorola developed the Sigma system because their workers, which were conditioned in academia, believed that 99% was close to perfections. However, if only 99,000 out of 100,000 lines of code produced by a certain process worked correctly that would not be viewed as a near perfect process. In order to achieve Six Sigma, the process cannot produce more than 3.4 defects per million opportunities (opportunity being defined as a chance for nonconformance or not meeting the required expectations). Note that Four Sigma is typical of most job shop processes such as physicians writing prescriptions, wait staff creating bills for restaurant customers, and IT professionals building software.

NOTES:



Process ControlKEY CONCEPT PRESENTATION GUIDE


Out of control process

NOTES:



Common Causes of VariationKEY CONCEPT PRESENTATION GUIDE


All processes contain some inherent variation, or common causes of variation. The amount of variation in a process is quantified with summary statistics.In a computer operation, abnormal terminations cause variation. Typical common causes of abnormal terminations included invalid data, no disk space, errors in operating or job control instructions, etc.

NOTES:



Special Causes of VariationKEY CONCEPT PRESENTATION GUIDE


• Special causes of variation are not present in a process • They occur because of special or unique circumstances• In the IT example of abnormal terminations in a computer operation, special causes might

include:- operator strikes- citywide power outages- earthquakes or hurricanes

NOTES:



How Are Processes Brought Under Control?KEY CONCEPT PRESENTATION GUIDE


• Understand process maturity• Continued process improvement• Testers should be a contributorDo Testers Need to Know Statistical Process Control (SPC)?• The concept of measuring and reducing variability is commonly called statistical process

control (SPC).• Testers need to understand process variability, because the more variance in the process

the greater the need for software testing.

NOTES:



What Is a Product Defect?KEY CONCEPT PRESENTATION GUIDE


A defect - or non-conformance to specifications - could include: • A product requirement that has not been met• A product attribute incorrectly implemented in the product• A function performed by a product that is not in the statement of requirements that define the

product• Anything that causes customer dissatisfaction, whether it is in the statement of requirements

or not

• Producer view- A deviation from specification; missing, wrong, or extra

• Customer view- Anything that causes customer dissatisfaction, whether in the specifications or not

• Two types of software product defects- Software design defects- Data defects

NOTES:



Defects Typically Found in Software SystemsKEY CONCEPT PRESENTATION GUIDE


Defects Typically Found in Software Systems Are the Results of:• IT improperly interprets requirements• Users specify the wrong requirements• Requirements are incorrectly recorded• Design specifications are incorrect• Program specifications are incorrect• Errors in program coding• Data entry errors• Testing errors• Mistakes in error correction• The corrected condition causes another defect

NOTES:





How might the unique aspects of the development processes used to develop the following projects impact your testing process?

Project 1: Maintenance (revision) of a Mainframe Project Project 2: E-commerce Project 3: New project with new technology

NOTES:



QuestionsKEY CONCEPT PRESENTATION GUIDE


NOTES:



SEI’s Five Levels of Process MaturityKEY CONCEPT PRESENTATION GUIDE


In other words, a look at management commitment is a type of culture management has established within the IT organization. The more mature the culture, the higher the probability of management commitment to activities such as tough software testing.Culture 1: Manage People – management only wants results and is not committed to support of how those results are achievedCulture 2: Manage Processes – management has accepted responsibility of the effectiveness of software testing and asks testers to follow the processes. However, since the processes are not fully mature, management support may waiver on the type of results achievedCulture 3: Manage Capabilities – at this level, management is committed to activities such as software testingCulture 4: Manage by Fact – management is now able to quantitatively calculate the effectiveness and efficiency of software testing in an effort to identify how to improve software testingCulture 5: Business Innovation – at this level, management recognizes that software testing represents the user/customer and is committed to testing, assuring that the customer’s true needs are met, not just the software specifications

NOTES:



Maturity DefinedKEY CONCEPT PRESENTATION GUIDE


The maturity of the product equals the maturity of the process

NOTES:



Factors Affecting Software TestingKEY CONCEPT PRESENTATION GUIDE


• People relationships• Scope of testing (producer vs. customer)• Understanding life cycle testing• Plan your work and work your plan • Testing constraints

NOTES:



Factors Affecting Software Testing (cont)KEY CONCEPT PRESENTATION GUIDE


People Related Challenges• Training in testing• Relationship building with developers• Using tools• Getting managers to understand testing• Communicating with users about testing• Making the necessary time for testing• Testing “over the wall” software• Trying to hit a moving target • Fighting a lose-lose situation• Having to say “no”From: Surviving the Top Ten Challenges of Software Testing, A People-Oriented Approach By: William Perry and Randall Rice

NOTES:





The Scope of Software Testing• Software testing can compensate for the fact that the software development process does

not identify the true needs of the user.• Finding defects early in the software development process.• Removing defects of all types prior to software going into production.• Identifying weaknesses in the software development process.

NOTES:





Why do we test software?List at least 5 reasons and be prepared to explain your answer

• To produce a quality product, which is defect free, and also to make sure all the requirements are satisfied and the best design system architecture is used.

• To Satisfy the Customer (customers are the king) • To make sure the software meets the quality attributes (you can list them here)• To achieve the goals of an organization as well as to make profit. • To reduce the possible risks associated with the software, then reduce the loss, which might

happen when/if the software is released with defects included

NOTES:



Factors Affecting Software Testing KEY CONCEPT PRESENTATION GUIDE


Misunderstanding Life Cycle TestingLife Cycle Phase• Requirements• Design• Program (build/construction)• Test• Installation• Maintenance

NOTES:




SUPPORTING / EXPLANATORY MATERIALSDevelop Your Test Plan

NOTES:




SUPPORTING / EXPLANATORY MATERIALSTesting Constraints Movement of one of the testing constraints will cause one or more of the other constraints to move

NOTES:

Scope Schedule

QualityResources Technology

ScopeScope ScheduleSchedule

QualityQualityResourcesResources Technology



Testing Cost CurveKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSAfter the optimum test point the cost of testing to uncover defects exceeds the losses from those defects.

NOTES:





NOTES:



Phases of the SDLC Testing ProcessKEY CONCEPT PRESENTATION GUIDE


NOTES:



Test MatricesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSTest matrices shows the interrelationship between functional events and tests.• The left side of the matrix shows the functional events and the top identifies the tests that

occur on those events. • Within the matrix cells are the process that needs to be tested.

NOTES:



Independent TestingKEY CONCEPT PRESENTATION GUIDE


Ideally independent testers will report to the project manager and have a reporting structure independent from the group designing or developing the application in order to assure that the quality of the application is given as much consideration as the project budget and timeline. The test manager’s responsibilities include:• Test planning and estimation• Designing the test strategy• Reviewing analysis and design artifacts• Chairing the Test Readiness Review• Managing the test effort• Overseeing acceptance tests

NOTES:



Independent Testing (cont)KEY CONCEPT PRESENTATION GUIDE


Testers are usually responsible for:• Developing test cases and procedures• Test data planning, capture, and conditioning• Reviewing analysis and design artifacts• Testing execution• Utilizing automated test tools for regression testing • Preparing test documentation• Defect tracking and reporting

NOTES:





NOTES:



What Is a Process?KEY CONCEPT PRESENTATION GUIDE


A process is a set of activities that represent the way work is performed. This work effort includes efforts of people and equipment guided by policies, standards, and procedures. The outcome from a process is usually a product or service.

NOTES:



The PDCA Cycle for Building and Improving Software Testing ProcessesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe PDCA cycle explains both how to build and improve software testing processes. To build processes:• Establish the process policies/objectives (i.e., plan)• Build that plan into procedures to do work• Build that plan to include procedures to check work• Build that plan for procedures which monitor and identify problems so that detected defects

in execution of the process can be executedTo improve the process, accumulate those defects and plan to minimize or eradicate them through process improvement (i.e., via planning), “Do” implement those improvements in the process, “Check” to see that those processes work, and if not, take “Action” to make changes.

NOTES:

CHECK • Confirm improvements• Measure performance i.e.

QC monitors the use of the template

ACT • Reassess process• ID next set of improvements

i.e. assess template usage and rework template and process as needed

PLAN• Identify business goals i.e. use

of a test plan template • Conduct assessment• ID needed improvements• Create action plan i.e. create

a test plan template

DO• Formulate improvements

i.e. train testers on the use of the test plan template

• Implement i.e. begin to use the test plan template

CHECK • Confirm improvements• Measure performance i.e.

QC monitors the use of the template

ACT • Reassess process• ID next set of improvements

i.e. assess template usage and rework template and process as needed

PLAN• Identify business goals i.e. use

of a test plan template • Conduct assessment• ID needed improvements• Create action plan i.e. create

a test plan template

DO• Formulate improvements

i.e. train testers on the use of the test plan template

• Implement i.e. begin to use the test plan template



WorkbenchKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe workbench is a graphic representation of the three attributes of a process. Note that the workbench has both inputs and outputs. Input initiates the process and output, or products, are a result of the process.

NOTES:



Verification and Validation TechniquesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSIt is the responsibility of every project team member, not just the test or QA team, responsibility to assure the quality of the products. Developers, managers, and support teams all have a role in assuring quality.Participation in verification techniques, such as walkthroughs and inspections, is critical to the delivery of quality applications. These activities can be supported -but not replaced - by automation.For example, automation can be used to verify programming syntax but can not assure the developer is following the design strategy outlined for the project. Subjective criteria requires expert opinion which only humans can provide.Techniques for quantifiably assuring that a work product meets its stated objectives: • Verification: Performed during development on key artifacts

- Walkthroughs, reviews and inspections- Mentor feedback, training, checklists and standards

• Validation: Performed after a work product is produced- Validating against established criteria - Ensuring product integrates correctly into the environment

• Reactive approach focused on defect detection and removal

NOTES:



Levels of TestingKEY CONCEPT PRESENTATION GUIDE


• Structural Tests- Tests that validate system architecture is structural testing. Also considered white box

testing because knowledge of the internal logic of the system is used to develop test cases.

• Functional Tests- Tests that validate system are called functional testing. This testing addresses the

overall behavior of the program by testing transacting flows, input validation, and functional completeness but no knowledge of the internal logic system is used (black box).

• Non-Functional Tests (Software Quality Factors)- Tests that validate system characteristics, such as performance, stability, maintainability,

usability, and security. Can be considered a user’s point of view.

NOTES:



Static Versus Dynamic TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe tools and techniques described will be divided into verification and validation. The verification, or static tests, are basically techniques, meaning that the practice is self-contained. The validation, or dynamic tests, are a combination of tools and techniques. For example, the technique of developing test cases might be added by a tool, which will record, and normally expand, the number of test cases based on input parameters to the tool.Verification – Static Test• Walkthroughs• Reviews• Inspections

Validation – Dynamic Test• Test cases• Scripting

NOTES:

• Feasibility reviews• Requirements reviews• Design reviews• Code walkthroughs• Code Inspections• Requirements tracing

• White Box• Black Box



A Life Cycle Quality ApproachKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe V-Model enables teams to significantly increase the number of defects identified and removed during the development life cycle by integrating inspection and test activities into all stages of developmentTest planning activities are started early in the project, and test plans are detailed in parallel with requirements. Various verification techniques are also utilized throughout the project to: • Verify evolving work products • Test evolving applications by walking through scenarios using early prototypesRemoving defects in the stage of origin results in:• Shorter time to market• Lower error correction costs• Fewer defects in the production system• Early test planning yields better test plans that can be used to validate requirementsThe V-Model Focuses on:• Testing throughout development life-cycle• Early development of test requirements• Early detection of errors• At pre-determined points• Involves static and/or dynamic test techniques

NOTES:



The “V” Testing ConceptKEY CONCEPT PRESENTATION GUIDE


NOTES:

Operational orBusinessNeed

VerifyOperational

or Business Need

DefineRequirements

VerifyRequirements

DesignSystem

VerifyDesign

BuildSystem

VerifyConstruction

AcceptanceTest

ValidateOperational

or Business Need

SystemTest

ValidateRequirements

IntegrationTest

ValidateDesign

UnitTest

ValidateConstruction

Static Dynamic

Validates

Validates

Validates

Validates

Operational orBusinessNeed

VerifyOperational

or Business Need

DefineRequirements

VerifyRequirements

DesignSystem

VerifyDesign

BuildSystem

VerifyConstruction

AcceptanceTest

ValidateOperational

or Business Need

SystemTest

ValidateRequirements

IntegrationTest

ValidateDesign

UnitTest

ValidateConstruction

Static Dynamic

Validates

Validates

Validates

Validates



Basic Test StagesKEY CONCEPT PRESENTATION GUIDE


• User Acceptance Test• System Test• Integration Test• Unit Test

NOTES:



Unit TestingKEY CONCEPT PRESENTATION GUIDE


Unit testing achieves confidence in individual units before integrating them into larger components. This is the only stage of testing where 100% code coverage is reasonably attainable. Effective unit testing detects the majority (>65%) of defects identified during testing.It is difficult to get developers to plan for unit testing or to report defects identified during unit test execution. Unit testing can require test harnesses, stubs or drivers for execution. Note that the development of these items adds time to the development schedule. What? Testing individual programs or components to validate that the logic works

according to specification Validates technical quality of the code

Who? Conducted by the developer who created the componentWhat? Begins once component development is complete

NOTES:



Integration TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSIntegration testing is conducted to validate that the design was implemented correctly. Integration testing is critical in client/server and web-based applications. Even if the individual components were successfully unit tested, integrating these components usually uncovers defects originating in design. For these applications, it is recommended that the major components be integrated first. For example: • integrate the server components and test• integrate the client components and test• test the network• integrate the server and network with the clientThis approach allows the development team to identify the source of integration errors more efficiently than if they integrated the entire application prior to testing.What? Tests the integration of components that have been successfully unit-tested

Validates the technical quality of the designValidates the integration of the application and the environment

Who? Conducted by development with support from the test teamWhen? Begins once the first components have passed unit test

NOTES:



System TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSSystem testing is primarily requirements-based testing. The goal is to validate that the requirements were implemented correctly. In client/server and web-based applications, system testing also verifies that the application integrates well with network and operating environment, as well as functions appropriately with all supported end-user platforms.Tests designed are both positive and negative, and should include scenarios that integrate the application with manual business procedures. It is also recommended that the test team run through the User Acceptance Test (UAT) plan to ensure that the UAT will run smoothly.What? Tests the entire assembled system

Validates delivery of functional and non-functional requirementsValidates interface to upstream and downstream applications

Who? Conducted by independent test teamWhen? Begins when integration testing has been successfully completed

NOTES:



User Acceptance TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe goal of User Acceptance Testing is to prove to the users or customer that you have delivered the correct application functionality - I.e. that the system is “fit for use”Multiple approaches can be selected, including:• Walkthrough business scenarios to demonstrate functionality to users in a test environment• Run the new application in parallel with the old one until they are confident that the new

application will meet their needs• Conduct pilots with the new application prior to full roll-outUsually, the users formally sign-off on the test results to indicate their acceptance of the system. Note that the user community may withhold sign-off until any issues identified during the test are resolved.What? Validates system is “fit for use”

Evaluates how system integrates with manual business processesWho? Conducted by end users, customers or designated representativesWhen? Begins when system test has been successfully completed

NOTES:



Test Techniques/StagesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSDuring test planning, the test manager must combine test techniques to achieve the maximum amount of coverage. This table illustrates that some techniques are more effective during certain stages of testing than others. The most effective test strategies utilize a combination of approaches to achieve the test objectives.

NOTES:

TestTechniquesTest Stages

WhiteBox

BlackBox

RegressionTesting

Unit Test 80%

IntegrationTest

60% X

System Test 80% X

AcceptanceTest

99%

X

X

20%

40%

20%

1%

TestTechniquesTest Stages

WhiteBox

BlackBox

RegressionTesting

Unit Test 80%

IntegrationTest

60% X

System Test 80% X

AcceptanceTest

99%

X

X

20%

40%

20%

1%





You feel that a Software Development Life Cycle testing model should be implemented in your testing approach. What justification would you provide to your manager to sell a defined life cycle approach?By implementing inspection and test activates throughout the development lifecycle, project teams are able to deliver better customer service. Key points to remember include:• Continuous verification ensures that the correct system is being built• Fewer defects are discovered in production• Prioritize inspection and test activities to yield highest quality software at lowest possible

cost. - Target high-risk development areas for early, increased testing- Identify areas of design complexity- Identify complex requirements, either structural or functional

• Using standard metrics enables management to easily measure project status and Quality attributes.

• Reduce the overall effort required for delivering new business functionality• Total testing time is reduced when problems are found early• Lower product development and maintenance costs by reducing rework

NOTES:



Exercise (cont)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSIn software testing there are at least four stages of testing. List the sequence in which those four stages of testing should occur from one, (which is the first), to four, (which is the last). Briefly explain the objective of each of these four stages of testing. • Unit Testing – Conducted by the Developer - Testing individual programs or components to

validate that the logic works according to specification and validates technical quality of the code

• Integration Testing – Conducted by development with support from the test team - Tests the integration of components that have been successfully unit-tested, validates the technical quality of the design and the integration of the application and the environment

• System Testing – Best performed by an independent test team - - Tests the entire assembled system and validates delivery of functional and non-functional requirements.

• User Acceptance Testing – Conducted by end users, customers or designated representatives - Validates system is “fit for use” and Evaluates how system integrates with manual business processes

NOTES:





NOTES:



Structural Test TechniquesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSStructural system testing is designed to verify that the developed system and programs work.

NOTES:



Functional Test TechniquesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSFunctional system testing ensures that the system requirements and specifications are achieved.

NOTES:



Special Test TypesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThese special test types are not applicable to all projects. The project test objectives and quality goals should be reviewed to determine if any of these are necessary. When designing a project test strategy, applicable test types must be scheduled and integrated into the basis test stages selected.• Usability Test - Identifies areas of poor human factors design that may make the system

difficult to use. Most beneficial if conducted with user representatives or customers during prototyping

• Vendor Validation Test - Validates functionality prior to acceptance of third party Commercial Off The Shelf (COTS) software. Should test all standard functions and data combinations

• Conversion Test - Validates effectiveness of conversion process (field-to-field mapping, data translation, etc.). Confirms application-to-application balancing and application- to-General Ledger balancing of converted data

• Stress/Load Test - System subjected to environmental conditions which defy normal expectations (high transaction volume, large database size, etc.). Intended to identify constraints and ensure there are no performance problems; results useful in capacity planning

• Performance Test - A variation on volume and stress tests that measures system performance against specified service-level objectives. Validates that on-line response time and batch run times meet performance requirements

• Configuration Testing - Necessary in client/server and web-based applications to validate that the application will run correctly on all supported end-user environments.

• Contingency Test - Evaluates the contingency features built into the application for handling interruptions and for returning to specific points in the applications processing cycle. Checkpoints, backups, restores, and restarts are tested here for execution and design deficiencies

• Security Test - Validates that the application is protected from unauthorized use. Ensures data protected from unauthorized access and/or modification. Validates network security checkpoints

NOTES:



White-Box TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThese tests answer the question: “How has the system been implemented?”White-box testing is most often used in program-based or structural testing. The tester must have detailed knowledge of the internal code structure in order to plan and execute these tests.There are certain types of errors that can only be detected using white-box tests. For example, decision point logic errors, code that is not called, etc. Skipping these tests will result in a high probability that some defects will go undetected during system test. • Structural test technique• Testing based on knowledge of internal code structure and logic - usually logic driven

NOTES:

Internal View

What is the system doing?How is it doing it?

Input

Output

Internal View

What is the system doing?How is it doing it?

InputInput

OutputOutput



White-Box TechniquesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSStatement Coverage:A white-box technique that measures the number of - or percentage - of statements executed by the test cases designed. 100% statement coverage would indicate that all statements had been executed at least once during testing.

Basis path Coverage:A Basis Path is a complete execution path through the application. This type of test is also known as “path coverage”, and measures the number of - or percentage - of decision directions executed by the test cases designed. 100% decision coverage would indicate that all decision directions had been executed at least once during testing.

Decision to Decision Point Coverage:This technique executes all possible combinations of decision points. It includes execution of all condition outcomes in each decision as well as all outcome combinations. It also assures that each point of entry has been invoked at least once by the executed cases.There are other white-box techniques that developers have to choose from. The technique selected should be driven by the complexity of the component. The more complex a component is, the higher the probability of errors.

NOTES:



Black-Box TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSBlack-box testing is also referred to as requirements-based, behavioral or functional testingThe tester does not have to understand the internal code structure in order to design and execute these tests. The tester utilizes requirements and analysis artifacts to design test cases. The focus is on achieving the correct output for a set of input conditions.• Functional test technique• Testing based on external specifications without knowledge of how the system is

constructed - usually data or business process driven• Validates that each input produces the appropriate output

NOTES:

External View

What did the application do?What should it have done?

Input

Output

External View

What did the application do?What should it have done?

InputInput

OutputOutput



Black-Box TechniquesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSExcept for the most high risk applications, such as air control and defense systems, exhaustive testing is not practical or achievable. Testers must choose test techniques that validate as much functionality as possible with the smallest number of test cases.• Equivalence partitioning • Test cases generated using a subset of data to represent a larger class of dataExample:A business rule edits credit limits within a given range ($10,000 - $15,000) could have three equivalence classes:

Less than $10,000 (Invalid)Between $10,000 and $15,000 (Valid)Greater than $15,000 (Invalid)

NOTES:



Black-Box Techniques (cont)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSBoundary analysis • Test cases designed using data from the limits of the input and output domain classesExample (continued)Testing the boundaries of a business rule that edits credit limits within a given range ($10,000 - $15,000) would test:• Low boundary +/- one ($9,999 and $10,001)• On the boundary +/- one ($10,000 and $15,000)• Upper boundary +/- one ($14,999 and $15,001)

NOTES:



Black-Box Techniques (cont)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSExploratory Testing (Ad Hoc or Error Guessing)• Test cases and data designed based upon experience and intuition of the testerExampleTesting the date-driven activities performed on credit reports, the tester might test the dates below based upon experience that year-end, month-end, and other dates usually cause problems:2/29/2006

NOTES:




SUPPORTING / EXPLANATORY MATERIALSUnit and Integration testing primarily focuses on which of the following?a. Verificationb. Validationc. Test planningd. Both a) and c).

Tests that validate system requirements are often called.a. Functional Testsb. Structural Tests

NOTES:



Incremental TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSTop-down: Start from the top of the module hierarchy diagram and work down to the bottom using interim stub modules. A stub module is a skeleton module that holds the place of a real module to be developed or added later.Bottom-upStart from the bottom of the module hierarchy diagram and work up to the top. Bottom-up testing requires the development of interim driver modules or test harnesses. Driver modules provide the test input, call the module(s) being tested, and display test output.Component (unit) tested programs are added to a given module one by one and each resulting combination is testedWhich to choose? Although both of these approaches are effective, bottom-up testing is often thought to be easier to use.Usually, driver modules are easier to create than stubs. Because a driver module can call another module a number of times, it is often necessary to create a number of different stubs for one module. Examining the output is often more direct when dealing with bottom-up testing. In top-down testing, the test output may come from a module that is far away from the module being testing. When using the bottom-up strategy, the output always comes from the module directly above the module being tested.

NOTES:



Thread TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSOnce the components required to deliver a piece of functionality have been tested using white-box methods, the development team should run sample functional tests through the components to validate that the function will execute correctly from a black-box perspective. This test is referred to as execution a “thread” through the application.• Often used during the integration of units or components• Demonstrates key functional capabilities by testing a string of units or components that

accomplish a specific function in the application• Can be performed simultaneously with incremental testing

NOTES:




SUPPORTING / EXPLANATORY MATERIALS• One of the software testing techniques used by many software testers is “boundary value

analysis.” • For boundary value analysis, explain the following:

- What is boundary value analysis? • Test cases that focus on the upper or lower (ranges) of boundaries of a given

function.- When should boundary value analysis be used?

• To ensure the a function responds according to specifications- Provide an example of the types of test data you would prepare for boundary value

analysis. • Low boundary +/- one unit• On the boundary +/- one unit• Upper boundary +/- one unit

NOTES:




SUPPORTING / EXPLANATORY MATERIALS• One of the software testing techniques used by many software testers is “boundary value

analysis.” • For boundary value analysis, explain the following:

- What is boundary value analysis?- When should boundary value analysis be used?- Provide an example of the types of test data you would prepare for boundary value

analysis. • Use a Life Insurance application which displays premium charts based on various age

ranges. • For example: 0-18, 19- 30, 31- 50, 51- 85

NOTES:




SUPPORTING / EXPLANATORY MATERIALSMatch the items to the numbered listA. InspectionsB. Black BoxC. Capture/PlaybackD. Stress TestingE. Regression Testing

NOTES:

1. Testing previously verified logic2. Testing Branches/paths3. Re-use of test data4. Test correct implementation of specifications5. Testing processing limits





NOTES:



Walkthroughs, Reviews, InspectionsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe tools and techniques described will be divided into verification and validation. The verification, or static tests, are basically techniques, meaning that the practice is self-contained. The validation, or dynamic tests, are a combination of tools and techniques. For example, the technique of developing test cases might be added by a tool, which will record, and normally expand, the number of test cases based on input parameters to the tool.• Walkthroughs and inspections are very disciplined procedures aimed at removing the major

responsibility for verification from the developer.• The main goal of a review is to identify defects within the stage or phase of the project where

they originate, rather than in later test stages; this is referred to as “stage containment.”- Feasibility reviews- Requirements reviews- Design reviews- Code walkthroughs- Code Inspections- Requirements tracing

NOTES:



What Are Reviews?KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSA process where key artifacts from each stage of the software development life cycle are inspected for defects by other team members or independent reviewers.• Feasibility reviews• Requirements reviews• Design reviews• Code walkthroughs• Code Inspections• Requirements tracing

NOTES:



Inspection Roles and ResponsibilitiesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSA test analyst or test manager should participate in the inspection as an inspector.• Inspectors are typically peers or stakeholders. • Participants in inspection• Peers: excellent for constructive criticism – learning• Stakeholders: include those who are impacted by the work productMichael Fagan describes five roles in his inspection process:1. Inspector2. Moderator (Facilitator)3. Reader4. Recorder5. AuthorNote: Multiple roles may be filled by the same individual, e.g. the Moderator may also serve as the recorder.

NOTES:



Sample Inspection ProcessKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSPlan:• Moderator selects the inspection team. Author distributes review material, including

pertinent input artifacts, 1-3 days prior to the review meeting. Prepare: • Inspectors review the material and return defects and issues to the moderator or author.

Everyone is required to report readiness to the moderator or the meeting can not be held. Review in the meeting is not allowed. Compare artifact to predecessor artifacts, standards, guidelines, and best practices as well as architecture strategies.

Meet:• The moderator facilitates the meeting, and addresses any defects or issues in question.

Defects that everyone agrees on are accepted and documented. Issues are documented for resolution outside the meeting.

Rework:• The author makes all agreed upon changes and returns updated materials to the moderator

for verification.Verify:• Inspectors verify that changes made are correct and no new defects have been introduced.

The work product is either accepted or rejected.

NOTES:



Inspection Meeting Rules KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSMeeting: Moderator and reader lead the review. Material may be submitted to the producer prior to the meeting to limit discussion to only open issues. Any comments accepted are not discussed. Meetings should be limited to two hours or less.• Accept as is • Accept with modifications• Revise and re-submit• Reject - start over• Record

• No preparation - no meeting!• Review the product - not the producer!• Identify defects and issues - don’t resolve them!• All participants are responsible for inspection success!

NOTES:




SUPPORTING / EXPLANATORY MATERIALSRecent studies you have read about shows that one of the most effective quality control practices is inspections.

Name three different roles that are performed during an inspection. • Inspector• Moderator (Facilitator) • AuthorList those roles and explain the purpose for each of the three roles you have identified.

NOTES:





NOTES:



Special Testing TechniquesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThese special test types are not applicable to all projects. The project test objectives and quality goals should be reviewed to determine if any of these are necessary. When designing a project test strategy, applicable test types must be scheduled and integrated into the basis test stages selected.• Usability Test - Identifies areas of poor human factors design that may make the system

difficult to use. Most beneficial if conducted with user representatives or customers during prototyping

• Vendor Validation Test - Validates functionality prior to acceptance of third party Commercial Off The Shelf (COTS) software. Should test all standard functions and data combinations

• Conversion Test - Validates effectiveness of conversion process (field-to-field mapping, data translation, etc.). Confirms application-to-application balancing and application- to-General Ledger balancing of converted data

• Stress/Load Test - System subjected to environmental conditions which defy normal expectations (high transaction volume, large database size, etc.). Intended to identify constraints and ensure there are no performance problems; results useful in capacity planning

• Performance Test - A variation on volume and stress tests that measures system performance against specified service-level objectives. Validates that on-line response time and batch run times meet performance requirements

• Configuration Testing - Necessary in client/server and web-based applications to validate that the application will run correctly on all supported end-user environments.

• Contingency Test - Evaluates the contingency features built into the application for handling interruptions and for returning to specific points in the applications processing cycle. Checkpoints, backups, restores, and restarts are tested here for execution and design deficiencies

• Security Test - Validates that the application is protected from unauthorized use. Ensures data protected from unauthorized access and/or modification. Validates network security checkpoints

NOTES:



Combining Test TechniquesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS• To effectively test systems, you need to use both functional and structural testing methods.• One of the more common combinations today is the merger of standard testing techniques

with formal verification.• Another technique combines dataflow analysis, symbolic execution, elementary theorem

proving, dynamic assertions, and standard testing into one systematic, efficient and effective testing method.

NOTES:



QuestionsKEY CONCEPT PRESENTATION GUIDE


NOTES:



Skill Category 1 - SummaryKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS• Vocabulary• Why Do We Test Software?• Factors Affecting Software Testing• Life Cycle Testing• Test Matrices• Independent Testing• Tester’s Workbench• Levels of Testing• Testing Techniques

NOTES:



Skill Category 2KEY CONCEPT PRESENTATION GUIDEBuilding the Test Environment

SUPPORTING / EXPLANATORY MATERIALSThe following topics will be discussed in this Skill Category:• Management Support• Test Work Processes• Test Tools• Testers Competency

NOTES:



What Is the Test Environment?KEY CONCEPT PRESENTATION GUIDE


All the conditions, circumstances and influences surrounding and effecting the testing of software.

NOTES:



What’s Included in the Test Environment?KEY CONCEPT PRESENTATION GUIDE


• Policies• Test processes• Culture• Attitudes• Organizational structure• Rewards• Test tools• Training• Test labs• Monitoring

NOTES:



What Is the Key Component to an Effective Test Environment?KEY CONCEPT PRESENTATION GUIDE


Management Support for Testing

NOTES:



What Is a Brief Description of Management Support?KEY CONCEPT PRESENTATION GUIDE


Management’s “tone” is representative of the environment that management has established that influence the way testers work.

NOTES:



How Does Management Communicate and Demonstrate Its Tone?KEY CONCEPT PRESENTATION GUIDE


• Integrity and ethical values- Incentives- Providing resources so testers can do an effective job

• Commitment to competency• Philosophy and operating style• Organizational structure

NOTES:



Integrity and Ethical ValuesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSKey Questions:• Do you do what you say you will?• Do you treat everyone equally?• Are you honest with your suppliers, colleagues and customers?• Are a team member?• Do you work hard every day?

NOTES:



Commitment to Competency KEY CONCEPT PRESENTATION GUIDE


Key Questions:• Do you know the skills you need to be fully competent at your job?• Would you spend your own money and time to improve your competency?• Do you tell your supervisor your skill deficiencies?

NOTES:



Philosophy and Operating StyleKEY CONCEPT PRESENTATION GUIDE


Management’s philosophy and operating style affect the way testing is managed, including the kinds of business risks accepted.Key Questions:• Do you operate within your organization’s risk appetite?• Do you use your resources effectively?• Do you work within the system of internal control including policies and work procedures• Are you an open and effective communicator?•

NOTES:



Organizational StructureKEY CONCEPT PRESENTATION GUIDE


An entity’s organizational structure provides the framework within which its activities for achieving entity-wide objectives are planned, executed, controlled and monitored.Key Questions:• Are your rules and responsibilities clearly defined?• Do you have the authority and resources to fulfill your responsibilities?• Does your organizational location facilitate fulfilling your responsibilities?• Do your human resource policies and procedures facilitate fulfilling your responsibilities?

NOTES:



Much of the test environment is communicated through the test work processesKEY CONCEPT PRESENTATION GUIDE


• A Test Objective• How the objective will be measured• Tasks to accomplish the objective• Verifies tasks were performed correctly

NOTES:



Process EngineeringKEY CONCEPT PRESENTATION GUIDE


All processes have three attributes. These are:1. The policy, which defines the process objective

- Answer the question “Why”. • Why is this process important?• Establishes measurable goals and objectives

2. The standards, which are used to measure whether or not the policy achieves its objective- Answers the question “What”.?

• What will we do to satisfy the objectives stated in the policy?3. Procedures to both do and check work

- Answers the question “How”.? • How will we satisfy the intent of the standard.? • Procedures represent tasks that are performed.

NOTES:



Importance of Work ProcessesKEY CONCEPT PRESENTATION GUIDE


• Improves communications• Enables knowledge transfer• Improves productivity• Assists in mastering new technology• Reduces defects and cost

NOTES:



Developing Work ProcessesKEY CONCEPT PRESENTATION GUIDE


• Standards and Procedures are Products• These Products Must Be Standardized ThemselvesPrior to creating any processes, the Process Engineering Committee must develop a standard and a procedure for developing standards and procedures.

NOTES:



Attributes of a Standard for a StandardKEY CONCEPT PRESENTATION GUIDE


• Defines Format• Defines Styles• Defines Attributes of a Document Called a Standard• Defines Attributes of a Document Called a ProcedureDeveloping a Test Standard• Testing Policies• Testing Standards• Testing Procedures (DO)• Testing Procedures (Check)

NOTES:



Tester’s WorkbenchKEY CONCEPT PRESENTATION GUIDE


NOTES:



How Is the Test Function Managed by Process? KEY CONCEPT PRESENTATION GUIDE


NOTES:

Test Policy Test Policy

ProceduresProcedures

Test Policy

Standard Standard

Test Mission

Test Policy Test Policy

ProceduresProcedures

Test Policy

Standard Standard

Test Mission



Responsibility for Building Work ProcessesKEY CONCEPT PRESENTATION GUIDE


It is important that organizations clearly establish who is responsible for developing work processes (i.e., policies, procedures, and standards.)

NOTES:



Key Concepts for Process EngineeringKEY CONCEPT PRESENTATION GUIDE


• Management provides an organizational structure for the workers to develop their own standards and procedures

• Program is driven by management policies• Absolute compliance to standards and procedures is required• Mechanism is provided for the continual maintenance of standards and procedures to make

them more effective

NOTES:



Test Process SelectionKEY CONCEPT PRESENTATION GUIDE


• Selecting, developing, and acquiring work processes is an overall IT organization responsibility.

• The specific components that need to be addressed include:- Building a Process Engineering Organization- Developing a Standard and Procedure for Standards- Planning for Standards- Writing, Storing, and Retrieving Standards and Procedures- Enforcing Standards

NOTES:



Professional Test StandardsKEY CONCEPT PRESENTATION GUIDE


Professionals in any industry need to be aware of industry standards that apply to their domain. Several organizations publish standards covering the test process, activities, and work products. Test professionals should be familiar with the standards published by organizations such as:• IEEE - Institute of Electrical and Electronics Engineers• ISO - International Organization for Standardization • NIST - National Institute of Standards and Technology• SEI - Software Engineering Institute

NOTES:



Analysis and Improvement of the Test ProcessKEY CONCEPT PRESENTATION GUIDE


Studies at many IT organizations have indicated that testers make more defects in performing test activities than developers do in performing developmental activities. For example, if the developers make three defects per function point of logic; testers would make more than three defects in testing that function point of logic.

NOTES:



Reasons for Test Process AnalysisKEY CONCEPT PRESENTATION GUIDE


Verifies:• The effectiveness and efficiency of test processes • That test objectives are applicable, reasonable, adequate, feasible, and affordable• The test program meets the test objectives• The correct test program is being applied to the project• The test methodology is used correctly• The task work products are adequate to meet the test objectives• Analysis of the results of testing to determine the adequacy of testing• The adequacy of testing

NOTES:



Effectiveness and Efficiency of the Test ProcessKEY CONCEPT PRESENTATION GUIDE


• Effectiveness means that the testers completed their assigned responsibilities.• Efficiency is the amount of resources and time required to complete test responsibilities.• Are the current test processes effective and efficient?• Who decides?

- Testers- Quality Assurance- Other Stakeholders

NOTES:



What Makes Process Improvement Work?KEY CONCEPT PRESENTATION GUIDE


• Examine the Organization’s Needs and Business Goals• Conduct Assessment• Initiate Process Improvement• Analyze Assessment Output and Derive Action Plan• Implement Improvements• Confirm Improvements• Sustain Improvement Gains• Monitor Performance

NOTES:



Test Process AlignmentKEY CONCEPT PRESENTATION GUIDE


The Assessment of the Test Process Alignment Map is two-fold:1. To determine that there are adequate processes in place to achieve goal2. To determine that there are no goals without processes and that there are no processes

without defined goalsTest Process Alignment Map

NOTES:



Software Development Methodology Impacts Test ProcessesKEY CONCEPT PRESENTATION GUIDE


• Waterfall • Prototype• Spiral • RAD• Incremental• V Model

NOTES:



Adapting Testing Processes to Different Software Development ProcessesKEY CONCEPT PRESENTATION GUIDE


• Understand the development process• Understand the deliverables• Understand the applicable and non-applicable parts of the test processes• Customize as needed

NOTES:



Test ToolsKEY CONCEPT PRESENTATION GUIDE


• Building an effective and economic test environment requires the selection of tools.• #8 of Top Ten Challenges was “Testing Without Tools”• It is difficult to perform testing economically and effectively without the aid of automated

tools.• The most important aspect of software testing tools is the process used to acquire those

tools.Key Concepts for Test Tools• Test processes specify what tools to use and how to use them• Test Tools should be selected to facilitate the performance of a test process

NOTES:



Selecting Test ToolsKEY CONCEPT PRESENTATION GUIDE• Define your test process• Prioritize automation needs• Identify top rated tools on the market• Evaluate and select tools• Integrate tools into the test process

SUPPORTING / EXPLANATORY MATERIALSTips:• Look for integrated tools to reduce training needs• Look for tools with an underlying powerful scripting language. Although immature

organizations will focus on capture/playback functionality to start, they’ll soon outgrow the sole reliance on this functionality. Mature teams will soon require the customization and flexibility provided by creating custom scripts with the scripting language - and the more powerful the language, the more powerful the scripts.

• Look for tools that function on as many of your supported platforms as possible. Platform Independence refers both to the platform where you create the scripts, as well as execution of them. Technology rapidly changes, and for internet applications you may be supporting many more client platforms than before. The ability to execute the scripts on multiple configurations will greatly enhance your configuration testing capability.

• Hire the appropriate resources and/or skill sets for effectively deploy automation. Note that the script automators do not have to be the same individuals who do the test planning. These are two different skill sets.

• Don’t implement the tool without a process to support it. Remember the old adage about fools with tools?

NOTES:



Tool UsageKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSClasses of Test Tools:• Automated Regression Testing Tools• Defect Management Tools• Performance/Load Testing Tools• Manual Tools• Traceability Tools• Code Coverage• Test Case Management• Common Tools Applicable to TestingStatic tools:• Measure complexity (e.g. McCabe’s tools) which can be used to validate maintainability

requirements• Identify “dead” or uncalled code• Can be used to map logic or basis paths (e.g. Battlemap) This helps with test planning as

each path will need a test case.

NOTES:



Test Tool GuidelinesKEY CONCEPT PRESENTATION GUIDE• Can’t use without training• Mandatory, not optional• Help to use readily available


Test generators automatically generate tests from various artifacts throughout the development process, including requirements, data flow diagrams, etc. For example, Bender’s test generator generates tests from requirements that are written in a prescribed format.Client/server applications introduced new requirements on the test process. The increased integration of networks into core application architecture adds additional requirements to the test process. Failure of the network can be as critical to the project as the failure of the application itself. The risk associated with the failure of networks supporting Internet applications is even higher, as this impact is felt directly by your customers.Tools to test various aspects of network performance and security are available. In these cases, the tools are essential, as manual test processes cannot identify some classes of defects, including encryption failures, and weak points in the telecom network.

NOTES:



Benefits of Using Test ToolsKEY CONCEPT PRESENTATION GUIDE• Decrease number of tests• Increase test coverage• Decrease regression test time • Increase test case reuse through use of

test repositories• Identify high risk areas of code with

complexity analysis• Reduce risk resulting from changes to

requirements and design

SUPPORTING / EXPLANATORY MATERIALSTools can help to accelerate testing and to widen test coverage, improving the ROI on test investments.Static tools can be effectively used to automatically identify problem areas in the code, and identify previously untested portions of code. Other benefits include:• Testing can be distributed among various groups - not just limited to test groups with

knowledge• Test scripts can be reused by production support groups to verify system recovery and

availability• Knowledge in repository rather than tied to specific individuals• Up to 50% reduction in person hours after initial investment• Reduces reluctance to execute necessary regression testing because the execution time is

significantly reduced.• Note: It will be critical to include tool training into the schedule. Other limitations include:• Does not replace the need for detailed test planning• The creation of the initial test cases will require additional time. NO time will be saved on the

first test cycle• Automated scripts must be updated each time changes are made to functionality• Requires complete reset facility to be able to execute multiple times

NOTES:



Effective Testing Necessitates Testers to Be Competent in Software TestingKEY CONCEPT PRESENTATION GUIDE


NOTES:





• Management Support• Test Work Processes• Test Tools• Testers Competency

NOTES:



Skill Category 3KEY CONCEPT PRESENTATION GUIDEManaging the Test Project


The following topics will be discussed in this Skill Category:• Test administration• Test team approaches• Test leadership• Managing change

NOTES:



Test AdministrationKEY CONCEPT PRESENTATION GUIDE


• Test Planning*• Budgeting • Scheduling• Staffing• Customization of the test process

* Test Planning will be covered in detail in Skill Category 4

NOTES:



Budgeting and SchedulingKEY CONCEPT PRESENTATION GUIDE


What is the challenge of producing a reasonable budget and schedule?The size of the work effort must be correctly estimated.

NOTES:



What Factors Influence Estimation?KEY CONCEPT PRESENTATION GUIDE


• Requirements• Past data• Organization culture• Selection of suitable estimation technique• Test management experience• Resources available • Tools at our disposal

NOTES:



What Techniques Are Used to Estimate Budget?KEY CONCEPT PRESENTATION GUIDE


• Top-down (cost, constraints, Function Points, COCOMO)• Expert judgment• Bottom-up (needs Work Breakdown Structure)COCOMO (Constructive Cost Model) based on inputs relating to the size of the system and a number of cost drivers that affect productivity.

NOTES:



SchedulingKEY CONCEPT PRESENTATION GUIDE


Answers the following questions:• What tasks will be done?• Who will do them?• When will they do them?

NOTES:



Staffing Approaches for Software Testing ProjectsKEY CONCEPT PRESENTATION GUIDE


• Developers test• Independent testers• Non-IT testers• Combination of aboveIt is not necessary for every member of the test team to possess all the skills, but in total the team should have all the needed skills.If the test team does not possess the necessary skills, it is the responsibility of the test manager to teach those individuals the needed skills.

NOTES:



Software Testing Staffing MatrixKEY CONCEPT PRESENTATION GUIDE


NOTES:

D

C

B

A

Tool YTool XTest Data Generation

PlanningStaff

Skills Needed

D

C

B

A

Tool YTool XTest Data Generation

PlanningStaff

Skills Needed



What Causes the Need for the Test Processes to Be Customized?KEY CONCEPT PRESENTATION GUIDE


• Release cycle schedules• Software development methodology• User schedules• Project status reporting• Interfacing with other projects• Interfacing with enterprise-wide databases• Different naming conventions/data definitions

NOTES:



Software Development ConstraintsKEY CONCEPT PRESENTATION GUIDE


Changes to one of the constraints will cause one or more of the other constraints to also change

NOTES:

Budget Schedule

QualityStaffing

BudgetBudget ScheduleSchedule

QualityQualityStaffingStaffing





NOTES:



Test SupervisionKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSSupervision is a combination of the supervisor possessing the skill sets needed to supervise, and the tasks that contribute to successful supervision.• Communication• Negotiation• Project relationships• Motivation, mentoring, recognition

NOTES:



Communication SkillsKEY CONCEPT PRESENTATION GUIDE


• Written & oral communications• Listening skills• Interviewing skills• Analyzing skills

NOTES:



Written and Oral CommunicationKEY CONCEPT PRESENTATION GUIDE


Guidelines for presentations are:• Present the best solution.• Emphasize team’s positives.• Sell team’s experience.• Sell management capabilities.• Sell technical expertise.• Sell enthusiasm.

NOTES:



3 Step Listening ProcessKEY CONCEPT PRESENTATION GUIDE


Oral communication (which includes listening) is rated as the number-one skill for the tester.The listening process involves three separate steps: 1. Hearing the speaker2. Attending to the speaker3. Understanding the speaker

NOTES:





Step 1. Hearing the speaker.Hearing the speaker requires an understanding of the five channels of communication incorporated into speech:1. Information Channel - The speaker’s subject.2. Verbal Channel - The words used by the speaker.3. Vocal Channel - The tone of voice associated with the various words.4. Body Channel - The body movements and gestures associated with the information being

conveyed.5. Graphic Channel - The pictures, charts, etc. that the speaker uses to emphasize or illustrate

the material being discussed.

NOTES:





Step 2. Attending to the Speaker (sometimes referred to as being an active listener).Some suggestions to help in attending to the speaker are:• Concentrate on the speaker.• Maintain eye contact (80%).• Provide feedback.• Restate what you heard.

NOTES:





Step 3. Understanding the SpeakerThere are five types of listening. The type chosen will have an impact on the ability to understand what the speaker is saying.Type 1: Discriminative ListeningType 2: Comprehensive ListeningType 3: Therapeutic ListeningType 4: Critical ListeningType 5: Appreciative or Enjoyment Listening

NOTES:



Interviewing SkillsKEY CONCEPT PRESENTATION GUIDE


A software tester will use interviewing skills for fact-finding and to convey information A finding – identifies a difference between what is and what should be. When documenting a finding, it should include:• Fact• Cause• Significance

NOTES:



Analyzing Skills and the Analysis ProcessKEY CONCEPT PRESENTATION GUIDE


NOTES:



NegotiationsKEY CONCEPT PRESENTATION GUIDE


Conflict can be defined as a breakdown in the decision-making process.Some sources of conflict requiring negotiations are:• Project Priorities• Administrative Procedures• Technical Opinions and Performance Trade-Offs• Human Resource• Cost• Schedule• Personality

NOTES:



Negotiation Resolution MethodsKEY CONCEPT PRESENTATION GUIDE


• Forcing• Withdrawal• Smoothing• Compromise• Problem solving

NOTES:



Resolving ComplaintsKEY CONCEPT PRESENTATION GUIDE


Research shows that complaints must be resolved within four minutes.*

4-Step Complaint-Resolution ProcessStep 1: Get on your customer’s wavelength

(Establish physical parity. Show empathy.)Step 2: Get the factsStep 3: Establish and initiate an action programStep 4: Follow up with your customer

NOTES:



JudgmentKEY CONCEPT PRESENTATION GUIDE


Judgment is a decision made by an individual.

Judgment is normally a decision based on three criteria which are:1. Facts2. Standards3. Experience

NOTES:



Constructive CriticismKEY CONCEPT PRESENTATION GUIDE


Constructive criticism should incorporate the following tactics:• Do it privately• Have the facts• Be prepared to help the worker improve his performance• Be specific on expectations• Follow a specific process in giving criticism

NOTES:



Project RelationshipsKEY CONCEPT PRESENTATION GUIDE


NOTES:



Project Relationships (cont)KEY CONCEPT PRESENTATION GUIDE


Listening+ Interviewing+ Resolving+ Criticism = Relations

NOTES:



MotivationKEY CONCEPT PRESENTATION GUIDE


Motivation has been defined as getting individuals to: • Do work tasks they do not want to do • Perform those work tasks in a more efficient or effective manner.The Four Most Common Motivators1. Personal challenge2. Respect3. Rewards4. Recognition

NOTES:



Mentoring and RecognitionKEY CONCEPT PRESENTATION GUIDE


Mentoring:Mentoring is helping or supporting an individual in a non-supervisory capacity. Mentoring can occur in any of the following three areas:1. Career counseling 2. Work tasks3. Professional advancementRecognition:• A key concept in supervision is that recognition is important.• People like to be recognized for the contribution they make to a project.

NOTES:



Testing LeadershipKEY CONCEPT PRESENTATION GUIDE


All test managers are part manager and part leader.

In discussing leadership, we will address these areas:• Chairing Meetings• Team Building• Quality Management Organizational Structure• Code of ethics

NOTES:



Team BuildingKEY CONCEPT PRESENTATION GUIDE


Team building needs to focus on the following items that an individual can achieve from working on a specific project:• Challenge• Growth• Credit“New loyalty is not to the boss or to the company, but to projects that actualize the mission and offer challenge, growth, and credit for results.”

R.M. Kanter

NOTES:



Team DevelopmentKEY CONCEPT PRESENTATION GUIDE


There are seven guidelines that are helpful in developing compatibility and motivation of a software project team:1. Communicate 2. Define roles and responsibilities 3. Empower team members 4. Hold team members accountable5. Ensure skills are present6. Provide training7. Reward successes

NOTES:



Team EthicsKEY CONCEPT PRESENTATION GUIDE


The following attributes of the team are associated with ethical team behavior:• Customer relations that are truthful and fair to all parties• Protecting company property• Compliance with company policies• Integrity of information• Attendance• Redefine standards of quality

NOTES:



Quality Management Organization StructureKEY CONCEPT PRESENTATION GUIDE


The quality revolution significantly impacted the traditional hierarchical structure. • The structure was flattened • Employees were empowered to make more decisions• New approaches to management were introduced.

NOTES:



Traditional vs Quality Management PhilosophyKEY CONCEPT PRESENTATION GUIDE


NOTES:

Accomplishment from long-term impact of improving processes

Accomplishment from meeting quotas, the monthly or quarterly bottom line

Continuous process improvementFire fighting

Management of process inputs—methods or sources of variation that focus on preventing defects

Management of outputs (results)—focusing on detection of defects

Motivation from within (self)Motivation from fear of failure

TeamworkCompetition between organizations

Management accountable to the customerManagement accountable to their manager

Refine the processEmployees are the problem

Reduce variation and prevent the errorCorrect the error

What allowed the error?Who made the error?

Use the processControls each result

Quality Management PhilosophyTraditional Management Philosophy

Accomplishment from long-term impact of improving processes

Accomplishment from meeting quotas, the monthly or quarterly bottom line

Continuous process improvementFire fighting

Management of process inputs—methods or sources of variation that focus on preventing defects

Management of outputs (results)—focusing on detection of defects

Motivation from within (self)Motivation from fear of failure

TeamworkCompetition between organizations

Management accountable to the customerManagement accountable to their manager

Refine the processEmployees are the problem

Reduce variation and prevent the errorCorrect the error

What allowed the error?Who made the error?

Use the processControls each result

Quality Management PhilosophyTraditional Management Philosophy



Why a Code of Ethics Is ImportantKEY CONCEPT PRESENTATION GUIDE


• Differentiates a profession from a job• Teaches how to conduct oneself as a professional• When followed, it gains the respect of colleagues

NOTES:



Managing ChangeKEY CONCEPT PRESENTATION GUIDE


• Managing change is a process.• The process of managing change is a primary responsibility of all IT staff.

NOTES:



Test Manager Must Manage ChangeKEY CONCEPT PRESENTATION GUIDE


• Software Change Management: Changes to software• Software Configuration Management: Controlling change over project artifacts (e.g. test

cases)- Software Version Control: Changed versions of software

NOTES:



Software Configuration ManagementKEY CONCEPT PRESENTATION GUIDE


The entire Test Environment and all it’s project artifacts must be managed including:• Source code• Requirements• Analysis models• Design models• Test cases and procedures• Automated test scripts• User documentation, including manuals and online Help• Hardware and software configuration settings

NOTES:



Software Change ManagementKEY CONCEPT PRESENTATION GUIDE


The testers need to know two aspects of change:1. The characteristics of the change so that modification to the test plan and test data can be

made to assure the right functionality and structure are tested.2. The version in which that change will be implemented.

NOTES:



Software Version ControlKEY CONCEPT PRESENTATION GUIDE


• Once dynamic testing begins, the project team must ensure that the appropriate versions of the software components are being tested.

• The Configuration Manager must develop both migration and back-out procedures to support this process. These processes can be either manual or automated.

• Often, the CM tool selected will support the creation of automated build procedures that reduce the level of risk associated with migration.

NOTES:





When people are presented with new ideas, their most common reaction is:a) Accept the ideab) Object to the new ideac) Ask for more information about the idead) Ask to be trained in the concepts related to the new idea

Which of the following is NOT the type of project artifact that must be managed and controlled in configuration management environment?

a) Source codeb) Requirementsc) Name of the project teamd) Test case and procedures

NOTES:





• Test administration• Test team approaches• Test leadership• Managing change

NOTES:



Skill Category 4KEY CONCEPT PRESENTATION GUIDETest Planning


The following topics will be discussed in this Skill Category:• Risk Concepts and Vocabulary• Risks Associated with Software Development• Risks Associated with Software Testing• Risk Analysis• Risk Management• Prerequisites to Test Planning• Create the Test Plan

NOTES:



Concepts and VocabularyKEY CONCEPT PRESENTATION GUIDE


Understanding the definitions of the following terms will aid in comprehending the material in this category:• Test Case - Test cases are how the testers validate that a software function, such as

deducting a payroll tax, or a structural attribute of software, such as code complexity meets the software specifications (i.e., expected results).

• Test Data - Test data is information used to build a test case.• Test Script - Test scripts are an online entry of test cases in which the sequence of entering

test cases and the structure of the entry system must be validated, in addition to the expected results from a single test case.

NOTES:



Concepts and Vocabulary (cont)KEY CONCEPT PRESENTATION GUIDE


The following definitions are helpful when discussing all of the factors with risk and risk analysis.• Risk - any potential loss to an organization, as for example, the risk resulting from the

misuse of its computer i.e. unauthorized modification, and/or the authorized but incorrect use of a computer.

• Risk Analysis - an analysis of an organization’s information resources, its existing controls, and its remaining organization and computer system vulnerabilities. It combines the loss potential for each resource or combination of resources with an estimated rate of occurrence to establish a potential level of damage in dollars or other assets.

• Risk Appetite - defines the amount of loss management is willing to accept for a given risk.

NOTES:



Concepts and Vocabulary (cont)KEY CONCEPT PRESENTATION GUIDE


The following definitions are helpful when discussing all of the factors with risk and risk analysis.• Threat - something capable of exploiting vulnerability in the security of a computer system

or application. Threats include both hazards and events that can trigger flaws.• Vulnerability - a design, implementation, or operations flaw that may be exploited by a

threat; the flaw causes the computer system or application to operate in a fashion different from its published specifications and to result in destruction or misuse of equipment or data.

• Control - anything that tends to cause the reduction of risk. Control can accomplish this by reducing harmful effects or by reducing the frequency of occurrence.

NOTES:



Risks Associated with Software TestingKEY CONCEPT PRESENTATION GUIDE


Risk is the probability that an unfavorable event will occur that results in loss.

It is the test manager’s responsibility to determine the appropriate amount of testing to perform based upon the risks associated with the application.

NOTES:

ProbabilityProbability XX==RiskRisk LossLossProbabilityProbability XX==RiskRisk LossLoss



Risks Associated with Software DevelopmentKEY CONCEPT PRESENTATION GUIDE


Some risks are associated with software functions, and other risks are associated with the process that develops the software.• Improper use of technology• Repetition of errors• Cascading of errors• Illogical processing• Inability to translate user needs into technical requirements• Inability to control technology• Incorrect entry of data• Concentration of data

NOTES:



Risks Associated with Software Development (cont)KEY CONCEPT PRESENTATION GUIDE


• Inability to react quickly• Inability to substantiate processing• Concentration of responsibilities• Erroneous or falsified input data• Misuse by authorized end users• Uncontrolled system access• Ineffective security and privacy practices for the application• Procedural errors during operations• Program errors• Operating system flaws• Communications system failure

NOTES:



Risks Associated with Software TestingKEY CONCEPT PRESENTATION GUIDE


• Lack of test competency• Adversarial mentality• Lack of test tools• Lack of management support• Lack of customer / user involvement• Inadequate time or budget• Lack of unit testing• Rapid change• Inadequate test environment• New technology implementation• Utilization of new developmental processes

NOTES:



Risks Associated with Premature ReleaseKEY CONCEPT PRESENTATION GUIDE


Premature release is defined as releasing the software into production under the following conditions: • The requirements were implemented incorrectly• The test plan has not been completed• Defects uncovered in testing have not been corrected In premature release of software, the testers should include in their report the most probable outcomes associated with premature release.

NOTES:



Risk AnalysisKEY CONCEPT PRESENTATION GUIDE


• Objective of risk analysis (as it relates to software testing) is to help allocate limited resources to those components that pose the greatest risk to the organization.

• Testing is a process designed to minimize software risks.

NOTES:



Risk Analysis ProcessKEY CONCEPT PRESENTATION GUIDE


Performing risk analysis during test planning is a four-step process as follows:1. Form the risk analysis team2. Identify risks3. Estimate the magnitude of the risk4. Select testing priorities

NOTES:



Form The Risk Analysis TeamKEY CONCEPT PRESENTATION GUIDE


Successful risk analysis requires the establishment of a Risk Analysis Team, whose responsibility will be to identify risks and prioritize the use of test resources.The team should be comprised of three to six members and at a minimum possess the following skills:• Knowledge of the user application• Understanding of risk concepts• Ability to identify controls• Familiarity with both application and information services risks• Understanding of information services concepts and systems design• Understanding of computer operations procedures

NOTES:



Identify RisksKEY CONCEPT PRESENTATION GUIDE


The risk team can use one of the following two methods to identify the application-oriented, not environmental, risks associated with the application system:• Risk analysis scenario• Risk checklist

NOTES:



Eliminate the Magnitude of RiskKEY CONCEPT PRESENTATION GUIDE


The magnitude of a risk can be determined by any of the following means:• Intuition and Judgment• Consensus• Risk Formula• Annual Loss Expectation (ALE) Estimation

NOTES:



Select Testing PrioritiesKEY CONCEPT PRESENTATION GUIDE


Risks are normally ranked by the severity of the risk. However, other considerations may impact the prioritization such as:• Compliance required to laws and regulations• Impact on competitiveness• Impact on ethics, values and image

NOTES:





Your team is developing an Electronic Bill Presentment application for the Internet. You will be providing this service to corporate customers who want a central location to present their invoices to consumers on the Internet.• The Corporate Customer will send you an electronic file of invoices to be presented on the

Internet.• Consumers will be given a user id and password that enables them to log on to the web site

and view their monthly invoices.• Consumers can select the invoice and payment type, and initiate electronic payment.

NOTES:





The Electronic Bill Presentment application is the first Internet application your company is developing and deploying. • Identify five risks that might be associated with testing an Internet application• Document five assumptions you might have to make to assure the test effort is successful

NOTES:



Risk ManagementKEY CONCEPT PRESENTATION GUIDE


Risk management is a totality of activities that are used to minimize both the frequency and the impact associated with risks.

• The first component of risk management is to understand, identify, and determine the magnitude of risks.

• The second component of risk management is determining risk appetite.

NOTES:



Risk Reduction MethodsKEY CONCEPT PRESENTATION GUIDE


To reduce the frequency:• Reduce the opportunity for error• Identify the error prior to loss

To reduce loss:• Minimize loss• Recover loss

NOTES:



Contingency PlanningKEY CONCEPT PRESENTATION GUIDE


• Many IT organizations fail to develop contingency plans for the processing events occurring in the software system.

• The role of testers is to evaluate the adequacy of the contingency plans associated with risk. • Contingency planning should be a part of the test plan and the testing process.

NOTES:



Test PlanningKEY CONCEPT PRESENTATION GUIDE


NOTES:



Prerequisites to Test PlanningKEY CONCEPT PRESENTATION GUIDE


If test planning is viewed as a process, or a workbench, there are entrance criteria to the test planning process. The following entrance criteria are prerequisites to test planning:• Test Objectives• Acceptance Criteria• Assumptions• People Issues• Constraints

NOTES:



Test PlanKEY CONCEPT PRESENTATION GUIDE


• Describes how testing will be accomplished. • Is essential to effective testing.• Takes about one-third of the total test effort.• Is a dynamic document. • Evolves as the developmental effort changes in scope.• Used to ascertain the status of the test effort.

NOTES:



Test Plan (cont)KEY CONCEPT PRESENTATION GUIDE


The test plan is a “contract” defining what testers agree to do.And a roadmap to tell testers how to conduct testing.

NOTES:



Attributes of a Test PlanKEY CONCEPT PRESENTATION GUIDE


• Repeatable• Controllable• Ensures adequate test coverage

NOTES:



Understanding the Characteristics of the Software Being DevelopedKEY CONCEPT PRESENTATION GUIDE


• Define what it means to meet the project objectives.• Understand the core business areas and processes.• Assess the severity of potential failures.• Identify the components for the system.• Assure requirements are testable.• Address implementation schedule issues.• Address interface and data exchange issues.• Evaluate contingency plans for this system and activities.• Identify vulnerable parts of the system and processes operating outside the information

resource management area

NOTES:



Build the Test PlanKEY CONCEPT PRESENTATION GUIDE


The development of an effective test plan involves the following tasks:• Set test objectives• Develop the text matrix• Define test administration

NOTES:



Set Test ObjectivesKEY CONCEPT PRESENTATION GUIDE


Test objectives should restate the project objectives from the project plan.• Test objectives need to be defined and assigned a reference number • Test objectives must be agreed upon by the test team. • These objectives must be measurable and the means for measuring defined. • The objectives must be prioritized so that approximately one-third are High, one-third are

Average, and one-third are Low.• Define the acceptance criteria for each objective. The more specific the criteria, the easier it

will be for the testers to follow through.

NOTES:



Develop the Test MatrixKEY CONCEPT PRESENTATION GUIDE


To develop a software test matrix, follow these steps:• Define tests as required• Define conceptual test cases to be entered as a test script• Define verification tests• Prepare the software test matrix

NOTES:

XGeneral Ledger Changes

XXTax Deduction

XXGross Pay

XXPayroll Deduction Calculation

Validate InputCode InspectionParallel TestDesk CheckSoftware Function

Test Used to Test Function

XGeneral Ledger Changes

XXTax Deduction

XXGross Pay

XXPayroll Deduction Calculation

Validate InputCode InspectionParallel TestDesk CheckSoftware Function

Test Used to Test Function



Sample Test MatrixKEY CONCEPT PRESENTATION GUIDE


The objective of this matrix is to illustrate how to document which test cases test which software function and which structural attribute.

NOTES:

Test F

Test E

Test D

Test C

Test B

Test A10987654321Test Cases

Software Function / Structural Attribute

Test F

Test E

Test D

Test C

Test B

Test A10987654321Test Cases

Software Function / Structural Attribute



Define Test AdministrationKEY CONCEPT PRESENTATION GUIDE


The administrative component of the test plan identifies the following: • Software Project Name• Project / Unit • Checkpoint for Administration• Schedule• Milestones • Budget• Resources• Testing Materials• Test Training

NOTES:



Guidelines to Writing the Test PlanKEY CONCEPT PRESENTATION GUIDE


• Start early• Keep the test plan flexible • Review the test plan frequently• Keep the test plan concise and readable• Calculate the planning effort• Spend the time to do a complete test plan

NOTES:



Test Plan StandardsKEY CONCEPT PRESENTATION GUIDE


Test Plans and their formats vary from company to company, but the best examples contain the following sections:• Test Scope• Test Objectives• Assumptions• Risk Analysis• Test Design• Roles & Responsibilities• Test Schedule & Resources• Test Data Management• Test Environment• Communication Approach• Test Tools

NOTES:





• Risk Concepts and Vocabulary• Risks Associated with Software Development• Risks Associated with Software Testing• Risk Analysis• Risk Management• Prerequisites to Test Planning• Create the Test Plan

NOTES:



Skill Category 5KEY CONCEPT PRESENTATION GUIDEExecuting the Test Plan


The following topics will be discussed in this Skill Category:• Test Case Design• Test Coverage• Performing Tests• Recording Test Results• Defect Management

NOTES:



Test Case DesignKEY CONCEPT PRESENTATION GUIDE


The test objectives established in the test plan should be decomposed into individual test cases and test scripts that will not only test the software during development, but can test changes during the operational state of the software.You can design and prepare the following types of test cases:• Functional• Structural• Erroneous• Stress• Scripts• USE CASES

NOTES:



Types of Functional Test CasesKEY CONCEPT PRESENTATION GUIDE


Program testing is functional when test data is developed from documents that specify a module’s intended behavior. The goal is to test the specified behavior for each software feature, including the input and output. Some types of functional test cases are:• Input testing• Equivalence partitioning• Syntax checking• Special value testing• Output result coverage• Decision tables

NOTES:



Types of Structural Test CasesKEY CONCEPT PRESENTATION GUIDE


In structural program testing and analysis, test data is developed or evaluated from the source code. The goal is to ensure that various characteristics of the program are adequately covered. Some types of structural test cases are:• Complexity measures• Data flow analysis• Symbolic execution• Statement testing• Branch testing• Conditional testing• Expression testing• Path testing

NOTES:



Types of Erroneous Test CasesKEY CONCEPT PRESENTATION GUIDE


Techniques that focus on assessing the presence or absence of errors in the programming process are called error-oriented. There are three broad categories of such techniques:• Statistical assessment - employs statistical techniques to determine the operational

reliability of the program. Its primary concern is how faults in the program affect its failure rate in its actual operating environment.

• Error-based testing - seeks to demonstrate that certain errors have not been committed in the programming process.

• Fault-based testing - attempts to show the absence of certain faults in the code.

NOTES:



Types of Stress Test CasesKEY CONCEPT PRESENTATION GUIDE


The objective is to verify that the system can perform properly when stressed, or when internal program or system limitations have been exceeded.The types of internal limitations that can be evaluated with volume testing include:• Internal accumulation of information, such as tables.• Number of line items in an event, such as the number of items that can be included within an

order.• Field size limitations, such as number of characters allocated for people’s names.

NOTES:



Test ScriptsKEY CONCEPT PRESENTATION GUIDE


Test scripts are a tool that specifies the step-by-step actions that will be performed during a test session. Test scripts contain expected results and may be manually prepared using paper forms, or may be automated. The following tasks are needed to develop, use, and maintain test scripts:• Determine testing levels• Develop the scripts• Execute the scripts• Analyze the results• Maintain the scripts

NOTES:



Process to Build a USE CASEKEY CONCEPT PRESENTATION GUIDE


• Build a system boundary diagram• Define USE CASES• Develop USE CASES

NOTES:



System Boundary DiagramKEY CONCEPT PRESENTATION GUIDE


NOTES:



Define a USE CASEKEY CONCEPT PRESENTATION GUIDE


An individual use case consists of:• Preconditions that set the stage for the series of events that should occur for the use case• Results that state the expected outcomes of the above process• Sequential narrative of the execution of the use caseUse cases are used to:• Manage (and trace) requirements• Develop application documentation• Develop training• Develop test cases

NOTES:



Information Needed to Define a USE CASEKEY CONCEPT PRESENTATION GUIDE


The information about each USE CASE that needs to be determined for defining the case is:• USE CASE name or ID• Actor• Objective• Preconditions• Results• Detailed description• Exceptions• Alternative courses

NOTES:



Develop Test CasesKEY CONCEPT PRESENTATION GUIDE


A test case is a set of test inputs, execution conditions, and expected results developed for a particular test objective. • There should be a one-to-one relationship between USE CASE definitions and test cases. • There needs to be at least two test cases for each use case:

- One for successful execution of the use case - One for an unsuccessful execution of a test case.

NOTES:



Correlation of USE CASES to Test CasesKEY CONCEPT PRESENTATION GUIDE


If you’re using Use Cases to represent your requirements, and detailing them into sunny and rainy-day scenarios, then creating test scripts should be greatly simplified. Scenarios can take the place of test scripts simply by adding a place to note actual results and comments.

NOTES:

Functional Requirement

Use Case

Scenario

Scenario

Test Objective

Test Case Test Case

Test ScriptTest Script

Functional RequirementFunctional

Requirement

Use CaseUse Case

ScenarioScenario

ScenarioScenario

Test Objective

Test Case Test Case


Test Objective

Test Case Test Case


Test ObjectiveTest Objective

Test CaseTest Case Test CaseTest Case

Test ScriptTest ScriptTest ScriptTest Script



Building Test CasesKEY CONCEPT PRESENTATION GUIDE


Optimizing testing through selecting the most important processing events is the key aspect of building test cases. Test cases can be built as follows:• Built manually• Created by a test case generator• Extracted from a production file

NOTES:



Process for Building Test CasesKEY CONCEPT PRESENTATION GUIDE


• Identify test resources.• Identify conditions to be tested.• Rank test conditions.• Select conditions for testing.• Determine correct results of processing.• Create test cases.• Document test conditions.• Conduct test.• Verify and correct.

NOTES:



Example Payroll Application Test CasesKEY CONCEPT PRESENTATION GUIDE


NOTES:

XXXXTo determine whether it is possible to change a field in an inactive master record.

Change a field in an inactive master record

3

XXXX

To determine whether the system will accept invalid data into employees’ master records.

Enter erroneous codes

2

XXXX

To determine whether the system will accept a master record with essential data missing. The master record should be rejected with the appropriate warning.

Leave a mandatory field blank on a new employee’s master record

1

Process without

cutback

Cut B

ack to Allow

able M

aximum

Reject in C

ertain C

ircumstances

Print Error Message

Reject

Purpose of Test Case

Nature of Test C

ase

Control #

Expected Results

XXXXTo determine whether it is possible to change a field in an inactive master record.

Change a field in an inactive master record

3

XXXX

To determine whether the system will accept invalid data into employees’ master records.

Enter erroneous codes

2

XXXX

To determine whether the system will accept a master record with essential data missing. The master record should be rejected with the appropriate warning.

Leave a mandatory field blank on a new employee’s master record

1

Process without

cutback

Cut B

ack to Allow

able M

aximum

Reject in C

ertain C

ircumstances

Print Error Message

Reject

Purpose of Test Case

Nature of Test C

ase

Control #

Expected Results





NOTES:



Test ScriptsKEY CONCEPT PRESENTATION GUIDE


Description• ID• Traceability info• Configuration Management info

NOTES:



Test CoverageKEY CONCEPT PRESENTATION GUIDE


There are many methods that can be used to define and measure test coverage, including:• Statement Coverage• Branch Coverage• Basis Path Coverage• Integration Sub-tree Coverage• Modified Decision Coverage• Global Data Coverage• User-specified Data Coverage

NOTES:



Performing TestsKEY CONCEPT PRESENTATION GUIDE


Each cycle of test execution needs to be planned, prepared for, executed and the results recorded. The activities involved in performing tests include:• Establish test platforms• Test cycle strategy• Use of tools in testing• Test execution• Executing the Unit Test plan• Executing the Integration Test plan• Executing the System Test plan• Defining when testing is complete• Identifying concerns

NOTES:



Performing Unit TestsKEY CONCEPT PRESENTATION GUIDE


Unit testing:• is normally performed by the programmer that developed the program• can be performed many ways but the result of unit testing should be that the unit is defect

free

NOTES:



Performing Integration TestsKEY CONCEPT PRESENTATION GUIDE


Integration testing should begin once unit testing for the components to be integrated is complete. Some common items to focus on during Integration testing include:• Validation of the links between the client and server(s)• Security controls• Performance and load tests on individual application components such as the database,

network, and application server• Sequences of adds, updates, views, and deletes within the application• Tests for empty database or file conditions• Output interface file accuracy

NOTES:



Performing System TestsKEY CONCEPT PRESENTATION GUIDE


This is a checklist of things that should have been done so we only have to test or that testing goes smoothly and efficiently• Test sets Helps in automation• Daily• Weekly• Monthly etc.

• Set up system test environment (mimicking production as closely as possible)• Set up test data bed (using production & manufactured data)• Identify test cycles needed to replicate production• Assign test sets to test cycles• Assign test scripts to testers for execution• Migrate build to test environment• Simulate start date on system test environment• Begin execution of test scripts• Document test results and record execution in the Test Log

NOTES:



The System Test ProcessKEY CONCEPT PRESENTATION GUIDE


• Review test results and determine whether problems are defects- We may actually cause problems- Planned wrong- Test environment is incorrect

• Record defect in tracking system, making sure the developer responsible for fixing defect is notified- Whether its a tool like Test Director or an Excel spreadsheet, record defects and their

origin• When defect is fixed and migrated to system test environment, re-test and verify fix:

- If fixed, close incident- If not fixed, return to developer

NOTES:



Example of a Tester’s DashboardKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSAn independent test manager should consider developing a tester’s dashboard. However, to do this, it would be necessary for testers to develop a measurement database regarding testing. This example of a tester’s dashboard shows five indicators that a test manager would monitor to help manage testing more effectively and efficiently.

NOTES:

To Be Tested Successful Test

ResourcesUsed

RequirementsSatisfied

RequirementChanges

To Be TestedTo Be Tested Successful TestSuccessful Test

ResourcesUsed

ResourcesUsed



RequirementChanges

RequirementChanges



When Is Testing Complete?KEY CONCEPT PRESENTATION GUIDE


When the test plan is completed.

NOTES:



Addressing General Testing ConcernsKEY CONCEPT PRESENTATION GUIDE


There are three general concerns testers have in performing tests:1. Software is not in a testable mode for this test level.2. There is inadequate time and resources.3. Significant problems will not be uncovered during testing.

NOTES:



What Is a Defect?KEY CONCEPT PRESENTATION GUIDE


• A variance from “what should be”• The impacted party must agree it’s a defect• Whatever an IT organization says is a defect

NOTES:



How Do We Describe a Defect?KEY CONCEPT PRESENTATION GUIDE


• Name the defect (e.g. wrong requirement)• Location where the defect occurred (e.g. requirements phase)• Describe the defect (i.e. document as many of the following attributes as appropriate for the

problem):- Activities involved – The specific business or administrated activities that are being

performed.- Procedures used to perform work – The specific step-by-step activities that are

utilized in producing output from the identified activities.- Outputs/Deliverables – The products that are produced from the activity.- Inputs – The triggers, events, or documents that cause this activity to be executed.- User/Customers served – The organization, individuals, or class users/customers

serviced by this activity.- Deficiencies noted – The status of the results of executing this activity and any

appropriate interpretation of those facts.

NOTES:



Test Problem Documentation ExampleKEY CONCEPT PRESENTATION GUIDE


NOTES:

The testers should enter any recommended action they believe would be helpful to the project team. If the testers feel unable to indicate the action needed, the project team would record the recommended action here. Once approved, then the action would be implemented. If not approved, an alternate action should be listed or the reason for not following the recommended action should be documented.

Recommended Action

Here, the testers should document where the problem occurred as closely as possible. It can be related to a specific instruction or processing section that is desirable. If not, the testers should try to find the location as accurately as possible.

Location of Problem

In this field the testers should indicate what they believe is the cause of the problem, if known. If the testers are unable to do this, the worksheet will be given to the development team and they should indicate the cause of the problem.

Cause of Problem

If this can be estimated, testers should indicate what they believe the impact or effect of the problem will be on computer processing.

Effect of Deviation

Write what the testers believe was the expected result from processing here.Statement of Criteria

Document the results of actual processing that occurred here.Statement of Conditions

Enter a brief narrative description of the variance here.Problem Description

Enter the name of the software system or subsystem tested here.Name of Software Being Tested

The testers should enter any recommended action they believe would be helpful to the project team. If the testers feel unable to indicate the action needed, the project team would record the recommended action here. Once approved, then the action would be implemented. If not approved, an alternate action should be listed or the reason for not following the recommended action should be documented.

Recommended Action

Here, the testers should document where the problem occurred as closely as possible. It can be related to a specific instruction or processing section that is desirable. If not, the testers should try to find the location as accurately as possible.

Location of Problem

In this field the testers should indicate what they believe is the cause of the problem, if known. If the testers are unable to do this, the worksheet will be given to the development team and they should indicate the cause of the problem.

Cause of Problem

If this can be estimated, testers should indicate what they believe the impact or effect of the problem will be on computer processing.

Effect of Deviation

Write what the testers believe was the expected result from processing here.Statement of Criteria

Document the results of actual processing that occurred here.Statement of Conditions

Enter a brief narrative description of the variance here.Problem Description

Enter the name of the software system or subsystem tested here.Name of Software Being Tested



Problem EffectKEY CONCEPT PRESENTATION GUIDE


• Effects should be stated in quantitative terms such as:- Dollars- Time- Units of production- Number of procedures and processes- Transactions

• Qualitative terms do not convey the appropriate message to management• Even intangible effects can have a major impact on a project

NOTES:



Problem CauseKEY CONCEPT PRESENTATION GUIDE


The cause is the underlying reason for the condition.The determination of the cause of a condition usually requires the scientific approach, which encompasses the following steps:• Define the problem (the condition that results in the finding).• Identify the flow of work and information leading to the condition.• Identify the procedures used in producing the condition.• Identify the people involved.• Recreate the circumstances to identify the cause of a condition.

NOTES:



Defect Management and What to ReportKEY CONCEPT PRESENTATION GUIDE


Severity vs PriorityDefect Management:• Requires a communication mechanism, either manual or automated• Facilitates communication between test and development teams What to Report?• Defect name and type• Severity and priority• Status• Date and time of detection• Location identified (e.g. component, GUI, etc.)• Detailed description• Component or program where defect was found• Screen prints, etc.• Stage of origination (may be added later)• Person assigned to correct• Correction effort in hours

NOTES:



The Defect Management ProcessKEY CONCEPT PRESENTATION GUIDE


• Prevent defects• Risk driven• Integrated into the development process• Used to improve the process.

NOTES:



Success Factors for TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS• Significant involvement of subject matter experts from the client or user community

- Have them walkthrough test scripts- If possible let them test (They may follow scripts but they will also do those user things

we didn’t anticipate)• Full functional coverage planned in the test plan

- Be prepared for full functional testing and then prioritize if constraints are encountered• Clear, unambiguous, documented expected results for tests

- Don’t make them guess if a script passed or failed. State expected results exactly.- Looks good means different things to different people

• Depth or intensity of testing based on risk or prior defect history• Orderly, controlled processes for test execution, defect tracking, coordination of re-work and

configuration & version control- One of the purposes of planning is establish necessary controls so that we can report

whether a system functions as required.- Testing is like a scientific experiment. If we lose control or there were no controls

established we will never know for sure if the system worked.• Collaborative working relationship with the systems developers, the QA staff and the clients

- Communications are critical all through the process. We are at the end of the process when everyone is stresses and time is short. Early communications will make us a part of the team and not the ones who are seen as stopping progress.

• Establishment of a regression test bed of data for on-going use in maintenance• Established policy for process improvement reviews

NOTES:





Name at least two ways to demonstrate the test process has been performed correctly.• Perform quality control on the test process by comparing the number of defects in the test

process to previous test process; or• Use entrance and exit criteria

NOTES:





• Test Case Design• Test Coverage• Performing Tests• Recording Test Results• Defect Management

NOTES:



Skill Category 6KEY CONCEPT PRESENTATION GUIDETest Status, Analysis, and Reporting


The following topics will be discussed in this Skill Category:• Prerequisites to Test Reporting• Test Tools used to Build Test Reports• Test Tools used to Enhance Test Reporting• Reporting Test Results

NOTES:



Prerequisites to Test ReportingKEY CONCEPT PRESENTATION GUIDE


Quality and Testing metrics should support project or departmental quality goals. Measure only what you’re really interested in and plan to act upon.

Absolute Values:• Useful when comparing actual vs. planned results at any point in time• “We are exactly 50 modules behind in test readiness.”• Useful when used as a gatekeeper: for example,• “No module may enter test if its cyclomatic complexity exceeds x without review/approval.”

The prerequisites to the process of reporting test results are:• Define the test status data to be collected• Define the test metrics to be used in reporting test results• Define effective test metrics

NOTES:



Measures and MetricsKEY CONCEPT PRESENTATION GUIDE


• Measure - Single unit of measurement, e.g. defect, line of code, man hour, second, inch, etc.

• Metric - The combination of two or more measures to provide meaningful information- A quantitative measure of the degree to which a system, component, or process

possess a given attribute.

NOTES:



Objective and Subjective MeasuresKEY CONCEPT PRESENTATION GUIDE


Objective Measure • a measure that can be obtained by counting.• Subjective Measure • A person’s perception of a product or activity.

As a general rule, subjective measures are much more important than objective measures.

NOTES:



How Do You Know a Metric Is Good?KEY CONCEPT PRESENTATION GUIDE


• Normalize data for comparison across projects. By showing percentages of totals, measures per LOC, etc. you can analyze trends and look for common process failures.

• Absolute and trend data can be presented together to communicate a more complete picture

• Reliability – two people, same result• Validity – Measures what it is suppose to • Ease of Use and Simplicity• Timeliness – Reported in time to make a difference• Calibration – Movement of metric so it becomes more valid

NOTES:



Productivity Versus QualityKEY CONCEPT PRESENTATION GUIDE


• Productivity - an attribute of a process.• Quality - an attribute of a product or service.

There are two ways in which quality can drive productivity.1. Lower or not meet quality standards (Undesirable)2. Improve processes so that defects do not occur, thus minimizing the need for testing and

rework (Desirable)

NOTES:



Test Metrics CategoriesKEY CONCEPT PRESENTATION GUIDE


• Metrics unique to test• Complexity measurements• Project metrics• Size measurements• Defect metrics• Product measures• Satisfaction metrics• Productivity metrics

NOTES:



Test Charts and Diagrams Used to Build Test ReportsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSPareto Charts – view the causes of a problem in order of severity. Known as the 80/20 rule.

NOTES:



Test Charts and Diagrams Used to Build Test Reports (cont)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSCause and Effect Diagrams (fishbone or Ishikawa diagrams) – Useful to visualize, clarify, link, identify, and classify possible causes of a problem

NOTES:




SUPPORTING / EXPLANATORY MATERIALSCheck Sheets – Record # of occurrences over a time interval

NOTES:




SUPPORTING / EXPLANATORY MATERIALSHistogram – Grouping of data by predetermined intervals to show the frequency of the data set

NOTES:



Test Charts and Diagrams Used to Build Test Reports (cont) KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSRun Charts – Graph of data in chronological order displaying shifts or trends in the central tendency

NOTES:




SUPPORTING / EXPLANATORY MATERIALSScatter Plot Diagrams – Shows the relationship that might exist between two variables

NOTES:




SUPPORTING / EXPLANATORY MATERIALSRegression Analysis – Shows the relationship between two variables

NOTES:




SUPPORTING / EXPLANATORY MATERIALSControl Charts – Technique to assess, monitor and maintain the stability of a process

NOTES:



Code Coverage AnalysisKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS• Illustrates % code coverage • Can illustrate test efficiency when compared to number of tests executed• Normalized data can be used to compare projects

NOTES:

14%

25%

45%

16%

0%10%20%30%40%50%60%70%80%90%

100%

Project A Project B

% Coverage

% Components

14%

25%

45%

16%

0%10%20%30%40%50%60%70%80%90%

100%

Project A Project B

% Coverage

% Components



Test Tools Used to Enhance Test ReportingKEY CONCEPT PRESENTATION GUIDE


• Benchmarking – Continuous process of measuring products, services, and practices against other projects

• Quality Function Deployment (QFD) – Comprehensive quality system aimed specifically at satisfying the end user

NOTES:



Reporting Test ResultsKEY CONCEPT PRESENTATION GUIDE


• Should be a continuous process• Should be prepared at pre-defined checkpoints• Prepared at end of testingQuestions to answer:• What information do the stakeholders need?• How can testers present that information in an easy-to-understand format?• How can I present the information so that it is believable?• What can I tell the stakeholder that would help in determining what actions to take?

NOTES:



Test ReportsKEY CONCEPT PRESENTATION GUIDE


• Function Test Matrix• Defect Status Report• Functional Testing Status Report• Functions Working Timeline• Expected versus Actual Defects Uncovered Timeline• Defects Uncovered versus Corrected Gap Timeline• Average Age of Uncorrected Defects by Type\• Defect Distribution Report• Unit Test Report• Integration Test Report• System Test Report

NOTES:



Guidelines for Report WritingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe following two guidelines are provided for writing and using the report information:1. Develop a baseline.2. Use good report writing practices. The following are examples of good report writing:• Allow project team members to review the draft and make comments before the report is

finalized.• Don’t include names or assign blame.• Stress quality.• Limit the report to two or three pages stressing important items; include other information in

appendices and schedules.• Eliminate small problems from the report and give these directly to the project people.• Offer to have the testers work with the project team to explain their findings and

recommendations.

NOTES:





As a test manager for a large IT department, you’ve been asked what information should be recorded about each defect that is uncovered.

Identify what you believe are the four most important characteristics that should be recorded about a defect.

NOTES:





• Prerequisites to Test Reporting• Test Tools used to Build Test Reports• Test Tools used to Enhance Test Reporting• Reporting Test Results

NOTES:



Skill Category 7KEY CONCEPT PRESENTATION GUIDEUser Acceptance Testing


The following topics will be discussed in this Skill Category:• Acceptance Testing Concepts• Roles and Responsibilities• Acceptance Test Planning • Acceptance Test Execution

NOTES:



Software Acceptance Testing ConceptsKEY CONCEPT PRESENTATION GUIDE


• Early detection of software problems• Preparation of appropriate test facilities• Early consideration of the user's needs• Accountability for software acceptance belongs to the customer or user of the software

NOTES:



Four ComponentsKEY CONCEPT PRESENTATION GUIDE


NOTES:

Structure

Data

Rul

es

FIT

The reliability, timeliness, consistency, and usefulness of the data included in the automated application.

People

People should have the skills, training, aptitude, and desire to properly use and interact with the automated application.

The structure is the proper development of application systems to optimize technology and satisfy requirements.

The rules are the procedures to follow in processing the data

Structure

Data

Rul

es

FIT

The reliability, timeliness, consistency, and usefulness of the data included in the automated application.

People

People should have the skills, training, aptitude, and desire to properly use and interact with the automated application.

The structure is the proper development of application systems to optimize technology and satisfy requirements.

The rules are the procedures to follow in processing the data



Roles and ResponsibilitiesKEY CONCEPT PRESENTATION GUIDE


User Role:• Decide if Acceptance testing will occur• Planning and conducting Acceptance testing• Define acceptance in a testable format• Provide USE CASES that will be used• Train user personnel in the new system• Provide necessary resources• Perform any additional work needed prior to placing the software into operation

NOTES:



Roles and Responsibilities (cont)KEY CONCEPT PRESENTATION GUIDE


User Responsibilities• Ensure user involvement in developing system requirements and acceptance criteria.• Identify interim and final products for acceptance, their acceptance criteria, and schedule.• Plan how and by whom each acceptance activity will be performed.• Plan resources for providing information on which to base acceptance decisions.• Schedule adequate time for buyer staff to receive and examine products and evaluations

prior to acceptance review.

NOTES:





User Responsibilities (continued):• Prepare the Acceptance Plan.• Respond to the analyses of project entities before accepting or rejecting.• Approve the various interim software products against quantified criteria at interim points.• Perform the final acceptance activities, including formal acceptance testing, at delivery.• Make an acceptance decision for each product.

NOTES:





Three Possible Roles for Software Testers:1. No involvement at all (user takes full responsibility)2. Advise the user on technical issues3. Active participant is testing

NOTES:



Acceptance Test PlanningKEY CONCEPT PRESENTATION GUIDE


Acceptance test planning will include:• Acceptance Criteria• Acceptance Test Plan• USE CASE Test Data

NOTES:



Acceptance CriteriaKEY CONCEPT PRESENTATION GUIDE


• Acquire full knowledge of the application for which the system is intended• Become fully acquainted with the application as it is currently implemented by the user’s

organization• Understand the risks and benefits of the development methodology that is to be used in

correcting the software system• Fully understand the consequences of adding new functions to enhance the system

NOTES:



Acceptance Criteria CategoriesKEY CONCEPT PRESENTATION GUIDE


• Functionality Requirements- These requirements relate to the business rules that the system must execute.

• Performance Requirements- These requirements relate to operational aspects, such as time or resource constraints.

• Interface Quality Requirements - These requirements relate to connections from one component to another component of

processing (e.g., human-machine, machine-module).• Overall Software Quality Requirements

- These requirements specify limits for factors or attributes such as reliability, testability, correctness,

NOTES:



Safety and Security Criteria KEY CONCEPT PRESENTATION GUIDE


By definition, all safety criteria are critical; and by law, certain security requirements are critical. Some typical factors affecting criticality include:• Importance of the system to organization or industry• Consequence of failure• Complexity of the project• Technology risk• Complexity of the user environment

NOTES:



Acceptance Test PlanKEY CONCEPT PRESENTATION GUIDE


• Project Description• User Responsibilities• Administrative Procedures• Acceptance Description

NOTES:



USE CASE Test DataKEY CONCEPT PRESENTATION GUIDE


An individual use case consists of:• Preconditions that set the stage for the series of events that should occur for the use case• Post-conditions that state the expected outcomes of the above process• Sequential narrative of the execution of the use case

NOTES:



Acceptance Test ExecutionKEY CONCEPT PRESENTATION GUIDE


Can be accomplished through: • Reviews• Interim products• Testing the executable software system

NOTES:



Possible Acceptance DecisionsKEY CONCEPT PRESENTATION GUIDE


1. Make all changes - Required changes are accepted and made before progressing 2. Defer all changes - Progress may continue and changes may be accepted at the next

review 3. Pick and choose - Some changes must be made and accepted before progressing, while

others may be made and accepted at the next major review 4. No changes are required - Progress may continue

NOTES:





When a software system is placed into production and there are defects in the system, who or what should be blamed for those defects?a. Development / test teamb. End usersc. Development / test processesd. Production processesWho is normally responsible for conducting acceptance testing?a. Programmerb. Testerc. End Userd. Business Analyst

NOTES:





• Acceptance Testing Concepts• Roles and Responsibilities• Acceptance Test Planning • Acceptance Test Execution

NOTES:



Skill Category 8KEY CONCEPT PRESENTATION GUIDETesting Software Developed by Outside Organizations


The following topics will be discussed in this skill category:• Challenges in Testing Acquired Software• COTS Software Test Process• Contracted Software Test Process

NOTES:



Challenges in Testing Acquired SoftwareKEY CONCEPT PRESENTATION GUIDEPurchased, Non-contracted


• Task or items missing• Software fails to perform• Extra features• Does not meet business needs• Does not meet operational needs• Does not meet people needs

NOTES:



Challenges in Testing Acquired Software (cont)KEY CONCEPT PRESENTATION GUIDEContracted


• Quality factors may not be specified• Non-testable requirements and criteria• Customer’s standards may not be met• Missing requirements• Overlooked changes in standards or technology• Training and deployment may be difficult

NOTES:



Challenges in Testing Acquired Software (cont)KEY CONCEPT PRESENTATION GUIDEOffshore


• Cultural differences• Communication barriers• Loss of employee morale and support• Root cause of the contractor IT organization not addressed

NOTES:



Software Tester’s Responsibility for Software Developed by an Outside OrganizationKEY CONCEPT PRESENTATION GUIDE


• Responsibility for the quality COTS or contracted software cannot be contracted• There must be a process to monitor the development and validate the correct functioning of

contracted software• The software tester needs to look at how the contractor tests in relation to the SEI CMMI®

Capability Maturity Model.

NOTES:



COTS Specific Testing ChallengesKEY CONCEPT PRESENTATION GUIDECommercial-Off-The-Shelf (COTS)


NOTES:



COTS Testing ProcessKEY CONCEPT PRESENTATION GUIDE


• Assure completeness of needs specification• Define critical success factor• Determine compatibility with your computer environment• Assure the software can be integrated into your business system work flow• Demonstrate the software in operation• Evaluate the people fit• Acceptance test the COTS software

NOTES:



COTS Testing Process (cont)KEY CONCEPT PRESENTATION GUIDE


• Assure completeness of needs specification• Output products and reports• Management decision information

NOTES:





Define critical success factor• Ease of Use• Expendability• Maintainability• Cost-effectiveness• Transferability• Reliability• Security

NOTES:





Determine compatibility with your computer environment• Hardware Compatibility• Operating System Compatibility• Software Compatibility

NOTES:





Assure the software can be integrated into your business system work flow• Current system based on certain assumptions• Existing forms, existing data, existing procedures• COTS based on certain assumptions• COTS uses a predetermined set of forms & procedures.• Current system & COTS may be incompatible• COTS is not going to change

NOTES:





Demonstrate the software in operationObserve during demonstration:• Understandability• Clarity of communications• Ease of use of manuals• Functionality of software• Knowledge to execute• Effectiveness of help routines• Evaluate program compatibility• Data compatibility• Smell Test

NOTES:





Evaluate the people fit• The software can be used as is• Additional training and support is necessary• The software is not usable with the skill sets of the proposed users

NOTES:





Acceptance test the COTS software• Same basic procedure as acceptance testing in-house developed software

NOTES:



Contracted Software Specific Testing ChallengesKEY CONCEPT PRESENTATION GUIDE


NOTES:



Contracted Software Test ProcessKEY CONCEPT PRESENTATION GUIDE


• Assure process for contracting software is adequate• Assure requirements & contract criteria are testable• Review adequacy of the contractors test plan• Perform acceptance testing• Issue a report on adequacy• Ensure knowledge transfer & intellectual rights protected• Incorporate copyrighted material into manuals• Assure ongoing operation and maintenance • Assure the effectiveness of contractual relations

NOTES:



Contracted Software Test Process (cont)KEY CONCEPT PRESENTATION GUIDE


Assure process for contracting software is adequateContracts should include:• What is done.• Who does it• When it is done• How it is done• Where it is done• Penalties for nonperformance

NOTES:




SUPPORTING / EXPLANATORY MATERIALSAssure process for contracting software is adequateContracts concerns:• Warranty• Deliverables• Delivery date• Commencement date• Installation• Updates• Contractor support• Costs• Foreign attachments• Penalties• Life of contract• Modification capability• Service discontinuance

NOTES:

• Manual/training discontinuance• Acceptance test criteria• Purchase versus lease• Fairness of contract• Performance of maintenance• Contractor training• Contractor manuals• Supplies• Transportability• Termination• Contractor employee rights• Governing Law• Contractor inspection• Access to developers & documentation





• Assure requirements & contract criteria are testable• Common term: performance-based contracting

NOTES:





Review adequacy of the contractors test plan• Evaluate the adequacy and completeness of testing that will be performed.• May perform a comparison against internationally excepted test plan standards (e.g., IEEE)

NOTES:





Perform acceptance testingMinimum the acceptance testing should validate:• The documentation is consistent with execution• Documentation is understandable• User will be adequately trained• It is operable within the operational constraints

NOTES:





Issue a report on adequacyFocus on two definitions of quality:• Meets requirements• Fit for use

NOTES:





Ensure knowledge transfer and intellectual rights are protected• Training programs for staff• Being advised of defects uncovered by other organizations using software• Ability to contact contractor’s help desk• Use care to protect contractor property rights

NOTES:





Incorporate copyrighted material into manualsContracting organization may have copyrighted material to protect

NOTES:





Assure ongoing operation and maintenanceMajor concerns:• Adequacy of control• Adequacy of documentation• Speed of service• Nearness of service• Competency of service• Adequacy of hardware• Skilled personnel• Multi-contractor problem resolution• Cost of services• Cost of operations• Error diagnosis• Error documentation

NOTES:





Assure effectiveness of contractual relations• Contractor obligations met• Contracting organization obligations met• Needs met• Limits on cost increases• Exercising options (e.g., added features)• Renegotiation• Compensation for error• Returns on termination

NOTES:





• Assume that your organization was asked to build an important software project, but your IT organization was currently overcommitted, so it was decided to outsource the development of the software to a contractor.

• Your contracting officer has asked you as a test manager to write the section of the contract that specifies how the contractor is to test the software.

• Rather than write that section, list what you think are the five (5) most important contractor test responsibilities that you think should be included in the contract and then describe those responsibilities in enough detail that if the contractor met those testing responsibilities you would be comfortable with the contractor’s ability to effectively test the software.

NOTES:





• Challenges in Testing Acquired Software• COTS Software Test Process• Contracted Software Test Process

NOTES:



Skill Category 9KEY CONCEPT PRESENTATION GUIDETesting Software Controls and the Adequacy of Security Procedures


The following topics will be discussed in this Skill Category:• Principles and Concepts of Internal Control• Internal Control Models• Testing Internal Controls• Testing Security Controls

NOTES:



Internal Control DefinedKEY CONCEPT PRESENTATION GUIDE


Four key terms should be understood1. Risk – The probability that an undesirable event will occur.2. Exposure – The amount of loss that might occur if an undesirable event occurs.3. Threat – A specific event that might cause an undesirable event to occur.4. Control – Anything that will reduce the impact of risk.

A process, effected by an organization’s Board of Directors, management and other personnel, designed to provide reasonable assurance regarding the achievement of objectives in the following categories:• Effectiveness and efficiency of operations• Reliability of financial reporting• Compliance with applicable laws and regulations.

COSO = Committee of Sponsoring Organizations

NOTES:



Internal Control TermsKEY CONCEPT PRESENTATION GUIDE


• Internal ControlThe interaction and/or interactivity within an organization that allows the entity to provide for information accuracy, the safeguard of assets, reliability and efficiency of operations, and adherence to management policies, procedures and processes.

• ProcessAn established methodology that has certain defined components which enable the activity to be consistently repeated without variability.

• TestingThe activity performed to reduce the risk and probability of an unfavorable event occurring.

NOTES:



Internal Control Terms (cont)KEY CONCEPT PRESENTATION GUIDE


• RiskThe probability that an undesirable event will occur.

• ExposureThe amount of loss that might occur if an undesirable event occurs.

• ThreatA specific event that might cause an undesirable event to occur.

• ControlAny activity, technique, method, or approach that minimizes or eliminates risk.

NOTES:



Internal Auditor’s ResponsibilitiesKEY CONCEPT PRESENTATION GUIDE


The International Standards for the Professional Practice of Internal Auditing, established by the Institute of Internal Auditors, specify that internal auditors should:• identify and evaluate significant exposures to risk and contribute to the improvement of risk

management and control systems• Monitor and evaluate the effectiveness of the organization’s risk management system• Evaluate risk exposures relating to the organization’s governance, operations, and

information systems • Assist the organization in maintaining effective controls by evaluating their effectiveness and

efficiency and by promoting continuous improvementIt should be recognized that the internal audit function does not have primary responsibility for establishing or maintaining the internal control system.

NOTES:



Risk Versus ControlKEY CONCEPT PRESENTATION GUIDE


To address risk, management must define control objectives and implement controls to accomplish that control objective.

NOTES:

ProbabilityProbability XX==RiskRisk LossLoss


ProbabilityProbability XX==RiskRisk LossLossProbabilityProbability XX==RiskRisk LossLoss




Levels of Organizational ControlKEY CONCEPT PRESENTATION GUIDE


NOTES:

Environmental controls are the means by which management “manages” the organization.

• Organizational policies

• Organizational structure

• Method of hiring, training, supervising and evaluating

• Processes provided for day-to-day work activities



Levels of Organizational Control (cont)KEY CONCEPT PRESENTATION GUIDE


NOTES:

Transaction Processing Controls

• System that Processes Transactions

• System that Controls the Transaction Processing


• System that Processes Transactions

• System that Controls the Transaction Processing



Levels of Organizational Control (cont)KEY CONCEPT PRESENTATION GUIDE


NOTES:


• Preventive Controls

• Detective Controls

• Corrective Controls


• Preventive Controls

• Detective Controls

• Corrective Controls



Internal Control ModelsKEY CONCEPT PRESENTATION GUIDE


COSO Enterprise Risk Model (ERM)The COSO ERM framework defines risk and enterprise risk management, and provides a foundational definition, conceptualizations, objective categories, components, principles and other elements of a comprehensive risk management framework. ERM consists of eight interrelated categories:• Internal Environment• Objective Setting• Event Identification• Risk Assessment• Risk Response• Control Activities• Information and Communications• Monitoring

NOTES:



Internal Control Models (cont)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSCOSO Internal Control FrameworkThe COSO Internal Control Framework chose to use “control objectives” as opposed to defining risk.

NOTES:



Internal Control Models (cont)KEY CONCEPT PRESENTATION GUIDE


CobiT ModelThe CobiT model is a generally applicable and accepted standard for IT security and control practices. Four-part Cycle:Part 1: Plan and OrganizePart 2: Acquire and ImplementPart 3: Deliver and SupportPart 4: Monitor

NOTES:



Testing Internal ControlsKEY CONCEPT PRESENTATION GUIDE


Testing the controls in a software system involves accomplishing these objectives:• The requirements for the controls have been defined. These are normally the risks that need

to be minimized or eliminated.• The defined controls are in place and working, which is traditional testing.• Test that the “enterprise” controls are included in the software system and are working.

Examples of enterprise controls include security controls and control documentation.

NOTES:



Testing Internal Controls (cont)KEY CONCEPT PRESENTATION GUIDE


The best practices for testing the controls in a software system involve two tasks:1. Perform Risk Assessment

Test the completeness of the control requirements, which involves evaluating the software system risks.

2. Test Transaction Processing ControlsFollow the flow of transaction processing to test whether or not the defined controls are in place, working, and that the transactions are controlled throughout transaction processing.

NOTES:



Testing Security ControlsKEY CONCEPT PRESENTATION GUIDE


Tester’s Tasks1. Understand points where security is most frequently penetrated2. Build a penetration point matrix3. Assess security awareness training program4. Understand attributes of an effective security control5. Understand the process for selecting techniques to test security

NOTES:



Testing Security Controls (cont)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSTask 1. Define points where security is most frequently penetrated

NOTES:




SUPPORTING / EXPLANATORY MATERIALSTask 2. Build a Penetration Point Matrix

NOTES:




SUPPORTING / EXPLANATORY MATERIALSTask 3. Assess the Security Learning Continuum

NOTES:





Task 4. Define the Attributes of an Effective Security Control• Simplicity• Fail Safe• Complete Mediation• Open Design• Separation of Privilege• Psychological Acceptability• Layered Defense• Compromise Recording

NOTES:





Task 5. Select Techniques to Test SecurityThe following security testing techniques are recommended for testing security:• Network scanning• Vulnerability scanning• Password cracking• Log review• Integrity checkers• Virus detection• War dialing• War driving (wireless LAN testing)• Penetration testing

NOTES:





• Principles and Concepts of Internal Control• Internal Control Models• Testing Internal Controls• Testing Security Controls

NOTES:



Skill Category 10KEY CONCEPT PRESENTATION GUIDETesting New Technologies


The following topics will be discussed in this Skill Category:• Risks Associated with New Technology • Newer IT Technologies that Impact Software Testing• Testing the Effectiveness of Integrating New Technology

NOTES:



Risks Associated with New TechnologyKEY CONCEPT PRESENTATION GUIDE


• Unproven technology• Technology is defective• Inefficient technology• Technology incompatible with other implemented technologies• New technology obsoletes existing implemented technologies• Variance between documentation and technology execution

NOTES:



Risks Associated with New Technology (cont)KEY CONCEPT PRESENTATION GUIDE


• Staff not competent to use new technology• Lack of understanding how to optimize the new technology• Technology not incorporated into the organization’s work processes• Obsolete testing tools• Inadequate vendor support

NOTES:



Risks Associated with Web-based ApplicationsKEY CONCEPT PRESENTATION GUIDE


Some questions to ask to determine these risks would be: • Which types of plug-ins are used by the application under test?• What are the testing implications associated with these plug-ins?• What issues should be considered during functionality and compatibility the plug-ins have

been integrated into the system?• How should the distribution of server-side components affect test design and strategy?• Which Web and database servers are supported by the application?• How Web-to-database connectivity is implemented and what are the associated

implications?• How can testing be partitioned to focus on problematic components?

NOTES:



Risks Associated with Distributed Application ArchitectureKEY CONCEPT PRESENTATION GUIDE


Some questions to ask to determine these risks would be: • If a thin-client system is utilized, is the server-side performance adequate? • If a thick-client system will be utilized, what bandwidth, performance and compatibility is

required?

NOTES:



Risks Associated with Wireless TechnologyKEY CONCEPT PRESENTATION GUIDE


Some questions to ask to determine these risks would be: • What standards will be used?• What is the expected coverage?• What security measures will be utilized

NOTES:



Risks Associated with New Business ModelsKEY CONCEPT PRESENTATION GUIDE


Some questions to ask to determine these risks would be: • e-commerce includes buying and selling over the Web, electronic funds transfer, smart

cards, digital cash, and all other ways of doing business over digital networks. What standards and security measures will be used to protect the business and the consumer?

• e-business is a more generic term than e-commerce because it refers to not only buying and selling but also servicing customers and collaborating with business partners. As with e-commerce, what standards and security measures will be used to protect the business and the consumer?

NOTES:



Risks Associated with New Communication MethodsKEY CONCEPT PRESENTATION GUIDE


Some questions to ask to determine these risks would be: What types of:• Voice and messaging devices• Hand-held and other Internet-enabled devices• Data networking will be used and what limitations and/or security risks do they pose?

NOTES:



Risks Associated with New Test ToolsKEY CONCEPT PRESENTATION GUIDE


Some questions to ask to determine these risks would be: • Will full or partial automated testing be utilized?• What levels of experience will testers need to fully utilize automated test tools?• If an automated test tool is used for the creation of test data or test cases, for example, is

there a process to ensure the information created is accurate and valid?

NOTES:



Testing the Effectiveness of Integrating New TechnologyKEY CONCEPT PRESENTATION GUIDE


Determine the process maturity level of the technology• Each time new technology is acquired an IT organization’s ability to optimize the use of that

technology reverts to Level 1.• Depending on the technology acquired the time span to mature the use of that technology

can vary from days to years.Testing the adequacy of the controls over the new technology to evaluate the effectiveness of the implementation involves these three tasks:• Testing actual performance versus stated performance• Test the adequacy of the current processes to control the technology• Assess whether staff skills are adequate to effectively use the technology

NOTES:





Assume that a software project within your organization is planning to use wireless technology. Further assume that the system using wireless technology is one that is processing financial transactions. As a tester, what issues (i.e. concerns that the technology might not be under control) do you believe need to be addressed before the software system using wireless technology is operational. List at least three (3) issues, describe those issues and then list one way that would increase your confidence that the wireless technology is under control.

NOTES:





• Risks Associated with New Technology • Newer IT Technologies that Impact Software Testing• Testing the Effectiveness of Integrating New Technology

NOTES:



Test Taking TipsKEY CONCEPT PRESENTATION GUIDE


NOTES:



On the Day of the ExamKEY CONCEPT PRESENTATION GUIDE


• Arrive at the examination location at least 30 minutes before the scheduled start time of the examination

• You must have the following on the day of the exam:• Confirmation Letter from Software Certifications• Photo identification • Two # 2 pencils• Don’t bring personal items into the room• Once exam starts, no one will be admitted

NOTES:



About the ExamKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSFour parts to the examPart Questions Type

1 50 Objective or multiple choice questions – each question has equal point value

2 10 Essay or Scenario based - each question has equal point value3 50 Objective or multiple choice questions – each question has equal

point value4 10 Essay or Scenario based - each question has equal point value

Minimum passing = 75% averaged over all 4 parts of the examination to become certified

NOTES:



After the ExamKEY CONCEPT PRESENTATION GUIDE


• Allow up to 12 weeks for results• Completed results are mailed the same day• Do not discuss the contents of the exam – The code of ethics• Do not call QAI or Software Certifications within those 12 weeks• Results will not be given over the phone.

NOTES:



First ThingKEY CONCEPT PRESENTATION GUIDE


Before you begin – • Remember you have plenty of time• Take a deep breath• Don’t be nervous

NOTES:



Tip #1KEY CONCEPT PRESENTATION GUIDE


• Remember one question may only be worth one point, if you are not able to answer questions later because you have lost time you could lose more points!

• Becoming nervous can make you lose your concentration and that leads to worse results.

• Do not insist on completing each question before going to the next. • This is extremely important. If you spend a lot of time on one question that you don't

understand there can be two negative outcomes: 1. Losing time 2. Becoming nervous

NOTES:



The How-to’sKEY CONCEPT PRESENTATION GUIDE


• For multiple choice parts, read each question’s stem twice, then read ALL the responses• For essay parts, read each of the essay questions thoroughly • As you read through each question (multiple choice and essay), determine whether:

- You absolutely know the answer to this question.- You believe you know the answer to this question. - You are not sure you know the answer, or it would take time to develop an answer.

• For both multiple choice and essay questions, answer the questions that you know the answers to first

NOTES:



Sample QuestionsKEY CONCEPT PRESENTATION GUIDE


The primary purpose of quality control is to:a. Prevent defects from occurringb. Assign quality responsibilitiesc. Uncover defectsd. Conduct testing

The most effective test approach is to begin testing when:a. A new project beginsb. After requirementsc. After external designd. After coding

NOTES:



More How-to’sKEY CONCEPT PRESENTATION GUIDE


If possible, go through the entire test answering the questions you are sure you know. • This results in your being more relaxed and feeling more confident.

NOTES:



Second RoundKEY CONCEPT PRESENTATION GUIDE


• Go through the test a second time, working out the answers to more difficult questions. • Now you will feel more confident and this will improve your test taking.

• Key Tip: Do not waste too much time on any one question!

NOTES:



Check BackKEY CONCEPT PRESENTATION GUIDE


• This should not be tried until you have finished the test and have some time remaining to try the questions you have had problems with again.

• Once you have gone through the test twice, see if any of the questions asked can help you answer those really difficult questions.

• This is a little used trick: Sometimes questions asked are answered in later questions asking for different things

NOTES:



Don’t Change AnythingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS• Going back to think about it usually makes you unsure and often causes an error. This is

very common so be very careful!

• If you have a strong feeling about a question when you first answer it, don't go back and change it later.

• Usually (but not always) a strong first impulse means we know the answer and we don't really have to think about it too much.

NOTES:



Change the ChancesKEY CONCEPT PRESENTATION GUIDE


• Play the odds • If you don't know the answer, write something. • If you are answering a 4 possibility multiple choice question you will still have a 25% chance

of being correct!

NOTES:



Fluff and StuffKEY CONCEPT PRESENTATION GUIDE


• As a test manager for a large IT department, you’ve been asked what information should be recorded about each defect that is uncovered.

• Identify and explain what you believe are the four most important characteristics that should be recorded about a defect.

NOTES:



The DistractersKEY CONCEPT PRESENTATION GUIDE


• Throw out the ridiculous answers in a multiple choice question • Many tests often include one or two completely ridiculous possibilities in any multiple choice

question. • Find these first and get rid if them. This will usually leave you with a choice between two.

Your odds will have already increased to 50%!

NOTES:



Test Taking Tip SummaryKEY CONCEPT PRESENTATION GUIDE


• Answer the questions that you believe you know the answer • Answer the questions that you think you know the answer• Answer the questions that you don’t know the answer

NOTES:



The EndKEY CONCEPT PRESENTATION GUIDE


NOTES:

.

ID_CSTE-EPC-3927 A-1.

How to Take the CSTE Examination

the Introduction of this preparation guide explained a process for you to follow to prepare for the examination. It emphasized familiarizing yourself with a Common Body of Knowledge (CBOK), the vocabulary of software quality, the activities performed by testing professionals, and reviewing several other different references

that are included on the software certifications Web site. Other references you might look at include articles in professional IT publications.If you feel you are ready to take the examination, you need to schedule the examination. Be sure to visit www.softwarecertifications.org for up-to-date CSTE examination dates and places. Once scheduled, the remaining event is to take the examination.

A.1 CSTE Examination OverviewThe four and a half hour examination consists of four written parts, including multiple-choice andessay questions. A typical CSTE examination is comprised of two parts for Software TestingTheory and two parts for Software Testing Practice:

CSTE Examination Overview page 1

Guidelines to Answer Questions page 2

Sample CSTE Examination page 4

Appendix

A

T


A-2 © QAI Worldwide ID_CSTE-EPC-3927

A.1.1 Software Testing TheoryPart 1 50 multiple-choice questions. Complete within 45 minutes.

Part 2 10 essay questions. Complete within 1 hour, 15 minutes.

Software testing theory evaluates your understanding of testing principles, practices, vocabulary andconcepts. In other words, do you have a solid foundation in testing basics? For example, can youdifferentiate between verification and validation?

A.1.2 Software Testing PracticePart 3 50 multiple-choice questions. Complete within 45 minutes.

Part 4 10 essay questions. Complete within 1 hour, 15 minutes.

Software testing practice evaluates whether you can apply testing basics to real-world situations. Forexample, a question may be: “What methods of quality control would you use to reduce defects for asoftware system under development? During which phase of development would you use thosemethods?”

You cannot bring any study or supporting materials to the examination site other than a pencil. Eachindividual will have an assigned seat. You may take a 15-minute break between each examination part,but not during the examination.

Proctors for the examination are not required to be certified. If not certified, they are ineligible to takethe examination for at least two years after proctoring. Proctors follow specific instructions to conductthe examination. Software Certifications’ policies and procedures enforce confidentiality and securityof the examination instrument, prior to and after the examination.

A.2 Guidelines to Answer QuestionsThe examination proctor will give you one part of the examination at a time. As you receive each part,use the following steps to answer the questions:

1. Read the entire examination part before answering any questions. - For multiple-choice parts, only read each question’s stem, not the four to five responses. - For essay parts, read all of the essay questions thoroughly.

2. As you read through each question (multiple-choice and essay) determine whether:- You absolutely know the answer to this question.- You believe you know the answer to this question.- You are not sure you know the answer, or it would take time to develop an answer.

3. For both multiple-choice and essay questions, answer the questions that you know theanswers; they should not take you much time to complete. This will give you the majorityof the examination time for the other questions you may need more time to answer.


ID_CSTE-EPC-3927 © QAI Worldwide A-3

4. Answer the questions that you believe you know the answer.- For multiple-choice questions, answer all of the questions.- For essay questions, answer the questions worth the most points first, followed by those

worth less points. Note that the points equal the percentage of score allocated to that essay question.

5. Answer the questions that you do not know the answer.- For multiple-choice questions, answer all of the questions.- For essay questions, answer the questions worth the most points first, followed by those

worth less points.Follow these recommended guidelines for answering essay questions. Remember that an individual grades your examination. Make your thoughts and ideas clear and concise. Also, be sure to write legibly.• Those questions worth the most points should be the longest essay response. For example, if

an essay question is worth 25 points, the response should be at least twice as long as an essay response for a question worth 10 points.

• You need not write complete sentences. Those grading your examination are looking for key phrases and concepts. You can highlight them in a bulleted form, underlined or capitalized. This will help ensure that the individual scoring your examination can readily understand your knowledge of the correct response to that essay question.

• Charts, graphs, and examples enhance your responses and enable the individual grading your examination to evaluate your understanding of the question. For example, use a control chart example to clearly indicate your knowledge of statistical process control.

Follow these recommended guidelines for answering multiple-choice questions.• Each multiple-choice question is comprised of a stem statement (or stem) and multiple

responses to the stem. Read the stem carefully to assure you understand what is being asked. Then without reading the given responses to the stem, create in your mind what you believe would be the correct response. Look for that response in the list of responses.

• You will be given four or five responses to each stem.• If you cannot create a response to the stem prior to reading the responses, attempt to

eliminate those responses which are obviously wrong. In most cases, after that you will only have two responses remaining.

• To select between what appears to be correct responses, rethink the subject matter in the stem and document what you know about that subject. This type of analysis should help you to eliminate a response or select the correct response among what should be two responses.

Follow these recommended guidelines when you do not know the answer to a question. Usually youcan take a few minutes to look over the choices or write down some key points to help you answer thequestions.

• For multiple-choice questions, answer all questions – there is no reduction to your score for wrong answers. If you do not know the answer, try to rule out some of the potential responses. Then select the response you believe might be correct.

• For essay questions, indicate any information that might be helpful. For example, you have a question on test policies. You have not actually written a test policy, but you could define a test policy, give an example of a test policy, and explain what you recommend to implement that policy in your IT organization.



A.3 Sample CSTE Examination The following CSTE examination is a sample of some of the questions, both multiple-choice andessay, which may appear on the actual examination. Use this sample examination to help you study.The multiple-choice questions are listed by skill category. If you miss a question in a particular skillcategory, simply refer back to that skill category in this guide for additional study. Use the essayquestion responses in the sample examination as a guide for responding to the actual CSTE essayquestions.

A.3.1 Part 1 and Part 3 Multiple-Choice QuestionsThis sample examination contains 20 multiple-choice questions – two questions for each category.While taking this sample examination, follow the steps described in “Guidelines to Answer Questions”on page page 2 to practice the recommended techniques.

Circle the correct answer. Compare your answers to the answer key that follows on page 10.

Skill Category 1 – Software Testing Principles and Concepts

1. The customer’s view of quality means:

a. Meeting requirements

b. Doing it the right way

c. Doing it right the first time

d. Fit for use

e. Doing it on time

2. The testing of a single program, or function, usually performed by the developer is called:a. Unit testing

b. Integration testing

c. System testing

d. Regression testing

e. Acceptance testing

Skill Category 2 – Building the Test Environment



3. The measure used to evaluate the correctness of a product is called the product:a.Policy

b.Standard

c.Procedure to do work

d.Procedure to check work

e.Guideline

4. Which of the four components of the test environment is considered to be the mostimportant component of the test environment:

a.Management support

b.Tester competency

c.Test work processes

d.Testing techniques and tools

Skill Category 3 – Managing the Test Project

5. Effective test managers are effective listeners. The type of listening in which the tester isperforming an analysis of what the speaker is saying is called:

a.Discriminative listening

b.Comprehensive listening

c.Therapeutic listening

d.Critical listening

e.Appreciative listening

6. To become a CSTE, an individual has a responsibility to accept the standards of conductdefined by the certification board. These standards of conduct are called:

a.Code of ethics

b.Continuing professional education requirement

c.Obtaining references to support experience

d.Joining a professional testing chapter

e.Following the common body of knowledge in the practice of software testing

Skill Category 4 – Test Planning

7. Which of the following are risks that testers face in performing their test activities:a.Not enough training



b.Lack of test tools

c.Not enough time for testing

d.Rapid change

e.All of the above

8. All of the following are methods to minimize loss due to risk. Which one is not a method tominimize loss due to risk:

a.Reduce opportunity for error

b.Identify error prior to loss

c.Quantify loss

d.Minimize loss

e.Recover loss

Skill Category 5 – Executing the Test Plan

9. Defect prevention involves which of the following steps:a.Identify critical tasks

b.Estimate expected impact

c.Minimize expected impact

d.a, b and c

e.a and b

10. The first step in designing use case is to:a.Build a system boundary diagram

b.Define acceptance criteria

c.Define use cases

d.Involve users

e.Develop use cases



Skill Category 6 – Test Reporting Process

11. The defect attribute that would help management determine the importance of the defect iscalled:

a.Defect type

b.Defect severity

c.Defect name

d.Defect location

e.Phase in which defect occurred

12. The system test report is normally written at what point in software development:a.After unit testing

b.After integration testing

c.After system testing

d.After acceptance testing

Skill Category 7 – User Acceptance Testing

13. The primary objective of user acceptance testing is to:a.Identify requirements defects

b.Identify missing requirements

c.Determine if software is fit for use

d.Validate the correctness of interfaces to other software systems

e.Verify that software is maintainable

14. If IT establishes a measurement team to create measures and metrics to be used in statusreporting, that team should include individuals who have:

a.A working knowledge of measures

b.Knowledge in the implementation of statistical process control tools

c.A working understanding of benchmarking techniques

d.Knowledge of the organization’s goals and objectives

e.All of the above

Skill Category 8 – Testing Software Developed by Contractors



15. What is the difference between testing software developed by a contractor outside yourcountry, versus testing software developed by a contractor within your country:

a.Does not meet people needs

b.Cultural differences

c.Loss of control over reallocation of resources

d.Relinquishment of control

e.Contains extra features not specified

16. What is the definition of a critical success factor: a.A specified requirement

b.A software quality factor

c.Factors that must be present

d.A software metric

e.A high cost to implement requirement

Skill Category 9 – Testing Internal Control

17. The condition that represents a potential for loss to an organization is called:a.Risk

b.Exposure

c.Threat

d.Control

e.Vulnerability

18. A flaw in a software system that may be exploited by an individual for his or her advantageis called:

a.Risk

b.Risk analysis

c.Threat

d.Vulnerability

e.Control



Skill Category 10 – Testing New Technologies

19. The conduct of business of the Internet is called:a.e-commerce

b. e-business

c. Wireless applications

d. Client-server system

e. Web-based applications

20. The following is described as one of the five levels of maturing a new technology into anIT organization’s work processes. The “People-dependent technology” level is equivalentto what level in SEI’s compatibility maturity model:

a.Level 1

b.Level 2

c.Level 3

d.Level 4

e.Level 5



A.3.2 Part 1 and Part 3 Multiple-Choice AnswersThe answers to the sample examination for Part 1 and Part 3 are as follows. If you missed a question,study that material in the relative skill category.

1. d Fit for use

2. a Unit testing

3. b Standard

4. a Management support

5. d Critical listening

6. a Code of ethics

7. e All of the above

8. c Quantify loss

9. d a, b and c

10. a Build a system boundary diagram

11. b Defect severity

12. c After system testing

13. c Determine if software is fit for use

14. e All of the above

15. b Cultural differences

16. c Factors that must be present

17. a Risk

18. d Vulnerability

19. b e-business

20. a Level 1



A.3.3 Part 2 and Part 4 Essay Questions and AnswersEssay questions on theory focus on “what to do” and essay questions on best practices focus on “howto do it.” Part 2 and Part 4 of the certification exam both have ten essay questions, one from each skillcategory. Five of the following essay questions are questions that could be included in Part 2, TheoryEssay Questions of the CSTE examination. The other five questions are questions that could beincluded in Part 4, Practice Essay Questions.

A.3.3.1Part 2 – Software Testing Theory Essay QuestionsAnswer these five essay questions following the guidelines in Guidelines to Answer Questions on pagepage 2. Note that on the actual examination, each page has just one essay question to give you plenty ofspace to write your response.


1. List 5 reasons why we test software.


2. The requirements for a major system to which you have been assigned as a tester includes avery complex data model with an extensive list of fields, codes, and data values that arehighly interdependent. What steps will you take throughout your test planning to assure thecompleteness and adequacy of your test coverage and what impact will these steps have onthe content of your test plans?



Skill Category 2 – Building the Test Environment

3. Your IT Director is concerned about the cost of software testing and the length of timeinvolved in testing a software system. An investigation indicates that a significant amountof testing is performed after it is known that conditions have occurred which makes theadditional testing of little value. You believe that if testers stop testing when certainconditions are met, the cost and duration of software testing could be reduced. Your ITDirector agrees with you and asks you to indicate what types of conditions you wouldinclude in a test plan that would justify stopping software testing. These conditions wouldhave been corrected prior to resuming the tests.

List below those conditions that you would recommend being incorporated into a test planfor testers to stop testing. Name and briefly describe each condition.

Skill Category 8 – Testing Software Developed by Contractors

4. Your organization has outsourced the development of a major software project. Yourmanager has asked you to develop a plan to test the software; but, before developing yourtest plan your manager wants you to list at least four differences between testing softwaredeveloped by contractors and software developed in-house. List these differences:

Skill Category 3 – Managing the Test Project

5. Developing compatibility and motivation with a test team helps assure effective testing.List below at least four guidelines you would follow to develop compatibility andmotivation within your test team.



A.3.3.2Part 2 – Software Testing Theory Essay Answers

The following responses are examples of responses expected to receive a good grade. Review these examples as responses that adequately answer the essay question, not

as the only correct response.

Essay 1.

a. To produce a quality product, which is defect free, and also to make sure all therequirements are satisfied and the best design system architecture is used.

b. Customers/user satisfaction (customers are the king).

c. To make sure the software is:

Correct

Reliable

Portable

Efficient

Interoperable

Usable

Maintainable

Re-usable

Secure

Flexible

d. To achieve the goals of an organization as well as to make profit.

e. To reduce the possible risks associated with the software, then reduce the loss,which might happen when/if the software is released with defects included.

Essay 2.

a.Since it is a very complex model and if I have access to source code, then try to test thecode with the following techniques:

Branch coverage

Statement coverage

Decision coverage

b.Would use Boundary Value Analysis, error guessing and Equivalence partitiontechniques to create 3 or more test cases for every field, in addition to the test casesthat are created for testing functional requirements.



c.Would test the control mechanism implemented for every field and make sure the fieldsaccept only the values that they are supposed to accept. For example, will make surealphabetic fields don’t accept numbers or special characters.

d.Check for lengths to make sure the input data cannot exceed the field length defined in thedatabase.

e.Would test for valid dates, if date fields were used.

f.Would create a data pool with all possible values for fields and generate quite a few testscripts to test the different fields at a faster speed.

g.Would make sure all possible error conditions are tripped and they are handledgracefully.

h.Would test all the required fields as mandatory.

i.Would alter the auto-generating fields in the database (if possible) and see how the systemhandles it.

Essay 3.

a.When the quality objectives are met.

b.When the exit criteria mentioned in the test plan is met.

c.When there is no potential show-stopper or show blocker in the software.

d.When the expected test coverage is achieved.

e.When all the test cases are executed.

f.When the project budget is depleted or when test time is not enough (this is not a goodpractice, but happens.)

g.When the remaining minor defects go below the accepted level.



Essay 4.

The differences between software developed by a contractor and software developed in-house are:

- Quality factors may not be specified.- There are many factors such as reliability and ease of use which are frequently not

included as part of the contractual criteria. Thus when the software is delivered it may not be as easy to use or as reliable as desired by the contractor.

- Non-testable requirements and criteria.- If the requirements or contractual criteria in measurable and testable terms then the

delivered result may not meet the intent of the contractor.- Customer’s standards may not be met- Unless the contract specifies the operational standards and documentation standards the

delivered product may be more complex to use than desired by the customer.- Missing requirements- Unless detailed analysis and contractual specifications work is complete the contractor

may realize during the development of the software that requirements are missing and thus the cost of the contract could escalate significantly.

- Overlooked changes in standards in technology- If changes in standards that the organization must meet, or the introduction of new

desirable technology is incorporated into the contract there may be significant cost to modify the software for those new standards in technology.

- Training and deployment may be difficult- If software is developed by another organization there may be inadequate knowledge in

the contracted organization to provide the appropriate training for staff and to ensure that deployment is effective and efficient.

Essay 5.

Guidelines are helpful in developing compatibility and motivation of a software project team:

2.Communicate the vision, objectives, and goals of the project.A software professional wants to know what the project is trying to accomplish. The visionindicates why the project is undertaken, the goals and objectives indicate what the projectis to achieve. For example, the vision of a bank commercial loan software project might beto increase profitability. This specific objective might be to provide the loan officer theinformation needed to make a good loan decision.

3.Define roles and responsibilities of team members.Software projects, unlike non-software projects, have roles which are heavily peopledependent and project scope dependent. It’s important for professional staff to have thoseroles and responsibilities clearly defined. The staffing matrix defines those roles andresponsibilities.

4.Empower team members to manage their responsibilities.Empowerment is a major motivator for professional people. Many of the agile conceptsrelate to empowerment. In other words, enable people to perform the tasks in the most



efficient and effective manner. This helps eliminate barriers that increase costs and helpproject schedule.

5.Hold team members accountable for their assigned responsibilities in the team process.Team members need to have their work tasks well defined and then held accountable forcompleting those work tasks. Managerial practices indicate that this process works bestwhen individuals accept responsibility for performing tasks. Thus, having the ProjectManager work individually with team members to assign team tasks they agree to perform,and then hold those individuals accountable for completing those tasks is an effectivemanagerial practice.

6.Ensure that all the required skills are present on the team.Projects cannot be completed successfully if the team members lack the skills to completethe project. It is not necessary for every team member to have all the needed skills, but theteam in total needs the skills. The staffing matrix helps assure that the appropriate skillsexist within the project team.

7.Provide the necessary technical and team training.If the team lacks technical and team skills, the project manager should provide thattraining. Technical skills include the skills necessary to design and build the software, teamskills to cover such skills as consensus building and conflict resolution.

8.Award successes and celebrate achievements.Establishing goals and objectives provides the basis for rewards and celebrations. Whileit’s appropriate to reward and celebrate individual achievements, the team buildingnecessitates team goals and team celebrations. These can be centered around milestonesaccomplished, as well as scoring high on customer satisfaction surveys.



A.3.3.3Part 4 – Software Testing Practice Essay QuestionsAnswer these five essay questions following the guidelines in Guidelines to Answer Questions on pagepage 2. Note that on the actual examination, each page has just one essay question to give you plenty ofspace to write your response.


1. Explain and give an example of each of the following black-box test case techniques.Equivalence partitioning:

Boundary analysis:

Error guessing:


Explain the difference between verification and validation.



Skill Category 4 – Test Planning

2. A study by a major software supplier indicates that software testers make more defects thansoftware developers. Since software testing in your organization is a costly and time-consuming process, your IT Director believes that software testers may, in fact, make moredefects than software developers. The IT Director has asked you to describe what youbelieve might be the five major defects that software testers make in planning, executingand reporting the results from software testing.

List below the name you would give to those five defects and briefly describe each defect.(10 points)


3. Assume you have been promoting testing throughout the development life cycle, but yourmanager does not really understand what specifically is involved. Your manager has askedyou to present the concept at an IT staff meeting. You chose the “V” concept of testingmodel to explain testing throughout the development life cycle.

Explain and provide a graphic below of the “V” concept of testing.




4. Recording and tracking defects uncovered during testing is an important test responsibility.Describe three attributes of a defect that you believe testers should include when theydocument a defect. Give specific examples of how you would describe each of the threedefect attributes you propose.



A.3.3.4Part 4 – Quality Assurance Practice Essay Answers

The following responses are examples of responses expected to receive a good grade. Review these examples as responses that adequately answer the essay question, not

as the only correct response.

Essay 1.

Equivalence partitioning:

This technique will help to narrow down the possible test cases using equivalence classes.Equivalence class is one, which accepts same types of input data. Few test cases for everyequivalence class will help to avoid exhaustive testing.

Boundary analysis:

This technique helps to create test cases around the boundaries of the valid data. Usuallyvalues passed are exact boundary values, + or – 1 at the lower boundary and + or – 1 atthe higher boundary. This is an excellent technique and has proven that software is error-proof to boundaries.

Error guessing:

This technique is used to find defects using the experience of the tester, who is very familiarto the module he/she is testing. Usually the history is used as an input to guess the valuesfor input. For example, if the software is always error prone to negative values, where onlypositive values should be accepted, the tester can easily guess to enter negative values forvarious tests that would help to identify all the defects related to them.

Essay 2.

Verification:

• Uses non-executable methods of analyzing the various artifacts.• More effective, it has been proven that 65% of defects can be discovered here.• Uses inspection reviews to verify the requirements and design.• Good examples are having checksheets, traceability matrix kind of documents to

verify the documents, requirements and software features.• Code reviews, walkthroughs also come under this category.

Validation:

• Can be used throughout the software development life cycle.• Uses executable methods – means the software will be used to analyze the various

artifacts and test software.• Effective, but not as effective as verification, for removing defects. It has been proven

that 30% of defects can be discovered here.• Software will be executed to validate the requirements and design features.• Using functional or structural testing techniques to catch defects.



• Unit testing, coverage analysis, black-box techniques fall under this category.• It can also be used throughout the life cycle.

Essay 3.

Inadequate test requirements: test requirements are created from functional requirements and the functional requirements are not good and complex enough.Testers are in lose-lose situation: if we find more defects we get blamed for slower project implementation, and if we find less defects, the quality of our team is in jeopardy.Dependence on independent testers: unit testing is not good enough and this causes testers to test and find all defects, which puts more pressure and workload on them.Inadequate test coverage: because of frequent software developments, testers did not get a chance to test all test cases at once; this causes too much duplication of work.Test plan updates not enough: test cases are not added because of time crunch and this caused some test cases to ship during regression.Changes not tested: changes made in the code were not tested, resulting in shipping defects to operations.



Essay 4.

Life cycle testing involves continuous testing of the system during the developmental process. At predetermined points, the results of the development process are inspected to determine the correctness of the implementation. These inspections identify defects at the earliest possible point.Life cycle testing cannot occur until a formalized SDLC has been incorporated. Life cycle testing is dependent upon the completion of predetermined deliverables at specified points in the developmental life cycle. If information services personnel have the discretion to determine the order in which deliverables are developed, the life cycle test process becomes ineffective. This is due to variability in the process, which normally increases cost.The life cycle testing concept can best be accomplished by the formation of a test team. The team is comprised of members of the project who may be both implementing and testing the system. When members of the team are testing the system, they must use a formal testing methodology to clearly distinguish the implementation mode from the test mode. They also must follow a structured methodology when approaching testing the same as when approaching system development. Without a specific structured test methodology, the test team concept is ineffective because team members would follow the same methodology for testing as they used for developing the system. Experience shows people are blind to their own mistakes, so the effectiveness of the test team is dependent upon developing the system under one methodology and testing it under another.The life cycle testing concept is illustrated below This illustration shows that when the project starts both the system development process and system test process begins. The team that is developing the system begins the systems development process and the team that is conducting the system test begins planning the system test process. Both teams start at the same point using the same information. The systems development team has the responsibility to define and document the requirements for developmental purposes. The test team will likewise use those same requirements, but for the purpose of testing the system. At appropriate points during the developmental process, the test team will test the developmental process in an attempt to uncover defects. The test team should use the structured testing techniques outlined in this book as a basis of evaluating the system development process deliverables.



Essay 5.

The three defect attributes I would propose are:

- Defect NamingName defects according to the phase in which the defect most likely occurred, such as arequirements defect, design defect, documentation defect, and so forth.

- Defect SeverityUse three categories of severity as follows:

• Critical – Would stop the software system from operating.• Major – Would cause incorrect output to be produced.• Minor – Would be a problem, but would not cause improper output to be produced,

such as a system documentation error.- Defect Type

Use the following three categories:

• Missing – A specification not included in the software.• Wrong – A specification improperly implemented in the software.• Extra – Element in the software not requested by a specification.

software tester’s certification exam prep€¦ · software tester’s certification exam prep...

Documents