leading with quality
DESCRIPTION
In the software industry you'll often hear the words Testing, Quality Assurance [QA] and Quality Control [QC] used interchangeably and very frequently. While it is usually enough to get your point across with a developer, it is certainly helpful to know more about how these apply to the world of software testing. So, what exactly the difference between QC and QA and how can you get from one destination to another? This presentation attempts to shed some light on this and will be taking you through various QA best practices and how we can collectively build and grow the next breed of intelligent QA EngineersTRANSCRIPT
www.unicomlearning.com
Topic: LEADING WITH QUALITY
Speaker Name: PRASANNA VEESpeaker Designation: Founder
Company: CrawlPal
Agendawww.unicomlearning.com
• What is Quality? • Quality Control Vs. Quality Assurance • Defect Prevention Vs. Defect Detection • Next Generation Testing• The Agile Tester
www.unicomlearning.com
What is Quality?
Definition of Quality
• Quality of Customer Experience– How well the functionality and end-to-end experience meets or exceeds customer
expectations under actual usage
– Do customers feel the product or service is consistently Reliable, Responsive, Compatible, Secure, aesthetically pleasing, and easy to use?
• Quality of Engineering– How effectively the architecture, design, code, testing, and integration meet high
engineering standards and product requirements
Quality of Customer Experience
Reliability Performance Compatibility
Security Completeness Usability
Aesthetics Market Relevance Availability
Supportability Manageability …
Quality of Engineering
Architectural Quality
Code Quality Design Quality
Agility Cost Effectiveness Defect Rate
Integration Maintainability Operational Cost
Resiliency Testability …
November 10, 2014 4
Quality of Customer Experience
What is Quality?
“Delighting the customer by fully meeting their needs and expectations”
What does Quality Include?–Functionality, Performance, Reliability, Availability, Usability/User Experience, Maintainability
Why is Quality Critical?Ariane 5 example, Disappearing Warehouse, McAfee 5958 example
How to accomplish Total Quality?
“Quality is the result of a carefully constructed cultural environment. It has to
be the fabric of the organization, not part of the fabric.” -Philip Crosby
Be a Customer Champion – Customer Empathy competency
November 10, 2014 5
The bottom line: Make customer experience the fabric of your organization.
Goal: Making the Quality of the customer experience THE reason for purchasing your product/service over competition.
Quality of Engineering
• Product Engineering Quality– How effective is each filter at
finding the right defects at the right time?
– How early do we remove defects?
– How efficient are the Testing efforts?
– Is this release or milestone better than previous releases? By how much?
– What is the cost, benefit, and risk analysis for each fix?
6November 10, 2014
SupportProgram Management
Quality – A Hub & Spoke model
Customers & Partners
Sustained Engg
Product Mgmt
Quality
QADevelopment
7
Who owns Quality?
Everyone!
www.unicomlearning.com
Quality Control vs
Quality Assurance
November 10, 2014 9
QA Vs QC
November 10, 2014 10
Quality Control Quality AssuranceA failure detection system that uses a testing technique to identify errors or flaws in products and tests the end products at specified intervals.
A failure prevention system that continually checks the product quality and then takes steps to control and prevent flawed products or services from reaching the advanced stages of development cycle
A series of analytical measurements usedto assess the implementation of the requirements & non-existence of unwanted behavior
An overall management plan to guarantee the Quality & Integrity of the System
More of a Reactive ApproachTypical Software ‘Testing’ is a Quality Control mechanism
More of a Proactive ApproachBuilding an ecosystem where bug prevention and a quality culture are inherent in the development lifecycleConstant and Continuous focus on QualityToyota Example
Quality Control is more about the Product itself Quality Assurance is more about the Process
Top Quality Principles
• Detect more defects closer to the phase of the product cycle where the defect was introduced rather than later in the cycle.
• Products should be designed from the start with quality in mind.
• Build Quality into the Development Process
• It helps with detecting product bugs as early as possible, prior to implementation.
• System Testing alone is not sufficient to Ensure Quality• Test Team can never find all the defects
• Example: VE + IE memory leak issue
• Quality Should be controlled at every node of a Development process
D.R.P principle: Detect early, Reduce and Prevent Defects!
November 10, 2014 11
System Testing alone is insufficient to ensure Quality
November 10, 2014 12
Testing the “Periphery” – Heathrow Terminal 5 example!
www.unicomlearning.com
Defect Detectionvs
Defect Prevention
Why Prevent Defects? – We have a QA team!
“In the beginning of a malady it is easy to cure but difficult to detect, but in the course of time, not having been either detected or treated in the beginning, it
becomes easy to detect, but difficult to cure.”
-Machiavelli
November 10, 2014 14
Defect Lifecycle : Early, Late & Too late
Early Found in early development activities like: Static Code Analysis, Personal Reviews, Peer
Reviews, Unit testing
Tracked through tools like Code Collaborator , Early Defect Recorder or Coverity Static Analysis Tool etc.,
Late Found after the code reaches System Testing, Integration Testing, Beta Testing etc.,
Tracked through Bugzilla and other similar defect management tools
Too Late Found by the Customer – either external or internal partner product
November 10, 2014 15
Shifting to Early Defect Discovery
Hours
Early Late Too Late
Early and Efficient Defect Removal is Key to Quality and Productivity
November 10, 2014 16
How can Test team help in PREVENTING defects?
Participate in Code Reviews & Design Reviews
Review Requirements with Quality in mind
Validate Unit Testing Are the unit tests making optimal use of the common harness, tools and libraries?
Do the unit test-cases overlap with your test-cases that can be avoided?
Customer Focus Look into Customer issues and Support issues.
Test often has a better view of how the component actually works and is consumed by partners and customers.
Test has an opportunity to call out unwanted side effects of a change
Root Cause AnalysisLearn From Mistakes to Prevent in the future
The Five Why’s of Toyota
November 10, 2014 17
From TEST plan to QUALITY Plan..
Quality isn’t just the responsibility of a single person
Quality Plans don’t have a single author. Rather, multiple people can contribute to them at various points throughout the feature development cycle
Dev, Test and PM’s work together on a Unified Quality strategy
Development: will describe how they plan to validate the code before committing changes, which scenarios will be covered by pre-check-in tests, any testability considerations in the code, etc.
Test team: will describe the additional testing to be done post-check-in and any other processes, procedures, test tools, to be used to get the job done.
This ensures Quality from Day one!
November 10, 2014 18
Test Plan: a plan that details how the test team will measure the quality of the product is and report it back to the team
Quality Plan: a plan that explains how various people working on a feature will work together to ensure it’s delivered with high quality
Summary: The Flow of Quality
• Define the Team Goals
• Identify the KPI’s to be measured
• Baseline from previous or similar releases
• Commit to not only features but Quality
Quality PlansEarly
IdentificationTest
ExecutionQuality Reviews
High Quality Release
• Implement and utilize tools to do the work.
• Peer Reviews, Unit tests, Static Analysis –Early Defect Discovery
• Security reviews
• Progress to Quality Plan
• Quality Heat maps to identify hotspots and action items
• Data Driven Frequent Project Quality Reviews to assess Risk & Readiness
• Build off early identification quality
• Automatedinstead of manual
• ExploratoryTesting focused on system and integration
• Builds Customer Confidence
• Reduces expenses
• Drives Increased Revenue
• Allows for more Innovation
• Unsettles Competition
November 10, 201419
www.unicomlearning.com
Next Generation Testing
Critical Shifts towards Engineering Excellence
• Problems with Manual Testing• Not easily repeatable
• Very Effort Intensive
• How long will it take to ‘re-run’ all the tests?
• How often can you run it?
• “Effective” & “Efficient” Automation• Lower maintenance: Minimal cost from extending test cases
• Maximum ROI
• Finds issues that cannot be found through Manual testing
• Increased Coverage
• Good Code coverage through System Testing
• Lot of White Box testing – Understand code changes
– Smarter approach to what needs to be tested
November 10, 2014 21
Solution: Decrease amount of Manual
Testing
Critical Shifts towards Engineering Excellence [cont.]
• Employ Intelligent Testing techniques– Equivalence Class Partitioning
– Pairwise Testing
– Model Based Testing
• More focus on Non-functional Testing– Performance Testing
– Security Testing
• Move away from being Quality Police!– Quality is a team wide effort
• Test Early, Test Often
• Keep in touch with latest Tools & Technologies
November 10, 2014 22
www.unicomlearning.com
The Agile Tester
How does Agile change the tester’s role?
Traditional Testing– In traditional methods, Test’s role is to expose risks in the system and
communicate them to management, who can then make informed release decisions
– In a sense, Test actively Verifies that product does not have any defects and exposes the risks to the management & to be addressed by Developers to make the product/service production ready
Agile Testing– In Agile methods, Test team is still responsible for exposing and
communicating risks, but Test also gets involved in the communication of specifications to developers via automated “acceptance” tests
– In a sense, Test actively Ensures that the product meets the specification by putting the spec in terms that developers understand – failing tests
November 10, 201424
Moving from Verifying Quality to Ensuring Quality
How does Agile change the tester’s role?
Traditional Testing– Quality Police: Test’s responsibility is
to be an impartial observer who reports findings to management
– Not my job: Test is not responsible for making the code match the spec –bugs are a developer’s problem
– Inspection: By definition, occurs after the code is written
– Combative: Success is measured by number of bugs found, which creates a combative relationship with Dev
– “There is a bug in your component.”
Agile Testing– Equal Partner: Test’s responsibility is
to collaborate with rest of team to build a quality system
– All for one: Test becomes a full-fledged member of the team – a bug is a failure of a test to communicate
– Injection: By definition, occurs before the code is written
– Collaborative: Success is measured by consistent delivery of running software to business
– “Can you help me figure out why this test is failing?”
November 10, 201425
www.unicomlearning.com
Topic:
Organized byUNICOM Trainings & Seminars Pvt. Ltd.
Speaker name: Prasanna VeeEmail ID: [email protected]
Thank You
www.unicomlearning.com
Appendix