hawkeye-lean process for product testing

15
Quality Checked Process Strategy and Guidelines for a better product

Upload: pramatiworld

Post on 10-Dec-2014

2.878 views

Category:

Business


1 download

DESCRIPTION

Hawk-Eye Methodology of Qlabs helps put a small but efficient process for Succesful testing of Software Products.

TRANSCRIPT

Page 1: HawkEye-Lean Process For Product Testing

Quality Checked

Process Strategy and Guidelines for a better productProcess Strategy and Guidelines for a better product

Page 2: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Building Blocks

Mission

“Work towards ensuring our customer’s products are used, usable easily and effectively by their end customers”

Vision

“To be the prefered testing partner of excellence for product ISV’s and enterprises in the chosen markets and technologies. Provide best in class, innovative testing services delivered on time, every time and any time”

Values

“Customer’s Success is Qlab’s Success and so is Pramati’s”

Mission

“Work towards ensuring our customer’s products are used, usable easily and effectively by their end customers”

Vision

“To be the prefered testing partner of excellence for product ISV’s and enterprises in the chosen markets and technologies. Provide best in class, innovative testing services delivered on time, every time and any time”

Values

“Customer’s Success is Qlab’s Success and so is Pramati’s”

Page 3: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Key Assumptions

• Intended client is a product firm/ISV who has outsourced/co-sourced product QA to pramati’s Qlabs

• Qlabs is in a position to suggest and improve an existing QA process/ propose and set up the QA process for an effective engagement

• Qlabs will always working towards being an extended arm of the product and its success rather than just offering a testing service

• To achieve over all product quality Qlabs can proactively propose for few mandatory things from the customer

• Believe in process as a tool for improvement in every aspect of our work rather than a step by step tutorial

• Process is not a hinderance to innovation, but a handle to increased customer satisfaction

• Intended client is a product firm/ISV who has outsourced/co-sourced product QA to pramati’s Qlabs

• Qlabs is in a position to suggest and improve an existing QA process/ propose and set up the QA process for an effective engagement

• Qlabs will always working towards being an extended arm of the product and its success rather than just offering a testing service

• To achieve over all product quality Qlabs can proactively propose for few mandatory things from the customer

• Believe in process as a tool for improvement in every aspect of our work rather than a step by step tutorial

• Process is not a hinderance to innovation, but a handle to increased customer satisfaction

Page 4: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Hawk-Eye Methodology-Precision Delivered

5 4

3

21

6

Planning Design

Coding

TestingRelease

Post-Release

Process Definition

Process Improvement

QA Planning

Risk Assessment

QA Certification

Release Inspection

Test Development

Test Suite Design

Product DefinitionInputs

Early ReleaseProgram

Test Execution

Code Review

Design Review

Page 5: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Process Strategy and guidelines- Focal Points

• Teams’n’Customers – Proactive Engagement (Together we win) Defined communication Assigned ownerships Refined delivery

• Cross Functional Responsibilities

Product is not just about writing code, its more than that Respect each other for what they do as part of the PDLC

• Prevention is better than cure QA has to be the eyes and ears of the product QA is the first customer (hear and understand what they say) Bugs and reports are not enemies/pointers towards development

• Excellence in Delivery Own what you released Metrics for Improvement Report what you did, Analyze what went wrong and Improve

• Key Value Adds Extended arm in product testing

• Teams’n’Customers – Proactive Engagement (Together we win) Defined communication Assigned ownerships Refined delivery

• Cross Functional Responsibilities

Product is not just about writing code, its more than that Respect each other for what they do as part of the PDLC

• Prevention is better than cure QA has to be the eyes and ears of the product QA is the first customer (hear and understand what they say) Bugs and reports are not enemies/pointers towards development

• Excellence in Delivery Own what you released Metrics for Improvement Report what you did, Analyze what went wrong and Improve

• Key Value Adds Extended arm in product testing

Page 6: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Teams’n’Customers – Proactive Engagement (Together we win)

Apart from the Project Schedules and Plans

• Plan and Define communication methodology

• Plan and Define communication frequency

• Plan and Define escalation methodology

• Plan and Identify Key stake holders

• Plan and Identify functional and technical leaders

• Plan and Identify Knowledge sharers

Apart from the Project Schedules and Plans

• Plan and Define communication methodology

• Plan and Define communication frequency

• Plan and Define escalation methodology

• Plan and Identify Key stake holders

• Plan and Identify functional and technical leaders

• Plan and Identify Knowledge sharers

Page 7: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Teams’n’Customers – Proactive Engagement (Together we win)

• Increased Peer to Peer Engagement through FeatureSpot meetings once in 2 weeks atleast/ whenever a new feature is being developed/planned

• Developers and QA to interact more on feature discussions through a common interface.

• Develops bonding, respect and sense of product ownership

• Once/Twice in a month cross functional call to get the sense of market and customer requirements (if possible)

• Bug severity levels, descriptions and escalation procedures to be defined before testing takes off (protocol/modus operandi)

• Bug owners from QA and Dev side module/feature wise to be identified before testing takes off (Ownership driven)

• Bug assigners from conflict resolution owners to be identified before testing takes off (ownership driven and effective communication)

-- Technical aspects of the bug (Engineering responsibility both Dev and QA)

-- Functional aspects of the bug (Product Management responsibility with Dev and QA)

• Increased Peer to Peer Engagement through FeatureSpot meetings once in 2 weeks atleast/ whenever a new feature is being developed/planned

• Developers and QA to interact more on feature discussions through a common interface.

• Develops bonding, respect and sense of product ownership

• Once/Twice in a month cross functional call to get the sense of market and customer requirements (if possible)

• Bug severity levels, descriptions and escalation procedures to be defined before testing takes off (protocol/modus operandi)

• Bug owners from QA and Dev side module/feature wise to be identified before testing takes off (Ownership driven)

• Bug assigners from conflict resolution owners to be identified before testing takes off (ownership driven and effective communication)

-- Technical aspects of the bug (Engineering responsibility both Dev and QA)

-- Functional aspects of the bug (Product Management responsibility with Dev and QA)

Page 8: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Key Cross Functional responsibilities and take aways for end to end ownership

Drive down the Bare need of the product and its positioning

Product Management/product marketing to provide business value and use case that drives the product into the market

Offshore QA to understand who, where and why the product is being used

Drive in the performance expectations of the product to ensure it works the way it should rather than the way it can

Product Management/product marketing to provide desired performance benchmarks against which (at the beginning of the product dev cycle)

Offshore QA would ascertain and provide results to ensure reliability

Dev/Architecture/Engineering to signoff the benchmarks provided by the product management and share the unit testing results to offshore QA towards measuring the performance (after code freeze)

Offshore QA would share the performance benchmark testing plan and strategy and share the key test results and trends (before GoToMarket testing)

It’s a Product and not a Project

Drive down the Bare need of the product and its positioning

Product Management/product marketing to provide business value and use case that drives the product into the market

Offshore QA to understand who, where and why the product is being used

Drive in the performance expectations of the product to ensure it works the way it should rather than the way it can

Product Management/product marketing to provide desired performance benchmarks against which (at the beginning of the product dev cycle)

Offshore QA would ascertain and provide results to ensure reliability

Dev/Architecture/Engineering to signoff the benchmarks provided by the product management and share the unit testing results to offshore QA towards measuring the performance (after code freeze)

Offshore QA would share the performance benchmark testing plan and strategy and share the key test results and trends (before GoToMarket testing)

It’s a Product and not a Project

Page 9: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Prevention is better than Cure

• Bugs are not our enemies neither they are a direct measure of the work we do.

• Dev not to take bugs as personal pointers

• QA not to make bugs are the only measure of their credibility

• Dev to see they find as many bugs as QA would have [Increases QA scope]

• QA to see they find as many bugs as Customer would have [Increases product quality scope]

• An effective Dev-QA is more important than individual Dev and QA

• Bugs are not our enemies neither they are a direct measure of the work we do.

• Dev not to take bugs as personal pointers

• QA not to make bugs are the only measure of their credibility

• Dev to see they find as many bugs as QA would have [Increases QA scope]

• QA to see they find as many bugs as Customer would have [Increases product quality scope]

• An effective Dev-QA is more important than individual Dev and QA

Page 10: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Excellence in Delivery

Own what you released

• Do not be in a hurry to release just because the date is there/it has reached

• Is it really ready (Introspect and think like the end user) ??

• Account for this testing in your Project Plan rather than Panic

• Account for User Driven GoToMarket testing for a specific period of time post functional sign

Separate use cases will be written/can be provided

Identified use cases/work flows/scenarios will be demoed one last time by QA before letting go to market

Product Management/Marketing to sign off both the scenarios and demo

Will be tested on a purely fresh environment rather than the same one that was used for individual feature testing

Its all about meeting the functional needs in its simplest form than being glossy and complex

Own what you released

• Do not be in a hurry to release just because the date is there/it has reached

• Is it really ready (Introspect and think like the end user) ??

• Account for this testing in your Project Plan rather than Panic

• Account for User Driven GoToMarket testing for a specific period of time post functional sign

Separate use cases will be written/can be provided

Identified use cases/work flows/scenarios will be demoed one last time by QA before letting go to market

Product Management/Marketing to sign off both the scenarios and demo

Will be tested on a purely fresh environment rather than the same one that was used for individual feature testing

Its all about meeting the functional needs in its simplest form than being glossy and complex

Page 11: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Excellence in Delivery (contd..)

Some Metrics and related process guidelines

• Define bug severity levels and descriptions at the begining

• Number of Sev-1 and Sev-2 bugs that can be allowed to be open for Ready to Go to be decided in advance

• Number of Sev-1 and Sev-2 bugs to be allowed to be open for beta release to decided in advance

• Number of Sev-1 and Sev-2 bugs that can be converted/down graded to Sev-3’s and Enhancements to be decided in advance

• Effective Bug Filing rate –> Direct measure of offshore QA’s credibility (Number of bugs posted – No.of invalid bugs)

• The lesser the Junk bugs the more its effective [Penny saved is worth more than a penny earned]

Some Metrics and related process guidelines

• Define bug severity levels and descriptions at the begining

• Number of Sev-1 and Sev-2 bugs that can be allowed to be open for Ready to Go to be decided in advance

• Number of Sev-1 and Sev-2 bugs to be allowed to be open for beta release to decided in advance

• Number of Sev-1 and Sev-2 bugs that can be converted/down graded to Sev-3’s and Enhancements to be decided in advance

• Effective Bug Filing rate –> Direct measure of offshore QA’s credibility (Number of bugs posted – No.of invalid bugs)

• The lesser the Junk bugs the more its effective [Penny saved is worth more than a penny earned]

Page 12: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Excellence in Delivery (contd..)

• Enhancement Quotient More number of valid enhancements that QA provides

and agreed upon proves the way QA thinks from a market and customer stand point- Key Differentiator towards product ownership rather than project ownership

Enhancements filed by QA which go in as features in the next releases/ which result in product enrichment must be recognized and rewarded too [Motivation and Recognition]

• Quality is more important than Quantity 1 Quality bug is better than 10 which convey the same

.. And Many More Metrics can be worked out both on Dev and QA front [Numbers should reflect the stage and status, not the

damage and demotivation]

• Enhancement Quotient More number of valid enhancements that QA provides

and agreed upon proves the way QA thinks from a market and customer stand point- Key Differentiator towards product ownership rather than project ownership

Enhancements filed by QA which go in as features in the next releases/ which result in product enrichment must be recognized and rewarded too [Motivation and Recognition]

• Quality is more important than Quantity 1 Quality bug is better than 10 which convey the same

.. And Many More Metrics can be worked out both on Dev and QA front [Numbers should reflect the stage and status, not the

damage and demotivation]

Page 13: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Excellence in Delivery (contd..)

• Automated, Standardized and Centralized reporting Reports at every mile stone [Agreed upon] Reports at regular intervals [Agreed upon] Publish to all the stake holders [Agreed upon] Bug triage meetings and reports [Archive and publish] Report and analyze Bug trends severity wise with a

correlation to modules and over all functionality rather than bugs in its individuality [Archive and publish]

Performance and Benchmark reports [Archive and publish]

• Automated, Standardized and Centralized reporting Reports at every mile stone [Agreed upon] Reports at regular intervals [Agreed upon] Publish to all the stake holders [Agreed upon] Bug triage meetings and reports [Archive and publish] Report and analyze Bug trends severity wise with a

correlation to modules and over all functionality rather than bugs in its individuality [Archive and publish]

Performance and Benchmark reports [Archive and publish]

Page 14: HawkEye-Lean Process For Product Testing

© 2007 Pramati Technologies. All Rights Reserved.

Key Value Adds- Extended arm in product testing

• Trends and Improvements [Post Mortem]

• List of enhancements and Open bugs [Release notes and next release]

• Contribute to beta programs and product launch demos

• Help Professional services, support and sales teams with product training if required

• Trends and Improvements [Post Mortem]

• List of enhancements and Open bugs [Release notes and next release]

• Contribute to beta programs and product launch demos

• Help Professional services, support and sales teams with product training if required

Page 15: HawkEye-Lean Process For Product Testing

Quality Checked

Thank you.Thank you.