agile testing - joe caravella 1 testing.pdf · agile testing - joe caravella 8 • a mind map is a...
TRANSCRIPT
Agile Testing - Joe Caravella 1
• From Long Island NY• Spent 20 years in USAF
• Sys Admin• Software Development
• Working in Software Development & Testing since 1993• Hobbies include motorcycles, boating, and water sports
2
• Waterfall – lots of documents and diagrams that become obsolete when coding begins. Plus long development cycles and annual releases
• Scrum-but – follows the basic tenants of Scrum, BUT inconsistent iteration lengths, no retros or reviews, and still doing annual or bi-annual releases
• Scrum – solid understanding and execution of all Scrum principles being led by a certified ScumMaster
• Kanban – workflow queues with WIP limits that encourage “pulling” story cards through your development process and into production
3Agile Testing - Joe Caravella
• My learnings were incremental and continue even today• After many retrospectives and tweaks, I’ll describe some adjustments I’ve made
4Agile Testing - Joe Caravella
• Product, Dev, and QA should sit and work together every day• Rule of Three for all product discussions (QA, Dev & Product)
5Agile Testing - Joe Caravella
• Create a Quality Development Process instead of trying to to test quality into yourproduct
• Unit testing and continuous integration + governance can really help • Get Devs an integrated environment to test their changes before going to QA• Create an automation framework that is dev friendly – they will use it as well
6Agile Testing - Joe Caravella
• The entire development team should create and have buy in to the testing & certification procedures
• Consider the Agile Testing Pyramid
7Agile Testing - Joe Caravella
• The test pyramid is a concept developed by Mike Cohn, described in his book Succeeding with Agile
• Mike’s point was to demonstrate that you should have more lower level unit tests in your testing strategy
• I added percentage goals, exploratory testing, and the need to always try and push testing lower
8Agile Testing - Joe Caravella
• A Mind Map is a diagram used to visually organize information• Exhaustive testing is rarely possible so strategies like Pairwise Testing can help
identify the combination of test cases that have the highest probability to find defects
9Agile Testing - Joe Caravella
• Don’t let your regressions grow so large that it takes more than 1 business day to certify a release
• Testing managers and test leads should add “regression pruning” to their monthly housekeeping procedures
• Querying your previous test results could help identify tests that never fail and are at a lower risk of
10Agile Testing - Joe Caravella
• Testing & development should be parallel activities• Avoid running a testing sprint after a development sprint• Acceptance should occur only after QA is done
11Agile Testing - Joe Caravella
• Avoid making defect fixes directly in QA envs – those change sometimes do not get checked in and the issue will occur in production
• All changes to QA envs should be applied by a versioned increment of your product (this included DB changes!)
• QA should initiate builds & deploys only when Continuous Integration is Green (meaning all UTs have passed)
12Agile Testing - Joe Caravella
• To avoid surprises, use the same people/team that will deploy to Production for all QA deployments
• Many times they will identify an issue that won’t work in Production
13Agile Testing - Joe Caravella
• Create a repeatable release cadence• Don’t sit on code – get value in front of your customers as soon as possible• This will also require teams to figure out how to be more efficient with their
release certification procedures
14Agile Testing - Joe Caravella
• Try to keep production as close to the trunk version of your code as possible• Consider using Release Trains & Feature Toggles
• Release Trains – code needs to be checked in by certain day to make it on the train
• Feature Toggles – configuration that controls when features are available to users
15Agile Testing - Joe Caravella
16Agile Testing - Joe Caravella
• Create a monthly data capture process so you stay informed about changes in your regression
• Try to minimize manual regression as much as possible• If you can keep your automated regression over 85% you are doing very well
17Agile Testing - Joe Caravella
• Create a simple graph of regressions growth and share with teams monthly• This helps teams remain sensitive to what’s being put in regressions• It also helps teams visually see when manual testing is starting to grow too high
18Agile Testing - Joe Caravella
• Identify a way to measure the effectiveness of your quality initiatives• At a minimum, however, an analysis should be performed on issues that escape
the testing phase and becomes customer facing – learnings should be shared across all teams
• Consider creating a company or department KPI that gets updated frequently so teams can react and adjust procedures immediately
19Agile Testing - Joe Caravella
20Agile Testing - Joe Caravella
• It’s difficult to be agile and produce frequent releases using manual regression tests – they just don’t scale well
• When done right, automation helps to create repeatable test execution process with a known timeline
21Agile Testing - Joe Caravella