continuous testing at scale - tapost...2016/10/02 · “continuous testing is the process of...
TRANSCRIPT
![Page 1: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/1.jpg)
Continuous Testing at Scale
October 12th 2016, RigaTAPOST Conference
![Page 3: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/3.jpg)
Introduction to Continuous Testing
![Page 4: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/4.jpg)
Continuous Delivery
• Get changes to production in fast and efficient way
• Deployment happens often and can be performed on demand
• Code is always in a deployable state
https://continuousdelivery.com
![Page 5: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/5.jpg)
Why CD?
• Frequent release cycle
• Decreased time to market
• Decreased delivery risks
• Commercial product development
![Page 6: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/6.jpg)
Cost of Defect Fixing
This chart is slightly different in CDTesting is performed as frequently as possible
![Page 7: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/7.jpg)
Continuous Delivery Pipeline
• Every code change needs to be verified
• Pipeline breaks the delivery process into stages
• Pipeline stages are not standardised
![Page 8: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/8.jpg)
Pipeline Example
Code Review
Build & Unit Test
Deploy to Test
Run Tests
Deploy to Stage
Deploy to Prod
Commit
Change
Confidence
![Page 9: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/9.jpg)
What is in “Test” Phase?
• Simplest scenario - Write tests in your favoured test framework and execute!
![Page 10: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/10.jpg)
Agile Testing Strategy
“Developers write some integration tests and execute them in the pipeline”
![Page 11: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/11.jpg)
Have Devs Considered?…environment live checkssmoke testingfunctional testingsecurity testingperformance testingfault-tolerance testingbrowser compatibility testingmobile testingtest environment preparationtest planningtest schedulingtest reportingtesting process traceability
![Page 12: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/12.jpg)
Tests Pyramid
http://martinfowler.com/bliki/TestPyramid.html
![Page 13: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/13.jpg)
Another Pyramid
https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html
Is it build right?
Is the right thing built?
![Page 14: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/14.jpg)
What tend to happen is
“Development focused CD”
![Page 15: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/15.jpg)
Development focused CDsprint 1 sprint 2 sprint 3
Version 1 Version 2
testing 1
testing 2Manual Testing
![Page 16: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/16.jpg)
Testers are lagging behind a
continuous deliverydevelopment train
![Page 17: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/17.jpg)
“Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.”
Shippable featureDEV
OPS
TEST
https://en.wikipedia.org/wiki/Continuous_testing
![Page 18: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/18.jpg)
Testers
Test management Test plans
Manual testing Non-functional testing Structured approach
Standards compliancy
Developers
Apply TDD No bureaucracy
All automated tests Ad-hoc decisions
Feature-level testing
How do you align?
![Page 19: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/19.jpg)
Is Continuous Testing easy?Not at all!
Some advices to follow…
![Page 20: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/20.jpg)
Story Begins
![Page 21: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/21.jpg)
Setting the Context
• Telecom company had a network management system developed for the last 20 years
• Time to do a complete rewrite for 4G/5G
• Continuous Delivery model selected
![Page 22: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/22.jpg)
100+Scrum teams
1000+Components
2000+Git Repositories
10000+Functional Tests
EverythingAutomated
200+Test Environments
![Page 23: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/23.jpg)
Expectations• Fast feedback cycle
• System quality statement few times a day
• Testing compliant to quality standards
• Traceability throughout the process
• Agile development model
![Page 24: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/24.jpg)
Traceability is the ability to verify the history, location, or application of an item by means of documented recorded
identification.
Boring bureaucratic stuff for non-agile
environments only, right?
![Page 25: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/25.jpg)
Artefact
Test Code
Requirement
Change
Relates to Covers
Build
Executes
ProducesJIRA ID
BUILD ID
ARTIFACT ID
SHA ID ???
TestDescribes
TEST ID
![Page 26: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/26.jpg)
Test ID in CodeEvery test should have a
unique test id
Test case relatedmeta-data
![Page 27: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/27.jpg)
Test Case ManagementRelates to a test
description in TMS
Test ID: TMS-2Title: Test loginRequirements: JIRA-678Priority: CRITICALComponents: AuthorisationTags: Regression, UIDescription: ...
Test ID: TMS-1Title: Test database connectionRequirements: JIRA-124, JIRA-234Priority: CRITICALComponents: DBTags: Regression, DBDescription: ...
![Page 28: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/28.jpg)
Allure Framework
https://github.com/allure-framework/
Test Cases
Test Results(XML Documents)
Aggregation
InteractiveReport
All popular test frameworks supported
Run
![Page 29: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/29.jpg)
Feature coverage by tests
![Page 30: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/30.jpg)
Drill-down to test step level
https://wiki.jenkins-ci.org/display/JENKINS/Allure+Plugin
![Page 31: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/31.jpg)
Test Run Summary
![Page 32: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/32.jpg)
Achievements
• Achieved a fully automated traceability in a single test run
• Possibility to extract and store test statistics
• Traceability is simple given right tools
![Page 33: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/33.jpg)
CD Meets Microservices
Build Test Deploy
Build Test Deploy
Build Test Deploy
Service A
Service B
Service C
x 100 Teams
![Page 34: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/34.jpg)
Service A
Service B
Service C
1.0 1.1 1.2 1.3
2.1 2.2 2.3 2.4
1.12 1.13 1.14 1.15 1.16
System Version
![Page 35: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/35.jpg)
System Level Testing
Build Test Deploy
End 2 End Tests
New package releases trigger E2E tests
![Page 36: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/36.jpg)
TeamTestwarePackage
Produces Contains
TestSuites
TestCases
Teams are producing testware packages
![Page 37: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/37.jpg)
Testware 1
Testware 2
Testware 3
Test Schedule
Test Executor
Team A
Team B
Team C
Testware packages are combined with
help of test schedules
![Page 38: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/38.jpg)
Test ScheduleTeam A
Testware
Team B Testware
Team C Testware
Consistency Check
Smoke Testing
Environment Cleanup
Schedule items are coming from different teams
Environment Preparation
![Page 39: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/39.jpg)
https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin
parallel ({ guard { build(“run-tests”, “testware1”) build(“run-tests”, “testware2”) } rescue { build(“run-tests”, “cleanup”) }})
Tests schedule produces build flow definition, which is
given to Jenkins build farm
Visualisation
![Page 40: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/40.jpg)
Aggregation
Full ExecutionReportNetwork Drive
Test ware LevelReports
Tests
![Page 41: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/41.jpg)
Achievements
• Tests from multiple teams are combined to a single test run
• Aggregated reports for all test cases are available
![Page 42: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/42.jpg)
How to run tests as quickly as possible to minimise the
feedback loop?
![Page 43: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/43.jpg)
Tests Quality Police• Ensuring tests are
• efficient (API vs UI, no waits)
• can run in parallel
• data-driven
• reusing other teams test API
![Page 44: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/44.jpg)
Distributed Test Executor
• Cloud based
• Scale-up/scale-down
• Health checks
• Parallelisation
![Page 45: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/45.jpg)
MASTER
SLAVE
MASTER
WORKER
Test Lab
SLAVE SLAVE
WORKER
MASTER
WORKER WORKER
MASTER
WORKER WORKER
![Page 46: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/46.jpg)
Testing Flow
SmokeTests
Quick Regression
Long Regression
Full Regression
ComponentTests Isolated runs
Environment up and running
Core use-cases
Important use-cases
All use-cases
ComponentTests Component
Tests
Decision is taken if to continue pipeline after every step
![Page 47: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/47.jpg)
Achievements
• Tests quality is measured and improved
• Tests execution time is minimised
![Page 48: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/48.jpg)
Failed test !=
Sometimes
![Page 49: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/49.jpg)
Interpreting Results• 100% green tests pipeline all the time - not realistic
at this scale
• what is failed test priority?
• is it a new test?
• is it a flaky test?
• was it rerun several times?
![Page 50: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/50.jpg)
Test Result Analytics
Need something like this, but with more information
![Page 51: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/51.jpg)
Internal Test Reporting Solution
![Page 52: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/52.jpg)
Achievements
• Identifying good enough builds
• Ignoring random faults
• Historical test results for analysis
![Page 53: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/53.jpg)
The Outcome
• Full testing traceability
• E2E pipeline automation
• Continuous testing
![Page 54: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/54.jpg)
Lessons Learned
![Page 55: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/55.jpg)
There are not too many tools out there to support large scale continuous testing
![Page 56: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/56.jpg)
Instead of using monolithic tools - aim for extensible independent services and
interoperability standards
![Page 57: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/57.jpg)
CD Standards• Some well-known standards and platforms:
• Jenkins - Pipelenes
• Docker Containers
• JUnit 5 - Test Runner
• WebDriver - Selenium HTTP API
• Allure Reporting - Common for all test tools
![Page 58: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/58.jpg)
Next 5 years are going to be about CD and Testing
tools interoperability
![Page 59: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/59.jpg)
This slide was intentionally left blank
![Page 60: Continuous Testing at Scale - TAPOST...2016/10/02 · “Continuous Testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate](https://reader030.vdocuments.net/reader030/viewer/2022041019/5eced4cb194eb40ca364614f/html5/thumbnails/60.jpg)
Copyright: nisanga / 123RF Stock Photo
martinm303 / 123RF Stock Photodundanim / 123RF Stock Photo
enjoyphoto / 123RF Stock Photo
pedrosala / 123RF Stock Photo
saranya2908 / 123RF Stock Photo
kzwwsko / 123RF Stock Photo