7 steps for digital app test automation success steps for...7 steps for digital app test automation...
TRANSCRIPT
![Page 1: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/1.jpg)
7 steps for digital app test automation success
▪October 2018
![Page 2: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/2.jpg)
Speakers
2
Ruth Zamir
Director of Marketing
Guy Arieli
CTO
![Page 3: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/3.jpg)
Agenda
3
About Experitest + Intro01 5 min
7 steps for digital app test automation success02 35 min
Summary & Q&A03 10 min
![Page 4: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/4.jpg)
SeeTest Continuous Testing Platform for web and mobile applications
4
DigitalLab
Web & Mobile
Codeless automation (e.g. Worksoft, Tricentis etc)
![Page 5: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/5.jpg)
SeeTest Continuous Testing Platform for web and mobile applications
5
DigitalLab
Web & Mobile
Codeless automation (e.g. Worksoft, Tricentis etc)
![Page 6: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/6.jpg)
SaaS or on-premise deployment available
Fully secure flexible deployment options to suite your needs
SaaS
Private / shared lab at Experitest datacenters
On-premise
Private lab at Customer datacenter
6
Safari test #1
![Page 7: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/7.jpg)
SaaS datacenters worldwide with real devices and browsers
All datacenters ISO 27001 and SOC 2 certified
7 7
Fremont, CA
Bracknell, UK
Tel Aviv, Israel
Frankfurt, Germany
New Delhi, India
NYC, NY
Toronto, Canada
Sydney, Australia
Singapore
![Page 8: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/8.jpg)
Analyst RecognitionForrester WaveTM: Omnichannel Functional Test Automation Tools, Q3 2018
8
• Top Ranked for market presence
• Best current offering for digital quality
• “a complete testing environment for both web and mobile, a robust mobile device cloud, and extensive cross-browser testing”
• One of the few players to offer a unified platform for both mobile and web
• Visual testing
![Page 9: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/9.jpg)
9
![Page 10: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/10.jpg)
10
~75% of global services decision makers are
investigating or implementing digital
transformation
The Forrester Wave™: Omnichannel Functional Test Automation Tools, Q3 2018
![Page 11: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/11.jpg)
Shorter cycles = More cycles
*Same or more features in app = More regression testsMore cycles
11
Achieving Speed
Continuous testing
3%
20%
40%
22%
8% 7%
What Precentage of your tests are automated?
Over 90%
50%-90%
10%-50%
less than 10%
Don't know
Don't measure
![Page 12: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/12.jpg)
12
52% biggest challenge in web and
mobile app testing is not enough time to test
61% find it difficult to automate because our applications change too much with every release
* World quality report 2018-2019, Capgemini
![Page 13: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/13.jpg)
13
It’s a scale issue
![Page 14: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/14.jpg)
Agenda
14
Intro01 5 min
7 steps for digital app test automation success02 35 min
Summary & Q&A03 10 min
![Page 15: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/15.jpg)
What is causing low automation rates?
➔ Slow test creation
➔ Test flakiness
➔High maintenance efforts
➔ Time consuming analysis
15
Going back to automation basics
![Page 16: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/16.jpg)
16
1 Object Identification
![Page 17: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/17.jpg)
Why object identification?
17
![Page 18: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/18.jpg)
18
XPath=//*[@class='android.widget.Switch' and ./preceding-sibling::*[./*[@text='Wi-Fi']]]
Workaround example: Complex object property combinations
Challenge:
Properties change across versions ➔
Broken identifiers ➔
Failed tests
![Page 19: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/19.jpg)
Best practice for object identification:
19
✓ Make creation of unique identifiers part of the design process– Create a well defined convention: what needs to be identified,
how to identify it
– Make identifier requests from automation engineers a high priority task
✓ Teach automation engineers to add identifiers to the application code
– (and grant them authority)
DevOps team Collaboration
![Page 20: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/20.jpg)
20
2 Break Test Chains
![Page 21: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/21.jpg)
21
What are test chains?
Tests that are dependent on the success of previous tests.
Example:• Register user• Log in• Perform action
![Page 22: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/22.jpg)
22
Why are test chains created?
Test “overhead” may be much longer than test itself
• Avoiding repetition:
– one test may be another test’s configuration
– Common steps at the beginning
• Lack of test data management
• No attention to test suite design
About test chains
So What’s the problem?
• A chain is only as strong as it weakest link
• Hard to debug tests at the end of the chain (execution time)
• “Hidden configurations” create lack of test visibility
![Page 23: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/23.jpg)
Breaking Test Chains
23
Use test data management to create the state
Add test parallelization
Use an API call to build an environment from scratch
– Will work only if the environment setup time is very fast
Use adaptive tests
Any test should be executable with no dependence on other tests
![Page 24: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/24.jpg)
24
3 Create Small Tests
![Page 25: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/25.jpg)
25
Make it easy to pinpoint an issue
Why do we want small tests?
Makes test simpler to create, simpler to read
Easier to identify what failed
*and you are really productive, creating a lot of tests…
Easier to debug
![Page 26: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/26.jpg)
26
✓ Every test should test the smallest functionality possible
✓ Use an API to do anything that is not the core test functionality
✓ Share setup between tests
– The test will identify the state and if needed will perform setup
Enabling small tests
Extend the number of tests as functionality grows
![Page 27: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/27.jpg)
27
4 Business driven tests
![Page 28: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/28.jpg)
28
Test like users, not like testers
Step 1: LoginStep 2: Check balanceStep 3: Make paymentStep 4: Check new balance
Step 1: Find the username field Step 2: Send the username, Step 3: Find the password field Step 4: Send the password, Step 5: Click on the login…
Business driven test script: UI driven test script:
![Page 29: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/29.jpg)
29
✓ Easily readable by everyone (coders and non-coders)
✓ Entire DevOps team can write tests – including business analysts
✓Abstraction enables cross-platform reuse for consistency
– Android/iOS/Web
✓Minimal code repetition
Benefits of business driven tests
![Page 30: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/30.jpg)
30
5 Dealing with environment
availability
![Page 31: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/31.jpg)
31
• Staging / QA environment may be available less than 20% of time
• Often hard to identify in test results
![Page 32: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/32.jpg)
Environment availability
32
✓ Execute your UI tests only after all of the API tests passed successfully
✓ Run your tests against a mocked environment (my favorite)
![Page 33: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/33.jpg)
33
6 Continuous execution
![Page 34: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/34.jpg)
34
• Once a test is create, start running it on the existing version to test it. Continue to do so to make sure your suite isn’t broken
Testing the testing
Your automation program is a very complex application
It takes around 1,000 executions to stabilize a test.
After that, if a test failed, I know the issue is with the application
![Page 35: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/35.jpg)
Development Engineer
The Continuous Testing process
Commit tests
Trigger test execution
Load app to cloud
Test Engineer
Commit app code
Consolidate test results
Continuous Integration
reporttest
build Version approved?
no yes
Debug / update test suite
1Pull app source2
Build app3
Pull test source4
Build test project5
6 7 8
9
10
Trigger build
![Page 36: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/36.jpg)
The Continuous Execution Process
Commit tests
Trigger test execution
Load app to cloud
Test Engineer
Consolidate test results
Continuous Integration
reporttest
build Test suite ok?
no yes
Debug / update test suite
Pull app source1
Build app2
Pull test source3
Build test project4
5 6 7
8
9
Build
![Page 37: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/37.jpg)
37
✓ Once a test is create, start running it on the existing version to test it.
✓ Run your entire test project periodically even when there is no new code, to make sure your suite isn’t broken
✓ Fix your test suite quickly, to be
Continuously execute
![Page 38: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/38.jpg)
38
7 Analyze quickly:
be ready to scale
![Page 39: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/39.jpg)
39
The test automation target is moving
How do I automate this test?How do I find the object and click it?
How do I run all my tests quickly?
* literally. This is an example from one of our customers.
I have results from a million* of tests.Now what? Should I release the code?
![Page 40: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/40.jpg)
40
Continuous testing of thousands of tests creates mountains of data
Analysis time grows exponentially with amount of test
Analyze quickly - be ready to scale
![Page 41: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/41.jpg)
Actionable Analytics to drive decisions
41
✓ Clear sharable reports
✓ Consolidated test results for analysis
✓ Dashboards for management decisions
![Page 42: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/42.jpg)
42
Demo
![Page 43: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/43.jpg)
Agenda
43
Intro01 5 min
7 steps for digital app test automation success02 35 min
Summary & Q&A03 10 min
![Page 44: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/44.jpg)
Collaborate on object identification
Break test chains
Create small tests
Business driven tests
Deal with environment availability
Execute continuously
Analyze quickly
7 Steps For successful test automation
AT SCALE
44
![Page 45: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/45.jpg)
![Page 46: 7 steps for digital app test automation success steps for...7 steps for digital app test automation 02 success 35 min 03 Summary & Q&A 10 min SeeTest Continuous Testing Platform for](https://reader034.vdocuments.net/reader034/viewer/2022042307/5ed3fa2e8d46b66d22633419/html5/thumbnails/46.jpg)
Thank You!
Thank You!Questions?
46