software testing & qa conference - quality engineering in the devops … · 2020-02-04 ·...
TRANSCRIPT
![Page 1: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/1.jpg)
Geoffrey van der Tas
@Gavdtas
Quality Engineering in the DevOps time period.
The new way of Testing
![Page 2: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/2.jpg)
2
Definition of Testing
DevOps Time Period
Old
“Independent judgement about the quality
of the testobject”
New
“Means of getting fast feedback on your
product”
![Page 3: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/3.jpg)
3
Waterfall DevOps
DevOps Time Period
Specs
Design
Code
Test
Maintain
Test
Test
Test
Test
Test Test
Test
Feedback
Feedback Feedback
FeedbackFeedback
Feedback
Code
Design
Specs
Maintain
![Page 4: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/4.jpg)
4
DevOps Time Period
![Page 5: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/5.jpg)
5
From Assurance towards Engineering
DevOps Time Period
“Quality Engineering is the management,
development, operation and maintenance of IT systems
and enterprise architectures with a high quality
standard.” - New Perspectives on Software Quality by Ruth Breu; Annie
Kuntzmann-Combelles; Michael Felderer (January–February 2014).
![Page 6: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/6.jpg)
6
What needs to happen
QA/ Testing to Quality Engineering
• Testers becoming Quality Engineers
• Quality from the start
• Automation, Automation, Automation
• Explore to test
• The Grand Tour
![Page 7: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/7.jpg)
7
Testers becoming Quality Engineers
![Page 8: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/8.jpg)
8
What is changing
Testers becoming Quality Engineers
The Team
Quality
Dev
Dev
Ops
Ops
Test
Analist
Test Quality
Engineer
Test
Quality Assistant
Agile Test Coach
![Page 9: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/9.jpg)
9
At other companies
Testers becoming Quality Engineers
Quality Engineering at Spotify
“The Quality Engineer will help developers and teams cultivate a quality mindset,
advocating for quality while assisting engineers with software development and test
automation best practices.”
Quality Assistance at Atlassian
“They optimised the process by empowering and educating developers to test their own
features to production quality standards.”
![Page 10: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/10.jpg)
10
Quality Engineer
Testers becoming Quality Engineers
• Advocate for quality
• Actively coach developers/business
• Make testing part of Development Process
• Coordinate testing efforts
• Maintain test reporting and visualization
• Spot opportunities for (Test) Automation
![Page 11: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/11.jpg)
11
Quality from the start
![Page 12: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/12.jpg)
12
Backlog Items – Sprint Backlog Items
Quality from the start
Crap in, Crap out
![Page 13: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/13.jpg)
13
Tell the story
Quality from the start
![Page 14: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/14.jpg)
14
Quality from the start
![Page 15: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/15.jpg)
15
Techniques
Quality from the start
Impact Mapping User Story Mapping
![Page 16: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/16.jpg)
16
Quality Engineering in Scrum
Understanding the big picture
![Page 17: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/17.jpg)
17
Big Picture to User Stories
Quality from the start
Specifications:Spec oneSpec two
…Acceptance Criteria:
Criteria oneCriteria two
Criteria three…..
![Page 18: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/18.jpg)
18
Communication
Quality from the start
![Page 19: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/19.jpg)
19
Refinements by examples
Quality from the start
![Page 20: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/20.jpg)
20
Specification By Example
Quality from the start
User story: As an employee I want to find other employees so I can find more
information about employees.
Example ATDD:Search parameter Result
Peter Peter Janssen
Jan Annie JansenJan van BurenPeter Janssen
Jans Annie JansenPeter Janssen
Bakkr (geen)
![Page 21: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/21.jpg)
21
Techniques
Quality from the start
![Page 22: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/22.jpg)
22
Quality Engineering in Scrum
Understanding the big picture
Powerful Refinements
![Page 23: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/23.jpg)
23
ATDD TDD
Quality from the start
Failing Acceptance Test
Passing Acceptance Test
Passing test
Refactor
Failing test
![Page 24: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/24.jpg)
24
Quality Engineering in Scrum
Understanding the big picture
Powerful Refinements
Good Development Practices
![Page 25: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/25.jpg)
25
Automation, Automation, Automation
![Page 26: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/26.jpg)
26
Why Automate
Automation, Automation, Automation
Fast
Reliable
Repeatable
Reuseable
Control
Money
Fun
Automation vs Manual
Manual Automation
![Page 27: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/27.jpg)
27
How to approach it
Automation, Automation, Automation
User Interface
Integration
Unit/Component
10 %
20 %
70 %
Slow, Big, Many Dependencies & High Maintenance
Fast, Bit Bigger, SomeDependecies, Easy Maintenance
ATDD
Fast, Small, Easy Maintenance
TDD
![Page 28: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/28.jpg)
28
What to automate
Automation, Automation, Automation
Context is key
Self build vs bought software
Java, Javascript, C++, C#
Employee vs Client facing
User Interface
Integration
Unit/Component
![Page 29: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/29.jpg)
29
Quality goes further then Test Automation
Automation, Automation, Automation
Continuous Delivery Pipeline - Xebia
![Page 30: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/30.jpg)
30
Even further
Automation, Automation, Automation
![Page 31: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/31.jpg)
31
Quality Engineering in Scrum
Understanding the big picture
Powerful Refinements
Good Development Practices
Automation
![Page 32: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/32.jpg)
32
Explore to test
![Page 33: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/33.jpg)
33
Explore to test
![Page 34: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/34.jpg)
34
Testing helps Checks
Explore to test
https://findingdeefex.com/2016/05/20/the-testing-checkingsynergy/
![Page 35: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/35.jpg)
35
Exploratory Testing
Explore to test
“Exploratory Testing is a style of testing in which you explore the software while
simultaneously designing and executing tests, using feedback from the last test
to inform the next.” – Elisabeth Hendrickson
• Timeboxed
• Charters
• Debrief
• In Pairs
![Page 36: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/36.jpg)
36
Exploratory Testing
Explore to test
Design
Execute
Learn
SteerCharters Debrief
![Page 37: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/37.jpg)
37
Release Strategy
Explore to test
Customers
Product
Canaries
Customer Selection
#Feature Toggles
Pilot
![Page 38: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/38.jpg)
38
Examples
Explore to test
![Page 39: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/39.jpg)
39
Examples
Explore to test
![Page 40: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/40.jpg)
40
Quality Engineering in Scrum
Understanding the big picture
Powerful Refinements
Good Development Practices
Automation
Explore
Explore
![Page 41: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/41.jpg)
41
![Page 42: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/42.jpg)
42
About the experience
The Grand Tour
![Page 43: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/43.jpg)
43
Quality Engineering
The Grand Tour
Functionality
Performance
Resilience
Security
Usability
Accesability
Other software components
Infrastructure
Servers
Databases
Monitoring
Build pipeline
![Page 44: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/44.jpg)
44
Quality Engineering with the customer in mind
The Grand Tour
![Page 45: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/45.jpg)
45
Quality Engineering in Scrum
Understanding the big picture
Powerful Refinements
Good Development Practices
Automation
Explore
Explore
Quality Engineers Grand Tour
![Page 46: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/46.jpg)
46
Key Principles
Quality Engineering Values
Customer first
Whole team is responsible for quality
Tested = Checked + Explored
Bugs are fixed when they are found
‘Done’ means fully tested
Quality before new functionality
Collaboration is key for quality
![Page 47: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/47.jpg)
47
How do I get from here to there?
Study and practice, years of it.
![Page 48: Software Testing & QA Conference - Quality Engineering in the DevOps … · 2020-02-04 · Quality Engineering in the DevOps time period. The new way of Testing. 2 ... QA/ Testing](https://reader034.vdocuments.net/reader034/viewer/2022050418/5f8dc96fa196f40f0569e18b/html5/thumbnails/48.jpg)
Geoffrey van der Tas
Explorer and Advocate for Quality!
E-mail: [email protected]
Twitter: @Gavdtas
LinkedIn: https://nl.linkedin.com/in/geoffreyvdtas