![Page 1: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/1.jpg)
Geoffrey van der Tas
@Gavdtas
Quality Engineering in DevOps
The new way of Testing
![Page 2: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/2.jpg)
![Page 3: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/3.jpg)
3
A changing world
DevOps Time Period
![Page 4: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/4.jpg)
4
What is DevOps
Quality Time Period
• Its a mindset
• Multidisciplinair teams
• Build it, Ship it, Test it;
![Page 5: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/5.jpg)
5
Waterfall DevOps
DevOps Time Period
Specs
Design
Code
Test
Maintain
Test
Test
Test
Test
Test Test
Test
Feedback
Feedback Feedback
FeedbackFeedback
Feedback
Code
Design
Specs
Maintain
![Page 6: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/6.jpg)
6
DevOps Time Period
![Page 7: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/7.jpg)
7
Definition of Testing
DevOps Time Period
Old
“Independent judgement about the quality
of the testobject”
New
“Means of getting fast feedback on your
product”
![Page 8: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/8.jpg)
8
From Assurance towards Engineering
DevOps Time Period
“Quality Engineering is the management,
development, operation and maintenance of IT systems
and enterprise architectures with a high quality
standard.” - New Perspectives on Software Quality by Ruth Breu; Annie
Kuntzmann-Combelles; Michael Felderer (January–February 2014).
![Page 9: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/9.jpg)
9
Quality Engineering
Quality Engineering
User story
(Automated) Testing
Development
Monitoring
InfrastructureDesign
Security
Performance
Customer Experience
![Page 10: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/10.jpg)
10
What needs to happen
QA/ Testing to Quality Engineering
• Testers becoming Quality Engineers
• Quality from the start
• Automation, Automation, Automation
• Explore to test
• The Grand Tour
![Page 11: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/11.jpg)
11
Testers becoming Quality Engineers
![Page 12: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/12.jpg)
12
What is changing
Testers becoming Quality Engineers
The Team
Quality
Dev
Dev
Ops
Ops
Test
Analist
Test Quality
Engineer
Test
Quality Assistant
Agile Test Coach
![Page 13: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/13.jpg)
13
At other companies
Testers becoming Quality Engineers
Quality Engineering at Spotify
“The Quality Engineer will help developers and teams cultivate a quality mindset,
advocating for quality while assisting engineers with software development and test
automation best practices.”
Quality Assistance at Atlassian
“They optimised the process by empowering and educating developers to test their own
features to production quality standards.”
![Page 14: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/14.jpg)
14
Quality Engineer
Testers becoming Quality Engineers
• Advocate for quality
• Actively coach developers/business
• Make testing part of Development Process
• Coordinate testing efforts
• Maintain test reporting and visualization
• Spot opportunities for (Test) Automation
![Page 15: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/15.jpg)
15
Quality from the start
![Page 16: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/16.jpg)
16
Backlog Items – Sprint Backlog Items
Quality from the start
Crap in, Crap out
![Page 17: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/17.jpg)
17
Tell the story
Quality from the start
![Page 18: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/18.jpg)
18
Quality from the start
![Page 19: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/19.jpg)
19
Techniques
Quality from the start
Impact Mapping User Story Mapping
![Page 20: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/20.jpg)
20
Quality Engineering in Scrum
Understanding the big picture
![Page 21: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/21.jpg)
21
Big Picture to User Stories
Quality from the start
Specifications:Spec oneSpec two
…Acceptance Criteria:
Criteria oneCriteria two
Criteria three…..
![Page 22: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/22.jpg)
22
Communication
Quality from the start
![Page 23: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/23.jpg)
23
Quality from the start
![Page 24: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/24.jpg)
24
Shared understanding
Quality from the start
• Shared vision, idea’s and alignment on what was discussed;
![Page 25: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/25.jpg)
25
Refinements by examples
Quality from the start
![Page 26: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/26.jpg)
26
Specification By Example
Quality from the start
User story: As an employee I want to find other employees so I can find more
information about employees.
Example ATDD:Search parameter Result
Peter Peter Janssen
Jan Annie JansenJan van BurenPeter Janssen
Jans Annie JansenPeter Janssen
Bakkr (geen)
![Page 27: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/27.jpg)
27
Techniques
Quality from the start
![Page 28: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/28.jpg)
28
Quality Engineering in Scrum
Understanding the big picture
Powerful Refinements
![Page 29: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/29.jpg)
29
ATDD TDD
Quality from the start
Failing Acceptance Test
Passing Acceptance Test
Passing test
Refactor
Failing test
![Page 30: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/30.jpg)
30
Quality First at ING
Quality from the start
Date/time Result
Current features
Advisor makes appointment 2017/10/05 12:13:54
- Appointment mailed to customer 2017/10/05 12:14:06
Summary of advice is mailed to customer 2017/10/05 12:14:46
This sprint
Make appointment with Specialist 2017/10/05 12:14:48
Customer details are shared with Specialist 2017/10/05 12:14:49
![Page 31: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/31.jpg)
31
Quality Engineering in Scrum
Understanding the big picture
Powerful Refinements
Good Development Practices
![Page 32: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/32.jpg)
32
Automation, Automation, Automation
![Page 33: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/33.jpg)
33
Why Automate
Automation, Automation, Automation
Fast
Reliable
Repeatable
Reuseable
Control
Money
Fun
Automation vs Manual
Manual Automation
![Page 34: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/34.jpg)
34
How to approach it
Automation, Automation, Automation
User Interface
Integration
Unit/Component
10 %
20 %
70 %
Slow, Big, Many Dependencies & High Maintenance
Fast, Bit Bigger, SomeDependecies, Easy Maintenance
ATDD
Fast, Small, Easy Maintenance
TDD
![Page 35: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/35.jpg)
35
What to automate
Automation, Automation, Automation
Context is key
Self build vs bought software
Java, Javascript, C++, C#
Employee vs Client facing
User Interface
Integration
Unit/Component
![Page 36: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/36.jpg)
36
Quality goes further then Test Automation
Automation, Automation, Automation
Continuous Delivery Pipeline - Xebia
![Page 37: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/37.jpg)
37
Even further
Automation, Automation, Automation
![Page 38: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/38.jpg)
38
Quality Engineering in Scrum
Understanding the big picture
Powerful Refinements
Good Development Practices
Automation
![Page 39: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/39.jpg)
39
Explore to test
![Page 40: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/40.jpg)
40
Explore to test
![Page 41: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/41.jpg)
41
Testing helps Checks
Explore to test
https://findingdeefex.com/2016/05/20/the-testing-checkingsynergy/
![Page 42: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/42.jpg)
42
Exploratory Testing
Explore to test
“Exploratory Testing is a style of testing in which you explore the software while
simultaneously designing and executing tests, using feedback from the last test
to inform the next.” – Elisabeth Hendrickson
• Timeboxed
• Charters
• Debrief
• In Pairs
![Page 43: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/43.jpg)
43
Exploratory Testing
Explore to test
Design
Execute
Learn
SteerCharters Debrief
![Page 44: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/44.jpg)
44
Release Strategy
Explore to test
Customers
Product
Canaries
Customer Selection
#Feature Toggles
Pilot
![Page 45: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/45.jpg)
45
Examples
Explore to test
![Page 46: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/46.jpg)
46
Examples
Explore to test
![Page 47: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/47.jpg)
47
Quality Engineering in Scrum
Understanding the big picture
Powerful Refinements
Good Development Practices
Automation
Explore
Explore
![Page 48: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/48.jpg)
48
![Page 49: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/49.jpg)
49
About the experience
The Grand Tour
![Page 50: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/50.jpg)
50
Quality Engineering
The Grand Tour
Functionality
Performance
Resilience
Security
Usability
Accesability
Other software components
Infrastructure
Servers
Databases
Monitoring
Build pipeline
![Page 51: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/51.jpg)
51
Quality Engineering with the customer in mind
The Grand Tour
![Page 52: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/52.jpg)
52
Key Principles for your team
The Grand Tour
Customer first
Whole team is responsible for quality
Tested = Checked + Explored
Bugs are fixed when they are found
‘Done’ means fully tested
Quality before new functionality
Collaboration is key for quality
![Page 53: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/53.jpg)
53
Quality Engineering in Scrum
Understanding the big picture
Powerful Refinements
Good Development Practices
Automation
Explore
Explore
Quality Engineers Grand Tour
![Page 54: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/54.jpg)
54
Becoming a Quality Engineer? Sounds impossible?
Quality Engineering Values
![Page 55: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/55.jpg)
55
![Page 56: Quality Engineering in DevOps - TestCon...Specification By Example Quality from the start User story: As an employee I want to find other employees so I can find more information about](https://reader033.vdocuments.net/reader033/viewer/2022050421/5f909a0987af765f0f4316f5/html5/thumbnails/56.jpg)
E-mail: [email protected]
Twitter: @Gavdtas
LinkedIn: https://nl.linkedin.com/in/geoffreyvdtas
Geoffrey van der Tas
May the Quality Force be with you!