from 0 to 100 000 test cases per day...docker puppet kvm cloudformation @haapalajani server room...
TRANSCRIPT
![Page 1: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/1.jpg)
From 0 To 100 000 Test Cases Per
DayGuide to successful Quality marriage
@HaapalaJani
![Page 2: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/2.jpg)
Me... In Scale
• Jani Haapala
• Dad, Husband, Thinker, Learner, Developer, Tester
• Senior Quality Consultant @ Qentinel
• Focus on software process improvements and automation
• DevOps and Lean advocate
• 15+ years in the field
• Nuts about measurements, data and feedback
@HaapalaJani
![Page 3: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/3.jpg)
Scaling Automation…
@HaapalaJani
Manual Testing
Test Automatio
n
Continuous Delivery
Waterfall
Agile DevOps
![Page 4: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/4.jpg)
Dedicated To People Flow
KONE
@HaapalaJani
![Page 5: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/5.jpg)
DatingFinding Your (Test Automation) partner
@HaapalaJani
![Page 6: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/6.jpg)
$ pep8 --first optparse.pyoptparse.py:69:11: E401 multiple imports on one lineoptparse.py:77:1: E302 expected 2 blank lines, found 1optparse.py:88:5: E301 expected 1 blank line, found 0optparse.py:222:34: W602 deprecated form of raising exceptionoptparse.py:347:31: E211 whitespace before '('optparse.py:357:17: E201 whitespace after '{'optparse.py:472:29: E221 multiple spaces before operatoroptparse.py:544:21: W601 .has_key() is deprecated, use 'in's
Why Do We TestPROBLEM SPACE VS SOLUTION SPACE
@Test
public void pageTitleAfterSearchShouldBeginWithDrupal() throws IOException {
assertEquals("The page title should equal Google at the start of the test.",
"Google", driver.getTitle());
WebElement searchField = driver.findElement(By.name("q"));
searchField.sendKeys("Drupal!");
searchField.submit();
assertTrue("The page title should start with the search string after the search.",
(new WebDriverWait(driver, 10)).until(new ExpectedCondition() {
public Boolean apply(WebDriver d) {
return d.getTitle().toLowerCase().startsWith("drupal!");}}));}
-----------------------------------------------------------
-----------------------------------------------------------
Open Page www.google.com
Insert Text searchbox Drupal!
Click Button Google-Search
Verify Title Starts With Drupal!
-----------------------------------------------------------
Customer must be able to buy shoes
Customer needs to put shoes in basket and complete payment
With browser xx and url yyopen. Icon zz must be clicked and then button ii pressed. If valid information is given into dd, transaction should be seen from db xx with query ss.
OR
@HaapalaJani
![Page 7: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/7.jpg)
Our ”He”, The Test Automation
www.robotframework.org
@HaapalaJani
![Page 8: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/8.jpg)
Moving TogetherBuilding solid (Infrastructure) Foundation To Live In
@HaapalaJani
![Page 9: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/9.jpg)
Infrastructure as Code == SafetyNet
Static checksDevelopment
Stakeholders
Unit tests
Smoke tests
Integration tests
System tests
Release
candidate
Commit
Acceptance tests
Release
1min
10min
20min
30min
60min
90min
10min
@HaapalaJani
![Page 10: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/10.jpg)
Complexity Explosion
Dev
Test
DevOps
+ +
@HaapalaJani
![Page 11: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/11.jpg)
Our ”SHE”, The Infrastructure (as Code)
Bare Metal Server
Virtual Machine
Artifactory
Jenkins
Nodes
Results
Metrics
Monitoring
AWS
EC2
Docker
Jenkins
Nodes
Results
Metrics
Monitoring
NodesNodes
Influxdb
MySQL
Inhouse Tools
Puppet Kubernetes
Docker Compose
Docker
Puppet
KVM CloudFormation
@HaapalaJani
Server Room Cloud
Physical Layer
Application Layer
Executor Layer
Status Layer
”Physical” Layer
Executor Layer
![Page 12: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/12.jpg)
Having KidsSpreading Quality DNA (Scaling Automation) To All
@HaapalaJani
![Page 13: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/13.jpg)
1
Testing Escalation…squared
0
50
100
150
200
250
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
MIN
UT
ES
MONTHS
TPT TPF
Create 100TC -> 200min -> 3h Debug 100TC -> 9700min -> 162h -> 20 days
0
100
200
300
400
500
600
0
1000
2000
3000
4000
5000
6000
1 3 5 7 9 1113151719212325272931333537394143454749
TE
ST
S F
AIL
ING
TE
ST
S C
RE
AT
ED
MONTHS
Tests Failures
@HaapalaJani
![Page 14: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/14.jpg)
0
50
100
150
200
250
300
350
400
450
500
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Co
st
Solutions
10 % 12 % 14 % 16 % 20 % 25 %
Infrastructure Explosion…squared
@HaapalaJani
![Page 15: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/15.jpg)
CSI Concept
Sw n
CSI TEAM
Producti
on
Support and Tooling
System QA
Team n
Team 1
Integration
Sw b
Team n
Team 1
Integration
Sw a
Team n
Team 1
Integration
Team 1
Team n
Team 1
Team n
Team 1
Team n
Integration
Integration
Integration
System QA
@HaapalaJani
![Page 16: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/16.jpg)
Happily Ever After?Happy Kingdom (With 100 000 Test Cases Per Day) For All?
@HaapalaJani
![Page 17: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/17.jpg)
Critical Red
@HaapalaJani
![Page 18: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/18.jpg)
Regression
Jail
Investigation
Court
Rehabilitation
Probation
Case Denied
Collateral Damage
RemoveTest
New TestBug In Test
Bad Environment
New Feature
Actual Bug
Flaky Test
Jailing Tests
@HaapalaJani
![Page 19: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/19.jpg)
Something To Be Proud Of
@HaapalaJani
Communi
ty
Commitment
Co-
operation
CultureQualit
y
![Page 20: From 0 To 100 000 Test Cases Per Day...Docker Puppet KVM CloudFormation @HaapalaJani Server Room Cloud Physical Layer Application Layer Executor Layer Status Layer ”Physical” Layer](https://reader035.vdocuments.net/reader035/viewer/2022081615/5fe22254798e9d309923969d/html5/thumbnails/20.jpg)
Q&A...I Congratulate You, New (DevOps) Marriage Counselors
@HaapalaJani