continuous performance testing
TRANSCRIPT
![Page 1: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/1.jpg)
Continuous Performance Testing
Mairbek Khadikov
![Page 2: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/2.jpg)
Story #1
Performance, we are
losing it
![Page 3: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/3.jpg)
Story #1
DeliveryManager
New version goes live soon
Let's check performance
![Page 4: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/4.jpg)
Story #1
Performance Tester
Performance degraded
=
![Page 5: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/5.jpg)
Story #1
Developer
Let's profile our app
![Page 6: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/6.jpg)
Story #1
Performance Tester
Performance increased on 10%
But it's still worse than the previous release
![Page 7: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/7.jpg)
Story #1
DeliveryManager
We can't wait much longer
CPU utilization is 50%
![Page 8: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/8.jpg)
Story #1
Alert is rased
too late
![Page 9: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/9.jpg)
Story #2
It's all tested, but...
![Page 10: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/10.jpg)
Story #2
Operations
We deployed new release
It passed all smoke tests
Works incorrect with load balancer
![Page 11: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/11.jpg)
Story #2
QA
We tested all this features
![Page 12: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/12.jpg)
Story #2
PerformanceTester
System has acceptable throughput
![Page 13: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/13.jpg)
Story #2
Developer
Doh, one singleton unit is not thread safe
![Page 14: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/14.jpg)
Story #2
Performance test
does not check that system works
correctly
![Page 15: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/15.jpg)
Story #3
Forgotten branches
![Page 16: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/16.jpg)
Story #3
PerformanceTester
Performance degraded in the last release candidate
![Page 17: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/17.jpg)
Story #3
Developer
What branch exactly impacted performance?
![Page 18: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/18.jpg)
Story #3
PerformanceTester
I've never heard about the half of this branches
![Page 19: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/19.jpg)
Story #3
Large projects need a comprehensive
performance testing process
![Page 20: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/20.jpg)
Story #4
Did you test that?
![Page 21: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/21.jpg)
Story #4
VP of Engineering
Production system failed this morning
![Page 22: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/22.jpg)
Story #4
Developer
Two nodes were restarted
simultaneously
![Page 23: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/23.jpg)
Story #4
Operations
Those nodes consumed to much memory
![Page 24: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/24.jpg)
Story #4
QA Lead
This is a complex scenario we can't do it continuously
![Page 25: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/25.jpg)
Story #4
Simulation of
maintenance operations and failures was not a
part of testing
![Page 26: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/26.jpg)
Performance Testing
Revised
![Page 27: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/27.jpg)
Performance Testing
Automatically test builds
Continuous
![Page 28: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/28.jpg)
Performance Testing
Rise alert flag in case of performance degradation
Continuous
![Page 29: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/29.jpg)
Performance Testing
Store all testing results
Traceable
![Page 30: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/30.jpg)
Performance Testing
Guaranty that system works correctly under workload
Validated
![Page 31: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/31.jpg)
Performance Testing
Result should include monitoring and profiling information
Transparent
![Page 32: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/32.jpg)
Performance Testing
Modules should be tested in isolation where possible
Transparent
![Page 33: Continuous Performance Testing](https://reader033.vdocuments.net/reader033/viewer/2022052601/558b3a86d8b42ad26a8b45be/html5/thumbnails/33.jpg)
Performance Testing
Simulate potential maintenance
operations and failure
Pessimistic