shake up the culture with automation!
TRANSCRIPT
![Page 1: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/1.jpg)
Shake up the Culturewith Automation!
Jul/06/2016Hiroyuki Ito / 伊藤 宏幸
![Page 2: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/2.jpg)
2
Hiroyuki Ito (The Hiro)@hageyahhooYahoo Japan Corporation Automation Coach Agile Coach
Who I Am
![Page 3: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/3.jpg)
3
Speaker of Agile2014
![Page 4: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/4.jpg)
4
Japanese #1 Internet Provider
Web Services -> Smartphone Apps
Aiming to become a Tech Giant
About Yahoo Japan Corporation
![Page 5: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/5.jpg)
5
What isDevOps?
![Page 6: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/6.jpg)
6
CALMS / CLAMS
General Definitions of DevOps
Culture
Automation
Lean
Measurement
Sharing
http://itrevolution.com/devops-culture-part-1/
![Page 7: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/7.jpg)
7
"DevOps" means:
The Hiro Says
Total Optimization
Whole Process Improvement
Based on Technical Foundation
![Page 8: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/8.jpg)
8
Successful DevOps requiresthe cultural changein your organization.
![Page 9: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/9.jpg)
9
VersionControl
CommitStage
AcceptanceTest Stage
ReleaseStage
ManualTest Stage
PerformanceTesting Stage
ArtifactRepository
Humble, Jez, and Farley David, Continuous Delivery, Addison-Wesley, 2010
Check-in
The Hiro’s Scope
![Page 10: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/10.jpg)
10
VersionControl
CommitStage
AcceptanceTest Stage
ReleaseStage
ManualTest Stage
PerformanceTesting Stage
ArtifactRepository
Humble, Jez, and Farley David, Continuous Delivery, Addison-Wesley, 2010
Check-in
The Hiro’s Scope
Cultural Change withTest Automation & CI
![Page 11: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/11.jpg)
11
4 Teams with 3 Products
Our Target
We coaches supported them
Lots of Immature Engineers
Few Automated Tests
Lots of Integration Troubles
![Page 12: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/12.jpg)
12
Agenda
1. Test Automation
3. Evaluation of Measures
2. Continuous Integration
4. Conclusion
![Page 13: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/13.jpg)
13
1. Test Automation
3. Evaluation of Measures
2. Continuous Integration
4. Conclusion
![Page 14: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/14.jpg)
14
Make product teamswrite test scripts voluntarily
Self-running Team
Cross-functional Team
Higher Quality
Higher Productivity
Our Goal
![Page 15: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/15.jpg)
15
Background
![Page 16: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/16.jpg)
16
Prerequisite
PHP
Cloud-based Infrastructure
Brownfield
![Page 17: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/17.jpg)
17
Legacy Code: No test scripts!
Brownfield: Our Common Challenges
Test Automationas a Technical Foundation
Improve design & program
Refactoring
Prevent regressions
![Page 18: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/18.jpg)
18
Lots of Immature Engineers
Engineers
Test Scripts
???
How to WriteTest Scripts
The Importance ofTest Automation
How to TestPrograms & Products
What should I do?
![Page 19: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/19.jpg)
19
Passive Culture
Front-endEngineers
DatabaseEngineers
APIEngineers
Waiting forInstructions
Silos Based onComponent TeamsLeader
![Page 20: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/20.jpg)
20
Concrete Actions
![Page 21: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/21.jpg)
21
Scope of Test Automation
Unit Testing
Acceptance Testing
Performance Testing
Security Testing
Other ility Testing
![Page 22: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/22.jpg)
22
Stages of Adapting to Unit Testing
1. Write simple test scripts
2. Adapt to TDD
3. Use Test Double (Mock)
4. Write characterization test if necessary
5. Refactor production code & architecture
![Page 23: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/23.jpg)
23
Schedule & Actions
Jan Feb Mar
Preparation• Investigate products• Create sample scripts
Execution• Lectures• Workshops• Pair Programming
ExecutionRemove architecturalimpediments
![Page 24: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/24.jpg)
24
A powerful toolto learn
Unit Testing & TDD!
Cyber Dojo: Our Weapon
http://cyber-dojo.org/
![Page 25: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/25.jpg)
25
1. Test Automation
3. Evaluation of Measures
2. Continuous Integration
4. Conclusion
![Page 26: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/26.jpg)
26
Make product teams set up CI processes
Our Goal
Nurture product teams
Avoid integration/deployment troubles
Achieve the target from CTO
Improve product teams
![Page 27: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/27.jpg)
27
Background
![Page 28: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/28.jpg)
28
e.g.) During integration & deployment process
Tons of Failures were There
Needed to avoid these troubles fundamentally
StagingENV
ProductionENV
CONFIG Filefor Staging
Troubles!
![Page 29: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/29.jpg)
29
Voice of CTO
Let’s do CI Companywide!
Top-down Approach
Bottom-up Approach
CTO
![Page 30: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/30.jpg)
30
Concrete Actions
![Page 31: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/31.jpg)
31
Accustomed to CI
WorkingSamples
Team Members Coach
Worked with Samples
Configurationof CI Server
CONFIG File
Did Pair Setting(like Pair Programming)
Team Members Coach
![Page 32: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/32.jpg)
32
Always added Automated Tests
Test! Test! Test!
Pull the real power of CI
Adapt to Test Automation more
Nurture product teams
![Page 33: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/33.jpg)
33
Simplified CI Processes
VersionControl
APServer
CONFIGFile
DBServer
WebServer
???
VersionControl
APServer
CONFIGFile
DBServer
WebServer
![Page 34: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/34.jpg)
34
1. Test Automation
3. Evaluation of Measures
2. Continuous Integration
4. Conclusion
![Page 35: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/35.jpg)
35
1. Results
![Page 36: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/36.jpg)
36
Updated on Mar/31/2016(during about 2 months)
Metrics
Item Quantity
Unit Tests Added/Updated 27
Software Bugs Detected 10
Infrastructure Bugs Detected 24
Prevented Troubles 3
It may be insufficient, but it’s REAL.
![Page 37: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/37.jpg)
37
Self-running Actions by Product Team Members
PO added “writing unit tests” as DoD
Started to write unit tests voluntarily
Established their own CI/CD strategy
Established their own testing rules
![Page 38: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/38.jpg)
38
2. Lessons Learned
![Page 39: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/39.jpg)
39
Coaches should NOT do everything
Role of Coaches
For retaining learning opportunity
For nurturing voluntary actions
Giving hints is sometimes appropriate
Total
![Page 40: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/40.jpg)
40
Should Create Unavoidable Situation
CI:Unavoidable from Test Automation
Top-down Approach:Unavoidable to achieve objectives
Bottom-up Approach:Unavoidable to scale improvements
Total
![Page 41: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/41.jpg)
41
Based on the production code
Sample Test Scripts Are Very Useful
Easy to adapt to Test Automation
Can use them as initial test scripts
Can learn their products & problems
TeamMembers
Coaches
Test Automation
![Page 42: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/42.jpg)
42
Need to Solve Architectural Bottlenecks
Unable to apply mocks
Side effectsby static mocks
Hard to identifyweb components(Selenium WebDriver)
Modified interfacesfor injecting mocks
Introduced Phake(Mock FW for PHP)
Defined locatorsto each component(like “id” or “name”)
Test Automation
![Page 43: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/43.jpg)
43
/** * @test */public function exampleOfTest() { $sut = new Foo(); $result = $sut->bar('abc'); $this->assertEquals('huga', $result);}
Pair Programming Clarifies Real SkillsTest Automation
![Page 44: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/44.jpg)
44
/** * @test */public function exampleOfTest() { $sut = Foo::new(); $result = …? /* Cannot write logics */
$result = Foo::bar('abc');}
Pair Programming Clarifies Real SkillsTest Automation
![Page 45: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/45.jpg)
45
Detected failuresup-front with Automated Tests
Prevented Troubles
Unexpected Environmental Changes
Failures of Merging Branches
Leakages of Merging Branches
CI
![Page 46: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/46.jpg)
46
Detected Misconfigurations on Servers
Only PHP
Package Hell
Don’t need to compileInterpreter
Unused Packages Leakages of Packages
CI
![Page 47: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/47.jpg)
47
Detected Misconfigurations on Servers
Can solve Package Hell
PHP + Automated Test + CI
CI
![Page 48: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/48.jpg)
48
Made team memberslearn a lot based on
automated working processes
Nurtured Product Teams
Fast Feedback
Learn from Failures
Lots of Confidence to Go ahead
CI
![Page 49: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/49.jpg)
49
Technical Foundation
Technology-Driven Development
-> Presentation at Agile2014
CI/CDTest Automation
Teams
Members
Organization
Products
![Page 50: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/50.jpg)
50
3. Next Actions
![Page 51: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/51.jpg)
51
Next Challenges
Skills of other languages
Acceptance Test Automation
Gather other useful metrics
![Page 52: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/52.jpg)
52
Example of Additional Metrics
Test &Test Automation
Code CoverageCyclomatic Complexity# of Bugs Detected# of Members Who Can Write TestsUpdate Times of Test Scripts(Lots of Updates = Valuable Test)
CI/CD Cycle TimeMTBFMTTR# of Releases per Week# of MembersWho Can Operate CI/CD Servers
![Page 53: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/53.jpg)
53
1. Test Automation
3. Evaluation of Measures
2. Continuous Integration
4. Conclusion
![Page 54: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/54.jpg)
54
What isDevOps?
![Page 55: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/55.jpg)
55
"DevOps" means:
The Hiro Says Again
Total Optimization
Whole Process Improvement
Based on Technical Foundation
![Page 56: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/56.jpg)
56
We are challengingthe cultural changein our organization.
![Page 57: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/57.jpg)
57
Test Automation with CIis sufficiently valuable
Adapting to Test Automation & CI in DevOps Context
Iterative & Incremental Improvement
Do Kaizen / 改善
![Page 58: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/58.jpg)
58
Adapting to Automationis a long way
It’s Not Easy to Continue…
It takes 6 monthto do the same thingfor 1 teamin Facebook
Photo by Improve It - Kent Beck no Workshop Mapping XP.(2006) / CC BY-SA 2.0
![Page 59: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/59.jpg)
59
But necessary to continuefor achieving
total optimization
![Page 60: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/60.jpg)
60
Shake up• product teams• working processes• ourselves
for• total optimization• huge success
Always Improve, Always Advance
![Page 61: Shake up the Culture with Automation!](https://reader031.vdocuments.net/reader031/viewer/2022021507/586f71941a28ab10258b4ff9/html5/thumbnails/61.jpg)
61
With DevOps!