bots on guard of sdlc
TRANSCRIPT
Bots on guard of SDLC
Alexey Tokar
VP of Engineering @ WorldAPP
2
3y 4y 5y 2y
Developer Developer & Founder Team Lead & Architect VP of Engineering
@ Various companies @ Startup @ Yandex @ WorldAPP
13 years in practical engineering
70 people in the department
10+ projects from initial commit to production
SDLC
3
Software
Development
Life Cycle
PHASE 2
Design
PHASE 1
Requirements
analysis
PHASE 3
Development
PHASE 5
Maintenance
PHASE 4
Testing
Complicated scenarios in living examples
4
Success scenario for a task:● Open → In Progress● In progress → For Review
○ Code review on Merge Request○ Merged MR○ Set branch name in the task
● For Review → Resolved○ Does the fix contain SQL changes?○ Comment which parts of the system
might be affected by the fix● Resolved → In Testing● In Testing → Tested
○ Set “fixed in” version● Tested → Closed
Additional requirements
● have crosslinks between different tools
● include all actual changes into a release changelog
● follow the process for each change
● have one and only one task in “In Progress” state at any moment
5
Major obstacles in the process
● Every new employee must be properly educated
● It is boring to follow a process with lots of steps
● It is so easy to forget about some steps
● Using a ton of tools calls to a constant context switching
● Each step must be “supervised” to check its correctness
6
Manual process improvements
7
Move to automation
8
What already have been automated
9
MaintenanceDevelop Test
● Unit and Integration tests run on every commit to MR branches
● Static code analysis on each push
● Cross references between GitLab and Jira
● HipChat notifications about created Merge Requests
What already have been automated
10
MaintenanceDevelop Test
● UI and Performance tests run on every commit to a develop branch
● Deploy a successful build to the test environment
● Check against different types of supported DBMS
What already have been automated
11
MaintenanceDevelop Test
● Deploy a successful build to the production environment
● Grafana alerting to HipChat
Our new member - Overlord
12
Bot design13
How does it work
dependencies {compile 'io.evanwong.oss:hipchat-java',
'org.eclipse.jgit:org.eclipse.jgit',
'org.gitlab:java-gitlab-api',
'com.atlassian.jira:jira-rest-java-client-core','com.atlassian.jira:jira-rest-java-client-api',
'com.google.inject:guice','io.dropwizard:dropwizard-core','commons-io:commons-io'
}14
How does it work. Entry points
WebHooks
ScheduledExecutorService
15
java.util.concurrent.
WebHooks16
How does it work
@POST
@Path( "/gitlab/callback" )
public void gitLabCallback(
GitLabEvent event
) {
...
17
Purpose: respond on external events
WebHooks usages
18
Task updates according to the workflow
19
● Transitions task status
● Assigns proper person for the next step
● Marks if task has SQL changes
● Adds a label with branch merged into
Improve cross references between tools
20
● Notifies about missed ticket key in MR title
● Fills MR with information from Jira
● Fixes common mistakes in MR creation
Propose the best reviewers based on MR changeset
21
● Who previously has edited the touched code lines
● Who has coded more than others in the files
● Who is team lead / owner of the service / package
Check that MR has 2 upvotes before merging
22
● Check that rules are followed
● Notify TeamLead / Dev manager about any
violation
● Push an author to ask colleagues to look at his
masterpiece
Predict possible bugs in the changeset
23
● Searches for possible non static bugs
● Provides unobvious observations about devs
habits
● Calls to an attention on hotspots
“Bug prediction based on your code history”A talk on JavaDay Ukraine 2017
ScheduledExecutor24
How does it work.
1 ScheduledExecutorService scheduler;23 scheduler = Executors.newScheduledThreadPool( 1 );45 scheduler.scheduleAtFixedRate(6 () -> yourRunnableAsATask,7 initialDelay, 8 period, 9 TimeUnit.MINUTES10 );
25
Purpose: call scheduled events
ScheduledExecutor usages
● Notifies author about old MR without any reactions
● Notifies assignee that MR can be merged
● Notifies if you have lots of “In Progress” tickets or don’t have them at all
● Provides a list of merged tasks in the particular branch
26
Gains27
Efficiency
Engineers spend less time on bureaucracy and can put more effort on
getting closer to the goal.
28
BUREAUCRACY ⇨
⇦ EFFICIENCY
● less context switching
● less human errors during process following
Integrations
Improved cross references between tools. Makes clearer any introduced
change and speed up bug triangulation and resolution in future.
29
● where the change was introduced
● how other tasks impacts this one
● what else could be broken by the bug
Education
Education instead of punishment. As there is no human on the other side of the wall
it is much easier to ask help. It can explain how to follow the process 100500 times
per day for no cost.
30
Collaboration
Collaboration is very important between members of the technical team. Rather
than working in functional silos, modern methodologies promote frequent and face-
to-face communication between team members.
31
htt
p:/
/ww
w.a
iim.o
rg/~
/media
/AII
M_R
eal_
Images/6
00x400/C
olla
bora
tio
n_lig
htb
ulb
-illu
str
atio
n.jp
g
Future improvements
● Gamification and employee engagement
● Bot queries instead of production requests, one-time scripts or “TestController”
mappings
● Production system statuses and operations with Hubot
● ...and any other automated routines
32
