How Database DevOps Will Help You

To Stay Ahead Of The Competition

Bob Walker, Solutions Architect at Octopus Deploy

Agenda

• Problem to solve

• DevOps

• Benefits

• Roadblocks

• Questions

What is DevOps trying

to Solve?

Previous Work Experience

• Lifting Controls for Oil Terminals

• Loan Origination System

• Electronic Pollbook

Software Development Life Cycle

Development Team

Operations Team

Knowledge Silos

• Development Team• Creates roadmap• Feature requirements• Knows how to fix the application

• Operations Team• Performance statistics• Support tickets submitted• Incidents

Typical Communication Between Teams

• “This service is down please restart it”

• “Your service is consuming 16 GB of RAM, fix it now”

• “I have a call from a customer and they say your

application is slow”

• “I got paged last night because this SQL Job

failed. Fix it.”

Developer’s Mindset

A Lack of Trust

Lack Of Trust

Slows Everything

Down

Real-World Example

• Loan Origination System

• Used Redgate Schema compare to compare test

and prod

• Scripts dropped into shared folder to be ran

• 2ish hour deployment

Real-World Example

• Lifting Controls Application

• Manually Written Database Scripts

• Manual Deployments

• 2 AM Saturday Morning Deployments

• 2 – 4 hour deployments on average

Expected

Perfection

Big Bang Deployments

• Multiple Sprints prior to going to production

• Multiple KanBan cards bundled together prior to

going to production

• Long Waterfall release cycles (quarter or months)

Perfect Storm

• Lifting Control Application: Quarterly Release with

six month rotation

• Loan Origination System: Quarterly Release

• Electronic Pollbook: Three Month - Eight Month

Release

Everyone Is Unhappy

• Business Owners• “This isn’t going to work now, we need to add the following features to

make this viable”• “Fine. Fine. Fine. Deploy it as is. I’ll add it my next release doc”

• Users• “This isn’t what I wanted”• How long until the application is no longer relevant• Head to competitors

• Security• Data Compliance• Patches

DevOps to the rescue!

Right?!?!? Right?!?!?!

DevOps: Development

and Operations Teams

Working Together

Towards a Common

Goal

Common Goal Examples

• 1 deployment a month

• 1 deployment a week

• Zero Downtime Deployments

• 10 deployments a day

• Fixing all critical bugs within 2 business days

• Performance tests before each release

• Decrease deployment time

• Increase response time on critical methods

• Decrease resource usage

Benefits

Automated Database Deployments

• Loan Origination System

• Database Developer, Lead Developer, DBA, Architect and Redgate Consultants collaborate on a initial process

• Pilot Team adopted process and tooling

• Iterate through the process

• 2ish hour deployments went down to 10ish minute deployments

Less Time Fixing Bugs

• 3200 schema objects

• 2 to 3 emergency fixes post release

• Missed views, stored proc updates, tables, etc

• Post automation this went down to zero

Multi-tenant Database Deployments

• Electronic Pollbook Reporting Application

• 100+ jurisdictions with unique databases

• Unique delta script per jurisdiction

• Prep-work minimum two days

• Deployment minimum four hours

• Post Automation 20 – 30 minutes of prep, 30 minute deployment

Fixing Critical Bug

• Customer reported bug to their process through

support channels at Octopus Deploy Tuesday

morning

• Discussed bug during the day

• Fix released late Tuesday night

• In customer’s hands by Wednesday morning

Better Performance

• Loan Origination System

• Endpoint averaging 2000 requests per minute

• After deployment average response time went

from 20 ms to 200 ms

• Identify slow performance and deploy release

before first support ticket came in

Staying Ahead Of Competition

• Respond to customer feedback

• Fix critical bugs

• Experiment and A/B testing

• Keep company relevant

Higher Employee Utilization

Getting Started

Octopus Deploy and Redgate to the

rescue!

Hold your horses…

Start Small

• Attend meetings

• Discuss issues and complaints

• Listen and Learn

• Pick a common problem

• Use existing tooling

Go Bigger

• Solve a larger problem

• Pilot Application / Group

• New Tooling

• New Processes

Tooling

• Build: TeamCity / VSTS / Jenkins / Bamboo

• Unit Testing: NUnit / MSTest / XUnit

• Integration Testing: Selenium / Postman

• Deploy: Octopus Deploy / Redgate (for DBs) / VSTS Release / DACPAC

• Monitoring: NewRelic / DataDog / DynaTrace / SQL Sentry

• Error Logging: Splunk / Seq / PaperTrail

DevOps is

Both People

and Tools

Roadblocks

People

• “We’ve been doing it this way for 10 years”

• “Sure what we have is a little bumpy, but it works”

• “I’m too busy to change”

• “This was tried once before and it failed”

• “What you proposed won’t work for me or my

team, we’re special”

And some people…

Time

• Time to implement changes

• Each new change requires multiple iterations

• Deadlines and other conflicting priorities

Cost

• Tools cost money

• Cost per development hour

• Consultants

• Months or years before an ROI

Breaking Through Roadblocks

• Technology is the easy part

• Pilot team / project

• Small changes

• Don’t shoot for perfection at first

• Provide clear and measurable stats

• Be Transparent

Anyquestions?

Thank You!