customer+business+analysts+qa+developer+dba+admin ... afkari irmac may2018.pdfhow devops can help...

How DevOps can help DATA and Information Management!

IRMAC

May 2018

Ahead Technology & Afsaneh Afkari (utg) 1

Dev+OpsCustomer+Business+Analysts+QA+Developer+DBA+Admin+Architects+

Service Desk+Security+Audit+Compliance+Supplier+...

Copywrite


https://www.ahead-technology.com

https://www.utg.caAfsaneh [email protected]

https://www.utg.ca/

High Level Agenda

• Introduction

• 2016 Data Quality Survey

• The Average Use of Software Features Survey

• DIKW

• Data Sources

• What is DevOps

• Other Frameworks

• DevOps Principles

• DevOps Practices

• DevOps Adoption – A Watch-list!


Bio – Afsaneh Afkari

A technology consultant and accredited trainer with 20 years of experience in variety of industries with focus on customer value, outcome, building long lasting and high performing teams.

• Certified DevOps Lead (DOL)

• Certified Agile Scrum

• Certified Lean IT

• Certified ITIL ® Expert & Accredited ITIL® instructor and consultant

• Project Management Professional (P.M.P)

• Certificate in Master Black Belt Lean Sigma

• Certified TOGAF 9

• Certified Associate Business Continuity Planner (A.B.C.P.)

• Contact Information: • [email protected] or [email protected]• Toronto DevOps Meetup: https://www.meetup.com/Toronto-DevOps-Meetup/events/247844511/

• www.linkedin.com/in/afsanehafkari


mailto:[email protected]

mailto:[email protected]

https://www.meetup.com/Toronto-DevOps-Meetup/events/247844511/

52% of Application Development Teams Choose to Avoid Their Organization’s Data Group.


8%

15%

46%

46%

46%

62%

Not allowed to

Don't know it exists

Don't know they should do so

Data group too hard to workwith

Data group offers little value

Data group too slow

Source: 2016 Data Quality Survey

The average use of Software FeaturesBased on The Standish Group in 2011 research

• 45% = never

• 7% = always

• 13% = often

• 16% = sometimes

• 19% = rarely• It means 64% is Wasted• Why: Customer thinks of all features up front in a Waterfall

method!


Let’s Talk DATA!


Where does DATA come from today?

• Every day that you live your life data about you is being generated. Turn on your phone, check your email, watch TV, listen to spotify, browse the web, fill up your car with gas, drive through a toll, go to the restaurant, see a doctor, talk to a friend on the phone. In all of these activities, data about your actions, behaviors, likes and dislikes, mood and feelings, purchases and choices are being recorded by a huge variety of organizations that never will inform you of the data they collect, how they process it, sell it to other organizations, retain and store it.

(Article Source: The Data Dividend by Steven Adler, Creator of the Data Governance Industry)


https://www.linkedin.com/in/stevenbadler/

https://www.linkedin.com/in/stevenbadler/

How Is DATA Generated?

• Every action we take generates DATA! • No longer direct interaction with devices and computer systems!• IOT! Clouds!

• And of course! • Any devices! • Applications

• Systems of Record; Systems of Engagement; Systems of Differentiation; Systems of Innovation; websites; Apps ….

• Business Processes• Hand-written notes!• Beloved Excel sheets! Many many others!

• Technology Processes• Code, System logs, batch jobs, test and QA artifacts, release notes, DBs, many others!


What is DATA?

• It is no longer just Customer Data or from Systems of Record (SOR)

• Everything about Humans!• Pictures! Habits! Behaviour! Background! Weight! Height! DNA!...

• In IT!• Software, the programs and other operating information used by a computer

• Application software (an application) is a set of computer programs designed to permit the user to perform a group of coordinated functions, tasks, or activities.

• Source Code, Data Models, Configuration Files, System logs, Chats, !


Where is DATA?

• Hardware and Infrastructure!• Internal systems, Private Clouds, Hybrid Clouds, Managed Clouds…

• Storage, SAN

• VMware

• Devices …..Smart Phones, Wearables, Tablets….

• Computers, Laptops, TVs, Cameras, Cars….

• Websites…

• Many others!


DIKW!

Wisdom

Knowledge

Information

Data


know-nothing, know-what, know-how, and know-why

https://en.wikipedia.org/wiki/Know-how

How good is the DATA?

Really don’t know!


What is DevOps?

• A “Movement”… not a set of standards, not a framework, not best practices or anything else!

• Builds on and leverages Lean principles, Agile Software Development methods and ITSM (ITIL) framework• Agile left out Ops!

• ITSM left out Dev!

• Lean was not in IT picture at all!

• Focuses on communication, collaboration, cultural and behavioural changes


Culture Change

•“You can’t directly change culture. But you can change behavior, and behavior becomes culture” – Lloyd Taylor VP Infrastructure.

• During his three year term at Google, Taylor was responsible for scaling the Google.com infrastructure including data centers and server operations to support Google's post-IPO growth. His efforts helped the company scale its infrastructure by more than 500% while growing staff responsible for machine maintenance by only 80 percent.


Why Development & IT Operations?

Because that is typically the value stream that is between the business (where requirements are

defined) and the customer (where value is delivered). Source: IT REVOLUTION: Top 11 Things You Need to Know About DevOps


Why the Business is Driving DevOps?• Every business has become a tech business

• IoT is rapidly increasing

• Intelligent data must shape direction quickly

• Consumers have developed “app” mentalities

• Customers value outcomes, not products

• Time to value is replacing time to market

• Customer delight is more important than customer satisfactionAhead Technology & Afsaneh Afkari (utg) 17

Why is IT Driving DevOps?

• Every business has become a tech business

• IoT is rapidly increasing

• Consumers have developed “app” mentalities

• Customers value outcomes, not products

• Time to value is replacing time to market

• Intelligent data must shape direction quickly

• Customer delight is more important than customer satisfaction


Output Versus Outcome


Source: https://www.businessillustrator.com/output-vs-outcome-cartoon/

The Wall of Confusion (1)

Dev wants change

Ops wants

stability

What about Security, Governance, Risk Management

and Compliance? What do they want?


The Wall of Confusion (2)

What are the dangers?Extreme Focus on Stability

Extreme Focus

on Change

What does the business want?

All of the above

Quality


What Problems DevOps is Solving

• Command-And-Control reporting structure

• Culture of us versus them

• Delayed customer feedback ..customer is anyone in the Value Stream

• Functional teams & Siloes

• Slow feature time to market

• Long deployment cycles, linear approach to traditional Waterfall …releases every 9 to 12 months!

• Unreliable data

• Poor quality

• Increasing numbers of outages

• Complex or bureaucratic processes

• Lack of agility

• Overly tight architecture

• Low IT performance

• Growing “Technical Debt” developed by Ward Cunningham pioneer in both design patterns and extreme programming(XP)

• Too much TIM WOODS in IT!


DevOps Values

• CALMS …

• Culture | Automation | Lean | Measurement | Sharing

• Coined by Damon Edwards (@damonedwards) and John Willis in 2010 as the acronym CAMS.

• Jez Humble (@jezhumble) later added an L, standing for Lean, to form CALMS.• DevOps Values …My perspective …LMSCA!

• Lean

• Measurement

• Sharing

• Culture

• Automation!


http://twitter.com/#!/damonedwards

http://twitter.com/#!/jezhumble

Cross-functional Teams & “BUS FACTOR”!

• I-Shaped – Most organizations! This is what we wanted!

• T-Shaped

• Pi-Shaped

• Comb-Shaped or E-Shaped


DevOps

Agile

Lean

ITSM

Other Frameworks


DevOps & Lean


Lean Concepts!

• Value-stream mapping is a lean-management method for analyzing the current state and designing a future state for the series of events that take a product or service from its beginning through to the customer. A Toyota Production System (TPS) concept• VA(Value-Add),NVA(Non-Value-Add), NNVA(Necessary- Non-Value-Add)

• Theory of Constraints • A methodology for identifying the most important limiting factor (i.e. constraint) that

stands in the way of achieving a goal and then systematically improving that constraint until it is no longer the limiting factor. The constraint is often referred to as a bottleneck.

• Continuous Improvement • Continuous Learning• Measurements: Cycle Time, Processing Time, Waiting Time, WIP and others


Lean Concepts!

• Identification of “WASTE” (MUDA)…

• Where is Timmy! TIM WOODS• T – Transport – Moving people, products & information

I – Inventory – Storing parts, pieces, documentation ahead of requirementsM – Motion – Bending, turning, reaching, lifting

• W – Waiting – For parts, information, instructions, equipmentO – Over production – Making more than is IMMEDIATELY requiredO – Over processing – Tighter tolerances or higher grade materials than are necessaryD – Defects – Rework, scrap, incorrect documentationS – Skills – Under utilizing capabilities, delegating tasks with inadequate training


Lean Concepts…continue

• TIM WOODS in IT:• Moving code, moving data, moving people and systems; too many hand-offs;

high WIP; unused features; storage; hardware; data; defects; Change Requests; delays between processes; too many projects and initiatives; too many tools that do the same thing; slow; work-arounds; lack of automation where necessary; Too much documentation; too much governance; Too many policies; and many others!

• Single Piece Flow!


Batch (vs. Single Piece Flow)! The Envelope Game!

Fold 1

Fold 2

Fold 3

Insert 1

Insert 2

Insert 3

Seal 1 Stamp 1

Seal 2 Stamp 2

Seal 3 Stamp 3

Mail to customers


Single Piece Flow

Insert 1 Seal 1 Stamp 1Fold 1 Mail to customers





DevOps & ITSM


ITSM and Agile Service Management

• IT Service Management

• Deployment and management of entire IT Operations’ life cycle

• Service Strategy, Service Design, Service Transition, service Operations and Continuous Service Improvement

• Change Management, Release Management, Problem Management, SACM (Service Asset and Configuration Management), Incident Management, and others


DevOps & Agile!


Agile Manifesto for Software Development!

We Value! Over!

Individuals and interactions

processes and tools

Working software comprehensive documentation

Customer collaboration

contract negotiation

Responding to change

following a plan


12 Principles behind the Agile Manifesto1. Our highest priority is to satisfy the customer

through early and continuous deliveryof valuable software.

2. Welcome changing requirements, even late indevelopment. Agile processes harness change for the customer's competitive advantage.

3. Deliver working software frequently, from acouple of weeks to a couple of months, with apreference to the shorter timescale.

4. Business people and developers must worktogether daily throughout the project.

5. Build projects around motivated individuals.Give them the environment and support they need, and trust them to get the job done.

6. The most efficient and effective method ofconveying information to and within a development team is face-to-face conversation.

7. Working software is the primary measure of progress.

8. Agile processes promote sustainable development.The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

9. Continuous attention to technical excellenceand good design enhances agility.

10. Simplicity--the art of maximizing the amountof work not done--is essential.

11. The best architectures, requirements, and designs emerge from self-organizing teams.

12. At regular intervals, the team reflects on howto become more effective, then tunes and adjusts its behavior accordingly.


Richness of Communication Channels

Ahead Technology & Afsaneh Afkari (utg)37

What else from Agile & Agile Scrum!

• Continuous learning through experimentation

• Continuous Improvement

• Self-organizing Teams

• Cross-functional teams

• MVP ..Minimum Viable Product

• Iterations

• Increments

• Others!


DevOps Principles & Practices!


DevOps Principles: The Three Ways!

• The First Way – Flow • Understand and increase the flow of work (left to right)

• The Second Way – Feedback • Create short feedback loops that enable

continuous improvement (right to left)

• The Third Way – Continuous experimentation and learning • Create a culture that fosters

• Experimentation, taking risks and learning from failure

• Understanding that repetition and practice is the prerequisite to mastery


DevOps Practices

• Continuous Integration*

• Continuous Delivery*

• Continuous Testing

• Continuous Deployment

• Rugged DevOps

• DevSecOps*

*will be discussed


Continuous Integration (CI)(Source: Devops.com)


CI ..Continue

• At least once per day code commit by each developer into a single CI server that is “Production Like” environment

• Brining forward System Integration (SI) testing

• Automated Unit Testing based on Test Driven Development (TDD) methods

• Feedback loop where all errors and bugs are stopped and will not be deployed to the next environment

• Case Study Google: in 2016, 25000 developers had 16,000 commits a day into a single repository (if I recall correctly, and around 500,000 per week!)


Continuous Delivery (CD)

• According to Martin Fowler:“Continuous Delivery is a software development discipline where you build

software in such a way that the software can be released to production at any time”.

• More important to keep the Deployment Pipeline working correctly and the selected features to be releasable than performing new work packages and features

• Feedback loop at any stage in the Pipeline


DevSecOps(Source: devsecops.org)

• The purpose and intent is to build on the mindset that "everyone is responsible for security" with the goal of safely distributing security decisions at speed and scale to those who hold the highest level of context without sacrificing the safety required.

• It's hardly ever the case that a Security Team has all the information it needs to render a security decision that makes sense at the tale end of the value creation life cycle.

• Security needs to be added to all business processes and a dedicated team needs to be created to establish an understanding of the business, tooling to discover flaws, continuous testing, and science to forecast how to make decisions as a business operator.


DevOps Adoption – A Watch-list!

My Perspective!


Roles!

We Have seen This!

Data Analyst Data Scientist!

DBA Data Architect!

Business Analyst Project Manager!

Developer Enterprise Architect!

We See in DevOps!

Tester DevOps Engineer!

Developer DevOps Engineer!

System Admin DevOps Engineer!

DBA DevOps Engineer!

Data Analyst DevOps Engineer!


Automation!

• Tools and Automation before understanding Value Stream!

• Too many of them!


Training, Transition Time, Coaching!

• Change ≠ Transition!

• No Lean, Agile and ITSM knowledge or training!

• Forgetting about cultural changes needed!

• Little support for Adoption!


I am still learning!Michelangelo

Thank you!


Sources and Credits • Ahead Technology Training Material (Copy-write) -DevOps Foundation Certificate

• www.ahead-technology.com

• The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations by Gene Kim (Author), Patrick Debois (Author), John Willis (Author), Jez Humble (Author), John Allspaw (Foreword)

• Scott Ambler + Associates

• https://www.gartner.com/newsroom/id/1923014

• ITIL and ITSM (Ahead Technology and Axelos)

• Source: https://www.businessillustrator.com/output-vs-outcome-cartoon/

• http://agilemanifesto.org/principles.html

• www.lean.org

• https://itrevolution.com/devops-culture-part-1/

• DevOps Culture (Part 1); May 1, 2012 by John Willis

• As noted on individual slides as well


http://www.ahead-technology.com/

https://www.amazon.ca/s/ref=dp_byline_sr_book_1?ie=UTF8&field-author=Gene+Kim&search-alias=books-ca

https://www.amazon.ca/s/ref=dp_byline_sr_book_2?ie=UTF8&field-author=Patrick+Debois&search-alias=books-ca

https://www.amazon.ca/s/ref=dp_byline_sr_book_3?ie=UTF8&field-author=John+Willis&search-alias=books-ca

https://www.amazon.ca/s/ref=dp_byline_sr_book_4?ie=UTF8&field-author=Jez+Humble&search-alias=books-ca

https://www.amazon.ca/s/ref=dp_byline_sr_book_5?ie=UTF8&field-author=John+Allspaw&search-alias=books-ca

https://www.gartner.com/newsroom/id/1923014

https://www.businessillustrator.com/output-vs-outcome-cartoon/

http://agilemanifesto.org/principles.html

http://www.lean.org/

https://itrevolution.com/devops-culture-part-1/

https://itrevolution.com/author/jwillis/

customer+business+analysts+qa+developer+dba+admin ... afkari irmac may2018.pdfhow devops can help...

Documents