devops 101 - ibm impact 2014
Post on 13-Sep-2014
666 views
DESCRIPTION
With Wai Lee, Vice President PrudentialTRANSCRIPT
© 2014 IBM Corporation
DevOps 101Concepts and OverviewWai Lee
Vice President
Digital Platform Engineering
Prudential Financial
© 2013 IBM Corporation
Sanjeev Sharma
IBM WorldWide Lead – DevOps Technical Sales
DevOps Blog: http://bit.ly/sdarchitect
Twitter: @sd_architect
Author: DevOps For Dummies –
http://ibm.co/devopsfordummies
Please Note
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
Agenda
• What is DevOps?• Adopting DevOps
o Peopleo Processo Technology
• Implementing Delivery pipelines
Agenda
• What is DevOps?• Adopting DevOps
o Peopleo Processo Technology
• Implementing Delivery pipelines
A lack of continuous delivery impacts the entire business enterprise in the new reality of “Systems Of Interaction”
>45%of customers experience production
delays
>50%of outsourcedprojects fail to
meet objectives
>70%of budgets devoted
to maintenanceand operations
4-6 weeks
to deliver even minor application changes
to customers
Systems of Interaction
Continuousclient
experience
Partnervalue chain
Cloud-basedServices
Systems of Engagement Systems of Record
OperationsRapid app releases impact
system stability and compliance
SuppliersDelivery in the context of
agile
Development/TestSpeed mismatch between faster moving front office and slower moving back office systems, delaying time to obtain feedback
Line-of-businessTakes too long to introduce or make
changes to mobile apps and services
HR
DB ERP
MF iSeries
CRM
Plan
Do
Check
Adjust
William Deming – American statistician
Major influencer of Japanese manufacturing and business
Famous for Plan-Do-Check-Act cycle (Deming Cycle)
PDCA cycles found in DevOps
6
William Edwards Deming
Plan
Do
Check
Adjust
Deming Cycle
Quality
Plan
Do
Check
Adjust
DevOps approach: Apply Lean principles to software innovation and delivery to create a continuous feedback loop with customers
Line-of-business
Customer
1
3
2
1. Get ideas into production fast2. Get people to use it3. Get feedback
Adopt DevOps approach to continuously manage changes, obtain feedback and , deliver changes to users
Eliminate any activity that is not
necessary for learning what
customers want
8
The Big Sources of Wasted Efforts: Find the Hidden Factory
Overhead, rework, over-production
Value-added production work
80%
20%
LeanTransformation
Type of Waste Create Feature Deliver Feature
Unnecessary Overhead
Communicating ideas/knowledge Communicating between development and operations
Unnecessary Re-work
Tasks assigned back to developers from testing
Tasks assigned back to developers from production rollbacks
Over-production Unnecessary functionality produced Unnecessary hardware, data center, personnel
60%
40%
The Lean Adoption Framework Uncovers the Hidden Factory Opportunities
Fat efforts to minimize Waiting Training
Reporting TraceabilityLate rework
Duplicate effortsMetrics collection
Regression testingChange propagationDocument generationMeetings/CheckpointsSystem administrationResource accountingHuman inspections
Streamline or automate
More Valuable efforts to improveScoping LearningFeedback
RefactoringDesigningTeamingCodingTesting
PlanningEngineeringEmpoweringPredictionDecidingSteering
Facilitate or smarten
Priorities of Indian Global System Integrators (200 responses)
Fat efforts to minimize
Late rework Waiting
Regression testing Duplicate efforts
Reporting
Document generation Training
Metrics collection Change propagation
Traceability Human inspections
Meetings/Checkpoints System administration Resource accounting
Streamline or automate
More Valuable efforts to improve
Scoping Designing Planning Testing
Reusing Deciding Steering Feedback Coding
Prediction Engineering
Learning Teaming
Refactoring
Facilitate or smarten
Enterprise capability for continuous software delivery that enables clients to seize market opportunities and reduce time to customer feedback
Acceleratesoftware deliveryfaster time to value
Acceleratesoftware deliveryfaster time to value
Balance speed, cost, quality and risk
increased capacity to innovate
Balance speed, cost, quality and risk
increased capacity to innovate
Reduce time to customer feedback
improved customer experience
Reduce time to customer feedback
improved customer experience
IBM DevOps
Continuous Release and Deployment
ContinuousCustomer
Feedback and Optimization
Monitor& Optimize
Develop& Test
Release& Deploy
Plan& Measure
Continuousinnovation,
feedback and improvements
ContinuousMonitoring
Collaborative Development
ContinuousBusiness Planning
ContinuousTesting
Systems of Engagement (SoE) Apps
RapidReleases AppStore
Monitorand Optimize
Releaseand Deploy
Developand Test
Inte
gra
tio
n T
est
12
Monitorand Optimize
Developand Test
Web Apps
FrequentReleases
ProductionEnvironment
Databases
Systems of Record (SoR) AppsFewer
Releases Databases
The need: Integrate systems of engagement with systems of record
By bringing together the culture, processes, and tools across the entire software delivery lifecycle – spanning mobile to mainframe platforms
A Lean View of DevOps
Idea/Feature/Bug Fix/Enhancement
Production
Development Build QA SIT UAT Prod
PMO
Requirements/Analyst
Developer
CustomersLine of Business
BuildEngineer
QA Team Integration Tester User/Tester Operations
Artifact Repository
Deployment Engineer
Release Management
Code Repository
Deploy
Get Feedback
Infrastructure as Code/Cloud Patterns
Feedback
Customer or Customer Surrogate
Metrics - Reporting/Dashboarding
Tasks
Artifacts
Demands on IT
April 7, 202314
Limited amount of features and changes being deployed to keep pace with rapidly evolving business requirements
NEW demand from business and technology
BAU, Maintenance, Break Fix, Lights On
Deliver systems of interaction for a consistent user experience across the
enterprise
Cloud
Customer Experience
Quality
Agility
Simplicity
Speed
DevOps
April 7, 202315
Collaboration
Communications
Standards
AGILE
•Bridges not Silos•S.M.A.R.T.•Risk Management
•Change and Release Management Policies and Procedures
•Architecture
•Continuous Integration•Continuous Delivery
• Transform Legacy processes• Simplify workflows• Automate!• provide agility to keep pace• Audit and Compliance• Pre-approved Routine Changes
• Service Catalog – Rapid Deployment and provisioning
• Increase frequency of changes• Systems of Interaction/Innovation vs.
Systems Record• smaller set of changes vs. large scale
periodic releases
• Do the things right vs. Do the right things• Tolerance of Risk for Development and
Production• Engagement oriented teams• Specific Measurable Attainable Realistic
Timely projects
Agenda
• What is DevOps?• Adopting DevOps
o Peopleo Processo Technology
• Implementing Delivery pipelines
17
• Common Business Objectives
• Vision Statement
• Common measures of Success
• Building a DevOps Culture
• There is no Silver Bullet
• Right People are needed
• Everyone is responsible for Delivery
Product Owner
Team Member
Team Lead
Team Member
Team Member
Senior Executives
UsersDomain Experts
Auditors
Gold Owner
Support Staff
External System Team
OperationsStaff
Adopting DevOps in the Enterprise: People/Culture
• Organizational Change
‘Shift Left’ – Operational Concerns‘Shift Left’ – Operational Concerns
Adopting DevOps in the Enterprise: People/Culture
• Do more upstream • App and infrastructure configuration management database • Dynamic Provisioning and Automated deployments • Automated testing – Test earlier and get feedback earlier• User testing early in QA and Staging. Not just Production.• Create test plans and test cases in Requirements Phase
Agenda
• What is DevOps?• Adopting DevOps
o Peopleo Processo Technology
• Implementing Delivery pipelines
Orchestration
End-to-End Software Delivery Process
Provision Hardware
•Virtual server configuration (S,M,L)
•Operating System version
Provision Server
Instrumentation
•Metering software
•Monitoring software
Provision Security
•Groups
•Service accounts
•User accounts
Provision Software
•Provision Application Enabling
•Software (AES)
•Configure AES software
•Archive and Catalog pattern image for reuse
Produce App. Build
•Build application version
•Create install package
•Create application configuration scripts
Deploy App. Build
•Install package
•Run application configuration scripts
Provision App. Data
•Acquire application data and archive
•Deploy seeded data
•Run data configuration scripts (if any)
I&O Processes
I&O Processes w/ BU IT configuration/settingsBU IT Processes
21
• DevOps as a Business Process• A Process to get Capabilities from Ideation to Value
• Apply Lean Thinking to Processes
Adopting DevOps in the Enterprise: Process
DevOps Solution: Adoption paths and key capabilities
Deployment
Provisioning
Release / Deploy
Develop /Test
Monitor / Optimize
Monitoring
Customer Feedback
Code
Test
Portfolio Management
Requirements
Plan /Measure
Change & Configuration Management
Dashboards/Analytics
Business Owner
PlatformOperations
ServiceDeveloper
Collaboration
TargetCustomer
Jazz, OSLC and Open Standards Platform
Where do you start: DevOps Adoption Roadmap
What am I trying to achieve?
• Think through business-level drivers for improvement • Align vision and pains to common business drivers across
silos• Look across silos, not just within the team, for improvements
Where am I currently?
• What do you measure and currently achieve• What don’t you measure, but should to improve• What practices are well scaled vs. incubating• Refine objectives to particular practice areas
Where are my bottlenecks
and priorities?
• Business-level drivers expose practice gaps across silos• Focusing outside of the bottleneck limits overall
improvement• It’s not just about tools, its about People, Practices,
Technology, and information
Ste
p 1
Ste
p 2
Ste
p 3
Current PracticeAssessment
Objective & Prioritized Capabilities
Business Goal Determination
Determine Activities Objective
How should I plan my practice
improvement?
Ste
p 4
• Identify improvements to skills, processes, and tools to achieve desired outcome
• Roadmap activity to define actionable plan• Target improvements which get the best bang for the buck
Roadmap
February 26, 2014
23
Understanding Maturity Levels
PracticedSome teams exercise activities associated with the practice, inconsistently. No enterprise standards defined. Automation may be in place but without consistent usage models.
ConsistentEnterprise standards for practice are defined. Some teams exercise activities associated with the practice and follow the standards. No core team or COE to assist with practice adoption. Automation, if used, follows enterprise standards.
Reliable Mechanisms exist to assist adoption and ensure that standards are being followed. Core team of mentors available to assist in adoption.
ScaledInstitutionalized adoption across the enterprise. COE is a matured and integral part of continuous improvement and enablement. Practices are mainstreamed across the enterprise. Feedback process in place to improve the standards.
Specific maturity levels are defined by how well an organization can perform practices. The levels look at consistency, standardization, usage models, defined practices, mentor team or center of excellence, automation, continuous improvement and organizational or technical change management.
February 26, 2014
24
Plan / Measure Development / Test Release / Deploy Monitor / Optimize
Scaled
Reliable
Repeatable
Practiced
Practice Based Maturity Model
Define release with business objectives
Measure to customer value
Define release with business objectives
Measure to customer value
Optimize applicationsUse enterprise issue resolution procedures
Optimize applicationsUse enterprise issue resolution procedures
Standardize and automate cross-enterprise
Automate patterns-based provision and deploy
Standardize and automate cross-enterprise
Automate patterns-based provision and deploy
Manage data and virtualize services for test
Deliver and integrate continuously
Manage data and virtualize services for test
Deliver and integrate continuously
Link objectives to releasesCentralize Requirements
ManagementMeasure to project metrics
Link objectives to releasesCentralize Requirements
ManagementMeasure to project metrics
Link lifecycle information Deliver and build with testCentralize management
and automated test
Link lifecycle information Deliver and build with testCentralize management
and automated test
Plan departmental releases and automate status
Automated deployment with standard topologies
Plan departmental releases and automate status
Automated deployment with standard topologies
Document objectives locallyManage department
resources
Document objectives locallyManage department
resources
Manage Lifecycle artifactsSchedule SCM integrations
and automated builds Test following construction
Manage Lifecycle artifactsSchedule SCM integrations
and automated builds Test following construction
Plan and manage releases Standardize deployments
Plan and manage releases Standardize deployments
Monitor resources consistently
Collaborate Dev/Ops informally
Monitor resources consistently
Collaborate Dev/Ops informally
Plan and source strategically
Dashboard portfolio measures
Plan and source strategically
Dashboard portfolio measures
Monitor using business and end user context
Centralize event notification and incident resolution
Monitor using business and end user context
Centralize event notification and incident resolution
Automate problem isolation and issue resolution
Optimize to customer KPIs continuously
Automate problem isolation and issue resolution
Optimize to customer KPIs continuously
Improve continuously with development intelligence
Test Continuously
Improve continuously with development intelligence
Test Continuously
Manage environments through automation
Provide self-service build, provision and deploy
Manage environments through automation
Provide self-service build, provision and deploy
Area of primary focus
February 26, 2014
25
We are
Here
Key Capabilities
1. Collaborative Development & Continuous
Integration
2. Continuous Business Planning
3. Continuous Release and Deploy
4. Continuous Testing
5. Continuous Feedback
Adopting DevOps in the Enterprise: Process
Region Configuration – To Be Environment
27
Agenda
• What is DevOps?• Adopting DevOps
o Peopleo Processo Technology
• Implementing Delivery pipelines
1. Collaborative Development and Continuous Integration
http://bit.ly/PRQ4a7
Mobile App Development
Teams
Enterprise Services
Development Teams
Adopting DevOps in the Enterprise: Process
Deployment Automation
http://bit.ly/1cr40TB
What one deploys may be anything from simple
configuration changes; to incremental code changes
towards a new feature; to Database schema changes; to
changes to the environment; to the whole stack.
Not just the application• Application Components• Middleware Configurations• Database Changes• Environment Configurations• Orchestrating the deployment
process
Investing in Release Automation … Introducing UrbanCode
Drive down cost
Remove manual effort and wasted resource time with push button deployment processes
Speed time to market
Simple, graphical process designer, with built-in actions to quickly create deployment automation
Reduce risk
Robust configuration management, coordinated release processes, audits, and traceability
Enabling clients to more rapidly deliver mobile, cloud, big data and traditional applications with high quality and low risk
IBM UrbanCode Deploy automates the deployment of applications, databases and configurations into development, test and production environments, helping to drive down cost, speed time to market with
reduced risk.
IBM UrbanCode Release is an intelligent collaboration release management solution that replaces error-prone manual spreadsheets
and streamlines release activities for application and infrastructure changes.
https://www.ibmdw.net/urbancode/
32
Continuous Integration Architecture – Current State
Jenkins 1.5 server
Artifactory Server 3.0
Drop-Off
PVCS Server
Source Repository(SVN Server)
1 Devloper Checkin in SVN
Server
2 Pull code From SVN and Display Result
3 Build source , create artifact(WAR , EAR ,
Jar , etc)
5 . Check in file in PVCS
7. Copy file to Drop Off
ACME Server
Portal Server
33
Provisioning Manager – Process Orchestration
Network
VIRTUALIZATION
CPU MEMORY I/O & NETWORK
AES
OS
Application
VIRTUAL SERVER
AES
OS
VIRTUAL SERVER
2b. Deploy Pattern
1b. Capture Pattern
Provisioning Manager
ConfigurationDatabase
(Regions, Tiers, Applications)
PatternRepository
Deploy Application
Application Content (Data)
Reporsitory
Application Build packages
Repository
Build Master Process
Process to clone DB Datafiles
Application Buildn,n+1,n-1revisoins
SeededApplication
Data
AD Security Provisioning
AESOS
AESOS
1a. Capture & Catalog Pattern Request
2a. Deploy Pattern Request
3a. Provision Security
4a. Deploy application build
5a. Deploy application data Source CodeLibrary Management
ApplicationDatabases
Agenda
• What is DevOps?• Adopting DevOps
o Peopleo Processo Technology
• Implementing Delivery pipelines
Implementing a DevOps toolchain
SCM
Build / CI Server
Unit testingTest
AutomationTest Stubbing
Delivery Pipeline
Environment Configuration
Automated Monitoring
Asset Repository
April 7, 202336
Continuous Delivery
• Create a repeatable, reliable process for releasing software
• Automate almost everything• Keep everything in version control• If it hurts, do it more frequently, and
bring the pain forward• Instrument and Audit Everything• Test Everything• Everybody is responsible for the delivery
process• Continuous improvement• Dynamic Infrastructure supports
continuous delivery
36
Source: “Continuous Delivery: Reliable Software Releases Through Build, Test and Deployment Automation”
Application Artifacts
ApplicationCode
Instrumentationand
Configuration
Runtime Environment
DefinitionVersion
Automate
Test
Track and Plan
Instrument and Audit
DashboardEnvironment
ApplicationConfiguration
Continuous Delivery Pipeline
PVCSSVN
Artifactory
SplunkJenkins MavenAnt
Puppetchef
HP ALMPolicy TesterSelenium
Smart Cloud Control DeskPMGServiceNow
Microsoft Team Foundation ServerSerena Dimensions
Improve Change and Release Management4-8-12-16-20 hours…
Testing is 1/3 of our release time
Lack of automation
Too many hand offs
38
More environments Test earlier Automate Process Optimization Monitoring Continuous Service Improvement
Testers
Developers
MiddleWare AdminsDBA
Ops
Users
Release Manager
App Managers
Are you done yet?! Why is it taking so long?!
Where are we with the release?!When can I test?!
Let’s fix it in production!
New “DevOps” team
39
Application
EngineeringOperations & Release
Management
Infrastructure EngineeringMiddleware
Web, Security, Application Server
Architecture
Automation and Tooling
DEV OPS
BU Aligned
DevOps Tiger Teams
40
QA, TEST STRATEGY & PLANNING
DEVELOPMENT BUILD
PROCESS
RELEASE & CHANGE
GOVERNANCE & BUSINESS OUTCOMES
CULTURE
ELASTIC INFRASTRUCTUR
E
• Automated monitoring & dash-boarding inclusive of business drivers & events
• Implementation of application enabling software
• Process for determining Capacity Planning and Capacity/Event Monitoring
• Future scalability and how to measure
• Orchestration process • Virtualization strategy
across applications
• Automation Strategy and Scope • Data Management• Baseline data for each application• After image baseline for each application• Test case transaction for each application.
• Process controls & measures
• Define Phase Gate/SDLC intersection
• Changes/Implications to costing models based on dynamic virtualization
• Knowledge Management
• Define how to achieve the management and relationship between source, manifest, and executables
Questions?
We Value Your Feedback
Don’t forget to submit your Impact session and speaker feedback! Your feedback is very important to us – we use it to continually improve the conference.
Use the Conference Mobile App or the online Agenda Builder to quickly submit your survey
• Navigate to “Surveys” to see a view of surveys for sessions you’ve attended
42
Thank You
Legal Disclaimer
• © IBM Corporation and The Prudential Insurance Company of America 2014. AllRights Reserved
• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM ’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.
• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.
• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.
• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.
• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.
• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.
• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.
• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:UNIX is a registered trademark of The Open Group in the United States and other countries.
• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.
• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only.