how we do devops at walmart: oneops oss application lifecycle management platform
TRANSCRIPT
DevOps in PracticeQINGSONG ZHANG, PH. D.
Something related to me
Ph. D.
Sr. I T Architect
Software Manager
Sr. DevOps Staff
Sr. DevOps Manager
Outline• Internet IoT• Information Data• DevOps DataOps
• Why DevOps• How DevOps• DevOps in Walmart• From DevOps to DataOps
• How DataOps
IoT50
petabytes
The Internet of Things (IoT) is a system of interrelated computing devices, mechanical and digital machines, objects, animals or people that are provided with unique identifiersand the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction.
Connect, Monitor, Search, Manage, Control, Play
4000Exabyte
Service / Informati
on Intelligent/Connected SystemsData service component in smart products driving increased value and differentiation
Big DataInsights on new products by more efficiently interpreting massive quantities of data
CloudDemand for apps requires fast, scalable environments for dev and test, as well as production
Instrumented ProductsIndustry requirements demand faster response to regulations and standards, with traceability and quality
Social BusinessBroader set of stakeholders collaborates to deliver continuous innovation and value
MobileModern workforce expects constantlyupdated data toconnect to enterprise systems
Data
What is DevOps
•Culture• Respect & Trust• Embrace Change• Transparent
•Automation• Continuous integration (CI) and delivery (CD)• Infrastructure as code
•Lean• Focus on producing value for end user• Small batch size
•Measurement (Metrics)• Measure everything• Show the improvement
•Sharing• Open information sharing• Collaboration & communication
Why do we need DevOps• Technical benefits:
• Continuous software delivery• Less complex problems to fix• Faster resolution of problems
• Business benefits:• Faster delivery of features• More stable operating environments• More time available to add value
(rather than fix/maintain)
How to DevOps• Collaborative Development
• Foster productive collaboration• lifecycle integrations• Development on the cloud for continuous
delivery• Continuous Testing
• Enhanced integrations and capabilities to synchronize software testing with deployment and operations
• Continuous Release and Deployment• Greater delivery speed and frequency for
complex applications
• Continuous Monitoring• Capabilities to improve service quality by
monitoring application performance
Agile development approach• It spans the entire lifecycle, includes business planning and creation to delivery and feedback.• Enable continuous delivery of software‐driven processes and innovation
Web Server
App Server
Database
Internet/FirewallEnd Users Lan/network Web front-end Services Middleware
Yesterday Waterfall development Weeks to provision Static resources Limited change windows $100k+ of compute Limited number of users, in
business hours
Now/Tomorrow Agile development Continuous deployment Dynamic resourcing Spans across data centers and
clouds Friction of compute costs Millions of users with 24x7 access Private
Public
Public
Web Server
Message Q
Database
In Memory Cache
App Server
App Server
What about DataOps… Data is not only considered as an Asset FOR COMPETITIVE ADVANTAGE; but now a Strategic Asset for COMPETITIVE SURVIVAL …
Chief Information Officer
Line of Business Executive
Chief Data Officer
Chief Analytics
Officer
Compliance
Marketing
Risk
Chief DigitalOfficer
Chief Information Officer
Devlopment(Software
Engineering)
Technology Operation
Infrastructure
Quality Assurance
DevOps
Definition:Collaboration between development, operations and quality assurance engineering across the entire application delivery pipeline --- from design and coding to testing and production support
Integration
Chief Data
Officer
Acquisition
PrivacyLegal
Security
Quality
DataOps
Definition:A data management method that emphasize communication, collaboration, integration and automation of process between data engineers, data scientists and other data professionals
• Mission:To enable system developers and managers to handle modern, web-based application development and deployment.Infrastructure requirements:Need to support the quantity, performance demands and diversity of systems available to a modern enterprise --- from on-premise servers to hybrid clouds and container environments.
• Mission:To enable data professionals to optimize for modern, web-based data storage and analytics.Infrastructure requirements:Needs to support highly scalable, highly available and diverse types of data and data storage environments.
Top 10 DevOps Companies
Mission StatementTo help people save money so they can
live better
GoalBecoming an
international brand
Vison“If we work together, we’ll lower the cost of living for everyone … we’ll give the world an opportunity to see what it’s like to save and have a better life.”
SloganSave money, Live
better
• Founded in 1962 by Sam Walton
• Ranked 1st by Fortune 500• $466 Billion sales• 2.2M associates• 11,500 stores in 28
countries
• E-Commerce in 11 countries
• 198M users @20M per day• 3417 Application in
production• ~ 1000 Deployments per
day• Share the
responsibilities and serve the customers
• It’s our problem!
• Founded in 2011, acquired by Walmart in 2013
• A new open source DevOps platform for cloud and application lifecycle management (ALM), which was developed by Walmart Labs.
• Works with multiple public and private cloud platforms out of the box, including Microsoft Azure, Rackspace, AWS and CenturyLink Cloud, as well as any OpenStack clouds.
• Works with RDMS and NoSQL Databases, Node.js, Electrode, Docker, ElasticSearch and many others.
• Includes monitoring tools, auto-healing, auto-replacing and auto-scaling.
• Integrated with enterprise identity services, quota management and configuration management systems.
• Released to public at the beginning of 2016
OneOps• OpenStack
• 170,000+ cores• 30+ cloud regions
• OneOps• 5000+ users• 3000+ applications/services• 60+ open source products• 40,000+ monthly deployment• 100,000+ monthly auto-repairs• 1000+ monthly auto-replace
Technologies:Ruby, Chef,Tomcat,Postgres, LogStash, ActiveMq,Cassandra, ElastiSearch,Etc.
Minimum Requirement:
8 GB Memory2 CPU
Sign up
Create Organizat
ion
Create
Cloud
Choose
Cloud
Cloud Servic
e
Cloud Connecti
ons
Assembly
Diagram
Transition
Deploy Plan
Monitor
Repair and
Replace
Database
Monitor