brighttalk understanding the promise of sde - final
TRANSCRIPT
Understanding the Promise of the Cloud
Mr. White has fifteen years of experience designing and managing the deployment of Systems Monitoring and Event Management software. Prior to joining IBM, Mr. White held various positions including the leader of the Monitoring and Event Management organization of a Fortune 100 company and developing solutions as a consultant for a wide variety of organizations, including the Mexican Secretaría de Hacienda y Crédito Público, Telmex, Wal-Mart of Mexico, JP Morgan Chase, Nationwide Insurance and the US Navy Facilities and Engineering Command.
Andrew White Cloud and Smarter Infrastructure Solution Specialist IBM Corporation
http://weheartit.com/entry/12433848!
Ground rules for this session… • If you can’t tell if I am trying to be funny…
– GO AHEAD AND LAUGH! • Feel free to text, tweet, yammer, or whatever.
Use • If you have a question, no need to wait until
the end. Just interrupt me. Seriously… I don’t mind.
Why are we here?
I am going to share some of what I have learned about
Software Defined,
Continuous Integration,
& Process Management
What is “Software-Defined?”
According to Enterprise Management Associates…
“Software-Defined” serves to “abstract app/service design and delivery away from the details of the hosting/delivery technologies.
This is delivered by making use of technical enablers including virtualization and programmability (API’s)
This approach drives Service-Aligned IT and allows for more flexible applications
Preference is given to open solutions that shift control from hardware to software and leave single purpose appliances for flexible capabilities managed from a central location
The 4 key principles for AT&T:
Domain 2.0
Open – APIs are the
perfect tool
Simple – More
common infrastructure
Scalable – Supports growth
Secure – Protect the Control Plane
Source: John Donovan, Senior Executive VP AT&T, Keynote Presenter at Open Networking Summit 2014, 4 MAR 2014
What is driving this move? Three Trends
Vitrualization Utilization and operation Cloud Computing Building blocks (compute, network, and
storage) with economies of scale Internet of Things Home Cinema, Connected Car
Two Industry Initiatives Software Defined Environments New architectural approach, Open Network
Foundation - OpenFlow/OpenDaylight, Open Data Center Alliance
Network Function Virtualization New architectural approach, leaving dedicated hardware for VMs, ETSI
Three Implications Network Cloud Lower cost, simplified operations, flexibility Use Cases CDN, video on demand, home automation Industry Status Wide support for ONF and NVF
Increasing Complexity
§ Heterogeneous environments § Organizational silos § Skill gaps
Massive Scale
§ Users, transactions, data § Rapid demand cycles § Unpredictable
Rapid Pace
§ Evolving ecosystem § Minimize time to value § Accelerating business needs
Today’s IT infrastructures are too complex, provide poor scalability, and are slow to keep up with today’s rapid rate of change
A new set of challenges
V1 V2 V3 V4 V5 V5 ... …. Vn
C C
W1 W2 W3 W4
R1 R2 R3
Traditional (Systems of Record)
Emerging (Systems of Interaction)
Workload View
Future
§ Rapidly changing workloads, dynamic patterns
§ Dynamic automatic composition of heterogeneous system
§ Autonomic and proactive management
Current
§ Diverse workload, limited patterns
§ Homogeneous resource pooling
§ Expert configuration and mapping of workload
Traditional
§ Few, stable, and well known workloads
§ Fixed System hardware, manual scaling
§ Hardwired workload, minimal configuration
W1 W2 W3 W4
R1 R2 R3
Volatile workload characteristics result from changing business requirements
V1 V2 V3 V4 V5 … Vn V1 V2 V3 V4 V5 V5 ... …. Vn
C C
Workloads are volatile
SDE is an Enabler
Software Defined Environment
Cloud Environment Traditional Environment
Social and Mobile
Big Data and Analytics
Other Business
Applications Workload
Service Delivery
IT Infrastructure
Programmable, open standards-based infrastructure foundation to enable cloud, mobile and other dynamic enterprise solutions
SDE is the infrastructure approach to provide the most efficient and scalable cloud solutions
SDE improves agility of business applications and accelerates the application lifecycle through rapid change
So@ware Defined Environments provides abstracEons of workloads, services and infrastructure and an end-‐to-‐end mappings
.
Workload Abstraction Based on pattern and
functional and non-functional requirements
Resource Abstraction Semantically rich abstractions of heterogeneous
resource capabilities and system components
Mapping to resource Map requirements to potential system architectures. Proactively orchestrate
infrastructure and workload
Continuous Optimization Autonomously construct available system
architecture to optimize workload outcome
Agility
Efficiency Consumability
IMG
IMG
IMG Agile Workload Development Services
Workload Abstraction
SSD HDD
Tape
PowerVM
x86 KVM
PowerVM
x86 KVM
RDMA
Ethernet
Software Defined Compute, Network and Storage Agility, Consumability, Efficiency (ACE)
Resource Abstraction
Software Defined Environments
Continuous, Autonomous Mapping
J2EE/OLTP
Transactional
Map/Reduce Analytics
Web 2.0 Pattern
Web
Increasing Automation
SDE fully integrates IT infrastructure across resource domains to maximize utilization, ensure compliance and decrease administration costs
BEFORE AFTER
Storage Network Compute Continuous Optimization
++
++
+
+
++
++
+
+
Compute
§ IT silos and costly specialization § Slow and manual § Reactive administration
§ Fully integrated management § Rapid, repeatable and automated § Proactive administration
Policy
Policy Policy
Policy Software Defined Environment
Application Aware Policy
SDE in Action
Software Defined Networking (SDN) moves the network control plane away from the switch to the software – for improved programmability, efficiency and extensibility
Virtualized Network
OS
OS
OS
OS
SDN API
Open Flow
Open Flow
Open Flow
Software Defined Control Plane
SDN Controller & Analytics
Routing API Traffic Engineering API Flow Insertion API Firewall API
rout
ing
VPN …
mon
itorin
g
Direct Access to Physical Network
Traditional Switches
Console Based HW Configuration
rout
ing
VPN
IPS
mon
itorin
g
OS
rout
ing
VPN
IPS
mon
itorin
g
OS
rout
ing
VPN
IPS
mon
itorin
g
OS
rout
ing
VPN
IPS
mon
itorin
g
OS
Network Services
Traditional switch and router vendors being disrupted by the emerging SDN
What is the promise of Software-Defined Everything?
http://www.interestingtopics.net/storage/8eac114f16575e001ad0a35999fe2502.jpeg
AGILITY
What the CIO is hoping • Economies of Scale from End-to-End Virtualization
– Develop a truly shared infrastructure – Eliminate “vendor lock-in” – Compete on cost with 3rd party IT providers
• Break Down the Silos – Align with services and not technologies (silos of virtualization
are still silos) – Improve time to value – Reduce the number of IT specialties in the workforce
• Empower the business – Enable the self service consumption of IT services – Simplify the services being offered – Enable continuous improvement
What the architects are hoping • Cleanly separate the environment into four layers (planes):
Management, Services, Control, and Forwarding - providing the architectural underpinning to optimize each plane within the network.
• Centralize the appropriate aspects of the Management, Services and Control planes to simplify the design and lower operating costs.
• Use the Cloud for elastic scale and flexible deployment, enabling usage-based pricing to reduce time to service and correlate cost based on value.
• Create a platform for network applications, services, and integration into management systems, enabling new business solutions.
• Standardize protocols for interoperable, heterogeneous support across vendors, providing choice and lowering cost.
Reality Sets in The current environment is not ready for change
The staff is overworked, staffing levels are dropping, open req’s go unfilled to an inability to find adequate talent.
The business won’t abandon “legacy tools”
Increasing amounts of governance are established to manage the chaos
Technical debt and security risks cause incidents that distract from deployments
Architecture by Accident
The Humble Start… Meeting Demand…
The First Bottleneck…
The Second Bottleneck…
Becoming Mission Critical…
Enabling SOA… The Fun Begins…
How Did We Get Here?
Game changers 1. Increased demands for high availability and
low latency 2. The visibility gap grows 3. Market forces drive increased velocity and
volume of changes 4. Productivity losses and customer satisfaction
decreases impact the business
Broken Promises The ultimate result in the exact opposite of what the CIO initially hoped for:
• Communication failures • Security incidents • Poor performance • Compliance failure • Higher costs
Sometimes we need to recognize when we have problems to solve
Software delivery is critical to success 86%
of companies believe so/ware delivery is important or cri5cal
25%
leverage software delivery effectively today
But only…
Source: “The Software Edge: How effective software development drives competitive advantage,” IBM Institute of Business Value, March 2013
69%
outperform those who don’t
of those who leverage software
delivery today
You Gotta Have Skillz…!
Starting the journey…!
Feedback Loops Unfortunately feedback has taken on both positive and negative indications. In reality, positive feedback is not “praise” and negative feedback is not “criticism.” Positive feedback reinforces while negative feedback balances.
Profits
Productivity
Cost Cutting Reinforcing
Balancing
The Agile Value Proposition
Availability
Change Frequency Change
Size
Change Capability Change
Risk
(-)
(+) (+)
(-) (-)
Adapted From: http://www.lean4it.com/2013/05/devops-cld-part-2.html
Customer Satisfaction
Availability
Change Frequency Change
Size
Change Capability Change
Risk
(-)
(+) (+)
(-) (-)
Business Value
Business Demand
Change Backlog (+)
(+)
(+)
(+)
(-)
(+)
Adapted From: http://www.lean4it.com/2013/05/devops-cld-part-2.html
Be Careful of Good Intentions
Availability
Change Frequency Change
Size
Change Capability Change
Risk
(-)
(+) (+)
(-) (-)
Business Value
Business Demand
Change Backlog (+)
(+)
(+)
(+)
(-)
(+)
Change Process
Release Process
(+)
(+)
(-) (+)
(+)
Adapted From: http://www.lean4it.com/2013/05/devops-cld-part-2.html
Be Careful of Good Intentions
Availability
Change Frequency Change
Size
Change Capability Change
Risk
(-)
(+) (+)
(-) (-)
Business Value
Business Demand
Change Backlog (+)
(+)
(+)
(+)
(-)
(+)
Change Process
Release Process
(+)
(+)
(-) (+)
(+)
Change Automation
Adapted From: http://www.lean4it.com/2013/05/devops-cld-part-2.html
(+) (-) (-)
Organizations don’t fail because they take the wrong path, they fail because they can’t imagine a better path than the one they are on. -- Marty Neumeier
Service Orientation 1
2
3
4
5
6
Goals of Service
Orientation
Abstraction
Loose Coupling
Autonomy
Standard Services
Composability
Reusability
Divide and Conquer
36
Small Problem
Small Problem
Service A
Service B
Service C
Your Application
Enlightenment Bias: Sub-parts of a complex system are simpler and easier to manage A stable system is made from very hard and durable sub-parts
Creating Composite Applications
Composite Application
Service A
Service E
Service F
Service G Service I
Service H
Service B Service C Service D
Turning Services Into Solutions
Service Interface
Automa5on
Orchestra5on
Choreography Business Service Offering
Billing
Customer Management
Add Customer
Order Management
Assign Service to Customer
Order Fulfillment
Provisioning
Deploy Device Configure Device
Palette of library assets enable easy
workflow composition through drag and drop
Access to rich libraries (toolkits) of reusable
automation assets that enable to speed
automation creation
Rich set of actions types, flow control, data handling
primitives that simplify creation of complex
automations
Easy workflow action editing for managing: data mapping,
error recovery options, implementation details , etc.
Graphical editor for composing and
connecting workflows
Rich tooling functions to edit, version, debug,
optimize workflows
Automating Processes
Stru
ctur
ed
Activ
ities
Ba
sic A
ctivi
ties
Flow Control Parallel Processing
Miscellaneous Exception and Error Handling
Event Processing and
Timers
Data Manipulation
Message Exchange
Business Process Execution Language (BPEL)
Invoke
Reply
Receive
Assign
Scope
Pick / Select
onEvent
Sequence
Throw
Compensate
Catch
Wait
Empty
Validate
For Each
Flow
If … Else
Until
While
Or�ches�tra�tion [AWR-kuh-strey-shun]
• A central process controls everything and coordinates the execution of different operations involved in the operation
• The services do not "know” that they are involved in a composite process
• Only the central coordinator of the orchestration is aware of the desired outcome,
• The orchestration leverages explicit process definitions to operate the services in the correct order of invocation
1. the act of arranging a piece of music 2. the planning or execution of events in order to achieve a desired effect 3. The technique of arranging or manipulating, especially by means of
clever or thorough planning or maneuvering
Orchestration Illustration
Orchestrator
Web Service 1
Web Service 4
Web Service 3
Web Service 2
Cho�re�og�ra�phy [kawr-ee-OG-ruh-fee] 1. the art of composing ballets and other dances 2. the method of representing the various movements in dancing by a
system of notation 3. The arrangement or manipulation of actions leading up to an event
• Choreography does not rely on a central coordinator. • Each service knows exactly who and when to execute • Focuses on the exchange of messages and information • All services need to be aware of the business process,
operations to execute, messages to exchange, timing, etc.
Orchestration Illustration Web Service
1
Web Service 4
Web Service 3
Web Service 2
Send Receive
Invoke
Invoke
Invo
ke
Choreography vs. Orchestration • From the perspective of composing services to
execute business processes, orchestration is a more flexible paradigm and has the following advantages over choreography: – The coordination of component processes is centrally
managed by a known coordinator. – Web services can be incorporated without their being
aware that they are taking part in a larger business process.
– Alternative scenarios can be put in place in case faults occur.
Page 46
Orchestration Requirements • Event-based processing • Coordinate asynchronously between services • Correlate messages being exchanged • Provide for parallel processing • Allow for transaction roll-back • Manipulate and transform data between messaging
partners • Be able to manage long running business
transactions and activities • Have a robust mechanism for fault and error
handling
Why use an event-based orchestration engine? to have the ability to receive real-time feedback to assist its decision making processes
When decisions are not made based on information, it’s called gambling.
Environments QA PROD
Banking Application Banking Application
Banking Application
DEV
IBM UrbanCode Deploy
OpenStack Heat IBM Platform Resource Scheduler
Server Storage Network
Application "Lifecycle
Applications
Heat Orchestration Template (HOT) Heat Orchestration Template (HOT)
OpenStack Heat IBM Platform Resource Scheduler
Server Storage Network
TEST
IBM Cloud Orchestrator
Architecture on Purpose
Public Dedicated Private Traditional
IT
Application template
Infrastructure template
Hardware
Completing the journey
Define
• Review the existing architecture • Review the business outcomes • Define the end state
Prioritize
• Consolidations • Technologies to virtualize • Business processes to model and workflows to automate
Execute
• Look for early wins • Evolve incrementally • Organize the teams effectively
You have to be realistic about how fast you can mature. Iterating helps form a cultural of continuous improvements
Iterative development
Let’s keep the conversation going…
ReverendDrew!
SystemsManagementZen.Wordpress.com!
systemsmanagementzen.wordpress.com/feed/!
@SystemsMgmtZen!
ReverendDrew!
614-306-3434!
