CLOUD ORCHESTRATION
WITH
RIGHTSCALE CLOUD WORKFLOW
• Raphael Simon
• Chief Architect, RightScale
• Ryan O’Leary
• Senior Director Product, RightScale
Speakers
• Why Cloud Orchestration Matters
• Use Cases
• Overview of Cloud Workflow
• Behind the Scenes
• Demo
• Extending Cloud Workflow with Plugins
Agenda
Cloud Workflow Orchestrating the Clouds
• Complexity of cloud applications has been increasing • Micro-services, SaaS, PaaS, IaaS, containers, etc
• Orchestration is a requirement for the entire application
lifecycle • Must be code-based (no manual steps)
• From provisioning to operational actions and maintenance to
shutdown/cleanup
• Orchestration technology must be vendor-agnostic • Agility to use new services and optimize targets is a requirement
Introduction
Use Cases Why Cloud Workflow
• Resource dependency
• Sequential/concurrent actions
• Dealing with failures/errors
• Provision “N” number of a given resource
Orchestrated Provisioning
Complex applications require multi-service
orchestration to launch and configure correctly
• Environment-specific policies
• Complex quotas (user/group, core/memory, cost)
• Multi-resource policy definitions
• Multiple policy levels, including automated and man-in-the-
loop
Smart Policies
Cloud usage must be governed to ensure compliance
and minimize waste
• Automated runbook execution for dealing with known
problems
• Resolution attempts with multiple escalation paths
• Correlating alert information from multiple sources for more
informed responses
Contextual Alert Actions
Increase operational efficiency by automating alert
actions
• Scale relative to the event
• Scale down intelligently, removing the ideal nodes for the
application
• Scale up cost effectively, leveraging spot and/or discounted
instance types
Optimized Array Scaling
Save cost and improve performance by reacting
appropriately to load/usage events
• Runbook automation for regularly performed maintenance
tasks
• Include dependencies on other parts of the
application/environment
• Notify stakeholders via multiple channels and trigger other
actions in case of unexpected behavior
Operational Scheduled Actions
Automate regular maintenance tasks with visibility and
reporting
Overview What is Cloud Workflow?
● Goal: robust orchestration of cloud services
● Mean: code centered around the concept of resources ○ RightScale resources (servers, volumes, networks, applications etc.)
○ Arbitrary service resources via plugins (UPCOMING)
● Code in RCL (RightScale Cloud Workflow Language ○ Built-in functions to control resources lifetime (provision and delete)
○ Resource specific actions that make requests to arbitrary APIs
○ Scoping, variables, control flow, error handling, concurrency ...
Overview
Example
@deployment = rs_cm.deployment.create(name: “CWF rocks!”) @deployment.lock() # make API request to “lock” action delete(@deployment) # delete destroys the resource
API resource action parameters
Where does it fit?
Cloud
Workflow
Cloud Templates (CloudFormation, Azure
Templates)
Config. Mgmt (Puppet, Chef)
Client Tools (Terraform, ad-hoc
scripts)
Code type Imperative Declarative Mix Depends
Error Handling Explicit Implicit Implicit Depends
Multicloud? Yes No Yes Yes (w/ caveats)
Scope Systems Systems Servers Systems
Execution Hosted Hosted Mix Client / CI
Failure Modes Catastrophic Catastrophic SPOF SPOF
Introspection Full Basic No Basic
● Script like syntax
● Robust error handling
● Built-in timeout support
● Simple and explicit concurrency control
● Functions for making arbitrary HTTP requests
● Extensible via plugins
● Can be used together with Self-Service Cloud Applications
(CATs) to mix declarative with imperative code
Cloud Workflow Benefits
Error Handling
01 define main() do
02 @server = rs_cm.get(href: "/api/servers/123")
03 $attempts = 0
04
05 sub on_error: rescue($attempts) do
06 $attempts = $attempts + 1
07 @instance = @server.launch()
08 end
09 end
10
11 define rescue($attempts) do
12 if $attempts < 3
13 $_error_behavior = "retry"
14 end
15 end
Timeout
01 define main() do
02 @server = rs_cm.get(href: "/api/servers/123")
03 $atts = 0
04
05 sub timeout: 30s, on_timeout: rescue($atts), on_error:
rescue($atts) do
06 $atts = $atts + 1
07 @instance = @server.launch()
08 end
09 end
10
11 define rescue($attempts) do
12 if $attempts < 3
13 $_error_behavior = "retry"
14 end
15 end
Concurrency
1 define launch($serv1, $serv2, $serv3) return @inst1, @inst2,
@inst3 do
2 concurrent do
3 @inst1 = provision($server1) # blocks until @inst1 is
operational
4 @inst2 = provision($server2)
5 end
6 @inst3 = @serv3.launch()
7 end
Making HTTP requests
1 define random_text() return $body do
2 $resp = http_get(url: "http://loripsum.net/api")
3 $body = $resp["body"]["value"] # $resp contains the parsed
JSON
4 end
Behind the Scenes Cloud Workflow
Process Execution Flow
RCL Cloud
Workflow
External
API
Parse
Store
AST
Run
one expression Cloud
Workflow
Load / Save
one expression
No external state affected until step #4 ⇒ step #2 can be retried idempotently
This algorithm guarantees that the state of a process is
always consistent even when system failures occur.
Process Execution Algorithm
1. Load expression
2. Evaluate
3. Save expression
4. Apply evaluation outcome: make API requests, sleep,
raise error ...
5. Repeat until all expressions have completed
● A single process can run a number of concurrent tasks
● Code is not run concurrently, API requests are made concurrently
● Only one expression runs at any given time in the scope of a process to
guarantee that the state remains consistent even when system failures
occur.
Use concurrency to run flows concurrently, think workflow split and join
Process Concurrency
Parent
task
Parent
task
exp exp
exp exp exp
concurrent
Process Concurrency (2)
Parent Parent
activity (API requests)
T 1 T 2 T 1
activity
T 2
Concurrent execution
Demo
define main() do
task_label("Creating deployment")
@deployment = rs_cm.deployments.create(deployment: {"name": "CWF webinar"})
task_label("Waiting for deployment name to change to 'GO'")
sleep_until(@deployment.name == "GO")
task_label("Attempting to delete deployment ")
delete(@deployment)
task_label("Done!")
end
define main() do
task_label("Creating deployment")
@deployment = rs_cm.deployments.create(deployment: {"name": "CWF webinar"})
task_label("Waiting for deployment name to change to 'GO'")
sleep_until(@deployment.name == "GO")
$attempt = 1
sub on_error: handle_error(@deployment) do
task_label("Attempting to delete deployment (attempt " + $attempt + ")")
$attempt = $attempt + 1
delete(@deployment)
sleep(5)
end
task_label("Done!")
end
define handle_error(@deployment) do
task_label("Unlocking deployment")
@deployment.unlock()
sleep(5)
$_error_behavior = "retry"
end
Plugins Extending Cloud Workflow
RCL has the following built-in namespaces (a.k.a. APIs):
● rs_cm: RightScale Cloud Management APIs
● rs_ss: RightScale Self-Service APIs
● rs_ca: RightScale Cloud Analytics APIs (UPCOMING)
RCL also supports making HTTP requests to arbitrary
endpoints via the http_xxx functions.
Built-in Support
Describe resources hosted in external services, including:
● List of fields required to create resource
● List of output fields contained in created resource
● JMESPath expression for extracting output fields from responses
● List of actions supported by the resource (HTTP endpoints and
mapping of resource output fields to HTTP request)
● RCL definitions for provisioning and deleting resources
Plugins
Plugins - Example
plugin "digo" do
type "droplet" do
provision "provd"
delete "deld"
field "name" do
type "string", required: true
end
field "api_version" do
type "string"
location "header"
end
output_path "droplet"
output "id", "name"
...
...
action "show", "create",
"destroy"
action "kernels" do
verb "GET"
path "$href/kernels"
end
end
end
define provd($droplet) return @droplet
do
@droplet =
digo.droplet.create($droplet)
sleep_until(@droplet.status ==
"active")
end
define deld(@droplet) do
@droplet.detroy()
end
Plugins - Usage
define launch($droplet) return @droplet do
@droplet = provision($droplet)
@action = digo.actions.create(type: "enable_ipv6")
sleep_until(@action.status == "completed")
end
define main() return $name do
$fields = { "name" : "my droplet" }
$droplet = { "namespace": "digo", "type": "droplet", "fields":
$fields }
@droplet = launch($droplet)
$name = @droplet.name
end
• Need help optimizing cloud costs?
• Learn More • Plugin Webinar: https://www.rightscale.com/webinars (search “plugins”)
Q&A
