a new paradigm for cloud: architecting control ... new paradigm for cloud: architecting control ...
Post on 06-Feb-2018
Embed Size (px)
2017 Fugue Inc. // www.fugue.co
A New Paradigm for Cloud: Architecting Control & Resilience at Scale with Fugue
W H I T E PA P E R
When you decide to migrate to the cloud, use a hybrid cloud
model, or architect from the start in the cloud, youre thinking
about your systems designthe infrastructure choices
youve made and will make to support application integrity.
Known problem spaces present thorny questions and partial,
often convoluted, answers for every cloud architect making
these choices. Mistakes are expensive. The path to a resilient
architecture is a labyrinth. Architecture needs to be flexible
enough to adapt to rapid, ongoing change but characterized by
a sustainable core design and by time-tested cloud principles.
A New Paradigm for Cloud: Architecting Control & Resilience at Scale with Fugue 2 / 14A New Paradigm for Cloud: Architecting Control & Resilience at Scale with Fugue
The specific challenges are familiar. Youve got to plan for a dynamic operations environment, where scaling pain isnt just about horizontal and vertical capacity. Right now and into the next decade, its about scaling smartly amid tool, service, and component complexity, where these tools, services, and components change frequently and unpredictably. Youve got to establish an ecosystem that accommodates those continuous infrastructure transitions and user growth. Youve got to future proof by making software that lasts because it implements best cloud practices and can be renewed and refreshed easily. Youve got to know infrastructure state with confidence and accuracy in order to make cost-effective decisions as you continue to tune and develop apps or respond to changing market demands. Youve got to ensure resilience against misconfiguration, drift, unexpected interference, and outage. Youve got to manage security responsibilities and compliance requirements in an automated, reproducible manner. Youve got a lot on your mind.
This paper zeroes in on cloud architecting pain points and demonstrates that Fugue is a system that responds comprehensivelynot in piecemeal, point-solution fashion. Fugue takes a new approach to infrastructure orchestration for enterpriseswith a typesafe, compiled language, a process model, and an enforcement enginethat reduces cloud footprint and drives application performance.
With Fugue, security isnt merely a bullet point in a list of features, but a principle upon which system design is centered. Controlled access to shared resources is critical, especially when multiple operators and accounts are involved. Policy as code through typesafe, compiled, language libraries; secure collaboration through role-based access controls; and fortified permissions through replicable and auditable language abstractionsall form layers of security for enterprises operating at scale in the cloud.
SEE FIGURE 1: An Overview of Fugue
Orchestrating Complexity at ScaleConsider the evolution and proliferation of cloud services and their relentless pace of development. Cloud providers offer improved and new options ev-ery year, sometimes multiple times a year, as do third party vendors. Some services are more successful than others. Most lend themselves to elastic scaling, with parameters that may require attention.
Horizontal and Vertical Growth
A common, elastic service is Amazon Web Services Auto Scaling Groups (ASGs), which easily automates horizontal scaling based on demand within limits a user can specify. Vertical scaling, on the other hand, requires a bit more in terms of manual intervention. With ASGs, you can create several launch configurations that specify different instance sizes, addressing your vertical needs, and perhaps build a Lambda function that updates the launch configuration. Its trickier if youre using reserved instances, as you may be stuck in the same instance family. And, when you resize instances in Amazon Web Services (AWS) in any scenario, there are a series of compatibility issues, considerations with attaching and detaching volumes, as well as manual stop/suspend/restart actions to keep in mind. The cloud architect has to think about those vertical scaling intricacies within enterprise systems. Theyre entirely doable, but they have a time-consuming, manual cost. You can be left wondering whether youre saving money by utilizing only whats needed or whether you have overemployed instance sizing to account for spikes, as happens routinely with servers in traditional data centers. Estimating should be a science fed by sound, abundant data, but it often ends up feeling like an art.
Tool, Component, and Services Growth
Tackling scaling problems comprehensively requires that we think about these typical modes
3 / 142017 Fugue Inc. // www.fugue.co
of scaling within the context of clouds proliferation of services. Amid a software release schedule of your own thats likely aggressive, youre making relatively straightforward choices about horizontal scaling and more complicated choices about vertical scaling in an environment of increasing, ubiquitous, overlapping services and the constant availability of new point solutions, some of which your developers are actively using. Youre concerned about which members of your team should manage each potential environment. These factors combined create complexity at scale. Thats the reality in the cloud that isnt easy to orchestrate. Those point solutions tend to tackle a specific problem in your cloud build stage or in your cloud monitoring or in your cloud management, but not in all of those at once.
Fugue Tackles Complexity
In this complicated context, Fugue approaches scaling realities holistically, taking advantage of the fact that cloud services are programmable resources. You can code cloud services succinctly with Fugues easy-to-learn language, Ludwig, using compositions to define architectural patterns and libraries already formulated to best practices, but also able to be customized to your systems needs in minutes. You can collapse an elaborate operational sequence into a simple function. You can import an elaborate operational pattern in a single statement. You can author a simple policy to define access and authorization throughout your entire organization. You can get helpful compiler error messages without spending a dime on provisioningmessages that teach you how to better tweak infrastructure configuration. You can run, update, terminate, and replicate your architecture-in-action across multiple AWS accounts with easy, logical commands. By these means, you orchestrate complexity at scale.
For example, with CloudWatch and SNS expressions in Ludwig, you can ensure a notification when your apps CPU utilization passes a mark, then you can update ASG instance parameters, like group size (horizontal scaling) and instance type (vertical scaling) with a Ludwig function, while at the same time swapping out or modifying other loosely coupled services, if desired, in moments. Then, you can replicate those changes globally via Ludwig
abstractions, with the assurance that your scaled infrastructure is exactly what youve specifiedno lingering configurations, no uncertainty.
Immutability Streamlines Complexity
Fugue employs immutable patterns across its interactions with your application infrastructure. It takes a view of cloud services as disposable resources and acts on them accordingly. Doing so, where its practical, is one of Fugues essential benefits. For example, using the `fugue update` command updates the infrastructure of a running process connected to version one of a composition to match the infrastructure specified in version two of a composition. Updates may be destructive or mutative. A mutative update is an update-in-place for resource properties that can be changed (e.g., VPC tags). A destructive update is one in which the resource properties cannot be changed, so the entire resource is destroyed and replaced with a new one (e.g., an entire ASG with t2.medium instances updated to m4.xlarge instances or a VPC CIDR block replaced with another). This strategy of resource replacementwhere it makes senseis immutable infrastructure in action. Immutable patterns automatically integrated into your system, via another Fugue mechanism, enforcement (discussed in the next section), ultimately optimize your costs and the integrity of your applications deployment. Cruft buildup and uncertainty are eliminated with clean, immutable destruction and replacement cycles that provide clarity when you orchestrate changes to your system at scale.
The same benefits of immutability hold true when replicating whole architectural patterns for development, testing, and production environments, for disaster recovery purposes as well explore shortly, and for enterprise-scale, geographically diverse deployments. When you assign cleanly abstracted configurations of numerous cloud services in a Ludwig file and then easily convert the given pattern into a Ludwig module, that module can be imported in one line into other Ludwig files. You can swap different modules in and out of composition versions and test them. Youre not modifying infrastructure patterns in the runtime. Youre efficiently versioning your architectural
A New Paradigm for Cloud: Architecting Control & Resilience at Scale with Fugue 4 / 14A New Paradigm for Cloud: Architecting Control & Resilience at Scale with Fugue
patterns that you can test with the compiler at no actual infrastructure costs, then terminating an old pattern and running a new one, again with no in situ modifications. Destroy and replace.
PART I I
Knowing Infrastructure State with CertaintyImagine the operations control center in a giant power pla