Three Degrees of Mediation: Challenges and Lessons in building Cloud-agnostic Systems
Post on 25-Jan-2015
DESCRIPTIONThe paper presented on Cloud slam 2014 cloud computing conference. It is based on the work I have done while working as a Technology Principal and Cloud Innovation Lead at Global Technology Consulting Firm. It describes unique challenges encountered during the development of a Cloud Broker. Designed to provide seamless and uniform user experience as well as additional enterprise-level capabilities over a broad range of public and private cloud platforms, the broker had to work with a wide variety of diverse cloud APIs. And, in order to allow rapid onboarding of new target clouds, we had to design the system to operate in a generic cloud-agnostic way. We have reviewed and tried quite a few of available cloud integration technologies, both commercial and open-source as a foundation for our cloud abstraction and mediation layer. Couple seemed to be a good choice and, in theory, should have served all our needs, but in the course of development we found that the problem was far more complex than we (and the rest of the market) had anticipated. As we were building our system, we discovered that there was not just one, but 3 different levels of mediation required to build a truly cloud-agnostic solution: syntactic, semantic and idiosyncratic and even the best off-the-shelf solution will not be able to give us that last two out of the box. The final part of the presentation describes the architecture that allows to realize our goals and build the universal broker.
- 1. Copyright 2010 Accenture All Rights Reserved. Accenture, its logo, and High Performance Delivered are trademarks of Accenture. Confidential & Proprietary. Do not Distribute Copyright 2010 Accenture All Rights Reserved. Accenture, its logo, and High Performance Delivered are trademarks of Accenture. Confidential & Proprietary. Do not Distribute Three Degrees of Mediation Challenges & Lessons in Building Cloud-agnostic Systems Copyright 2014 Alex Maclinovsky All Rights Reserved. Alex Maclinovsky, Principal Engineer, Sears Holdings
2. 2Copyright 2014 Alex Maclinovsky All Rights Reserved. Confidential & Proprietary. 3. What is Cloud-Agnostic and why should I care? 3Copyright 2014 Alex Maclinovsky All Rights Reserved. Confidential & Proprietary. Cloud-Agnostic System consumes cloud services while being loosely coupled to the underlying cloud platforms and providers. Common CAS traits: Integrates with the underlying cloud rather than just running on it Large contact surface with the cloud Leverages Cloud API for the integration Orchestrates cloud operations and capabilities Typically integrates on the lower (IaaS, STaaS) levels of abstraction 4. Degrees of Cloud-Agnostic behavior 4Copyright 2014 Alex Maclinovsky All Rights Reserved. Confidential & Proprietary. Works with a TBD Cloud Works with multiple versions of a Cloud Can work with one of several clouds Can work with more than one cloud Can support new clouds Uses the same code to talk to multiple clouds Will support future features and capabilities of target clouds Marginal Useful Valueofmediation Technology Parallels: 5. Approaches for building Generic Clients 5Copyright 2014 Alex Maclinovsky All Rights Reserved. Confidential & Proprietary. Lowest Common Denominator Implements only functionality which is present and consistently implemented in all target systems Leaves all deviations out of scope Reflection Builds rich canonical domain model encompassing majority of the features found in the target systems Uses meta-model + discovery APIs to allow users to discover feature set supported by specific target Do This Implements only a single operation doThis() that takes an XML document describing the request 6. Popular Multi-Cloud Integration Options Apache jclouds often seen as the leader of the pack, VM-centric no networking, support for cloud-specific features is largely done via provider contexts Apache d-cloud - even more basic, with no networking support. Is a REST API not a Java library Apache Libcloud a python library that lacks even most basic canonical relying on the dynamic language to hide feature differences between cloud drivers Dasein Cloud the only one built on a real canonical model. Supports broad variety of clouds. Has rich networking. OSS foundation for Dell Cloud Manager Cisco CIAC - Cisco Intelligent Automation for Cloud 6Copyright 2014 Alex Maclinovsky All Rights Reserved. Confidential & Proprietary. 7. The Importance of Canonicals Much more variability between clouds than RDBMS Whether cloud abstraction layer uses a rich, well- defined canonical determines its mediation value and, ultimately, ability to write cross-cloud code Next 2 slides compare code snippets launching VMs with default configurations in EC2 and Terremark eCloud, highlighting: common, parameterizable and divergent code and showing overall mediation score between two integration libraries: one uses the other: 7Copyright 2014 Alex Maclinovsky All Rights Reserved. Confidential & Proprietary. 8. jclouds What Canonical? + = 8Copyright 2014 Alex Maclinovsky All Rights Reserved. Confidential & Proprietary. ComputeServiceContext context = ContextBuilder.newBuilder("aws-ec2") Template template = context.getComputeService().templateBuilder().osFamily(OsFamily.CENTOS).build(); options.as(AWSEC2TemplateOptions.class).subnetId(subnetId); template.getOptions().as(EC2TemplateOptions.class).noKeyPair(); Set