ogf cloud standards: current status and ongoing interoperability efforts with dmtf, snia and other...

©2011Open Grid Forum

OGF Cloud Standards:

Alan Sill, Ph.D Vice President of Standards, Open Grid Forum

Florian Feldhaus GWDG Göttingen

Current Status and Ongoing Interoperability Efforts With DMTF, SNIA and Other Organizations

CSCC Meeting – Dec 2011

2

OGF standards are part of the Grid Final Document series (GFDs), which has a strong track record of adop=on in large-‐scale commercial and academic distributed compu=ng products.

OGF’s success in cloud and grid standards development spans more than a decade.

3 © 2011 Open Grid Forum

OGF Cloud Standards Background

•  OGF has an extensive set of applicable standards and documents related to topics in cloud computing: -  Federated Identity Management -  Managing the Trust Eco-System (CA operations, AuthN/AuthZ) -  Virtual Organizations (VOMS) -  Job Submission and Workflow Management (JSDL, BES) -  Network Management (NMI, NSI, NSF) -  Secure, fast multi--party data transfer (GridFTP, SRM) -  Data Format Description (DFDL) -  Service Agreements (WS-Agreement, WS-Agreement Negotiation) -  Cloud Computing interfaces (OCCI) -  Firewall Traversal (FiTP) -  Others under development

•  Working to gather this information to form an organized description of OGF work - an OGF “Cloud Portfolio” - and to start new working groups on related topics.

4

OCCI Positioning


Status of OGF OCCI Standards

•  GFD.183: “Open Cloud Computing Interface - Core”, GFD.184: “Open Cloud Computing Interface - Infrastructure” published April 2011; slight revisions to textual portions released June 2011 (no changes to normative content).

•  GFD.185: “Open Cloud Computing Interface - RESTful HTTP Rendering” published June 2011

•  These standards are now stable and complete and being implemented by the community.

•  OCCI JSON Rendering being worked on now. •  Many implementations now exist - very strong uptake! •  InfoQ article summarizing interoperability published. •  Clients being written to supplement server implementations.


Updates On Recent OCCI Implementations

•  New OpenNebula implementation in progress - full OCCI v1.1 compliance! OCCI client tools, instructions and tutorials on OpenNebula developer site including examples

•  uptake of OCCI work within SLA@SOI by FP7 “Future Internet” FI-Ware project.

•  R2AD: New Android and JavaFX Cloud Client apps including OCCI and CDMI: design done, demos available and beta implementation in progress.

•  occi-py Python framework developed by Ralf Nyren: RDBMs-friendly backend; draft JSON rendering implemented and demo instance available for testing.

•  OpenStack implementation code work started, new client and server libraries available.

•  More info at http://occi-wg.org/community/implementations/


New R2AD Cloud Client (Info. from Michael Behrens)

Major Functions: • OCCI: Status/Control Resources, Create and

Update Resources (VMs) • CDMI: Create/Delete

Containers, Browse Storage Containers via CDMI, Upload content

Data Transport Compute Policy Billing

Example Interfaces to Standardize

Pluggable Scalable Secure Simple Dynamic Metadata RESTful

Desirable Features

OCCI CDMI

Other Specs

Standardized IaaS/PaaS Layer

OVF

•  Building to OCCI, CDMI, and other standards -  Goal: Interact with cloud

providers using the standard RESTful APIs, thereby reducing the code development and maintenance cost

•  Participating in SNIA Plug-Fests for OCCI/CDMI/OVF Integration

•  Visit http://r2ad.net -  Info & Screen Shots -  Android and JavaFX -  Beta: Nov 2011

•  65% compete

moc

kup


New OCCI Work in OpenStack

•  Based on pyocci (Thijs Metsch/Platform Computing) w/ input from other developers

•  Allows for the provisioning of VMs -  OCCI Infrastructure specification -  OCCI v1.1 compliance

•  Implemented by EU FP7 SAIL project in cooperation with an industrial partner -  www.sail-project.eu

•  Discussed at OpenStack Design Summit Oct 3-5, 2011; work currently in progress.

•  Current goal is to be in Essex release, March 2012.


Cloud Interoperability Efforts •  SNIA “Cloud Plugfests” have been used for combined

interoperability efforts: April, July, Sept 2011, Spring 2012. •  Several active OCCI and CDMI implementations! •  Integration of OCCI and CDMI reportedly “works well.” •  OVF integration under active development & progressing. •  Next steps and new testing work planned:

-  Feed back experience from implementations, write implementation experience documents and user guides.

-  Develop combined OCCI / CDMI / OVF verification suite. -  Stress/scaling testing, performance studies at scale. -  Interoperability testing w/ OGF Cloud Interoperability efforts. -  Extending OCCI with monitoring / reservation capabilities. -  Integration with other OGF & relevant standards.

•  OGF forming new Cloud Interoperability Working Group

10

OCCI / CDMI Client

OVF

Template

OS Resource

Storage

Network

Compute

OCCI Server CDMI Server

SLA Management Webservice

OCCI/CDMI/OVF Interoperability Demonstration:

Credit: F. Feldhaus, TU Dortmund

Why OCCI?

• mature, open standard •  several implementa=ons

– Python, Ruby, Java – OpenNebula, OpenStack, libvirt – DeltaCloud (under development)

• very flexible, extendible •  supports vendor specific features • working on support for monitoring and SLAs

OCCI Extension mechanisms

12

OCCI

CDMI OVF

Vendor Specific Feature

Vendor Specific Feature

Mixin Mixin

Link Link

OCCI Query / Filtering Mechanisms

• OCCI allows the discovery of provider specific features / extensions –  customer queries OCCI endpoint –  OCCI endpoint returns e.g. special hardware for HPC or graphics extensions for VMs

• OCCI allows filtering a subset of resources / en==es –  customer wants to manage only high-‐availability VMs or VMs with 4 cores

13

NFS Server

OCCI Server

CDMI Server

OpenNebula

VM Host

1. Create CDMI Container

2. Upload Image to Container

3. Create OCCI Compute object 4. Deploy VM

5. Deploy VM

5.a Get NFS export forCDMI Container

5.b Mount NFS export

5.c Acess image from NFS directory

VM

6. Start VM

7. Access VM using VNC

OCCI/CDMI Interoperability Demonstra=on

15

OCCI/CDMI Interoperability Demonstration:

Credit: F. Feldhaus, TU Dortmund

OpenSource Repository

• Ruby OCCI Framework with OpenNebula 3.0 backend –  h`p://dev.opennebula.org/projects/ogf-‐occi/wiki/

• OCCI / CDMI Client „Pollux“ –  h`ps://github.com/gwdg/Pollux

16


Other Work and Publications •  A. Edmonds, T. Metsch and A. Papaspyrou, “Open Cloud Computing Interface in Data Management-related Setups,” Springer Grid and Cloud Database Management, Jul. 2011.

•  A. Edmonds, T. Metsch and E. Luster, “An Open, Interoperable Cloud,” InfoQ article, http://www.infoq.com/articles/open-interoperable-cloud

•  OpenNebula Interoperability group formed. •  OGF Cloud Interoperability working group (CI-WG) forming. •  OGF WS-Agreement, WS-Negotiation have been published. •  Firewall Traversal Protocol in Public Comment. •  OGF Distributed Computing Infrastructure Federation and

Federated Security groups started.


Summary and Conclusions •  OCCI and other OGF standards are gathering substantial

implementation experience in real-world cloud and grid settings.

•  Production e-Infrastructure and commercially-oriented projects are implementing these in a wide range of real-world settings for user communities, writing usage guides, deployment scenarios and instructions.

•  OGF is engaged at several levels in developer outreach, standards interoperability demonstrations and ongoing roadmapping activities in cloud computing.

•  We do NOT view this work as in conflict with other standards efforts due to the differing demands and needs of our user communities; stand ready to engage in mapping activities and other mutual work as other standards mature.


More information

•  OGF Standards: -  http://ogf.org/standards -  http://ogf.org/documents

•  OCCI WG website: http://www.occi-wg.org •  OCCI specifications: (for other OGF standards, see OGF

documents link above) -  Core: http://ogf.org/documents/GFD.183.pdf -  Infrastructure: http://ogf.org/documents/GFD.184.pdf -  RESTful HTTP Rendering:

http://ogf.org/documents/GFD.185.pdf •  OCCI Mailing List

-  http://www.ogf.org/mailman/listinfo/occi-wg

E-mail: [email protected]


Full Copyright Text

Copyright (C) Open Grid Forum (2011). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. The limited permissions granted above are perpetual and will not be revoked by the OGF or its successors or assignees.

Backup Slides

•  Screenshots and project descriptions for recent implementations of OCCI and other OGF standards

21


OpenNebula OCCI v1.1 Ruby Framework

23

* (Note: “OCMI” is just a name given by the authors to this sofware project; it does not denote a name given to any current standard.)

*

(New, skeleton...)

Developer: Ilja Stevenson (VENUS-‐C) with input from OCCI-‐WG

24

Web site s=ll shows old version; new version and examples under ac=ve development at h`p://dev.opennebula.org/projects/ogf-‐occi/ by Florian Feldhaus and TU Dortmund group

OpenNebula 3.0 predates OCCI v1.1 specifica=ons; new code work in progress!

25

FIware Project and OCCI

•  FIware is an FP7 project that will provide a platform for several large use case projects.

•  Within FIware there is the Cloud Hosting chapter in which provisioning of compute, storage and networking services will be demonstrated and accomplished.

•  OCCI will be one of the APIs used to manage resources in the Cloud Hosting chapter. h`p://www.fi-‐ware.eu/

26

D-Grid Scheduler Interoperability Initiative and OGF Standards:

D-‐GSI Project aims to integrate latest OGF standards for informa=on discovery, job submission, infrastructure management, basic execu=on services and SLA agreement and nego=a=on into a single, modern up-‐to-‐date implementa=on involving mul=ple partners and the poten=al for reuse.


OCCI Tools from CompatibleOne:

- Described at OWF, ETSI and OpenStack In Action conferences by Iain James Marshall (PROLOGUE). - C libraries, example code - LGPL+CC Licensed. - OCCI interfaces for OpenNebula, OpenStack, Azure and several others.

http://www.compatibleone.fr

ogf cloud standards: current status and ongoing interoperability efforts with dmtf, snia and other...

Technology