infn occi implementation on grid infrastructure

INFN OCCI implementation

on Grid Infrastructure

Michele Orrù

INFN-CNAF

OGF27, 13/10/2009

1M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure

Outline

Farm virtualization: WNoDThe need for IaaS integrationIaaS Application internal

architectureIaaS Application flow of controlFuture works


3

Virtualization within INFN INFN Tier-1 currently supports ca. 20 Virtual

Organizations: each one has different software and resource requirements, and well-established workflows that should not be changed

To satisfy their needs INFN has developed a solution called WNoD (Virtual Worker Nodes on Demand)

With this kind of approach the installed software is not coupled anymore with physical hardware, and the resources are always dynamically shared thanks to the creation of VO-specific VMs at job execution time.

M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure

Farm

vir

tualiz

ati

on

: W

NoD

WNoD: high level design


Farm

vir

tualiz

ati

on

: W

NoD

Emerging Cloud requirementsUsage of Grid resources in a flexible way,

without the complex workflow that a typical grid job might have

Third-party users (academia, industry, government) needing dynamic virtual execution environments

INFN developers needing customized platforms to deploy and test their applications

The idea is to keep the same LRMS that is already handling jobs from the Grid computing interface, and then expose a second service to manage requests of cloud-like resources.


Th

e n

eed f

or

IaaS

in

teg

rati

on

Grid & Cloud: high level overview


Th

e n

eed f

or

IaaS

in

teg

rati

on

A mixed Grid infrastructure (both real and virtual worker nodes): only for job submission

Grid & Cloud: high level overview


Th

e n

eed f

or

IaaS

in

teg

rati

on

The same pool of resources is used to provide cloud-like services on the actual Grid infrastructure

This allows to have homogeneous sharing with the resources used through the Grid interface

Machines don’t need to be dedicated to either service

The IaaS layerWe have implemented OGF's OCCI API as the

interface of our application, due to its open and slim nature

This solution works today, with a small subset of the OCCI API: the user requests a computing resource, gets an IP address back from the application and can then connect to the VM via SSH.

The accounting and the authentication/authorization layers of the application will be completely integrated with the existing Grid infrastructure.

Java Enterprise application built with industry leader technologies such as Spring, Hibernate, JPA, Restlet, Jetty, Maven


IaaS A

pp

licati

on

in

tern

al

arc

hit

ect

ure

IaaS Application architecture


IaaS A

pp

licati

on

in

tern

al

arc

hit

ect

ure

We can fully leverage on unique Spring features such as Dependency Injection and Inversion of Control

OCCI implementation


IaaS A

pp

licati

on

in

tern

al

arc

hit

ect

ure

RESTful ws endpoints are defined in Spring’s application context XML files: following the “compute” descriptor. <bean id="root" class="org.restlet.ext.spring.SpringRouter">

<property name="attachments"><map>

<entry key="/compute/{computeId}"> <bean class="org.restlet.ext.spring.SpringBeanFinder"> <lookup-method name="create" bean="computeResource"/> <property name="beanName" value="computeResource" /> </bean> </entry>

</map></property>

</bean>

<bean name="computeResource" class="com.infn.cnaf.resources.compute.ComputeResource" scope="prototype"> <property name="batchSystemController" ref="bsController" /> </bean>

OCCI implementation


IaaS A

pp

licati

on

in

tern

al

arc

hit

ect

ure

Operations are mapped to HTTP verbs with annotation metadata (@Get, @Post, @Put, @Delete)

Right now only plain-text format is supported on responses

We are integrating ATOM and JSON formats:

@Post

public Representation storeRepresentation(Representation entity) {

if (entity.getMediaType().equals(MediaType.APPLICATION_WWW_FORM, true)) {…}}

@Get(“json”)

public Representation toJson() {…}

Flow of control: create compute resource


IaaS A

pp

licati

on

flow

of

con

trol

0/8



IaaS A

pp

licati

on

flow

of

con

trol

1/8



IaaS A

pp

licati

on

flow

of

con

trol

2/8



IaaS A

pp

licati

on

flow

of

con

trol

3/8



IaaS A

pp

licati

on

flow

of

con

trol

4/8



IaaS A

pp

licati

on

flow

of

con

trol

5/8



IaaS A

pp

licati

on

flow

of

con

trol

6/8



IaaS A

pp

licati

on

flow

of

con

trol

7/8



IaaS A

pp

licati

on

flow

of

con

trol

8/8

On the way…Support for multi-core VMsUI improvements (Web based UI)Full compliance with the OCCI APIBilling


Futu

re

work

s

Questions???


Quest

ions?

infn occi implementation on grid infrastructure

Documents

grid infrastructurethe

needs infn

mixed grid infrastructure

existing grid infrastructure

iaas integration grid

typical grid job

grid computing interface

actual grid infrastructurethis