Technical Introduction to caGrid Service Development

caGrid 1.3

Justin Permar

caGrid Knowledge Centerhttps://cabig-kc.nci.nih.gov/CaGrid/KC

Agenda

• Introduction to “Introduce”• Service Categories• Secure Services

Introduce Vision

• Introduce is the caGrid service development toolkit aimed at software developers building Grid services

• The vision is to be one stop shop for grid service development• Provide a simple, yet powerful, graphical user interface

(GUI) to encapsulate complexities of grid service development

• Provide an extensible toolkit to create and modify grid services programmatically

• A GUI front-end is a user interface for the Introduce engine

Introduce Overview

• caGrid uses the Globus toolkit “under the hood”• Introduce is a framework which enables fast and easy creation of Grid

services based on Globus that incorporate caGrid innovations• Hide all “grid-ness” (distributed computing infrastructure) from the

developer• Utilize best practice “separation of concerns” patterns, implemented

as a layered grid service architecture• Messaging layer, serialization layer, business logic layer

Introduce Overview (2)

• Integration with other core grid services and architecture components to enable common Grid use cases• GAARDS Security Infrastructure• Grid service directory (the Globus Index Service)• XML schema storage (the Global Model Exchange Service)• Service metadata annotation (the Metadata Model Service Service)• NCI’s Cancer Data Standards Repository

• Extension Framework for integrating with other architecture components• Extensions provide much of the functionality in Introduce• Extensions typically interact with the service (under development)

and with external Grid services to add new capabilities to your Grid service

Introduce Extensions Grid Usage

Introduce Features

• Definition of Grid objects types (based on XML schema)

• Creation of service interface• Modification of operations

• Adding operations• Removing Operations• Updating Operations• Importing Operations (via WSDL)

• Graphical Service Configuration• Advertisement• Security• Service Metadata Specification• Service Metadata Editing• Service Configuration Properties

• Auto Generates Service Code and client API

• Graphical Service Deployment• Globus, Tomcat, and JBoss

Introduce Requirements

• Utilize best practice “separation of concerns”• Enable development of strongly-typed services

• discovery and use of published Grid data types (e.g., type definitions in caDSR or XML schema)

• self describing services via metadata• advertisement and registration configuration (Index Service endpoint)

• Generate object-oriented client APIs• Customizable and extensible via the use of plug-ins • Create secure services

• provide customizable service-, method-, and resource- level security

Service Layers

Web Server (Apache/ Tomcat): Binds to server port(s)

Web Application Server (Tomcat): Hosts web applications connected to the web server

SOAP Engine (Axis): Interprets SOAP requests, installed as a web application

Web/ Grid Service (Globus): Binds “protocol” to operations on local application resources

Security (GSI)* Secure Communication* Authentication* Authorization

Metadata (WSRF – Resource Properties)* caGrid Service Metadata* caGrid Service Security Metadata* (caGrid Data Service Metadata)* (Custom Metadata)

Service Implementation

Service Definitions* WSDL* XSDs

Resources (WSRF Resource)

Configuration Properties

Advertisement (WSRF-SG)

Business Logic

Addressing Strong Typing

Strongly typed interfaces:• Utilizes XML schema to define Grid

object types• Uses the XML schema for object

binding layer (XML-Java class mapping framework)

• Business logic layer that Introduce emits uses only Java objects (no XML exposed)

Service

Core Services

Client

XSDWSDL

Grid Service

Service Definition

Data TypeDefinitions

Service API

Grid Client

Client API

Registered In

Object Definitions

SemanticallyDescribed In

XMLObjectsSerialize To

ValidatesAgainst

Client Uses

Cancer Data Standards Repository

Enterprise Vocabulary

Services

Objects

GlobalModel

Exchange

GMERegistered In

ObjectDefinitions

Objects

Example from caGrid

Service Advertisement and Discovery

When a grid service is deployed to caBIG, it registers its availability and service metadata with a central indexing service (Index Service).

The Index Service maintains a list of registered services and aggregates their metadata. It is the “white” and “yellow” pages of the grid.

Once the service has registered with the Index Service, users can discover the service using caGrid Portal, Globus command line tools, or incorporate the Discovery Client API in to their own applications.

Addressing Configurable Security

• Security:• Provide a service operation to return the

security configuration to each client that wants to invoke service operations

• Introduce generated clients will automatically retrieve this metadata and use it to configure the client before making a call to the service.

• Customizable authentication and authorization options

Service Metadata

All caGrid Services are expected to publish a set of standard metadata comprised of metadata registered in caDSR and EVS; metadata describes the meaning of service operations and data types and also provides details of the hosting institution that deployed the service. Metadata is used to discover Grid resources.

Standard Service Metadata

Instances of this model describe the grid service operations and parameters as well as the semantics of the data models used in the service. Standard service metadata also describes basic information about the service, such as the hosting information, including a point of contact.• Editable in Introduce as the ServiceMetadata Resource Property.

Standard Data Service Metadata

caGrid Data Services require additional standard data service metadata (the grid Domain Model), which details not only the UML Classes exposed by the service, but their relationships such as associations and inheritance. This information describes the logical model over which data service queries are executed.• Viewable in Introduce as the DomainModel Resource Property.

Service Contexts

• A service context is a service, same as the primary (main) service

• The purpose of a service context is to provide operations within a context, such as an analytical service job request

• Each service context has an associated resource that holds data for the context

• Each context is typically conceptually tied to an owner of that context• As an example, when a user submits a job to an analytical

service, the service creates a new job context and returns a reference to the user

• The user can then check job status using operations on the job context

Stateful Web Services

A resource holds state. The context provides operations that typically act on that state. The combination of context (operations) + resource (data) comprises a stateful web service. The service returns an end-point reference (EPR) that points to the context (and resource). The client uses the EPR to connect to their context.

Agenda

• Introduction to “Introduce”• Service Categories• Secure Services

Service categories

Introduce allows you to create two primary categories of strongly typed, semantically annotated Grid services:

1. An analytical service provides access to analysis routines.

2. A data service provides access to data repositories.

Note: technically, a “data service” is an “analytical service”. It is a specialization of an analytical service for the purposes of Grid data sharing. A data service extends an analytical service with a standard “query” method and the standard data service metadata

Analytical Services

• caGrid Analytical Services provide the ability to process, transform, and analyze the data available from a Data Service

• Analytical services all detail with strongly typed, semantically harmonize objects (take these objects as input and return these objects as output)

• Analytical service aspects• Definition of one or more Grid types• One or more service contexts• One or more operations that utilize those types• Authentication and authorization policies at service-, operation-, and

resource-level• Service properties• Use of one or more service extensions for additional functionality

• For example, the caGrid Transfer service to efficiently transfer large amounts of data

Data Services (1)

• Data services are added to Introduce with an Introduce extension. caGrid 1.3 supports building data services using caCORE SDK (up to caCORE SDK version 4.2). The extension provides an easy to use wizard to create a Data service using the artifacts generated by the caCORE SDK.

• The query language for data services is an object-centric language called CQL.

Data Services (2)

The wizards provides steps to:• Select the client interface to the caCORE SDK• Configure the properties to access your caCORE-like system• Generate a caGrid domain model from the XMI (also used by caCORE SDK)

• This domain model represents the UML model for Grid use

• Allows you to select the schemas that represent objects in the domain model

When deployed, the caGrid Data Service will use your caCORE-like system to execute your CQL query against the data in your database.

Example CQL Query

<ns1:CQLQuery xmlns:ns1=http://CQL.caBIG/1/gov.nih.nci.cagrid.CQLQuery>   <ns1:Target name="gov.nih.nci.training.BootCamp.domain.Protein”>       <ns1:Association roleName="organismCollection“

name="gov.nih.nci.training.BootCamp.domain.Organism”>           <ns1:Attribute name="commonName" value="Mouse" predicate="EQUAL_TO”>       </ns1:Association>   </ns1:Target>

</ns1:CQLQuery>

Agenda

• Introduction to “Introduce”• Service Categories• Secure Services

GAARDS Security Infrastructure

GAARDS provides services and tools for the administration and implementation of security policy in caGrid.

Authentication in Introduce

Introduce allows the user to customize security for the entire service or specific methods on an service context. The user can select any of the options as determined by your security policy.

Introduce Options• “None”: no authentication is performed• “Custom”: select authentication options

•Transport Layer Security – Encryption equivalent to https and requires authentication to grid.

• Secure Conversation – Ensures a secure conversation (multiple messages) between a client and service

• Secure Message – A message is signed or encrypted with your grid credential.

Authorization in Introduce

Authorization

For secure services (with “custom” authentication specified), Introduce allows users to select authorization options to implement authorization policy. These authorization options use the client’s Grid identity to perform an authorization check. Using Introduce, users can specify an authorization check on a resource, either a service context or an operation. The user must pass the authorization check in order to access the resource.

• CSM Authorization Plug-in • An extension that allows users to specify a CSM check at the service,

method, or resource level.• Grid Grouper Authorization Plug-in

• An extension that allows users to specify a Grid Grouper authorization expression at the service or operation level. A typical authorization expression checks whether a user is a member of a particular group.

GAARDS Overview

• Grid Authentication and Authorization with Reliably Distributed Services (GAARDS)• GAARDS provides services and tools for the administration and

enforcement of security policy in an enterprise Grid. • Developed on top of the Globus Toolkit • Extends the Grid Security Infrastructure (GSI) • Provide enterprise services and administrative tools for:

• Grid User Management • Identity Federation• Trust management• Group/VO management• Access Control Policy management and enforcement• Integration between existing security domains and the grid security domain• Delegation• Web Application integration

GAARDS Security Infrastructure

Dorian

• Account Management• Administrative interface for account provisioning and

management.

• Built in Certificate Authority• Manages/Issues PKI Credentials for each user.• Enables users to authenticate and obtain PKI credentials,

which they may use to access the grid.• Users may request service/host credentials for the Grid

services they administer.

• Identity Management and Federation• Integration point between external security domains and the

grid.• User may use their existing credentials to authenticate to the

Grid.• Automated Account Creation and Provisioning

• Built in Identity Provider (maintains user accounts)

• Comprehensive Administrative UI

Organizational Identity Providers

OSUActive

Directory

UTLDAP

• Institutions leverage varying technologies for Identity Management• Active Directory• LDAP• Custom / Proprietary / Home Grown

• Grid applications cannot be expected to know how to authenticate users with every possible identity management technology.

• A standard interface / API is needed such that applications can authenticate users against any identity management technology

Authentication Service

• Provides a uniform web service interface providing applications with a single approach for authenticating users across a federation.

• Allows existing identity providers to be seamlessly integrated into a production Grid environment.

• Users that are registered with an identity provider may use their existing credentials to access resources on the Grid.

• Together the Authentication Service and Dorian provide a solution for federating identity in a grid environment, allowing users to use their existing credentials to access secure grid resources.

How do we know which CAs to Trust?

Should I trust this CA?

Should I trust this CA?

Grid Trust Service (GTS)

• Grid Trust Service (GTS)• Creation and Management of a federated trust fabric.• Supports applications and services in deciding whether or not signers of digital credentials can be trusted.• Supports the provisioning of trusted certificate authorities and corresponding CRLS• Define and manage levels of assurance.• Distributed GTS, Enabling the creation of a scalable trust fabric.

Authorization

• GAARDS gives service providers complete control of enforcing access control policy.

• Service providers may integrate any authorization/access control system(s).

• caGrid provides tools/services for enforcing access control.

• Grid Grouper• Group management web service.• Hierarchal organization of groups

• Common Security Module (CSM)• Enforces local access control on

resources• Policy can be based on groups

managed by Grid Grouper

Credential Delegation Service (CDS)

• Credential Delegation Service (CDS) - A Web/Grid service that enables users/services (delegator) to delegate their Grid credentials to other users/services (delegatee) such that the delegatee(s) may act on the delegator's behalf.

Web Single Sign On (WebSSO)

• Provides a comprehensive, Single Sign On (SSO) solution for web applications using GAARDS.

• Uses the caGrid’s GAARDS framework in back end to authenticate and validate the user

• Allows users to use single set of local credentials to both navigate amongst different web applications and also invoke various grid services

• Provides an automated mechanism for delegation and retrieval of user’s grid credentials there by avoiding transfer of grid credentials around

GAARDS Admin UI

• GAARDS provides a comprehensive UI for using and administrating GAARDS

GAARDS In Action

To access secure Grid resources, a

user needs to obtain a Grid

credential

GAARDS In Action

GAARDS In Action

Authenticate with local institution

and obtain proof of authentication (SAML Assertion)

GAARDS In Action

Obtain Grid Credential from

Dorian using SAML Assertion

GAARDS In Action

Invoke SecureGrid Service

Using Credential Provided by

Dorian

GAARDS In Action

Validate that theCredential

provided by the user is issued bya trusted provider

GAARDS In Action

Determine if user is authorized to

access requested resources.

BACKUP

Service Contexts Details

• Every Grid service has a main service which has operations that are not associated with state (no associated resource). The main service context is unique in that it is a singleton (i.e., it always exists for clients to invoke).

• One or more service contexts can be added to a service, where each context has associated state. The operations on a service context have access to that state.

• Service contexts share the service configuration (i.e. service properties)• Service contexts know how to access every other service context and the

primary service resource home in order to factory new resources, find resources, and get the currently addressed resource.

• Service contexts will each independently register with index service using the same configuration as the primary service.

• Service contexts do have resource properties just like every other service and have operations just like every other service.

• Service contexts all have their own unique endpoints and to the outside world are unique web services indistinguishable from the fact they were developed and bundled together.

• Service contexts are all deployed together with the primary service to the grid service container (a deploy command deploys all contexts).