DoDDoD Software Transformation Software Transformation

SOASOAThe Road Toward Net-centric OperationsThe Road Toward Net-centric Operations

Michael BehrensMichael BehrensR2AD, LLCR2AD, LLC

Chief Technology OfficerChief Technology OfficerSupporting DISA GCCS-J ISupporting DISA GCCS-J I33

April 2005April 2005

Rob VietmeyerRob VietmeyerNCES Chief EngineerNCES Chief EngineerDefense Information Systems AgencyDefense Information Systems AgencyApril 2005April 2005

R2AD®, LLC

2

Agenda

• DoD Software Transformation - DISA

– Network Centric Enterprise Services(NCES) Overview

– DISA’s Pilot Effort & Lessons Learned

• Standardizing the Grid - GGF

– Open Grid Services Architecture (OGSA)

• Deployment and Maintenance

-- Albert Einstein

“We can’t solve problems by using the same thinking we used“We can’t solve problems by using the same thinking we used

when we created them”when we created them”

3

IT Complexity

• More than 400 different softwareapplications support JTF commanders

• Duplicity coupled with lack ofinteroperability and integration

• Tightly coupled interfaces preventingmigration and modernization

• Lack of full visibility and total access toinformation and functions

4

The New Imperatives

• How to Improve, Design, Implement, and

Manage large, complex Net-centric (SOA)capabilities?

• How to Lower Risks inherent in designing anddeploying and operating large, complex Net-

centric capabilities?

Shape Evolution of Enterprise ITvice

Build Systems

5

Why SOA: Technical View

• Interoperability: Integration across the networks, heterogeneous technologies,organizational boundaries

– Exposing S/W functions as consumable services, easing application and data sharing

– Transportability of application functions across organizational boundaries

– Platform for evolution of enterprise policies, standards, and functions

• Agility: Ability to dynamically reconfigure processes to meet changing operationalrequirements

– Separation of responsibilities and dependencies at the service interface

– Componentization of reusable software

– Technical and business abstraction

• Visibility: Common understanding of requirements and capabilities among consumers,planners, and providers

– Consumer: What information and functions are available

– Providers: What information and functions are needed

– Operations: Ability to identify and respond to events

– Business planning: Ability to identify service utilization patterns, ROI

• Transformation: Closer alignment of IT with business and warfighting processes– Abstraction of technical complexity

– Faster deployment of new technologies and processes, building block approach

– Enabling new capabilities and processes

6

Lessons we’re learning

• SOA is not just about technology…

– Business

– People & Culture

– Organization

– Information Philosophy

• Challenges remain

– Maturing technology and evolving standards

– Performance and security

• Web services = easy but deceptive

• Enterprise SOA = challenging but necessary

– Potential for HUGE benefits ….

7

GIG Enterprise Services

8

Architecting the Service Environment

• Services Environment– Service Producers

– Service Consumers

• Services Infrastructure– Utility and environmental

services to facilitate andgovern producer andconsumer interaction

• Computing Infrastructure– Hardware and software

platform for hosting serviceproviders and serviceconsumers

• Network Infrastructure– Ubiquitous connectivity

UA UA

SMTA

IMTA IMTA

ROOTDSA

Global

DSA

Regional

DSA

MFI BMTA

MLA

BMTA

MLA

Policies, procedures, middleware and services tosupport enterprise -- Discovery and access -- Security -- Management and evolution ofAvailable information, functions and capabilities

People, Processes, Information,

Applications

NCES Increment 1 FOCUS

9

Oktoberfest Demonstration

10

Service Discovery

Service Consumer

Service Consumer

PublishEnterpriseServices

DiscoverEnterpriseServices

Service Provider

Invoke

Enterprise Service Registry

LocalServices

AdvertisedServices

LocalRegistry

NCES Hosted

DOD Root Registry

Standards Base: Universal Description, Discovery

and Integration (UDDI)

11

Service Discovery

• UDDI Maturing– Previous implementations

• Complex

• Focused on technical abstraction and localimplementation

– Future direction• Foundation for enterprise SOA management and

governance

• Support for business (as well as technical) functions

• Improved support for integrating services registrywith associated artifact repositories

– Challenges• Registry federation and synchronization

• Supporting potentially conflicting needs of runtimeenvironment, development environment andmanagement environment

12

Pilot Security Services Architecture

ApplicationApplication

IdentityStore

Service

Consumer

Au

then

tica

tio

n Service ProviderService Provider

WebService

WebService

NCES Security ServicesNCES Security Services

PolicyStore

CertificateValidation

Service

PrincipalAttribute

Service

PolicyDecision

Service

PolicyAdmin

Service

PolicyRetrieval

Service

PolicyEnforcement

Point

PolicyEnforcement

Point

DOD PKI &

Global Directory

Standards Base: WS-Security, SAML, XACML

13

Challenges & Opportunities

• Today, false sense of comfort in our existing securityapproaches and mechanisms?

• SOA technologies are enabling abstraction of securitychallenges

– Mitigate vulnerabilities

– Improve functionality and dependability

• But we need …

– Standard trust models that extend beyond point-to-point / per-hoptrust

– Message addressing and routing standards. There are couple ofthem out there but they

• Need to converge

• Need to incorporate security into their design

– Policy standards beyond authorization (e.g., XACML)

• QoS, privacy, governance

• Support for policy discovery

14

Service Management

Standards Base: Web Services Distributed Management (WSDM)

Service Provider

EnterpriseService Registry

LocalServices

AdvertisedServices

Monitor

Management

Agent

Alerts System

Enterprise Service Manager

NET OPS

Network MgmtSystem Mgmt

Service ConsumerInvoke

Service Level Mgr

15

Service ManagementIntegrating the Services Environment with Computing Infrastructure

• Monitoring and management of the runtime environment

• Evolving marketplace and standards

• Potential for redundant or conflicting capabilities

– Sub-optimization through optimization?

CommunicationsManagement

ComputingInfrastructure Management

Serv

ice

Serv

ice

Serv

ice

Serv

ice S

erv

ice

Serv

ice

Serv

ice

Serv

ice

Serv

ice S

erv

ice

Serv

ice

Serv

iceServ

ice

Serv

ice

Serv

ice

Applications/SystemsManagement

Service Management

16

Engage with developers andstakeholders across your enterprise

• A Web-based global collaborative

software development solution

• Strengthens relationships with internalstakeholders, business partners, andclients

Project Tools• Administration

– Add members– Create roles

• Communication– Announcements– Discussion Forums– Mail Lists

• Artifact Repositories– File Sharing– Document Library– Version Control– Source Code Management

• Bug and Issue Tracking

17

Beware of the realities

Web services SOA– Ease of development can lead to poor implementation

• Services that are too granular causing inefficient communications and

overhead

• Tightly coupled designs tied to underlying technologies and

implementations

• Services tightly coupled to legacy business processes

– Limited understanding within the developer community

• Rapidly evolving technology

• Evolving standards and commercial implementations

• Lack of implementation guidance and best practices

– Limited understanding at a business level

• Impact of SOA technologies on the business

• Ability to align service evolution with business needs

• Approaches to SOA management and governance

• Need for new approaches to IT architecture, funding, oversight

18

SOA requires governance

• Enterprise policies and standards

– Maintain interoperability

– Maintain and improve information assurance

– Manage enterprise interdependencies

• New roles and authorities required

– Who (and how) establishes service identities and designatesthem authoritative

– Support for different rules and mechanisms for differentportions of the service’s life-cycle (proposed, developmental,testing, operational, deprecated, retired)

– Management of business processes that span traditionalorganizational boundaries. Judiciary functions to supportSLA processes??

• Ability to align service development and implementationwith business requirements

• Balance bureaucracy with need to support market driveninnovation and evolution

19

This is going to be interesting ….

• Systems

• Software interoperability

• Duplication of functions

• Data element standardization

• Common operating environment

• COTS

• Host applications locally

• Heavy clients

•• Service providers and consumersService providers and consumers

•• Business process integrationBusiness process integration

•• Optimization and specializationOptimization and specialization

•• Standardized business productsStandardized business products

•• Integrated network environmentIntegrated network environment

•• Commercial service providersCommercial service providers

•• Provide service globallyProvide service globally

•• Application delivery over networkApplication delivery over network

Yesterday TomorrowTomorrow

Migration to a SOAMigration to a SOA

-- David Alberts and Richard Hayes

““With the coming of the Information Age, there is an opportunity toWith the coming of the Information Age, there is an opportunity to

provide widespread access to information related services andprovide widespread access to information related services and

capabilities only dreamed about in previous eras.capabilities only dreamed about in previous eras.””

20

SOA…NCES…Grid

Standardizing the Grid Frontier

R2AD, LLC

21

Grid and WS: Convergence

Grid

Web

The definition of WS-Resource Framework (WSRF)means that Grid and Web communities can moveforward on a common base. There is a migrationpath for NCES to Grid Computing via web servicesstandards. A new paradigm is being born.

WSRF

Startedfar apartin apps& tech

OGSI

GT2

GT1

HTTP

WSDL,

WS-*

WSDL 2,

WSDM

Have beenconverging

TODAY

OGSA

Security

GT3GT4

22

Grid Operating EnvironmentDynamic Automated Virtual Global Grid

Grid Operating Environment (GOE)

A concept infrastructure framework

system supporting coordinated resource

sharing and problem solving in dynamic,

geographically dispersed secure virtual

organizations.

Open Grid Services

Architecture (OGSA):

an open source,

community supported set

of services and protocols

including resource access

and security.

R2AD

Dynamic Automated Virtual Global Grid

R2AD, LLC

23

Automated Grid Computing in a GridOperating Environment (GOE)

1. Grid Computing

– Virtualization

– High PerformanceComputing (HPC)

2. Automation

– Installation

– Fielding

– Management

GOE brings two concepts together

• Data Grid

• Computational Grid (agent support)

• Resource Virtualization

• Scalability

R2AD, LLC

24

Typical Grid Layers

Grid-Enabled C4ISR Applications

Grid Visualization Grid Security

Grid Workflow

Grid VMs, Cache

Manager

Grid Middle Ware Engine(GT4 + OGSA + NCES)

High Performance & Secure Grid Networking (GIG-BE)

P

ac

kag

in

g-

ACS

GridDevelopment

- Grid RPC

- Grid MPI- MDA Scheduler

EMS

RSCs DECCs COCOMs DISA

network

Ref: Adapted from NaReGI-PSE National Grid Initiative slides from GGF - ACS-WG, OGSA Standardization Process

R2AD

25

What is OGSA

• Open service-oriented architecture based onWeb services for addressing Grid scenarios

• Component-oriented architecture

– Interchangeable components

• Meta OS functionalities

– Distributed and heterogeneous environment

• A rendering of these functions, based on Webservice architecture and specifications

• A GGF’’’’s flagship architecture and the blueprintfor industry standard grid computing

26

Context Services

InfoServices

InfraServices

SecurityServices

Rsrc Mgmt Services

Execution Mgmt

Services

DataServices

Policy

Mgmt

VO

Mgmt

Access

Integration

Provisioning

Cataloging

Boundary

Traversal

Integrity

Authorization

Authentication

WSRF WSN WSDM

Event

Mgmt

Trouble-

shootingDiscovery

Job

Mgmt

Logging

Execution

Planning

Workflow

Mgmt

Workload

Mgmt

Provisioning

Application

Mgmt

DeploymentConfigurationReservation

Naming

Self MgmtServices

Heterogeneity

Mgmt

Service Level

Attainment

QoS

Mgmt

Optimization

Information Services

Infrastructure Services

SelfMgmtServices

SecurityServices

Resource Mgmt Services

Execution Mgmt Services

DataServices

Context Services

27

OGSA Design Philosophy

• Service Oriented Architecture– Interface Extension (WSDL 2.0 ‘extends’ attribute)

– Resources as First Class Entities

• Expressed as WSRF-Resource Properties

– Data type extensibility and introspection

– Dynamic service/resource creation, migration, & destruction

• Component Based– Elements of the Architecture are pluggable

• Customizable– Support for dynamic, domain specific content, ...

– Within the same standardized framework

• Distributed Specification and Standardization– Identify and/or develop open and accessible standard

specifications• Active current work in GGF, OASIS, W3C, and DMTF.

28

Architecture OverviewInfrastructure Services

SYSTEMMANAGEMENT

UTILITYCOMPUTING

GRIDCOMPUTING

Core Services WSDM

OGSA-EMSOGSA Self Mgmt

GRID Computing, Utility Computing and System Management are different views of the same important problem domain.

Discovery

DAIS

VO Management

Information

Distributed query processing

ASP

Data Centre

Use Cases &Applications

Multi MediaPersistent Archive

InfrastructureServices

WS-Addressing

Privacy

WS-Base Notification

CIM-XML/JSIM

WSRF-RAP WS-Security

Naming

GFD-C.16

GGF-UR

Data Model

HTTP(S)/SOAP

SAML/XACML

WSDL

WSRF-LT

Trust

Data Transport

WSRF-RP

X.509

29

Execution Management Service

• WS-Agreement• Job Submission Description Language (JSDL)• Configuration Description Language• Application Contents Service (ACS)

Provisioning

• Deployment

• Configuration

App. Contents

Service

Information

Services

Service

ContainerData

Container

Accounting

Services

Execution

Planning

Services

Candidate Set Generator

(Work -Resource mapping)

Job Manager

Reservation

A typicalconfiguration

of EMS

30

Resource Management

• OASIS WSRF, WSN• OASIS WSDM-TC

– Management of Web Service (MOWS)– Management using Web Service (MUWS)

• GGF CMM WG– Resource Management in OGSA

DataService

OGSA ServicesLevel

Domain-specific capabilities

OGSA Capabilities

SecurityService

Infrastructure Level

Resource Level

ExecutionMgmt

Resources

WSRF, WSDMWSRF, WSDM

31

Security Services

• Authorization, Roles, and Access Privileges– Locally (site) managed– SAML and XACML Basis, Proxy Certs– Credential mapping provided by implementations.

32

Data Service Architecture

• Components of Grid Data Service

– Engine (Grid Data Service Factories)

– Activities

• Replicate, Query, Transform, Deliver

– Data Resource Implementations

– Role Mappers

XSLTTransform

Web RowSet, xml

DeliveryToURL

Ref: Open Grid Services Architecture Data Access and Integration (OGSA-DAI)

33

Enabling Technologies

Requirements& Design

Develop & LAN Testing

System IntegrationWAN Testing

Operational

R2AD

34

Services Around the World

• Runtime environment

– Geographically dispersed grid containers

• Development Environment

– Also geographically dispersed

– Setup to including installing all locally required components

• Configuration Management, Version history/access

• Compilers

• Editors/IDEs (and plug-ins)

• Project Management, Design tools

• Test grid containers

• Production Environment

– Software Release

• Unit, Stress Testing

• Installation Description

• Documentation, Delivery

– Maintenance

R2AD

Virtualization

R2AD

35

Fielding Considerations

• Patches/Version

– Where is what installed

– Scheduled/Timed with world wide usage in mind

– Rollback support? Backwards Compatibility

– common vulnerabilities and exposures (CVE) - IAVA

• System Comparisons

– What is different between two systems which should bethe same? Support automated re-hosting of services

• System Updates and Migration

– Incorporate old-version support (via service brokers)

– Monitor process, metric gathering & status, alerting

• Content Distribution

– Large file distribution

– Data and Service edge caching

R2AD

36

Example workflow

Grid Container

Distributes to nodes

1. Develop application

and Store in repository

Grid nodes

3. Submit

Job to

Scheduler

2. Edit application,

make final build.

3. Retrieve (recursively) all components not local which

are needed, comparing, checking, security, etc

Entry Point

ACS

ESM, JSDL, CDDLM,

GRAM

R2AD

ARI AAF

37

Service Installation & Management

• ACS working group within GGF

– IBM’s Solution Installation, NAREGI-PSE Grid (Japan)

– Possible Integration with Sun’s N1?

• Application (Service) Repository

– Grid Provisioning Appliance

– Secure (Signed & Trusted applications)

– Used by provisioning systems - a trusted repository

• Configuration Description, Deployment and LifecycleManagement (CDDLM)

– Working Group within GGF

– Deploying complex, distributed services

– Dynamic service configuration and management

– Automatically deploy, manage and remove services

• WSDM (Web Services Distributed Management)

– Technical Committee within OASIS, standard Mar05

R2AD

38

Example, from draft ACS Specification

Register

AA File

AA

AADescriptor

Transfer

ACS (site 1)

AA Repository

EPR

ACS (site 2)

AA Repository

EPR

<acs:AAID> <acs:name>

http://example.org/application1 </acs:name> <acs:version> <acs:major>1</acs:mager> <acs:minor>0</acs:minor> <acs:revision>0</acs:revision> </acs:version></acs:AAID>

<wsa:EndpointReference> <wsa:Address>

http://site1.example.org/ACS <wsa:Address> <wsa:ReferenceProperties> <acs:AAID> ... </acs:AAID> </wsa:ReferenceProperties> </wsa:Address></wsa:EndpointReference>

<wsa:EndpointReference> <wsa:Address> http://site2.example.org/ACS <wsa:Address> <wsa:ReferenceProperties> <acs:Originator>

http://site1.example.org/ACS </acs:Originator> <acs:AAID> ... </acs:AAID> </wsa:ReferenceProperties> </wsa:Address></wsa:EndpointReference>

Terms: Application Archive (AA), Application Archive ID (AAID)

39

Possible Usage at DISA

submitsegment

submission parameter

program binaries,initial data,configuration data,deployment procedures,self-management policies,and so on...

Deployed Systems

register

develop

archive

Development & Test

Environment

test

AA File

AA File

Application

Developer

Business Activity

Manager (e.g.: DISA)

R2AD

Secure CM’d Repository(Accredited Software)

PullSegment

Global InformationGrid (GIG)

40

Automated Deployment Example

Processor LayerNetwork Layer

WAN

1. Deploy Zero Administration aRchive (AAF) into the Grid Manager

Global Grid Framework Manager ReplicaAAF

Network Layer Processor Layer

AAF

WAN

2. Deployed Zero Administration aRchive (AAF) into the Grid Fabric

Global Grid Framework Manager Replica

Processor LayerNetwork Layer

WAN

3. Configuration of the Network Fabric Grid

Subnet

AAF Global Grid Framework Manager Replica

Processor LayerNetwork Layer

Solaris Solaris Solaris

Solaris

Linux

WAN

4. Configuration of the Processor Layer with OS

Solaris Solaris

W2KS W2KS W2KS

AAF Global Grid Framework Manager Replica

Processor Layer

cluster

Network Layer

WWW J2EE

WAN

5. Configuration of the Processor Layer with OS

WWW J2EE

W2KS W2KS

Oracle

Oracle

MSQL

Load

Bal.

AAF Global Grid Framework Manager Replica

Processor Layer

cluster

Network Layer

WWW J2EE:80

:80

:80

SUN SUN

WAN

6. Activation of Routes

WWW J2EE

W2KS W2KS

Oracle

Oracle

MSQL

Load

Bal.

AAF Global Grid Framework Manager Replica

R2AD

41

References

• GGF Standards Body

– http://www.ggf.org/

– OGSA Working Group

• https://forge.gridforum.org/projects/ogsa-wg

– ACS Working Group

• https://forge.gridforum.org/projects/acs-wg

ACS Structure of Application Archive

R2AD

Acknowledgements

The content of this presentation includes information

created by companies or groups below

OGSA-DAI

43