addressing systemic complexity with soa and cloud

Addressing Systemic Complexity with SOA and Cloud

????? SOA + Cloud

?????

Tony Shan

[email protected]

July 15, 2011

mailto:[email protected]

Contents at a Glance

• Introduction

• What is SOA

• What is Cloud

• Why SOA

• Why Cloud

• Why Combination

• State of SOA

• State of Cloud

• Barriers to Successful SOA

• Barriers to Successful Cloud

• State of Art of Complexity Management

• Need Pragmatic Method

• Increasing Disparate Representations

• Increasing Dynamics

• Increasing Fragmented Activities on Specifications

• Increasing Components

• Best Practices

• Wrap-up

Addressing Systemic Complexity © 2011 Tony Shan. All rights reserved. Proprietary and Confidential. 2

Introduction

Addressing Systemic Complexity 3 © 2011 Tony Shan. All rights reserved. Proprietary and Confidential.

Concept of SOA

The Open Group

• Service-Oriented Architecture (SOA) is an architectural style that supports service orientation, which is a way of thinking in terms of services and service-based development and the outcomes of services.

OASIS

• SOA is a paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains. It provides a uniform means to offer, discover, interact with and use capabilities to produce desired effects consistent with measurable preconditions and expectations.

4 © 2011 Tony Shan. All rights reserved. Proprietary and Confidential. Addressing Systemic Complexity

Definition of Cloud Computing

Cloud Computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics, three service models, and four deployment models.

-- NIST Definition, V15


SOA Benefits Greater

interoperability

• SOA, and the industry standards underpinning it, enable existing siloed applications to interoperate seamlessly and in an easier to maintain manner than any traditional EAI solution.

Increased reuse

• Once legacy systems and applications are service enabled, these services can be reused, which results in reduced ongoing development costs and results in reduced time to market. Further, business processes built as an orchestration of services can also be exposed as services, further increasing reuse.

More agile business

processes

• SOA reduces the gap between the business process model and implementation. This enables changes to business processes already implemented as orchestrations of services to be to be easily captured and implemented.

Improved visibility

• SOA can give improved business visibility by enabling business capabilities exposed as services, and the status of in-flight business processes automated with Business process management (BPM) technology, to be rapidly integrated into service-enabled enterprise portals aiding business decision-making.

Reduced maintenance

costs

• SOA development encourages duplicated overlapping business capabilities (services) that span multiple applications and systems to be consolidated into a small number of shared services.

Compliance and governance

• By realizing better and more standardized operational procedures, SOA provides the basis for a comprehensive security solution, and enables better visibility into business operations and exception conditions.


Value Proposition of Cloud •Enhance quality of service: improve the stability and reliability of services via Cloud-based solutions

•Improve flexibility: scale infrastructure resources on demand, or develop a hybrid approach, maintaining some fixed computing capacity in private cloud or your own data centers while using public cloud for additional needs

•Product-agnostic and vendor-independent: non-biased approach enables a reliable foundation of open technology and methodology for future growth and evolution

Enriched

capabilities

•Empower the alignment of IT and business: the increased adaptation and agility of IT enable better alignment

•Increase speed to value: reduce learning-curve time significantly

•Strategize IT planning: balance the immediate needs with the long-term goals

Increased

agility

•Reduce expenses: achieve economies of scale through the right mix of use of cloud computing infrastructure

•Avoid capital expenditure: shift IT spending to operational investments

•Implement greener IT: reduce your business’ carbon footprint, as the compute capacity of the cloud is higher than traditional data centers

Lowered

cost


Why SOA+Cloud Combined?

SOA

• Service Orientation

• Mostly software-intensive

• Application integration

• Loose coupling

• Reuse via shared services

• Asset-based use

• Service components and composition

• Predominantly used within the enterprise

Cloud

• XaaS

• Mainly hardware-focused

• Resource provisioning

• On-demand scalability

• Multi-tenancy

• Utility-based consumption

• Infrastructure pooling and outsourcing

• New business model for service delivery


•Increased agility •Faster time to market

•More cost savings •Reduced integration •Easier outsourcing

SOA Landscape


State of Cloud


Source: Gartner

Barriers to Successful SOA


Initiatives solely led and driven by techies

Insufficient semantics in service composition

Disconnection between traditional education and real-world SOA

implementation needs

Absence of holistic roadmaps with

specificity

Lack of well-defined service models (business and

technical)

Gap between logical architecture and

infrastructure

Ad-hoc governance (dictatorship or

anarchy)

Home-grown reference models

Product lock-in with no or limited

interoperability

Inability to quantify ROI/TCO and

improper-sizing

Project-centric execution without

reuse/sharing disciplines

Immature specifications and

standardization

Top 10 Cloud Adoption Inhibitors Risk-Testing

•How is the cloud computing vendor managing risk?

Data Location

•Where is the data being stored? In-country or out of the country? What restrictions and laws are placed upon the data dependent on location of stored data?

Data and Code Portability

•Once the data has been put onto the cloud computing system, how difficult will it be to get the data back out?

Data Loss

•Does the cloud computing system back-up and restore data?

Data Privacy

•Does the data become more vulnerable when it is located on an external cloud computing system?

Source: Adapted and extended from Gartner and InformationWeek reports in 2009 and 2010.

Vendor Viability

•What will happen if the cloud computing vendor goes out of business?

Performance

•How can SLA guarantee performance?

ROI

•Does operational expense always trump capital expense, at least in technology investment?

Security

•What are the vulnerabilities in the Cloud architecture?

Control

•How can a cloud provider be trusted that they care about your IT processing as much as you do?

© 2011 Tony Shan. All rights reserved. Proprietary and Confidential. Addressing Systemic Complexity 13

How are cloud solutions crafted?


What these lead to…


Page 1 of 2

How to Effectively Deal with Complexity?


State of Art


Source: Wikipedia

Wisdom


The essential complexity is

irreducible.

Fred Brooks - Author of “The Mythical Man-Month”

- Coined “Computer Architecture”

- 1999 Turing Award

In Search for a Pragmatic Approach


Tomorrow’s computing systems

cannot be built using methods of

today. [Computing Research

Association (CRA) report]

We can't solve problems by

using the same kind of

thinking we used when we

created them. [Albert Einstein]

Conquering Complexity – one of five

“deliberately monumental" research

challenges, each requiring "at least a

decade of concentrated research in

order to make substantive progress”.

[“Grand IT Research Challenges” report

supported by NSF]

http://www.facebook.com/album.php?profile=1&id=93496868329

Key Areas

Notation

Technologies

Specification

Modules


21

Increasing Disparate Representations

Addressing Systemic Complexity © 2011 Tony Shan. All rights reserved. Proprietary and Confidential.

Semantic Notations


Source: The “Physics” of Notations


Increasing Dynamics

SIP

MTOM REST SCA XOP

XPATH

DSL AJAX

ITIL CMM

WOA COBIT POX

PaaS SaaS

IaaS

Service-Oriented Design Accelerator


Access & Interaction Layer

Integration/Communications Layer

Services & Components Layer

Composite Services Layer

Business Process Layer

Enterprise Resources Layer

Reference Model of Solutions Architecture for N-Tier ApplicationsDesigned by Tony Shan

Jets

peed

Life

ray

JOSSO

Yale

CAS

Acegi

Mo

de

lin

g &

De

ve

lop

me

nt

To

ols

Eclip

se,

Ne

tbe

an

sA

nt, M

ave

nC

VS

, S

ub

ve

rsio

n,

Bu

gzill

a, F

xC

op

JU

nit,

NU

nit,

Cru

ise

Co

ntr

ol

Arg

oU

ML

, S

tarU

ML

Cro

ss

cu

ttin

g A

sp

ec

ts &

Pa

tte

rns

Mic

roso

ft E

nte

rprise

Lib

rary

Mic

roso

ft

Ap

plic

atio

n B

locks

Go

F d

esig

n p

atte

rns

Ja

va

EE

pa

tte

rns

Asp

ectJ

, S

prin

g,

JB

oss A

OP

Ap

plic

ati

on

& S

erv

ice

Fra

me

wo

rks

JS

F,

Ta

pe

str

y,

Wic

ke

tR

IFE

, S

ea

mK

ee

l S

prin

gD

WR

, E

ch

o2

,

JS

ON

-RP

C,

Do

jo

We

bW

ork

, S

tru

ts,

Be

eh

ive

Ho

sti

ng

En

vir

on

me

nt

JB

oss A

pp

Se

rve

rT

om

ca

tX

en

Grid

En

gin

e,

Glo

bu

sL

AM

P

Se

cu

rity

Op

en

SS

HJa

cksu

mO

pe

nS

AM

LE

nh

yd

ra O

yste

rO

pe

nS

SL

Op

era

tio

na

l M

an

ag

em

en

t

Na

gio

sX

MO

JO

DM

TF

CIM

JM

X/S

NM

PW

EB

M S

erv

ice

s

ActiveB

PEL

jBPM

Ope

n-Xch

ange

Shark

JaW

E

WS-B

PEL

jUDDI

ebXM

L Reg

istry

SCA

SDO

Axis

REST, A

jax

Dro

ols

Swee

tRul

es

NxB

RE

Service

Mix

Mul

e

ActiveM

Q

Ope

nJM

S

Ope

n ESB

Der

by

Ope

nCM

S

Sugra

CRM

Mon

dria

n

Wek

a

Increasing Fragmented Activities on Specifications


WS-Security

WS-Attachments

WS-Addressing

WS-Policy WS-PolicyAttachment

WS-AtomicTransaction

WS-BusinessActivity

WS-RX WS-CDL WS-Trust

WS-Federation

WS-Coordination WS-TX WS-Discovery WS-SX

WS-Resource WS-Eventing

WS-Notification WS-Topics

WS-SecureConversation WS-ReliableMessaging WSRF WSE

WS-Transfer

WS-Enumeration

WS-MEX

Stack of Standards


Foundation

Presentation

Management

Composition/Orchestration/Construction

Process

Messaging

XML Processing

· DOM

· SAX

· XPath

· XSLT

· XQuery

· .Net XML

Serialization

· JAXB

· SDO

· StAX

Description

· XML

· XML Schema

· WSDL

· XML Info Set

· XOP/MTOM

· SML

· DMCBX

· RELAX NG

· Schematron

Assertion Lang

Communications and Events

· Transport: SSL/

TLS

· Network: IPSec

· BEEP

· HTTP/IIOP/MQ

· WS-Eventing

· WS-Notification

· WS-Addressing

Security

· WS-Security

· WS-

SecureConversation

· WS-Federation

· SAML

· Liberty Alliance IDFF

· WS-Trust

· XKMS

· XACML

· XrML

· EPAL

Interoperability

· WS-I Basic Profile

· WS-I Basic Security

Profile

· WS-I Reliable Secure

Profile

· Governance

Interoperability

Framework (GIF)

· Reusable Asset

Specification (RAS)

· DMTF CIM

Resources

· WSRF

· WSRF-

ResourceProperties

· WSRF-

ResourceLifetime

· WSRF-ServiceGroup

· WSRF-BasicFaults

· WS-Transfer

· RRSHB

· WS-Enumeration

Transaction

· WS-Coordination

· WS-Business Activity

· WS-Atomic Transaction

· WS-Context

· WS-CF

· WS-TXM

· WS-TX

Semantics

· RDF

· WSDL-S

· SA-WSDL, SA-REST

· OWL-S, RDF/S

· SWSO, WSMO

· SWSL, WSML

· SOA-S, FEARMO,

ODM

QoS

· WS-

ReliableMessenging

· WS-Reliability

· WS-RX

Discovery

· OWL

· WS-Discovery

· WS-

MetadataExchange

· UDDI

· ebXML

· SwSA

· WS-Policy

· WS-PolicyAttachment

· WS-SecurityPolicy

· WS-Manageability

· WS-Management

· WSDM

· WS-Provisioning

· WSDM

· WSRP

· XUL

· XAML

· XBL

· XForms

· MXML

· Ajax

· WS-Choreography

· BPMN

· BPDM

· BPML/BPQL

· XPDL

· WSCI

· CDL4WS

· BMM

· UML

· OAGIS

· BPEL

· WS-CAF

· WSE

· WCF

· JAX-WS

· SAAJ

· SCA

· Axis

· SOAP

· REST

· JSON

· SwA

· WS-I Attachment Profile

· XML Security: XML Encryption, XML Signature

28

Increasing Modules

Enterprise Portal: Role based portal that

is available 24x7. Provides single point of

entry for all users, multi-channel support,

consistent look and feel, access to business

capabilities based on role.

Custom Applications: These are either built on an

App Server, Portal or proprietary thick client.

Application Framework required to leverage reuse.

Examples: Logging, Exception handling, data services,

application configuration, monitoring, search framework,

notification framework, service proxy, Single Sign-On

Packaged Applications: These are the best of the breed

packaged application that also act as the system of record for

a particular business function.

Enterprise Services: Basic services required across the

enterprise. Examples: Directory Service, Content Management,

Search, eMail, Calendar, IM, Discussion Forum, White Board, etc. Business Process

Manager: Configure

and automate business

process. Provide

business users the

capability to modify the

business process &

policies.

Enterprise Service Bus: Route services to the appropriate

destination; receive and transmit messages in any protocol, provide

message transformation, routing, validation, auditing, security,

monitoring and reporting services.

Service Registry:

Service registry

containing service

properties such as

service capabilities,

parameters, service

levels, etc.

Shared Data

Services: Extract,

Transform & Load (ETL),

Electronic Data

Interchange (EDI),

Enterprise Information

Integration Data Quality

(Matching Engine, Master

Data Management)

Service Manager:

Manage service

lifecycle across the

enterprise.

Enterprise Application

Integration: Traditional

enterprise integration approach.

Provide Application Adapters,

Business Process, Messaging,

Security, etc. capabilities. Mostly

proprietary in nature and

application integration generally

implemented as a point-to-point

integration on a Hub..

Mainframe Application : Access data via gateways

Enterprise Security: Provide

user authentication,

authorization, identify

management, profile

management, delegated admin,

etc.

Business Service

Management: Monitoring,

capacity planning, utility

computing

Mapping SOA Reference Architecture

to the Enterprise SOA Maturity Model

Traditional

Development

Develop Web

Applications

Composite

Applications

Automate

BP

© 2011 Tony Shan. All rights reserved. Proprietary and Confidential. Legacy Application: Applications that do not have open APIs & are not web based

Source: SOA Practitioners’ Guide

Cloudonomic Paradigm


Service Integration & Management Platform and Lifecycle Engineering

Cloud Computing Foundation

Principles Methodology Process Techniques Tools Patterns Policy Standards Practices Maturity

Dev

Support

•Collaborati

on

•Asset

Mgmt

•Build Mgmt

•Test Mgmt

•Release

Mgmt

Biz

Support

•Customer

Mgmt

•Partner

Mgmt

•Revenue

Mgmt

•Billing

Mgmt

Operation

Support

•Incident

Mgmt

•Change

Mgmt

•Config

Mgmt

•SLA Mgmt

Provider

Support

•Product

Mgmt

•Inventory

Mgmt

•Capacity

Mgmt

•Resource

Manager

Interaction

Enablement

Virtual Hosting

Runtime

Cloud Service Factory Plan Define Model Implement Deploy

Channel Device User exp Participate

Onboard Catalog Provision Reports

UI Process Service Data

Security Repository Metering Monitor

Federated Composite Cloud Fabric

Aggregate Mashup

Bundle

Best Practices


Hybrid

Attitude Program

Case Study of Healthcare Vertical


Business Challenge

• Healthcare cost containment services to U.S. payers and insurance providers

• An average annual increase of over 40% for the last four years

• Requires continuous management of and investment in processing capacity and utilities, including power and cooling

• Using traditional servers and suffering from occasional sub-par system performance

Solution

• The cloud solution provides a comprehensive set of building blocks giving the company the flexibility to build systems in a way that was most applicable to their business

• The public cloud has all the pieces needed at a price point well below the competition

• Utilizing the computing, storage, and queuing capability in the public cloud to enable the claim processing system capacity to increase and decrease as required to satisfy the service level agreements (SLAs)

• Meet the needs of requiring massive amounts of computing resources that far exceed the existing system, without purchasing, housing and maintaining the servers themselves

Result

• Providing retrospective claim processing services which is essentially processing multiple years’ worth of data at a time

• Encrypt and send the data that need processed to the cloud while the majority of the PHI is retained in-house and is reconnected with the processed data upon its return

• The firm was able to produce a HIPAA-compliant application that scales as they do

• The public cloud is a great fit for the company because it allows them to economically expand and contract their computing resources to match the ebb and flow of the claim volume without making any long-term financial commitments or capital expenditures

Make it Simple…


Thank You

Merci

Grazie

Gracias

Obrigado

Danke Japanese

French

Russian

German Italian

Spanish

Brazilian Portuguese

Arabic

Traditional Chinese

Simplified Chinese

Hindi

Romanian

Thai

Korean Multumesc

Contact: Tony Shan

Email: [email protected]

Web: http://tonyshan.com

© 2006-2011 Tony Shan. All rights reserved. Duplication, reproduction or disclosure of the contents in this document is prohibited without prior written permission of the author.


mailto:[email protected]

http://tonyshan.spaces.live.com/




addressing systemic complexity with soa and cloud

Technology