keys to soa success draft - dovel...
TRANSCRIPT
1
Copyright © 2006, ZapThink, LLC 1
Keys to SOA Success
Jason BloombergZapThink, LLC
Take Credit Code: SOAKEY
Copyright © 2006, ZapThink, LLC 2
Agenda
SOA Level Set
SOA Foundations
SOA Governance
The SOA Roadmap
Rolling Out SOA
Organizational Issues
Case Studies
2
Copyright © 2006, ZapThink, LLC 3
SOA: Level Set
Copyright © 2006, ZapThink, LLC 4
SOA: Definition
An approach to building and managing distributed computing infrastructures that considers IT resources as Services available and discoverable on a network.
3
Copyright © 2006, ZapThink, LLC 5
SOA Features
• SOA is architecture – a set of best practices for the organization and use of IT
• Abstracts software functionality as loosely-coupled, business-oriented Services
• Services can be composed into business processes (which are also Services) in a declarative manner
Copyright © 2006, ZapThink, LLC 6
Characteristics of SOA
• Services are loosely coupled – making a change to a service provider does not mandate changing any service consumers.
• Business processes are composed of services, and are in turn exposed as services.
• Services are policy-driven – business users can change how a service behaves.
• Systems are inherently integrated by virtue of composable services – not through layers of middleware.
4
Copyright © 2006, ZapThink, LLC 7
Characteristics of SOA(Cont.)
• Services leverage legacy systems – SOA does not mandate replacement of runtime infrastructure.
• In SOA, metadata controls how the system behaves instead of code – business logic trumps application logic.
• In SOA, it’s the contracted interface that matters, not the underlying runtime environment.
Copyright © 2006, ZapThink, LLC 8
SOA is Enterprise Architecture
What is Architecture?The fundamental organization of a system embodied by its
components, their relationships to each other and to the environment and the principles guiding its design and evolution. (IEEE P1471/D5.3)
Enterprise architecture includes:• An aggregated architecture of all the individual IT systems
within an organization • The human element within the enterprise• Systems, people, and organizational constructs at other
companies that have relationships with the enterprise• Individual consumers who are that enterprise’s customers• Corporate governance
5
Copyright © 2006, ZapThink, LLC 9
CompositeApplication
Delivery
BusinessServices
BusinessProcesses
SOA Fabric AtomicServices
Data Integration
Legacy Applications
and Middleware
Databases
Governance and Security InfrastructureGovernance and Security Infrastructure
SecuritySecurity
ManagementManagement
RoutingRouting
TransformTransform
AA
BBCC
DD
EE
AA
BBCC
DD
EE
MessagingMessaging
Rich Rich ClientsClients
Composite Composite AppsApps
DiscoveryDiscovery
Source: MW2 Consulting
SOA: A Technology View
Copyright © 2006, ZapThink, LLC 10
Service orientation…the next big thing?
Approach TimeframeProgramming
ModelBusiness
Motivations
Mainframe timesharing1960s –1980s
Procedural (COBOL) Automated business
Client/server 1980s-1990s
Database (SQL) and fat client (PowerBuilder, Visual Basic)
Computing power on the desktop
n-Tier/Web 1990s-2000sObject-oriented (Java, COM)
Internet/eBusiness
Service orientation 2000sService-oriented (SOAP, WSDL, UDDI)
Business agility
6
Copyright © 2006, ZapThink, LLC 11
SOA shifts the way we think
Compose ServicesIntegrate Silos
Loosely Coupled, Agile and Adaptive
Tightly Coupled
Designed to changeDesigned to last
Favors Heterogeneous TechnologyFavors Homogeneous Technology
Architecture makes it workMiddleware makes it work
Interactive and iterative development
Long development cycle
Metadata OrientedCode Oriented
Service Oriented ArchitectureTraditional Distributed Computing
Copyright © 2006, ZapThink, LLC 12
How to Think Service-Oriented
• Think of software functionality as Services
• Think of Services as sending and/or receiving messages
• Think of integration as a byproduct of Service composition
7
Copyright © 2006, ZapThink, LLC 13
The Best Technology…
• Is complex on the inside yet simple on the outside
• The secret is the abstraction layer
Copyright © 2006, ZapThink, LLC 14
What about Coupling?
• Coupling – the level of common knowledge necessary in a distributed computing exchange
• Tight coupling – one participant must have detailed knowledge about the other participant
• Full decoupling – the two participants need have no knowledge about each other in order to interact
• Loose coupling – the two participants may have specific, limited knowledge about each other
8
Copyright © 2006, ZapThink, LLC 15
Service Contracts & Loose Coupling
• Service contract – a document external to each participant that provides the information each participant needs to interact with the other
• The service contract is the mechanism that enables loose coupling
• Web Services Description Language(WSDL) – standard Service contract language
Copyright © 2006, ZapThink, LLC 16
What’s in a Contract?
Functional Requirements
Non-Functional Requirements
Service Operations Infrastructure URI
Security
ServiceLevel
Transactions Commerce / Biz Rules
Semantics Process
9
Copyright © 2006, ZapThink, LLC 17
Service Model Drives Contract-First Development
• Service contracts specify required functionality to IT and provided functionality to the business
• Service model represents the clearinghouse for information about IT environment
• Contracts go beyond WSDL:– Usage policies– Security policies– Consumer delivery contracts– Service-level agreements, etc.
Copyright © 2006, ZapThink, LLC 18
SOA Foundations
Business Model(Use Cases) Service Model Platform
Dependent Models
10
Copyright © 2006, ZapThink, LLC 19
The SOA Metamodel
Business Model(Use Cases) Service Model Platform
Dependent Models
Deployment View
System Architects &System Engineers
Implementation View
Technical Architects & Developers
Process View
Business Analysts
Logical View
Line-of-Business Users
TechnologyViews
BusinessViews
Use-case View
Service-Oriented Architects
Copyright © 2003 ZapThink LLC
Copyright © 2006, ZapThink, LLC 20
SOA Foundation: The Zachman Framework
11
Copyright © 2006, ZapThink, LLC 21
Copyright © 2006, ZapThink, LLC 22
SOA & Zachman
DATA FUNCTION NETWORK
SOA
12
Copyright © 2006, ZapThink, LLC 23
IBM Service Integration Maturity Model (SIMM)
Source: IBM
Copyright © 2006, ZapThink, LLC 24
HP SOA Maturity Model
Source: HP
13
Copyright © 2006, ZapThink, LLC 25
SOA Governance
Copyright © 2006, ZapThink, LLC 26
Corporate Governance
• Establishing and communicating the policiesthat employees must follow
• Giving employees the tools they need to be compliant with those policies
• Providing visibility into the levels of compliance in the organization
• Mitigating any deviations from established policy
From project to program to sustainable processFrom project to program to sustainable process
14
Copyright © 2006, ZapThink, LLC 27
Creating the Governance Model
• Who is responsible for creating policies?• Which policies are within scope?• How will they create and communicate those
policies?• How will policies be represented?• What tools will people use to follow policies?• How will management get visibility into policy
compliance?• How will mitigation issues be addressed?
Copyright © 2006, ZapThink, LLC 28
IT Governance Activity Types
• IT principles– Clarifying the business role of IT
• IT architecture– Defining integration and standardization requirements across IT, as
well as organizational principles & best practices that govern IT infrastructure
• IT infrastructure– Determining shared technology within the organization
• Business application needs– Specifying business need for purchased or internally developed IT
applications
• IT investment and prioritization– Choosing which initiatives to fund & how much to spend
15
Copyright © 2006, ZapThink, LLC 29
Elements of IT Governance Strategy
• Architecture board– Establish a cross-organizational architecture board with
the backing of top management to oversee implementation of IT governance strategy
• Architectural principles– Delineate comprehensive set of architectural principles to
guide, inform and support organization
• Architecture compliance strategy– Adopt specific measures to ensure compliance with the
architecture, including project impact assessments, a formal architecture compliance review process, and the involvement of architecture team in product procurement
Copyright © 2006, ZapThink, LLC 30
Architectural Governance Processes
• The architecture review and approval process– Defines structured approach for review process & approves
changes to the existing architecture– Makes decisions in accordance with architecture roadmap
• The architecture exceptions and escalation process– Provides for ability to appeal architectural decisions– Allows exceptions to the architecture to meet specific
business requirements
• The architecture maintenance process– Maintains the architecture & communicates architectural
changes to the stakeholders – Focuses on communicating new & changed Services to
stakeholders– Documents & communicates exceptions to architecture
16
Copyright © 2006, ZapThink, LLC 31
Architectural Governance Processes
• The architecture communication process– Makes the architecture available to everyone who needs
access to it – Promotes understanding of the importance of SOA
• The architecture compliance review process– scrutinizes the compliance of a specific project against
established architectural criteria and business objectives. Questions cover system engineering, information management, security and systems management. Formal review processes form the core of an enterprise architecture compliance strategy.
Copyright © 2006, ZapThink, LLC 32
SOA Governance
–Policy management• SOA configured & controlled via metadata,
including policy
–Visibility• Services abstract heterogeneous data sources,
providing necessary business intelligence
–Flexibility• Ability to build Services that address compliance
issues and adjust them as regulations or business needs change
Not just governance of SOANot just governance of SOA……governance in the context of SOAgovernance in the context of SOA
17
Copyright © 2006, ZapThink, LLC 33
SOA Guiding Principles
• Reuse, granularity, modularity, composability, and “componentization”
• Compliance to standards, both horizontal and industry-specific
• Identification and categorization of Services• Service provisioning and delivery• Service monitoring and tracking
Copyright © 2006, ZapThink, LLC 34
SOA Architectural Principles
• Encapsulation of application functionality• Separation of business logic from underlying
technology• Leveraging existing assets wherever an
opportunity exists• Service lifecycle management• Efficient use of system resources• Service performance
18
Copyright © 2006, ZapThink, LLC 35
Enabling Service Domains
• A Service Domain is a logical grouping of shared Services with a common business context
• Examples: customer-facing Services,purchasing-related Services
• Manage Services by managingthe Domains
• Move away from traditional IT silos for the purposes of managingServices, but retain technical teamsas needed
Copyright © 2006, ZapThink, LLC 36
Service Domain Roles
SOA governance introduces new roles that the company must provide for:
• The domain owner– Manages the direction of the domain and the business
relationships between the domain and business units, as well as other domains
– Helps business process owners in various business units understand the business application of the Services within the domain.
– Tracks the usage of Services for management purposes and ROI calculations
• Domain SO business analyst– Identifies abstracted, normalized business Services– Translates business requirements into Service definitions– Works closely with IT personnel to direct Service implementation
19
Copyright © 2006, ZapThink, LLC 37
Service Domain Roles
More new service domain roles:
• Line of business representative– Communicates business requirements– Identifies business Services for each of the domains
• Domain developer– Builds and maintains Services consistent with the SOA
lifecycle– Implements Services consistent with implementation
guidelines and SOA
• Service tester– Certifies that each Service conforms to the business
requirements that apply to that Service– Builds test cases for Service interface
Copyright © 2006, ZapThink, LLC 38
The SOA Roadmap
20
Copyright © 2006, ZapThink, LLC 39
The ZapThink SOA Roadmap
Copyright © 2006, ZapThink, LLC 40
ZapThink’s SOA Implementation Roadmap
Heterogeneous Systems with Proprietary Interfaces
Manage Services
Implement the SOA Metamodel
Service-Oriented ProcessSemantic Integration
Dynamic Service Discovery
Secure Service Interfaces
Wrap Legacy Systems in Services Interfaces
Service-Oriented Enterprise
Enterprise SOA Buildout
Mission-Critical SOA
SOA Pilots
“Grass Roots” Web Services Implementations
Create a Governance Framework
Contract-First Development
Cross-Departmental SOA
21
Copyright © 2006, ZapThink, LLC 41
Emerging Best Practices
• Architecture is a discipline– SOA mandates new practices
and methods
• Best Practice: Iterative development– Start small, and let it grow
• Best Practice: Contract-first, top-down as well as bottom-up
• Best Practice: Implementing governance sooner than later
You can’t get SOA from a software vendor!
Copyright © 2006, ZapThink, LLC 42
Starting Point: Architectural Visioning Session
• Attendees: architecture team, business analysts responsible for business processes
• Provide the SOA perspective
• Processes drive the Services, Services drive the technology
• Split into two initiatives: process definition followed by Service definition
• Balance “big picture” with realistic pilot starting point
22
Copyright © 2006, ZapThink, LLC 43
Bring Together Different Mindsets
• Developer Mindset: “Bottom-Up”– Everything is a Service or an Interface– Goal: connect Services– Method: Use objects and App Servers– Problem: Too many things to connect!
• Business Mindset: “Top-Down”– Everything is a Process– Goal: Run business efficiently: manage
processes– Method: Use diagrams and flowcharts– Problem: How can you turn “shelf-
ware” into software?
Copyright © 2006, ZapThink, LLC 44
SOA Pilots
• A few high ROI Services• Build acceptance for SOA• Get team up to speed• Work out the kinks• Pilot the governance model• Part of an iterative approach to SOA
• Piloting only the Services instead of the architecture• Remember, the pilot is one step on the roadmap
DANGER! Avoid the SOA Pilot PitfallDANGER! Avoid the SOA Pilot Pitfall
23
Copyright © 2006, ZapThink, LLC 45
Cross-Departmental SOA
• Organizational issues of governance and control become paramount
• Long-term architectural plan critical• Increased focus on semantic issues
Copyright © 2006, ZapThink, LLC 46
Rolling Out SOA
24
Copyright © 2006, ZapThink, LLC 47
How do you manage change?
• SOA is all about continuous and sometimes unpredictable change
• Development issues
– How to handle versioning?
– How to handle metadata management?
– How to develop changing policies?
• Runtime issues
– Service availability
– Policy enforcement
– Guarantee Service-level agreement
– Maintain low TCO
Copyright © 2006, ZapThink, LLC 48
Key Elements of Successful SOA Projects
• Building the right SOA team– A diverse group that bring different perspectives and
wide-ranging support to the implementation
• Handling organizational/people issues– Human resistance to change more challenging than
the technical issues
• Tackling the project iteratively– Won’t have full spec as you get started
• Managing the Service lifecycle– Very different from the traditional software
development lifecycle (SDLC)
25
Copyright © 2006, ZapThink, LLC 49
Building for ongoing change:The Death of the SDLC
• Traditional Distributed Software Development– “Waterfall” – Gather requirements,
design, develop, test, deploy as separate steps
– Works great when things don’t change– But, over time, usually fails to meet
ongoing business requirements• What if things are never “done”?
– Need Iterative approaches– Same order, with overlapping cycles– Better, but still assumes project
completion• SOA – applications are never complete,
Services are always in flux– Traditional SDLC wholly inadequate
Copyright © 2006, ZapThink, LLC 50
The Agile SOA Lifecycle
• Agile architectures demand agile development approaches
• The Agile Manifesto:– Working software over documentation– People over programming– Begin with metadata
• Agile SOA:– Iterate between architecture and
implementation– The business drives the applications– Contract-first development
Software lifecycle at the Service levelSoftware lifecycle at the Service level
26
Copyright © 2006, ZapThink, LLC 51
Implementing SOA:Bottom-Up
Put Service wrappers around existing applications
• Pros:– Reduces cost of integration
• Cons: – May not be reusable– May be redundant– Management challenges
Copyright © 2006, ZapThink, LLC 52
Implementing SOA:Top-Down
Create architectural plan & detailed design
• Pros:–Agility, reuse, flexibility
• Cons: –May not be implementable–Difficult to budget
27
Copyright © 2006, ZapThink, LLC 53
SOA Must be Both
• Develop the vision (but not the details) ahead of time
• Decompose some processes to identify target Services
• Build modest set of Services• Compose applications to enable
flexible processes• Refine architectural plan• Repeat
SOA should be iterative
Copyright © 2006, ZapThink, LLC 54
Challenge: Service Granularity
• The trick of building composable Services is building at the right level of granularity
• Challenges:– Engraining business logic into code– Decomposing legacy services that are not fine-
grained enough
• Method– Top-down process decomposition, vs. bottom-
up Service development– Must be iterative
28
Copyright © 2006, ZapThink, LLC 55
Service Lifecycle Considerations
• Lines of business should work with metadata — no IT involvement
• Services go thru individual lifecycles – development, test, production, revision
• Service development driven by Service contracts
Service Model
Service Metadata
ExistingInfrastructure
Lines ofBusiness
Copyright © 2006, ZapThink, LLC 56
The Roles of SOA
Use Case ViewSOAs
Use Case ViewSOAs
Use Case ViewSOAs
Logical ViewFunctional requirements
BusinessStakeholders
Information View
InformationSpecialists
Business AnalystsLOB Experts
Process ViewProcess ViewProcesses
Use Case ViewSOAs
Enterprise Architects
Deployment ViewPlatforms
Systems Experts
Data View
Data Experts
Implementation ViewComponents
Component ArchitectsDevelopers
Copyright (C) 2004, ZapThink, LLC
29
Copyright © 2006, ZapThink, LLC 57
The Services Lifecycle Team
• Business analysts– Adjust Service specs depending on business requirements– Reconfigure Services & composite applications
• Enterprise/SO architects– Manage Service model
• Technical/project architects– Specify implementation changes
• Service provider/consumer developers– Implement requirements and policy
• Testers– Insure conformance, simulate Service providers & consumers
• Support personnel– Respond to user issues
Copyright © 2006, ZapThink, LLC 58
An Instance of the Service Lifecycle in Action
New require-
ment
Simulatechanged Service
New/changedcontract
Implem’tchangedService
Problemresolution
Processconfig
Change?
Deploychanges
TestService
Businessuser
Support
Operations
Tester
Architect
Servicedeveloper
Businessanalyst
Consumerdeveloper
30
Copyright © 2006, ZapThink, LLC 59
Organizational Issues
Copyright © 2006, ZapThink, LLC 60
Building the right SOA team
• Shared Services cross organizational boundaries
• Siloed IT management styles are becoming obsolete
• The new role for enterprise architects
31
Copyright © 2006, ZapThink, LLC 61
People, Change and Fear
• People are inherently resistant to change
• People consider job security, authority and responsibility when asked to share
• Fear is the strongest emotion of all!
Copyright © 2006, ZapThink, LLC 62
Interaction Challenges
• Cultural Issues– Network Ops and Developers don’t talk to each other
• Budget issues– Who pays for Service Infrastructure / intermediaries?
• Responsibility issues– Who is in control of policy?
Developer/ArchitectDeveloper/Architect Network OperationsNetwork Operations
Services blur the Application / Network Boundary!Services blur the Application / Network Boundary!
32
Copyright © 2006, ZapThink, LLC 63
More Interaction Challenges
• Management issues– People tend to avoid risk, stay within “comfort zone” - may appear stubborn
• Technical issues– Architecture is a difficult subject
• Cultural issues– The “Ivory Tower” problem…
ArchitectArchitect Development/TestingDevelopment/Testing
Architecture is difficult to mandateArchitecture is difficult to mandate
Copyright © 2006, ZapThink, LLC 64
Convincing Technical Specialists
• Among the most risk-averse are technical specialists – mid to late-career experts in a (typically legacy) technology (e.g., “COBOL Jockeys”)
• Architectural change threatens their careers• Solution:
– Work with younger developers to build acceptance for SOA (eventually the TS’s will come around)
– Take a “leave and abstract” approach over “rip and replace”
33
Copyright © 2006, ZapThink, LLC 65
Working with IT Middle Management
• Middle managers threatened by SOA because of the Service domain reorganization
• Solution:– Technical specialties still required– New opportunities for Service domain
management
Copyright © 2006, ZapThink, LLC 66
The “Ivory Tower” Problem
• Architects create design and other artifacts, but don’t have the authority or mandate to require their use
• Development team considers them optional
• Business likes idea of architecture in principle, but short-term needs trump best practices
• When architects are external consultants, the “not invented here” syndrome makes the Ivory Tower worse
34
Copyright © 2006, ZapThink, LLC 67
The Power of the SOA Center of Excellence
• SOA experts who maintain a knowledge base of best practices– General and company-specific– Design time and runtime
• Drives SOA policy (either explicitly or implicitly)
• Can unify approaches across a large organization
Copyright © 2006, ZapThink, LLC 68
Case Studies
35
Copyright © 2006, ZapThink, LLC 69
Telco Example
Verizon
Large regional telecommunications provider formed thru the merger of Bell Atlantic and GTE
• Eliminated redundant systems inherited from the merger of Bell Atlantic and GTE
• On average, each transaction had been developed five to 25 times; one was deployed 45 different times
Challenge
• 2.5 million to 3 million Web Services transactions a day
• Helped Verizon slash its IT budget by 50 percent
• Included managing and securing the Services, charging for reuse and monitoring the performance of Service-enabled transactions
Results/Benefits
• IT Workbench SOA project, operational in 2004
• Thousands of developers, .NET and Java
• Focused on 250 business transactions – incl. verifying customer credit histories & looking up customer info
• 57 Service-oriented applications with 200 transactions
Solution
Copyright © 2006, ZapThink, LLC 70
Government Example
The Defense Finance & Accounting Service (DFAS)US Federal Government agency responsible for all Department of Defense accounting
• Very large, complex organization that handles accounting for Army, Navy, Air Force & Marines
• Process $1 Billion dollars of payments per day
• Challenge of how to bring together DoD finances
• “Analysis Paralysis” – too much time devoted to design
Challenge
• SOA effective in environments of extreme complexity
• Information architecture essential to resolve semantic issues in complex environments
• SOA appropriate in environments where there are many stakeholders with many needs
Results/Benefits
• Reduced hardware costs by streamlining operations
• Achieved “semantic alignment” across organizations
• Collated and rolled up information in the face of enormous complexity
• Focused on information architecture in the context of SOA
Solution
36
Copyright © 2006, ZapThink, LLC 71
Energy Company Example
Large Global Petroleum Firm
• Different divisions had different concepts of SOA
• No unified approach to Enterprise Architecture (EA)
• Lines of business often had very little in common
Challenge
• Moved toward establishing enterprise-wide EA committee
• Undertaking SOA pilot projects
Results/Benefits
• Brought together EAs from across company for first time in 6 years
• Worked to develop a common vocabulary
• Identified areas of redundancy suitable for shared Services
• Developed a SOA maturity model
Solution
Copyright © 2006, ZapThink, LLC 72
Read the Book!
• Service Orient or Be Doomed! How Service Orientation will Change your Business
• Published by Wiley
• AVAILABLE EVERYWHERE!
37
Copyright © 2006, ZapThink, LLC 73
Thank You!
Photos © Lisa Polucci
Jason Bloomberg