download or view powerpoint presentation slides
DESCRIPTION
TRANSCRIPT
Realizing Service Oriented ArchitectureRealizing Service Oriented Architecture
Anna LiuArchitect AdvisorMicrosoft [email protected]
PreamblePreamble
What are the takeaways?What are the takeaways?Practical advice for implementing SOA Practical advice for implementing SOA based on real world case studiesbased on real world case studies
Actionable advice on key design issuesActionable advice on key design issues
Relevant insight into the proven practices Relevant insight into the proven practices of customers who have successfully built of customers who have successfully built SOA solutionsSOA solutions
Global Bank – a reference Global Bank – a reference implementation for SOAimplementation for SOA
Multi-National bankMulti-National bank
Grown through acquisitionGrown through acquisition
Heterogeneous environmentHeterogeneous environmentMainframe systemsMainframe systems
J2EE systemsJ2EE systems
Microsoft systems (COM and .NET)Microsoft systems (COM and .NET)
Provides many products & servicesProvides many products & servicesChecking, SavingsChecking, Savings
Bill Payment, Insurance, Investing, Bill Payment, Insurance, Investing, Institutional and Personal BankingInstitutional and Personal Banking
Internet Banking
Wireless
Aggregation
Branch Banking
CRM
Core Banking
Wealth Management
Treasury / Forex
Trading / Back office
Payment Systems and Card Mgmt
3D Secure
Business Intelligence
EAI
Straight through Processing
ATM / POS
Global Bank ArchitectureGlobal Bank Architecture
SituationSituationFragmented customer viewFragmented customer view
Difficult to get complete picture of Difficult to get complete picture of customer’s relationship with the bank in customer’s relationship with the bank in one placeone place
Islands of Data and Business LogicIslands of Data and Business LogicIntegration is an afterthoughtIntegration is an afterthought
Difficult/Expensive/Takes too long to Difficult/Expensive/Takes too long to meet new business demandsmeet new business demands
Business rules duplicated across many Business rules duplicated across many systemssystems
Complex, brittle silo-ed projects difficult Complex, brittle silo-ed projects difficult to evolveto evolve
Realizing SOA at Global BankRealizing SOA at Global Bank
Top SOA IssuesTop SOA Issues1. Schema Rationalization 1. Schema Rationalization 2. Service Design 2. Service Design 3. Reliable Messaging 3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
Top SOA IssuesTop SOA Issues1. Schema Rationalization1. Schema Rationalization2. Service Design 2. Service Design 3. Reliable Messaging 3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
Schema RationalizationSchema RationalizationServices need to agree on certain Services need to agree on certain
conceptsconceptsWhat is the format for an Address?What is the format for an Address?
What does an Expense Report look like?What does an Expense Report look like?
How do we define what a Contact looks How do we define what a Contact looks like? like?
Schema rationalization allows Schema rationalization allows commonality of many business commonality of many business conceptsconcepts
e.g. How would you provide a unified view e.g. How would you provide a unified view of the Customer in the absence of such of the Customer in the absence of such rationalization?rationalization?
1
Schema Rationalization CRM Architecture
Schema Rationalization CRM ArchitectureSQL ServerSQL Server
(in house)(in house)
SiebelSiebel(commercial)(commercial)
OtherOther(J2EE based)(J2EE based)
ContactTb
PK CntctI
FK2 CompanyIFirsNameMiddleNameLastNameSpouseNameAlteranateNameSalutationHobbiesInterestFunctionsEventsKeyContactFlgDecisionMakerFlgCitivisionIDType
FK3 BusinessAddressIFK4 OtherAddressIFK5 HomeAddressIFK9 CellPhoneI
EmailEmail3Comments
FK7 LastModifiedByLastModifiedTimeDeletedFlgDeleteComment
FK6 OwnerCreationTime
FK8 CreatedByFK1 SystemI
ContactTb
PK CntctI
FK2 CompanyIFirsNameMiddleNameLastNameSpouseNameAlteranateNameSalutationHobbiesInterestFunctionsEventsKeyContactFlgDecisionMakerFlgCitivisionIDType
FK3 BusinessAddressIFK4 OtherAddressIFK5 HomeAddressIFK9 CellPhoneI
EmailEmail3Comments
FK7 LastModifiedByLastModifiedTimeDeletedFlgDeleteComment
FK6 OwnerCreationTime
FK8 CreatedByFK1 SystemI
ContactSchema
PK CntctI
KnownAsMIIndexLastAlteranateNameSalutationKeyContactFlgDecisionMakerFlgTypeBusinessAddressOtherAddressHomeAddressMobileEmailEmailOtherCommentsLastModifiedByLastModifiedTimeDeletedFlgDeleteCommentOwnerCreationTimeCreatedOwnerSiebelIndex
ContactSchema
PK CntctI
KnownAsMIIndexLastAlteranateNameSalutationKeyContactFlgDecisionMakerFlgTypeBusinessAddressOtherAddressHomeAddressMobileEmailEmailOtherCommentsLastModifiedByLastModifiedTimeDeletedFlgDeleteCommentOwnerCreationTimeCreatedOwnerSiebelIndex
MyBusinessContacts
PK ContactID
FirstMiddleLastAlteranateNameSalutationBusinessAddressOtherAddressHomeAddressMobileEmailEmailOtherCommentsLastModifiedByLastModifiedTimeOwnerCreationTimePrimaryContactOpportunityIndexEmail2Email3
MyBusinessContacts
PK ContactID
FirstMiddleLastAlteranateNameSalutationBusinessAddressOtherAddressHomeAddressMobileEmailEmailOtherCommentsLastModifiedByLastModifiedTimeOwnerCreationTimePrimaryContactOpportunityIndexEmail2Email3
1
Schema Rationalization CRM Architecture
Schema Rationalization CRM ArchitectureSQL ServerSQL Server
(in house)(in house)
SiebelSiebel(commercial)(commercial)
OtherOther(J2EE based)(J2EE based)
ContactTb
PK CntctI
FK2 CompanyIFirsNameMiddleNameLastNameSpouseNameAlteranateNameSalutationHobbiesInterestFunctionsEventsKeyContactFlgDecisionMakerFlgCitivisionIDType
FK3 BusinessAddressIFK4 OtherAddressIFK5 HomeAddressIFK9 CellPhoneI
EmailEmail3Comments
FK7 LastModifiedByLastModifiedTimeDeletedFlgDeleteComment
FK6 OwnerCreationTime
FK8 CreatedByFK1 SystemI
ContactTb
PK CntctI
FK2 CompanyIFirsNameMiddleNameLastNameSpouseNameAlteranateNameSalutationHobbiesInterestFunctionsEventsKeyContactFlgDecisionMakerFlgCitivisionIDType
FK3 BusinessAddressIFK4 OtherAddressIFK5 HomeAddressIFK9 CellPhoneI
EmailEmail3Comments
FK7 LastModifiedByLastModifiedTimeDeletedFlgDeleteComment
FK6 OwnerCreationTime
FK8 CreatedByFK1 SystemI
ContactSchema
PK CntctI
KnownAsMIIndexLastAlteranateNameSalutationKeyContactFlgDecisionMakerFlgTypeBusinessAddressOtherAddressHomeAddressMobileEmailEmailOtherCommentsLastModifiedByLastModifiedTimeDeletedFlgDeleteCommentOwnerCreationTimeCreatedOwnerSiebelIndex
ContactSchema
PK CntctI
KnownAsMIIndexLastAlteranateNameSalutationKeyContactFlgDecisionMakerFlgTypeBusinessAddressOtherAddressHomeAddressMobileEmailEmailOtherCommentsLastModifiedByLastModifiedTimeDeletedFlgDeleteCommentOwnerCreationTimeCreatedOwnerSiebelIndex
MyBusinessContacts
PK ContactID
FirstMiddleLastAlteranateNameSalutationBusinessAddressOtherAddressHomeAddressMobileEmailEmailOtherCommentsLastModifiedByLastModifiedTimeOwnerCreationTimePrimaryContactOpportunityIndexEmail2Email3
MyBusinessContacts
PK ContactID
FirstMiddleLastAlteranateNameSalutationBusinessAddressOtherAddressHomeAddressMobileEmailEmailOtherCommentsLastModifiedByLastModifiedTimeOwnerCreationTimePrimaryContactOpportunityIndexEmail2Email3
1
Reading Service
Schema Rationalization Options
Reader-Makes-RightReader-Makes-RightWhen an incoming message When an incoming message arrives, the receiver fixes itarrives, the receiver fixes it
The source schema and The source schema and destination schema are destination schema are comparedcompared
Stuff that easily maps is Stuff that easily maps is movedmoved
The best fit to transform the The best fit to transform the mismatch is hand-craftedmismatch is hand-crafted
N-Squared CombinatoricsN-Squared CombinatoricsEvery reader must know about Every reader must know about all of its partnersall of its partners
For N partners, N*(N-1) hand-For N partners, N*(N-1) hand-crafted transformations must crafted transformations must existexist
As N gets large, this gets too As N gets large, this gets too largelarge
Some datarequires
transformation
Svc Svc
Svc
Svc
Svc
Svc
Svc
Svc Svc
Svc
Svc
Svc
12 Services12 X 11 = 132
message transformers
Other datacopies through
1
Schema Rationalization Recommendation - Define a Canonical SchemaModel the EntitiesModel the EntitiesCreate a static, Create a static, Canonical SchemaCanonical Schema
Agree on XML namespacesAgree on XML namespaces
Only expose XSD data typesOnly expose XSD data types
Agree on naming conventionsAgree on naming conventions
Keep it simpleKeep it simple
Canonical SchemaCanonical SchemaRepresents the authoritative, common definitionRepresents the authoritative, common definition
Other definitions can be derived from this – TraceabilityOther definitions can be derived from this – Traceability
InteroperabilityInteroperability
It is not one large XML-Schema It is not one large XML-Schema really a bucket of schemas that collectively form the Canonical Schemareally a bucket of schemas that collectively form the Canonical Schema
1
Schema Rationalization Canonical Schema Usage
Schema Rationalization Canonical Schema Usage
MessageMessageUsed for communication between servicesUsed for communication between services
Requires Canonical SchemaRequires Canonical Schema
Reference DataReference DataCached data – usually de-normalizedCached data – usually de-normalized
Requires Canonical SchemaRequires Canonical Schema
Outside the Service vs. Inside the ServiceOutside the Service vs. Inside the Service
Requires Open Schemafor Interoperability Yes No
Outside Inside
1
Top SOA IssuesTop SOA Issues1. Schema Rationalization 1. Schema Rationalization 2. Service Design2. Service Design 3. Reliable Messaging 3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
Service DesignService Design
FactoringFactoringData ownershipData ownership
Coarse grain servicesCoarse grain services
Service AnatomyService AnatomyHow do we expose Services?How do we expose Services?
Interface designInterface design
Multi-channel access / transport Multi-channel access / transport infrastructure independence infrastructure independence
2
Ref Vers#23of Employee
DataUpdate!
Ref Vers#24of Employee
Data
Service Design Factoring - When There’s Data Needed By ManyService Design Factoring - When There’s Data Needed By Many
Data May Be Needed by Many ServicesData May Be Needed by Many ServicesCustomers, Employees, Parts, etcCustomers, Employees, Parts, etc
Each Piece of Data Needs an OwnerEach Piece of Data Needs an OwnerOnly the Owner May Change ItOnly the Owner May Change It
Owner Publishes Changes to OthersOwner Publishes Changes to OthersOthers Receive Updates and Cache VersionsOthers Receive Updates and Cache Versions
Sales Service
AuthoritativeCustomer
Data
HR Service
AuthoritativeEmployee
Data – Vers#23
Ref Vers#24of Employee
Data
AuthoritativeEmployee
Data – Vers#24UpdateEmployees
UpdateEmployees
Vers#24Vers#24
2
22RequestRequestUses: Vers-XUses: Vers-XPlease MakePlease MakeData ChangeData Change
Service DesignFactoring - Requesting The Owner Make ChangesService DesignFactoring - Requesting The Owner Make Changes
If a Non-Owner Wants a Change It Must Ask for the If a Non-Owner Wants a Change It Must Ask for the ChangeChange
This is a Request Sent to the Owning ServiceThis is a Request Sent to the Owning ServiceThe Owning Service May Agree to Change the DataThe Owning Service May Agree to Change the DataIf It Changes, This Affects the Next VersionIf It Changes, This Affects the Next Version
Service-BService-B33A’s-DataA’s-Data
Vers-YVers-Y
A’s-DataA’s-DataVers-XVers-X
11
OwningOwningService-AService-A
2
Service Design Factoring – Service GranularityService Design Factoring – Service Granularity
Service granularity refers to scope of Service granularity refers to scope of functionality a service exposesfunctionality a service exposesSmall scope – e.g. data access – small Small scope – e.g. data access – small business valuebusiness valueBusiness Services as composite Business Services as composite interfacesinterfacesCoarse grained services to more closely Coarse grained services to more closely match business capability - provide match business capability - provide greater business valuegreater business valueconstructed by composing lower-level constructed by composing lower-level services to meet business process services to meet business process requirementsrequirements
2
Service
Service Design Service AnatomyService Design Service Anatomy
Ser
vice
Fac
ade
Web Services
Remoting
MSMQ
Agent MessagesMessages
COM+
2
Service Design Method-centric interfaceService Design Method-centric interface
[WebMethod][WebMethod]
Customer GetCustomer (int custId)Customer GetCustomer (int custId)
{{
Customer retVal;Customer retVal;
......
return retVal;return retVal;
}}
Pros•Simple to developCons•Encourages RPC like behavior (near model)•Encourages exposing internal state
2
Service DesignMessage-centric interfaceService DesignMessage-centric interface
[WebMethod][WebMethod]
GetCustomerMsgResponse GetCustomer GetCustomerMsgResponse GetCustomer (GetCustomerMsgRequest req)(GetCustomerMsgRequest req)
{{
GetCustomerMsgResponse retVal;GetCustomerMsgResponse retVal;
req.Process();req.Process();
......
return retVal;return retVal;
}}
Pros•Encourages encapsulating internal state•Encourages message-based communication (far model)Cons•More time-consuming to develop
2
Service DesignCommand MessageService DesignCommand Message
[WebMethod][WebMethod]CmdResponse DoCommand (CommandRequest req)CmdResponse DoCommand (CommandRequest req){{
switch (req.Command)switch (req.Command){{
case Cmds.QueryCustomer:case Cmds.QueryCustomer:retVal = DoQueryCustomer(req);retVal = DoQueryCustomer(req);
case Cmds.UpdateInvoice:case Cmds.UpdateInvoice:retVal = DoUpdateInvoice(req);retVal = DoUpdateInvoice(req);
......}}
return retVal;return retVal;}}
Pros•Expose a single service which accepts many kinds of messages•Easier to secure a single URL•Dynamic command routingCons•More time-consuming to develop•Solution is more complex
2
Top SOA IssuesTop SOA Issues1. Schema Rationalization 1. Schema Rationalization 2. Service Design 2. Service Design 3. Reliable Messaging3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
Reliable Messaging Reliable Messaging
In the absence of Reliable Messaging infrastructureIn the absence of Reliable Messaging infrastructureRequests get lost…Requests get lost…
Requests arrive more than once…Requests arrive more than once…
Idempotent means it’s OK to arrive multiple timesIdempotent means it’s OK to arrive multiple timesAs long as the request is processed at least once, the As long as the request is processed at least once, the correct stuff occurscorrect stuff occurs
IdempotentIf not yet withdrawal#XYZ then withdraw$1 Billion and label
as #XYZ
NaturallyIdempotent
Read Record X
Not IdempotentWithdrawing
$1 Billion
3
Reliable MessagingReliable MessagingSome queuing systems may offer some form Some queuing systems may offer some form of guaranteed deliveryof guaranteed delivery
message is delivered or the sender is notifiedmessage is delivered or the sender is notifiedat most once, exactly once semanticsat most once, exactly once semanticsNo need for idempotent message processing?No need for idempotent message processing?
However, only tell you that the message got However, only tell you that the message got to the destination systemto the destination system
has the destination business capability processed it?has the destination business capability processed it?
For synchronous Request/Response For synchronous Request/Response interaction it may not help!interaction it may not help!
You still have to implement a timeout and retry if you You still have to implement a timeout and retry if you don’t hear back from the business capability!!don’t hear back from the business capability!!Now that you’re retrying, the request had better be Now that you’re retrying, the request had better be idempotent!!!idempotent!!!
3
Top SOA IssuesTop SOA Issues1. Schema Rationalization 1. Schema Rationalization 2. Service Design 2. Service Design 3. Reliable Messaging 3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
Entity AggregationEntity Aggregation
Why Entity Aggregation?Why Entity Aggregation?Properties for an Entity (Contact or Properties for an Entity (Contact or Customer) may come from more than Customer) may come from more than one Legacy systemone Legacy systemSame Entity may be replicated across Same Entity may be replicated across Legacy systemsLegacy systems
Common ScenariosCommon ScenariosSchema is different across Legacy Schema is different across Legacy systems – typically subsettedsystems – typically subsetted
Schema Reconciliation is requiredSchema Reconciliation is required
Schema is different - Keys are differentSchema is different - Keys are differentInstance Reconciliation requiredInstance Reconciliation required
4
Entity AggregationSchema ReconciliationEntity AggregationSchema Reconciliation
CUSTOMERFirstNameLastNameAddressInfoSSIDMaritalStatus
Canonical Schema
Service Interfaces
Agents
Storage
EntityAggregation
Service
Legacy Service
Legacy Service
CUSTOMERFirstNameLastNameAddressInfo
CUSTOMERFirstNameLastNameSSIDMaritalStatus
SchemaTransformations
4
Entity AggregationInstance ReconciliationEntity AggregationInstance Reconciliation
Service Interfaces
Agents
Storage
EntityAggregation
Service
Legacy Service
Legacy Service
CUSTOMERKey1FirstNameLastNameZIP
CUSTOMERKey2FirstNameLastNameSSIDMaritalStatus
CUSTOMERFirstNameLastNameAddressInfoSSIDMaritalStatus
Insert
Operation - InsertCustomer
FirstName
LastName
ZIP
Response – Key1
Operation - InsertCustomer
FirstNameLastNameSSIDMaritalStatus
Response – Key2
4
Entity AggregationInstance Reconciliation #2Entity AggregationInstance Reconciliation #2
Aggregation Service holds a Redundant Aggregation Service holds a Redundant CopyCopy
Issue of Synchronization of CopyIssue of Synchronization of CopySingle Master – Multiple ReplicasSingle Master – Multiple Replicas
Legacy system should send a Notification eventLegacy system should send a Notification eventBatch Notification when Batch Update is allowed Batch Notification when Batch Update is allowed
Replicas must synchronize by listening to notificationsReplicas must synchronize by listening to notificationsAlternatively for some Entities, it could be pull-basedAlternatively for some Entities, it could be pull-based
Multi-master Multi-master
Entity Aggregation Service as MasterEntity Aggregation Service as MasterPost update events so that legacy systems can be Post update events so that legacy systems can be synchronizedsynchronized
4
Entity AggregationGlobal Bank Account Aggregation Service
Entity AggregationGlobal Bank Account Aggregation Service
Return a document which contains Return a document which contains summary data from all key backend summary data from all key backend systemssystems
Cache summary as per policyCache summary as per policyCredit CardsCredit Cards
AccountsAccounts
InvestmentsInvestments
Summary ServiceSummary Service
Summary Summary CacheCache
4
Entity Aggregation ServiceEntity Aggregation ServiceMaps multiple schemas from Maps multiple schemas from different back ends to a single different back ends to a single schema which is presented to schema which is presented to “outsiders”“outsiders”
The internal schema The internal schema differences are visible from the differences are visible from the “inside” only“inside” only
Service must account for what Service must account for what to do if one of the providers is to do if one of the providers is unavailableunavailable
Entity Aggregation PatternsRecommendation: Use the Patterns from the PAG Integration Patterns catalog
Entity Aggregation PatternsRecommendation: Use the Patterns from the PAG Integration Patterns catalog
4
PAG Entity Aggregation Patterns PAG Entity Aggregation Patterns http://http://msdn.microsoft.com/library/default.asp?urlmsdn.microsoft.com/library/default.asp?url=/library/en-us/=/library/en-us/dnpag/html/intpatt.aspdnpag/html/intpatt.asp
Top SOA IssuesTop SOA Issues1. Schema Rationalization 1. Schema Rationalization 2. Service Design 2. Service Design 3. Reliable Messaging 3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
Legacy IntegrationLegacy IntegrationThe world is full of independently designed The world is full of independently designed systemssystems
Differences happen all the way from the Differences happen all the way from the hardware though the OS and middleware, up to hardware though the OS and middleware, up to the application semanticsthe application semantics
Rationalizing these disparate systems is a Rationalizing these disparate systems is a huge challengehuge challengeRecommendation Recommendation
Treat Legacy systems as Business ServicesTreat Legacy systems as Business Services
Surround Legacy systems and build Surround Legacy systems and build messaging interfaces to themmessaging interfaces to themDon’t just Bridge them with Services – Don’t just Bridge them with Services – Adapt them!Adapt them!
5
Legacy IntegrationLegacy IntegrationSimulate Request/Response or One way Interaction Simulate Request/Response or One way Interaction over the Legacy access modelover the Legacy access modelTransform incoming data (Canonical Schemas) to a Transform incoming data (Canonical Schemas) to a representation that Legacy systems can representation that Legacy systems can understandunderstandAnalyze the App to Identify Its OperationsAnalyze the App to Identify Its Operations
Humans Perform Operations with the AppHumans Perform Operations with the AppMany of These Are CancelableMany of These Are Cancelable
The Cancellation May Take Many StepsThe Cancellation May Take Many Steps
The Goal Is to Capture “Low-Hanging-Fruit”The Goal Is to Capture “Low-Hanging-Fruit”Identify Easy to Automate Human InteractionsIdentify Easy to Automate Human InteractionsWrap Those as ServicesWrap Those as Services
If Too Hard to Automate, Enqueue for HumansIf Too Hard to Automate, Enqueue for HumansEnsure the Requests Aren’t LostEnsure the Requests Aren’t LostOK to Get Human HelpOK to Get Human Help
Try to Automate Cancellation and ConfirmationTry to Automate Cancellation and ConfirmationMany Times These Can Be Automated, TooMany Times These Can Be Automated, Too
5
InternetInternet
DMZDMZ
TrustedTrusted
Global Bank TodayGlobal Bank TodayEmployees
Billing Billing ServiceService
ClearingClearingHouseHouse
CreditCreditBureauBureau
AccountsAccountsInvestmentsInvestmentsCustomerCustomer
Tellers Server Apps
Partners
Customers
Phase 2: New OpportunitiesPhase 2: New OpportunitiesPhase 1: Service EnablementPhase 1: Service EnablementPhase 3: New PartnersPhase 3: New Partners
Top SOA IssuesTop SOA Issues1. Schema Rationalization 1. Schema Rationalization 2. Service Design 2. Service Design 3. Reliable Messaging 3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
Process ExternalizationProcess ExternalizationMultiple Services are usually required to Multiple Services are usually required to
work together to fulfill a business requestwork together to fulfill a business requestRecommendation - Use a Process Service to Recommendation - Use a Process Service to Orchestrate business services to fulfill a requestOrchestrate business services to fulfill a request
Usually corresponds to a user-task or a business Usually corresponds to a user-task or a business transactiontransaction
BenefitsBenefitsEasy customization – Externalized definitionEasy customization – Externalized definition
Business Analyst friendly representationBusiness Analyst friendly representation
Tools for effective communicationTools for effective communication
Robust Exception Handling - CompensationRobust Exception Handling - Compensation
6
Clients and Agents (Service Consumers)Clients and Agents (Service Consumers)
Entity Entity
Activity Activity
Process Process
DatabaseDatabase ComponentComponent PartnerPartner LegacyLegacy
Infra
stru
ctu
rIn
frastru
ctu
ree
Even
Even
tt 6
Entity Services represent simple atomic Entity Services represent simple atomic operations on an Entityoperations on an Entity
Process ExternalizationProcess Externalization
Entity Entity
Activity Activity
Process Process
DatabaseDatabase ComponentComponent PartnerPartner LegacyLegacy
Infra
stru
ctu
rIn
frastru
ctu
ree
Even
Even
tt 6
Clients and Agents (Service Consumers)Clients and Agents (Service Consumers)
Activity Services coordinate several Activity Services coordinate several Entity Services to enable Business Function Entity Services to enable Business Function execution (UpdateCustomer, AcceptPO)execution (UpdateCustomer, AcceptPO)
Process ExternalizationProcess Externalization
Entity Entity
Activity Activity
Process Process
DatabaseDatabase ComponentComponent PartnerPartner LegacyLegacy
Infra
stru
ctu
rIn
frastru
ctu
ree
Even
Even
tt 6
Clients and Agents (Service Consumers)Clients and Agents (Service Consumers)
Process services represent long Process services represent long running running business processes that may business processes that may involve involve complex workflow and human complex workflow and human interactioninteractionConsider BizTalk Server 2004 for Consider BizTalk Server 2004 for these servicesthese services
Process ExternalizationProcess Externalization
Entity Entity
Activity Activity
Process Process
DatabaseDatabase ComponentComponent PartnerPartner LegacyLegacy
Infra
stru
ctu
rIn
frastru
ctu
ree
Even
Even
tt 6
Clients and Agents (Service Consumers)Clients and Agents (Service Consumers)
Infrastructure Services enable Security, Infrastructure Services enable Security, Management and Metering/MonitoringManagement and Metering/Monitoring
Process ExternalizationProcess Externalization
Entity Entity
Activity Activity
Process Process
DatabaseDatabase ComponentComponent PartnerPartner LegacyLegacy
Infra
stru
ctu
rIn
frastru
ctu
ree
Even
Even
tt 6
Clients and Agents (Service Consumers)Clients and Agents (Service Consumers)
• Event Services notify subscribers of Event Services notify subscribers of interesting events triggered interesting events triggered • Invalidate Reference DataInvalidate Reference Data• Publish Reference DataPublish Reference Data
Process ExternalizationProcess Externalization
Process ExternalizationRecommendation - Use BizTalk Orchestration
Process ExternalizationRecommendation - Use BizTalk Orchestration
6
Top SOA IssuesTop SOA Issues1. Schema Rationalization 1. Schema Rationalization 2. Service Design 2. Service Design 3. Reliable Messaging 3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
Service AgentService Agent
Also called “Agent/ServiceAlso called “Agent/Service””, , “Agent”, “Agent”, “Emissary”“Emissary”
““Smart Proxy”Smart Proxy”
Communicate with Services on user’s behalfCommunicate with Services on user’s behalf
Provides additional capabilitiesProvides additional capabilitiesSimplify interfaceSimplify interface
OfflineOffline
CachingCaching
QueuingQueuing
Service location resolutionService location resolution
IdentityIdentity
7
Service AgentService AgentService AgentService Agent
Manages Activity-Oriented-DataManages Activity-Oriented-DataLives for a Single Long-Running-OperationLives for a Single Long-Running-Operation
Uses Only:Uses Only:Activity-Oriented-DataActivity-Oriented-DataRequests/Responses (Incoming & Outgoing)Requests/Responses (Incoming & Outgoing)Reference-DataReference-Data
ReferenceReferenceDataData
Read-Only
ActivityActivityOrientedOriented
DataData
Read/Write
Service LogicService Logic
ServiceServiceAgentAgent
7
Service AgentAgent DesignService AgentAgent Design
Agent Manager
Service Connecti
onInformati
on
Populate Populate and queryand query
Executor
Cache
Queue
WSDL gen
Proxy
Agent
AddAdd
PullPull
NotifyNotify
Service Service requestsrequests
Client Client method callsmethod calls
7
Top SOA IssuesTop SOA Issues1. Schema Rationalization 1. Schema Rationalization 2. Service Design 2. Service Design 3. Reliable Messaging 3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
Service Management
Challenges
Service Management
ChallengesWeb Services Availability, Versioning, Web Services Availability, Versioning, Monitoring, DeploymentMonitoring, Deployment
Policy-driven routing of Web Service Policy-driven routing of Web Service requests and responsesrequests and responses
Web Service traffic loggingWeb Service traffic logging
Providing Value added services Providing Value added services (Metering, Billing etc.)(Metering, Billing etc.)
Web Services Security*Web Services Security*
8
Service Management Recommendation – Create a Common Services Framework – Not One-off Ad-hoc solutions!
Service Management Recommendation – Create a Common Services Framework – Not One-off Ad-hoc solutions!
Web Services
Providers
Web Services Consumers
UsersCommon Services
Framework
Common Services
Framework
8
Company A(Web Service Provider)
1. Register Organization with CSF
Company BWeb Service Consumer
CSF Administration
4. Register Organization with CSF
Common Service Framework
2. Register Web Service3. Define access policies5. Subscribe to Company
A’s Web service
CSF RuntimeCSF Client Toolkit
Secure Log Route6. Consume web service
7. Web service response
Service Management
Basic Flows
Service Management
Basic Flows
8
Service Management Service Management
Policy-based RoutingPolicy-based Routing - Goal is to enable - Goal is to enable Service differentiationService differentiation
Use policy-based routing to enforce service Use policy-based routing to enforce service differentiationdifferentiationRouting policy could be based on any defined Routing policy could be based on any defined attributes:attributes:
Class of service. e.g. Silver, Gold, Platinum Class of service. e.g. Silver, Gold, Platinum subscriptionsubscription
Logging and MonitoringLogging and MonitoringLog web service requests, responses, security Log web service requests, responses, security events etc.events etc.Logging level can be changed by configurationLogging level can be changed by configuration
Enterprise Instrumentation Framework (EIF)Enterprise Instrumentation Framework (EIF)
Use Microsoft Operations Manager (MOM) for Use Microsoft Operations Manager (MOM) for Collection and AnalysisCollection and AnalysisFoundation for building other value added Foundation for building other value added services, e.g. Metering and Billingservices, e.g. Metering and Billing
8
b
Service Management CSF Runtime Deployment ScenariosService Management CSF Runtime Deployment Scenarios
As a Web service intermediaryAs a Web service intermediary
.NETWeb Service
Client
.NETWeb Service
Authenticate LogPolicy-based
Routing
CSF Runtime
Web Service Intermediary
J2EEWeb Service
J2EEWeb Service
Client
8
As a chain of web service intermediariesAs a chain of web service intermediaries
Distribute processing across intermediariesDistribute processing across intermediaries
AKA “The Message Bus” to some peopleAKA “The Message Bus” to some people
CSF Runtime
•Authenticate•Route
Web Service Intermediary
CSF Runtime
•Authorize•Log•Route
Web Service Intermediary
.NETWeb Service
Client
J2EEWeb Service
Client
.NETWeb Service
J2EEWeb Service
Service Management CSF Runtime Deployment ScenariosService Management CSF Runtime Deployment Scenarios
8
Point-to-point processingPoint-to-point processing
.NETWeb Service.NET
Web Service Client
CSF Runtime
•Authenticate•Encrypt/Decrypt
CSF Runtime
•Authenticate•Encrypt/Decrypt•Authorize•Log
Service Management
CSF Runtime Deployment Scenarios
Service Management
CSF Runtime Deployment Scenarios
8
Flexibly combine all modelsFlexibly combine all models
.CSF Runtime
.NETWeb Service
J2EEWeb Service
J2EEWeb Service
Client
CSF Runtime
.NETWeb Service Client
Web Service
Intermediary
Web Service
Intermediary
CSF Runtime
CSF Runtime
Service Management CSF Runtime Deployment ScenariosService Management CSF Runtime Deployment Scenarios
8
Top SOA IssuesTop SOA Issues1. Schema Rationalization 1. Schema Rationalization 2. Service Design 2. Service Design 3. Reliable Messaging 3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
SecuritySecurity
Security #1 concern for customers with Security #1 concern for customers with Web ServicesWeb ServicesAuthorizationAuthorizationAuthenticationAuthenticationConfidentiality – EncryptionConfidentiality – EncryptionIntegrity – Digital SignaturesIntegrity – Digital SignaturesPolicyPolicyServices a trust boundaryServices a trust boundary
Authenticate service consumerAuthenticate service consumerAuthorize service consumerAuthorize service consumer
9
Security Point-to-point vs End-to-endSecurity Point-to-point vs End-to-end
HTTPS/IPSEC HTTPS/IPSEC Point to pointPoint to pointMore performantMore performant
WS-SecurityWS-SecurityEnd-to-endEnd-to-endMessage levelMessage levelXML signing and encryption more costlyXML signing and encryption more costly
RecommendationRecommendationWS-Security preferredWS-Security preferredHTTPS/IPSEC for higher performance HTTPS/IPSEC for higher performance requirements if appropriaterequirements if appropriate
9
Great applications in financial Great applications in financial scenariosscenarios
Inherent support in .NET frameworkInherent support in .NET framework
Flexible and extensibleFlexible and extensible
Key concepts: principal, identity…Key concepts: principal, identity…bool IsInRole = bool IsInRole =
MyPrincipal.IsInRole("Manager");MyPrincipal.IsInRole("Manager");
MSDN Developer’s guide: MSDN Developer’s guide: http://msdn.microsoft.com/library/default.asp?url=/library/en-http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguide/html/cpconintroductiontorole-basedsecurity.aspus/cpguide/html/cpconintroductiontorole-basedsecurity.asp
9 Security Role-Based SecuritySecurity Role-Based Security
Security AuthorizationSecurity Authorization
Authorization Manager (Win2K3)Authorization Manager (Win2K3)Hierarchical tasks and roles – for handling Hierarchical tasks and roles – for handling complex scenarioscomplex scenarios
WSE 2.0 – role-based authorization model WSE 2.0 – role-based authorization model for secure access to a servicefor secure access to a service
IPrincipal interface on security tokens, IsInRole()IPrincipal interface on security tokens, IsInRole()
Declaratively via Policy statementsDeclaratively via Policy statements
PAG PAG ““Designing Application managed Authorization”Designing Application managed Authorization”
http://msdn.microsoft.com/library/?url=/library/en-us/dnhttp://msdn.microsoft.com/library/?url=/library/en-us/dnbda/html/damaz.aspbda/html/damaz.asp
Authorization & Profile Application BlockAuthorization & Profile Application Block
9
SecurityAuthentication - SSO and Authentication using Mediated Third Party
SecurityAuthentication - SSO and Authentication using Mediated Third Party
Architecturally speaking, typical SSO Architecturally speaking, typical SSO solutions use third party mediated solutions use third party mediated authentication design that is similar to authentication design that is similar to KerberosKerberos
Trusted Third Party
Application Service
Credential PolicyStore
1. Authenticate
with User Credential
2. Validate Credential
3. Issue Ticket-Granting-Ticket (TGT)
4. Request for Service Ticket, using TGT as proof of previous authentication5. Service
Ticket
6. Application request with service ticket as
proof of authentication
Smart Client
9
SecuritySSO ApproachesSecuritySSO Approaches
2 Common Architecture Approaches:2 Common Architecture Approaches:Proxy Authentication FirewallProxy Authentication Firewall
Application Authentication AgentApplication Authentication Agent
9
SecuritySSO Approach IProxy Authentication Firewall
SecuritySSO Approach IProxy Authentication FirewallA central point of policy enforcementA central point of policy enforcement
Simplify managementSimplify management
Propagate user identity to web applicationsPropagate user identity to web applications
Smart Client
App
licat
ion
Aut
hent
icat
ion
Fire
wal
lWeb Service 1
Web Service 2
Web Service 3
Propagate App Request + User Identity
Trusted Third Party
9
SecuritySSO Approach IIApplication Authentication Agent
SecuritySSO Approach IIApplication Authentication AgentSSO Agent installed at the Web ServiceSSO Agent installed at the Web Service
Smart Client
Trusted Third Party
Web Service 1Web SSO
Agent
Web Service 2Web SSO
Agent
Web Service 3Web SSO
Agent
9
Top SOA IssuesTop SOA Issues1. Schema Rationalization 1. Schema Rationalization 2. Service Design 2. Service Design 3. Reliable Messaging 3. Reliable Messaging 4. Entity Aggregation4. Entity Aggregation5. Legacy Integration 5. Legacy Integration 6. Process Externalization 6. Process Externalization 7. Service Agents7. Service Agents8. Service Management8. Service Management9. Security9. Security10. Transaction Management10. Transaction Management
Transaction ManagementTransaction Management2 Phase commit does not work well 2 Phase commit does not work well
across loosely coupled SOA linksacross loosely coupled SOA linksYou may not have tight control over your You may not have tight control over your business partner’s resourcebusiness partner’s resource
WS-TransactionWS-TransactionWS-Coordination, WS-AtomicTransaction, WS-Coordination, WS-AtomicTransaction, WS-BusinessActivityWS-BusinessActivity
Design architecture away from the Design architecture away from the need of tightly coupled distributed need of tightly coupled distributed transaction processingtransaction processingSome support for long running Some support for long running transactions in BizTalk Servertransactions in BizTalk Server
10
SummarySummary1. Service Design 1. Service Design
Factoring, Message based interfacesFactoring, Message based interfaces
2. Reliable Messaging2. Reliable Messaging Synchronous Request/Response: Idempotency; Fire/Forget: today use Synchronous Request/Response: Idempotency; Fire/Forget: today use
MSMQ or BizTalk MessagingMSMQ or BizTalk Messaging
3. Schema Rationalization3. Schema Rationalization Create Canonical SchemaCreate Canonical Schema
4. Entity Aggregation4. Entity Aggregation Design for Entity AggregationDesign for Entity Aggregation
5. Legacy Integration5. Legacy Integration Adapt Services, Service TaxonomyAdapt Services, Service Taxonomy
6. Process Externalization6. Process Externalization BizTalk OrchestrationBizTalk Orchestration
7. Service Agents7. Service Agents Value-added communication, Offline etc.Value-added communication, Offline etc.
8. Service Management8. Service Management Address service management needs for all Services – not ad-hoc solutionAddress service management needs for all Services – not ad-hoc solution
9. Security9. Security Trust boundary, WS-Security for end-to-endTrust boundary, WS-Security for end-to-end
10. Transaction Management10. Transaction Management Avoid 2PC, use WS-TransactionAvoid 2PC, use WS-Transaction
ReferencesReferences
ArchitectureArchitecturehttp://www.microsoft.com/architecturehttp://www.microsoft.com/architecture
patterns & practicespatterns & practiceshttp://www.microsoft.com/practiceshttp://www.microsoft.com/practices
Thank You
Thank You