soa vs eda 2 (discussions with jeppe)

31
SOA vs EDA 2

Upload: jean-jacques-dubray

Post on 22-Apr-2015

317 views

Category:

Software


0 download

DESCRIPTION

 

TRANSCRIPT

SOA vs EDA 2

Let me also correct something

• Obviously you have done a lot of research and you have a lot of practical experience,

• So I apologize, your presentation on SOA vs ESA did not reflect that, and I tend to get very upset at that style of communication, which brings absolutely nothing to the discussion and in general has a negative impact on our industry creating a large amount of FUD

Protocol vs Semantics• Jeppe, I truly believe that you are mixing two levels:

– Protocol– Semantics of the interaction

• When you say “Systems are more fragile. […] Suddenly the stability of the service I call is dependent upon the stability of the combined list of services“• What is the alternative? Aggregating the 3 services into a single back-end? • Use reliable protocols? • Dependencies and unreliable systems is a given, there is no way around it

– The semantics of an interaction are very well defined (e.g. the ebXML BP specification – I was one of the editors of that spec):• Notification• Business Transaction• Request/Confirm• Request/Response• Query/Response• Information distribution• Data Exchange

– These semantics cannot be dependent on the protocol which maybe “reliable” or not depending on what/how you want to achieve, you can choose a protocol more or less aligned with the semantics of the interactions, but you cannot pick interactions as you please

Protocol vs Semantics (cont’d)

• “That is, we want to eliminate temporal coupling between services by limiting ourselves to asynchronous communication between services.”– Really, you can eliminate temporal coupling

because you decide so? – How can someone writes this kind principle? When

you go rent a car, there is no temporal coupling? You can wait until a car becomes available, right?

Service Coupling

• “loose coupling is what gives us business agility, which is the key business driver for SOA”– Again the CAR theorem states that you can’t have

Consistency, Agility and a high degree of Relationality at the same time, you can only have two, that has nothing to do with temporal coupling, or even a protocol

– Service interaction semantics sit above the protocol

Note: I am the author of the CAR theorem, that’s why Google didn’t return anything

Service Coupling

• “Clemens explained that Autonomous Services is what allows the Azure teams to scale development or operations”– But when do you have a choice? When there is no consistency

and no relationality?– Yes, there are some services like that, typically the Azure Service

Bus / AWS services are autonomous, I agree, but that’s not true in the enterprise

– Is a service defined by the protocol you use, or by a higher definition?

– SOA is not “plumbing”, there are a lot of interesting things to do with the “plumbing” but we should call it “plumbing” not “SOA”

Service Coupling

• “NetFlix has MANY autonomous services/bounded contexts which seems to work for them. The same seems to be the case for Azure (and as far I can tell also for Amazon)”– I am not sure it is helpful to take examples that are non

relational• That being said, even Netflix had to fight the chattymess of their APIs

– SOA was designed for the enterprise not for the Cloud, we might call them “Services”

– I can find hundreds of examples of truly autonomous services, it does not help me designing the one that can’t be made “autonomous”

The Key to SOA …(I could not say it more clearly)

• … is to make the Systems of Record autonomous from one another

• That is what is very desirable, but this cannot happen by itself, it is the services above the integration points that make it happen

• So, if you call an integration point “a service” you might come to the conclusion that services are indeed autonomous, but integration points are not services by a very large margin

ESB

• “It takes too long to get anything across the ESB service team, let’s just bypass them for now, build what we need and deal with it later”– The truth is that you can expose an “integration points” in

minutes (e.g. Apache Synapse)– Once you do that you create a “stable interface” that

prevents changes in the SoR to propagate to that interface– An ESB is the only way (I know of) to decouple SoR from

consumers, if you don’t do that you’ll keep pushing changes to the SoR to all consumers (or break them)

– This is simply not true

ESB (cont’d)

• “but most companies seem to end up in a broker/hub-spoke ESB model”– There are ESBs and ESBs. For me an ESB is “Apache

Synapse”, and I use Synapse to manage the interface to a service not its implementation

– You cannot be successful at SOA if you do not decouple the service interface from its implementation (see Reuse section)

• An ESB greatly simplify security, monitoring, scalability…

ESB (cont’d)

• “A bunch of webservices that communicate with other services using Request/Response over HTTP (i.e. RPC) and an ESB put in the middle as a broker.”– This is not the view, the ESB is a mediation layer essential

to manage (and decouple) the service interface from its implementation

– You cannot handle changes in the SoRs without mediation, I don’t know any other way• There is Compatible interfaces, but it seems too complex for

people to deal with it• In any case, you would still need to deal with horizontal aspects of

a service (security, monitoring)

ESB• “A centralized ESB’s is one of the problem I see many organizations

battle with. In theory it should work (with clustering and all), but I’ve so many times heard “the ESB is slow again”, “none of our services work”, “who forgot to increase the DB quota for the ESB log tables”, etc. “– Again, Jeppe, you assume that because someone does this, everyone does it

or should do it– We can’t make this kind of reasoning– Many ESBs are fully decentralized (thanks to SCA for which I was also an early

author of the spec)• But the Hipsters didn’t like SCA either• We have to stop this wolf pack mentality that drives our industry in the worst possible

direction• Nowadays, ESBs like Apache Synapse make it easy to deploy mediation capability

wherever you need it• An ESB is only centralized because you say so

About “a bunch of Services”

• that is the key to a successful SOA, you can never aim at building the “right” service, your SOA must start as “just a bunch of services”

• It is through “reuse” (as I define it later) that services evolve towards becoming “enterprise services”• It is under the pressure of the changes requested by new

consumers that eventually the service becomes an enterprise service

• It is not via governance, years of analysis, it is slowly but surely, one version after another

Reuse

• Some don’t see reuse as the end goal. Udi Dahan made a comment sometime ago stating “For me, I’ve given up on the whole reuse thing. I think it’s a crock. Has been since OO• Once again, if you don’t understand that in SOA reuse happens the

other way around, you can never “reuse” anything• In SOA new consumers rarely can consume a service as-is, they need

changes to be made to the SoR and hence to the interface would need to change. The most sensible way to deal with this situation – Keep the existing interface stable– Change the service implementation– Expose a new service interface– When you do that, it is old consumers who reuse the new version of the

service, not the other way around

The fallacy of Reuse

• Udi “Reuse may make sense in the most tightly coupled pieces of code you have, but not very much anywhere else.”– Yep, and let’s try to not reuse “customer change

of Address service” … – Guys come on…

• Intentional interfaces are reusable• Interfaces which aim at driving a consistent outcome

are reusable• Yes, not all interfaces are reusable

Service Autonomy“You see, this synchronous producer/consumer implies a model where services are not able to fulfill their objectives without calling other services. In order for us to achieve the IT/Business alignment promised by SOA, we need services which are autonomous, ie. able to fulfill their objectives without that kind of external help” UDI Dahan• Again, the GOAL of SOA, the only goal of SOA is not to achieve service

autonomy, this approach is completely bogus. – Some services are autonomous of course by nature, but “Autonomy” means nothing in a

relational data model

• The goal of SOA is to achieve autonomy of the System of Record, – This is what has never worked in EAI and created the spaghetti that we all know– Services have a “back interface” in which SoRs can be plugged in, this is when you

reach agility– EDA is topologically and semantically equivalent to File based integration

• Synchronization and Replication of data• These patterns don’t scale, they are the problem, they are the patterns that caused EAI to fail

Service Autonomy• I think it’s a good idea to create autonomous services where the service owns its

own data– If you can yes, but on average you can’t data is relational, there is nothing you can do

about• Customers, Orders, Invoices, Shipping Notices are all a big ball of mud that you can’t fight• An Order has customer data, no? how do you define “autonomy” then?

– Is Google Maps autonomous with respect to my Order Entry system, you bet, but that’s irrelevant

• Autonomy is never, I repeat never a design goal, it is incidental– Autonomy was introduced randomly by the Microsoft team who was looking to be smart

with “tenets” and needed some form of encapsulation to match the WCF Object-Oriented reification of SOA

– Microsoft never understood a single to SOA (WCF, WPF, BizTalk are all horrendous middleware products that systematically lead to disasters and massive cleanups)

• It doesn’t just work on paper, but also in real life.– Again some services are autonomous in nature, but please tell me how a relational data

model leads to “autonomous” services

Arnon believes that services can be autonomous.

• “If it’s the exact same kind of autonomy as Clemens, Udi and Bill I’m not sure”

• Well, we have to define “autonomous”, for me if two services rely on the same SoRs/integration points they are not “autonomous”, I can’t take one down entirely and the other one keeps functioning, I can’t scale one without the other

• That’s why I think it is very important to show the SoR on the sequence diagram

Orchestration

• “The reason why I mentioned WebServices and Synchronous communication (Request/Response over HTTP) specifically is, as mentioned before, because that’s what 95% of all Services are (my experience – yours may vary).”– What I find disturbing in your rationale is that Orchestration is

precisely what fixes this problem and yet to demolish– Worse you replace it with “asynchronous” protocols and you try

to eliminate temporal coupling• But Orchestration is the best way possible to eliminate

temporal coupling– That’s why you got me so fired up,

Orchestration (cont’d)

• “What I’ve seen in many organizations is that many developers are NOT familiar with the fallacies of distributed computing, eventual consistency, orchestration, compensations, idempotence, etc.”– So and because they are not familiar, let’s scare

them a bit more towards events– Please, hipsters can throw just about any argument

and see what sticks? Is that how our industry function?

Orchestration (cont’d)

• “The reason why I say BPEL spaghetti is because that’s what I typically see in companies where they try to mend badly designed services using BPEL (this is not BPEL’s fault – if anything it’s a statement of what can be accomplished using BPEL). “– So because some people write spaghetti code, with BPEL, all

people must write spaghetti code?– Come on Jeppe, you are better than this kind of argument– This is why I get so mad at some people because they make

this kind of generalization that does not pass basic logic tests and yet everyone stops as “I shouldn’t use XXX the hipsters said so”

Event• “The reason why the event driven approach does not suffer

from the issues of the command message approach is that the service publishing the event does not make any assumptions as to which services(s) are subscribed or how they will behave in response to the event.” Bill Poole– Again this is totally incorrect, on paper it sounds good, but that is a

fallacy– Imagine, you add a new subscriber, and the processing of an event

happens to fail (business failure), how is does the clean up proceeds? • In business you have no choice as to the semantics of the interaction, if your

interaction is a “notification” yes, that sentence is correct. If it is a “request/confirm” that sentence is incorrect, a negative response will have implications poorly represented by the pub/sub mechanism

What is a service?

• “What this means is that even when services are publishing and subscribing to each other’s events, we always know what the authoritative source of truth is for every piece of data and rule.”– My definition is “Services are software components that govern a

consistent outcome of activities”• That requires an Intentional Interface• And a consistency-driven implementation

– Services have no boundary per se, – Operations are the unit of composition, I can compose operations

opportunistically to deliver a specific intent (e.g. the pump gas service has a “providePaymentInfo” operation, because I can’t govern the outcome without the payment, period

That is SOA

Pumping Station

Diesel 8987 91

Full

Empty

Intent

InterfaceProvide Payment

Info

SelectGas

PumpGas

Implementation

PaymentProcessor

POS Pump

OrchestrationOn (finished): v = getVolumePumped() p = v *price-per-gallon processPayment(creditCardInfo,p)

The back-end of the Service is as important as the font

Service

SoRSoRSoRSoRSoR

Consumer Consumer Consumer

You need to be able to:a) Plug and Play SoRs, b) Change SoR and control change on the service interfaceThat is how you can be agile

Layered Service Models are bad

• “Open any number of books out there on SOA and you'll find guidance on defining the service model for your enterprise in terms of process, task and entity services. This in essence sets up three layers of services, with process services as the top layer, task services underneath the process services, and entity services at the bottom.” Bill Poole – You bet, that is bad, really bad - we agree on that– However the title is misleading the proper layering is:

• Service Interface Management (mediation)• Service Implementation (container)• Integration Points (Systems of record/truth)• That is layered …

All this guidance about boundaries, autonomy, cohesion, blah, blah, blah is maddening

• SOA is simple think “Change of Address”1. Intent2. Consistency3. Done!

• Services emerge from the need to make the information system (as implemented by SoRs) consistent, period, nothing less, nothing more

• But …?– 1/ Intent 2/Consistency 3/Done

• How you achieve it, I don’t care as much, I personally like Mediation/Orchestration over events, Java, …

Microsoft and SOA

• I fancy that Microsoft knows nothing about SOA, I have followed them since about 2001 and I have never seen somebody building a great SOA just with .Net, it may be possible, only came across failed SOAs:– WCF, WF, Biztalk were all a conceptual disaster– They made a big mess in the standards (I was there)– The only thing I have seen done well is Service Versioning,

actually perfect, cannot do better, but it is not standard, only works with .Net consumers

– Everything else, you can throw away

How many “SOA” approaches?

• “I hope we can agree that there are many styles of SOA (it seems each person has his)”– Well I am not sure about that, I may be wrong but I

don’t see too many ways to do that• Personally, the outcome that I would like us to

reach is to agree on patterns (event, orchestration, mediation…), then you can slice patterns in different SOA models, I don’t care

• “I hope we can continue a good discussion forward ”

• I hope so to, this is the kind of discussion that has been missing hopefully Udi and Arnon can also chime in