10 soa commandments - agile business consultants 10 soa commandments v1 3.pdf · 10 soa...

24
AGILE BUSINESS CONSULTANTS BV 10 SOA commandments Getting the most out of Service Oriented Architectures Hans Wierenga 23/3/2010 Service Oriented Architecture (SOA) is an approach to organizing information processing. It is capable of lowering the costs of information systems interoperability, by greatly reducing the number of aspects of these systems that have to be brought into agreement with each other for them to work together. If it can be successfully applied on a large scale, the systems landscape will differ from that of today in much the same way as today’s freight industry differs from that of the pre-container era. However, it is currently being used in ways which result in additional overheads but fail to deliver on these interoperability advantages. Paradigms which are appropriate to database era are being applied to SOA, resulting in counterproductive, often stupid and sometimes downright dangerous designs. These paradigms must be replaced by patterns of thought and behavior which ensure that SOA leads to simpler interfaces, functionally better IT solutions and more manageable projects. This can be achieved by adhering to ten basic commandments.

Upload: lekhuong

Post on 18-Jun-2018

236 views

Category:

Documents


3 download

TRANSCRIPT

AGILE BUSINESS CONSULTANTS BV

10 SOA commandments Getting the most out of Service Oriented

Architectures

Hans Wierenga

23/3/2010

Service Oriented Architecture (SOA) is an approach to organizing information processing. It is capable of lowering the costs of information systems interoperability, by greatly reducing the number of aspects of these systems that have to be brought into agreement with each other for them to work together. If it can be successfully applied on a large scale, the systems landscape will differ from that of today in much the same way as today’s freight industry differs from that of the pre-container era. However, it is currently being used in ways which result in additional overheads but fail to deliver on these interoperability advantages. Paradigms which are appropriate to database era are being applied to SOA, resulting in counterproductive, often stupid and sometimes downright dangerous designs. These paradigms must be replaced by patterns of thought and behavior which ensure that SOA leads to simpler interfaces, functionally better IT solutions and more manageable projects. This can be achieved by adhering to ten basic commandments.

Ten SOA commandments

Author: Hans Wierenga Page 2 of 24 23-02-2010

Introduction

The potential impact of SOA

Service Oriented Architecture (SOA) is an approach to organizing information processing. The

approach consists of describing all interactions in terms of services, in which a requestor asks an

agent for something to be done, and the agent ensures that it gets done and delivers a response to

the requestor. This way of thinking can be applied at a business level, in order to describe

interactions between organisations, at a functional level, to describe how the activities of which

business processes are comprised interact, and at the level of information systems, in order to

describe how systems and parts of systems interact. At each of these levels the principle is the same:

how the agent does what needs to be done is no concern of the requestor, not even whether it is

done completely automatically, completely manually or something in between. Nor is the requestor

concerned whether the agent delegates part or even all of the work to others. All the requestor

needs to agree on with this agent is how the request and the response should be formulated, and

what the effect of the service is.

SOA is widely touted as a paradigm with enormous potential to reduce the costs of systems

development, testing and maintenance. In particular, it promises to lower the costs of getting

information systems components to work together, by greatly reducing the number of factors that

have to be brought into agreement with each other. Using SOA, differences in things such as

computing platforms and data formats are much less of a barrier to communication between

systems than with earlier paradigms. That makes cooperation on a larger scale possible, because it

minimises the hindrances imposed by the need to get systems designers to agree with each other

and, for that matter, to get systems configurators to agree with each other. If the promise can be

realised, the consequences will be revolutionary. Like the car changed the shape of urban regions,

the shipping container revolutionised the freight industry, and the lowering of transaction costs

enabled modern free market economies to develop, SOA will enable new patterns of cooperation.

When SOA dominates the way we apply IT, the systems landscape shall differ from that of today, as

a city designed around cars differs from a city designed around trains. For those of us whose thinking

is dominated by current technologies, it is difficult to imagine how big a difference SOA can make.

But the flexibility advantage that SOA offers is like the advantage of cars over trains: whilst trains can

be made to go as fast as cars, they can never support door to door transportation in the ways cars

can. It is simply not practical to put a train station at the end of every driveway, or even rails in every

street.

Why this impact has not (yet) materialised

To reap the benefits of the new paradigm, we must make use of the new possibilities it offers.

Unfortunately, most of the current hype around SOA has little concern for these possibilities. The

major discussion seems to be concerned with establishing how SOA enables individual information

systems to be developed more rapidly. However, that is not where SOA adds most value. In fact, it’s

debatable whether SOA is really an improvement on previous approaches in which separate

functions cooperate by sharing a common pool of data, normally implemented in a database. Using

SOA to build an individual, isolated information system is rather like using shipping containers to

Ten SOA commandments

Author: Hans Wierenga Page 3 of 24 23-02-2010

move goods around your manufacturing plant: certainly, it imposes order and structure on your

internal logistics, but the containers get in the way more than they help. SOA results in better

interoperability between information systems, rather like shipping containers enable interoperability

between carriers. That’s an important advantage, because the lead time between requirements

finalisation and the information system becoming operational is often largely determined by the

interoperability. Getting an information system to work together with the systems in its operational

environment normally costs more time and energy than building the system as such.

The focus on the use of SOA within, rather between information systems is a symptom of a wider

problem: because SOA is seen as a new way of doing what we have always done, we are unable to

reap its benefits. SOA concepts and technologies are being used with current systems development

paradigms. These paradigms have been developed in the database era, and incorporate the

limitations of database technology. Applying SOA with these limitations will result in additional

overheads, but fail to deliver on the additional benefits. But these “databased” paradigms are so

pervasive and pernicious that we are often not aware of how much they affect our thinking. They

are so ingrained that we regard them to be just plain common sense. Unfortunately, they are

generally counterproductive, often stupid and sometimes downright dangerous. They lead to

solutions that combine the disadvantages of the databased era with the down side of SOA, whilst

failing to deliver on the advantages that SOA has to offer.

What needs to change

The SOA paradigm has its own common sense rules, and these are often completely different to

those of databased paradigms. There are ten basic ones. The first five are concerned with making

things simpler than the databased paradigms dictate – simpler in the sense of sticking to the

essentials. When we make things simpler in this way, there is a dramatic improvement in the

probability that different solutions to the same problem will be interoperable. The following four are

concerned with making IT solutions better than their databased equivalents, by avoiding the blinkers

that lead databased thinkers to develop ineffective solutions. The last is concerned with making IT

more manageable, and in particular with organising systems development so as to reduce project

complexity and risks. SOA makes this possible – but also necessary – because it allows more

functionality to be delivered as infrastructure.

Making it simpler

The rationale

In a trapeze act, effective cooperation is based on each trapeze artist being in complete agreement

as to where the other is going to be at each point in time. Some trapeze artists are so confident of

knowing this that they regularly perform blindfolded. They can find each other because they have

ensured that at any given time they can each have only one possible location.

The successful application of SOA in order to maximise interoperability is very much like a trapeze

act. Interoperability demands that the millions of possible solutions as to how parties communicate

have been reduced to just one, on which both parties can rely. That doesn’t have to mean that there

is something wrong with all the other solutions: whether we drive on the left or on the right is

arbitrary, but it is important that we all drive on the same side of the road.

Ten SOA commandments

Author: Hans Wierenga Page 4 of 24 23-02-2010

When just one party performs a service for just one other party, it makes little difference as to which

solution the parties agree, as long as they agree. Idiosyncrasies from one side or the other can

determine the solution, without making the total communication effort significantly greater. After

all, these idiosyncrasies will need to be handled somehow, whatever the solution. But if there are

many parties requesting the service or many parties performing the service, the picture changes.

Then it is important that the communication solution is reduced to the essentials, in such a way that

each party has to handle its own idiosyncrasies and no other party is confronted with them.

It is illustrative to compare information communication with transplantation surgery. For a

successful transplantation of an organ from one person to another, the transplanted organ must

match the recipient on a wide range of factors, the vast majority of which have nothing to do with

the biological function of the organ. In other words, the transplanted organ must have the same

idiosyncrasies as the recipient. As a result, our organs cannot be transplanted as if they were Lego

blocks. Currently, information communication is very similar. For one information system to perform

a service for the other, they must agree on a wide range of things. They must use the same

vocabulary (metadata), the same set of functions which can be invoked by the one and executed by

the other, the same expectations for the data content of the questions and answers for each of

these functions, the same coding systems, the same technical communications protocols, the same

addressing scheme in order to channel messages to and fro, compatible notions of the tempo in

which a reaction is expected, compatible techniques for ensuring that messages don’t get lost,

compatible authentication mechanisms in order to ensure that they are talking to each other rather

than to imposters, compatible encryption techniques and key management in order to ensure that

messages cannot be listened to or intercepted, and so on. In order to facilitate interoperability, we

must ensure that parties that start standardising on these things independently of each other arrive

at compatible standards. This is possible only when some very rigorous rules are followed, so that

interfaces are reduced to the essentials. This way there is no room left over for idiosyncrasies.

The rigorous rules are all concerned with making service interfaces simpler. The less we need to

specify, the smaller the room for disagreement.

1. Don’t know what you don’t need to

What you don’t need to know can’t hurt you

The essence of the SOA paradigm is that it enables maximal cooperation with a minimum of

agreement between the cooperating parties or systems. That is an enormous advantage, because

anything you don’t need to know doesn’t need to be tested or maintained either. What you don’t

need to know can’t hurt you. Given that 40 percent of the systems development costs go into testing

and up to 80 percent of the lifecycle costs of an information system are incurred in the maintenance

phase, anything that the SOA paradigm liberates you from having to know represents money you

can save.

Metadata

The most important thing you don’t need to know is the structure, meaning and allowed values –

the metadata – of any data which is not used by logic that is built into your system for selection,

sequencing or performing calculations. You don’t need to know this, because SOA technologies

Ten SOA commandments

Author: Hans Wierenga Page 5 of 24 23-02-2010

make the metadata available wherever the data is. Your system can interpret this metadata on the

fly, so that you don’t need to build the knowledge of that metadata into your system if all you want

to do is capture, present or pass on the corresponding data. With a sufficiently sophisticated

presentation facility it is even possible for users to perform all sorts of ad hoc selections and

calculations using only the metadata that has been provided together with the data, rather than

inbuilt metadata.

By interpreting the metadata that is provided along with the data, rather than building the metadata

into the system, your system does not need to be changed when the metadata changes. Only the

source system needs to be changed. Think of the savings in development, testing and maintenance if

you can apply this principle! Remember that making a change in two systems rather than one is, on

average, four times as complex, because of all the coordination involved.

For many consumer-facing systems, presentation and ad hoc selection covers most of their

functionality. Such systems require inbuilt metadata for only the most basic customer data. That

does not include current and past orders, customer contacts, photos, correspondence and whatever

other data may be available for display, all of which can be presented in a manner which does not

require any knowledge as to the nature of this data to be built in to the system.

Technology

Many of the things you don’t need to know are technology-related. With SOA, you don’t need to

know whether the system with which you are interfacing is implemented using a Software-as-a

service or not, where the computer on which it is implemented is located, what type of computer it

is or which operating system it is running on, how the firewalls are configured, which database

management system is being used, or which transaction management system is available. Other

things that you don’t need to know concern the internals of the systems with which yours

communicates. In particular, you don’t need to know anything about the metadata which it uses for

its internal data storage, because any transformations that the other system may require in order to

conform to the XSD are its problem, not yours.

Having said that, there remains a wide choice of technology-related standards on which parties that

communicate using SOA must come to agreement. In particular, there are so many that standards

concerning web services that SOA practitioners refer to them collectively as WS-* hell (with the *

indicating that there are many possible labels that can replace it). In part, these standards have

arisen precisely because the SOA community has not been content to not know what it didn’t need

to know; this white paper presents guidelines to reduce the impact of the problems that gave rise to

such standards. When these guidelines are followed, SOA requires substantially less prior agreement

than other approaches.

Designing stable interfaces

If you want to reap the benefits of SOA, not knowing what you don’t need to know must become

your second nature. Apply it everywhere! For example, when designing a service to order goods,

remember that the service requestor needs to know whether the goods will be available when he

wants them, but does not need to know the current stock level. And if your application invokes a

security service in order to determine whether a requested action is authorised, don’t build in any

more knowledge of the workings of the service into your system than is necessary. For example, if

Ten SOA commandments

Author: Hans Wierenga Page 6 of 24 23-02-2010

the security service uses the credentials supplied with the input to the application, all it has to do is

pass them on! As far as you are concerned, they are just a single data item which is encapsulated in

your incoming message. Don’t even check whether the credentials are well-formed XML. If, for

reasons known only to them, the security boys have chosen to deviate from standard SOA practice

or to encrypt the credentials, that is their problem, not yours. If they change anything to the

credentials, your application should not need to be changed or adjusted in any way. Anything you

don’t need to know can’t hurt you. Unless, of course, you are so stubborn as to want to know it

anyway, in which case perhaps everybody is better off if you don’t waste your time on SOA.

Not knowing what you don’t need to know is harder than you might think. If you develop

information retrieval services specifically for a particular information system with which yours

communicates, you are already on the wrong track, because you have incorporated some knowledge

of that other system into yours. Any change in those requirements will necessitate a change in both

systems. In general, a better approach is to expose your system’s data using a limited set of retrieval

services, which, when used in combination, cover all the information retrieval requirements of

attached systems. For example, a product database may be exposed by separate services: a simple

one with just the code, description, units and list price of the product, a service which exposes all the

financial aspects of the product, and a service which exposes all the logistical aspects of the product.

Many systems will be satisfied by the simple service, and most will need some but not all of the data

of either the financial or the logistical service, and some will need to use both together, but there

will be no system that needs a special interface on top of these. This way of working is known as the

McDonalds approach: the customer assembles his own meal from standard products. Supporting

this approach is no hardship, because you need such a set of services anyway in order to support

customer-facing applications. You can even support very specific information requirements in this

way, because data that is not needed can be filtered out before consumption. If you don’t like the

gherkin in your big Mac, remove it! The idea is that providing too much information is much less of a

problem than providing too little, for the recipient’s systems can easily be programmed to ignore

information he doesn’t need, but are stuck if anything is missing.

Not knowing what you don’t need to know also results in profound simplifications of the information

exchanges which are required in order to support business processes. In the SOA paradigm, when

you ask another agent to do something for you, that is all that you do. You don’t provide the agent

with additional information which may be useful or even necessary for the agent to carry out the

task. You ask for the dish, and it is the task of the cook to ensure that the ingredients which go into it

are available. You ask what you want, and shut up. The agent may in turn request some or all of the

information from you using an information retrieval service, but it’s his problem to decide what

information to retrieve, when he wants to retrieve it and where he wants to retrieve it from. You

don’t need to know about it, and still less do you need to incorporate that knowledge into your

system. This way, changes on his side will seldom require a change on yours. For example, nothing

needs to change on your side if he decides to discontinue maintaining his own copy of your data.

It is of course true that not knowing what you don’t need to know results in inefficiencies. Some

operations that can be carried out in a single exchange will now result in multiple steps. The

McDonalds approach often results in several services being required where one would have been

sufficient, and leaving the other side to retrieve the information that it requires almost always

results in more messages than strictly necessary. There will be situations in which it makes good

Ten SOA commandments

Author: Hans Wierenga Page 7 of 24 23-02-2010

business sense to optimise these communication patterns. There will also be many occasions in

which you will want to optimise the user interface, if only because current presentation facilities are

not very good at providing users with attractive interfaces. But before you optimise, consider what

flexibility you are losing. And never, ever optimise anything for which the functional requirements

have yet to be stabilised.

2. Don’t know what you can’t know yet

Premature specifications freezes

One of the real bugbears of databased paradigms is that it requires you to fix data structures in

concrete before you know enough to be able to do that properly. That’s because they ignore a fact

of life: users don’t know what they want until they see what they don’t want.

It works this way: once you have finalised a data structure design, any subsequent changes are going

to require messy database conversions, in addition to changes in each system which accesses the

database. All of these changes must be coordinated, which is hard enough if all use of the database

is limited to a single system, but gets much harder when there are many systems, particularly if

some of them are managed by parties not under your control. In fact, making these changes is

already a problem in the systems development phase. In consequence, database designs are frozen

early on in the systems development phase, and data analysts bend over backwards to get the

design right.

The problem here is that data analysts have an impossible job. They have to finalise a design before

the users can understand it, let alone appreciate how it will work out in practice. It is only long

afterwards - once the system has been built - that users can get a feel for the system and assess how

well it meets their needs. If there are any major problems with the data structure, it is far too late to

fix them.

SOA prototyping

“How is SOA different? “, you might ask. When all is said and done, doesn’t SOA need to structure

data just as much as databased paradigms? The short answer is that SOA works just as well when

requested data is processed by a human agent as when it is handled by an automated system, and

people can interpret data even when it is not optimally structured. For example, users can

determine whether a letter is being sent to another country even when its address is presented as

line 1, line 2, line 3 and line 4, whereas an information system would need at least the country to be

an identifiable part of the data structure.

The long answer involves a discussion of SOA prototyping, which is carried out as follows:

1. Identify the metadata which should be built in to the system. Put it in a primary namespace,

and store its data in a DBMS according to its structure in the conventional manner. For

example, for a customer this metadata may consist of the customer-ID and the customer

name. Because this metadata is built into the system, it is possible to build logic into the

system in order to use these fields, for example to retrieve records using the customer-ID

and to sort and select records on the basis of the customer name.

Ten SOA commandments

Author: Hans Wierenga Page 8 of 24 23-02-2010

2. For each database record, put all the data that wasn’t included in the primary namespace

into a single XML-string which is appended to the database record as a single field. For each

such XML-string, make a secondary namespace, develop an XSD, and add a single data item

for the string to the primary namespace. For the initial implementation of a customer

record, this string may include metadata for address lines 1, 2, 3 and 4. The XSD of the

customer record itself will then contain metadata for three fields: the customer-ID, the

customer name and the XML-string containing the additional customer data. The XSD of the

additional customer data will include metadata for each of the address lines.

3. Capture and present all data using XSD-based logic for each relevant hierarchy of database

records, augmented by a single, standard validation service per XML string (including the

string for the primary record). If at all possible, use a tool to generate interfaces using the

XSD’s, rather than programming it yourself. This tool must unpack the secondary XML-

strings using the metadata that they contain, and must capture new records using both the

primary and the secondary XSD’s. The result is a working prototype which the user can use

to assess the usefulness of the intended system.

4. Adapt and change the XSD’s and the validation services as necessary until the users are

happy with the system. In the case of the customer record, a new XSD for the string may

contain metadata for street address, zip-code, state and country. Data stored using the old

XSD will still display correctly, so there is no immediate need to convert it. New records will

be captured using the new XSD.

5. Consider migrating data to conventional database structures and to the primary namespace

once both the users and you are convinced that the metadata is stable.

Of course, SOA prototyping is not always necessary. Apply it when the requirements are subject to

so much uncertainty that it is more economical to prototype first and stabilise later. But remember

that it is much easier to do than databased prototyping. In the databased era, prototyping was an

option, but in the SOA era, it’s a way of life. Don’t fix designs in concrete until you know they are

right; it’s as simple as that.

3. Don’t do more than asked

Product-driven service decomposition

One of the main strengths of SOA is that it is a business concept which is translated into technology,

and not, as is the case for the databased world, a technological concept trying to catch up with the

business. In SOA, each service must clearly add value, not just in some abstract sense, but specifically

to those who invoke it. Everything that happens does so because a service requestor asked for it to

happen. By not implementing anything within a service that the service requestor did not explicitly

ask for, services can be limited to their essential functionality. If we all do that, the result is an

enormous increase in interoperability.

The highest level services in SOA are the business transactions: a customer places an order for goods

or services, which – unless the order is rejected by the vendor – results in their delivery. In the SOA

paradigm, everything that happens between order acceptance and order delivery can and should be

implemented as services too. For each intermediate product or state that is required in order to

Ten SOA commandments

Author: Hans Wierenga Page 9 of 24 23-02-2010

deliver on the order, the vendor requests somebody or some organisational unit – not necessarily

within the vendor’s own organisation – to deliver it. Each of these intermediate services in turn may

themselves be decomposed into services, and so on, right down to the level of elementary units of

value addition.

The hierarchical service decomposition is an essential part of the SOA paradigm, because it enables

the delivery of services to be managed by means of service level agreements. It ensures that there is

somebody responsible for each service, and that service consumers know what they are getting.

Common intermediate products and common services

Note that the service decomposition is all about products. Processes only come into it when the

subservices of which a service is composed cannot just be carried out all at once. For example, if the

service determineCustomerOrderAcceptability requires that the subservice determineCustomer-

Creditstatus is carried out after the subservice determineOrderValue, then the implemention of the

service involves a process. Each such process is concerned with only one service. If you find you have

a process which cannot be limited to a single service in this way, you have probably forgotten to

model the initial customer request as a service.

It is possible that the same intermediate product – and hence the same service – may be required

for different higher level products. For example, business products that require substantially

different processes to be delivered will normally differ hardly at all when it comes to the

intermediate product in which the customer pays – in order to extract money from the customer we

need an amount, a date and a statement of what gives us title to ask for the money, regardless of

what that was. In such cases, the normal line is that if there is a common intermediate product, it

should be implemented by a single service that can be invoked from multiple higher level services.

Only the result of such a service matters – not the starting point – because collecting the information

that is necessary to deliver the service is part of the service, not of the environment from which it is

requested. Note that the decision whether to have a single service is a business decision; it has

nothing whatsoever to do with information technology. If it is practical and useful to deliver the

same intermediate product both now and in the relevant future, there should be a single service.

But if there are business forces at work which now or in the relevant future will result in structurally

different intermediate products, we are better off with separate services for each set of

requirements. The bias is, however, towards single services. In SOA, things are the same until they

are different, whereas in databased approaches, things are different until they are the same.

Common services with multiple behaviours

What, you may ask, is the point of making something one service if it requires two drastically

different types of behaviour? Aren’t we busy with the same “One size fits all” restrictiveness that has

bedevilled the databased world? For example, if we sell two types of product, one of which employs

fixed prices whilst the other calculates them afresh according to some complicated formula, why not

have two billing services, each tailor-made for specific types of product?

These are the right questions, asked at the wrong place. They are the right questions, because any

design approach which cannot adequately handle the variation that occurs within the problem

domain is doomed to failure. But they are asked at the wrong place, because the flexibility to adapt

Ten SOA commandments

Author: Hans Wierenga Page 10 of 24 23-02-2010

solutions according to the variation in the problem domain should be built into the services, rather

than be built around them. In the case of the billing service, it should determine which of the two

billing algorithms should be employed, each time it is invoked. That way, if a third billing algorithm

were to be introduced, only the billing service needs to be modified. Note that this does not mean

that the billing service has to be designed as some sort of “all purpose” machine; it could equally

well invoke separate subservices for each algorithm. This choice – the choice between a

multipurpose solution and a framework into which different components can be slotted – is a choice

that can be made per service, because it does not need to be known by the service requestor. In

SOA, this choice will more often than is currently the case result in framework solutions, because the

perceived need for multipurpose solutions is in large part the result of the failure to separate service

requests from the information which is needed to carry them out. The days of one-size-fits-all

multipurpose solutions with so many parameters that it requires highly qualified specialists to

implement them are numbered.

Sticking to the essentials

The process driven approach also enables us to distinguish between the things we do on behalf of

the service requestor and the things we do on our own account. The things we do on behalf of the

service requestor should be carried out as part of the service, and the rest should not. By making this

distinction, we can keep the service as simple as possible, in order to be able to replace it later

without having to change all sorts of other things as well. For example, we may manufacture

products in order to satisfy an external service request, but maintaining the bookkeeping system is

something we do for our own needs, not those of the requestor. If we were to develop a service in

which a customer order is translated into both the manufacturing activities and the bookkeeping, we

would need to amend the service whenever we implemented a new bookkeeping system. That may

not sound so bad, but when we consider all the things we do on our own account it is. Bookkeeping,

logging, stocking data warehouses with up-to-date data, maintaining employee productivity data: all

of these things and more are typically carried out using systems that differ from one organisation to

the next and from one time to the next, so that incorporating knowledge of them within a business

service reduces interoperability and increases the difficulty of replacing the service with an

implementation via another system. We can avoid this problem by generating notifications

whenever something that is important to these functions occurs, and letting them retrieve the

information they need to process the event using common services.

Another category of business activities that should not be carried out as part of a service consists of

those activities which you will not reverse if the service request itself were to be retracted. Typically,

this includes activities such as ordering supplies because the customer order resulted in stock levels

under the replenishment level, registering a new customer and updating the information of an

existing customer. These activities form separate steps in the process, which should be executed

using separate services.

Of course, this way of thinking can be bolted on to databased paradigms. But it is not inherent to

them, as it is with SOA. In consequence, many SOA implementations betray the databased thinking

which inspired them by identifying services in a bottom-up fashion, rather than the SOA top-down

way. In a bottom-up approach, a service that was initially developed for one problem can be re-used

for another problem as well, thanks to the fact that the person who designed it gave serious thought

as to how to make it more generally applicable. In SOA, the use of a single service in multiple

Ten SOA commandments

Author: Hans Wierenga Page 11 of 24 23-02-2010

contexts is the result of deliberate design, not of intuition, and it is designed for all of those contexts

from day one. To talk about re-using a service which delivers a common intermediate product is like

saying that you re-use your front door if you use it to you enter your house regardless of whether it

is to access the living room, the kitchen or the bathroom. Can you imagine a kitchen designer saying:

“Isn’t it wonderful? The guys who designed the exterior entry to the living room just happened to

make something which I can use to provide exterior entry to the kitchen!”? Anybody who talks

about “re-use” of, say, a billing service just hasn’t got it. Mind you, it’s not wrong, just strange and

not very illuminating.

Services follow the business

One consequence of the SOA way of thinking is that SOA results in services that are expressed in

terms of their propositional meaning, not their mechanical implementation. For example, a service

to add a customer record is databased, whereas a service to register a new customer is SOA, even

when they do exactly the same thing. The label we give to a service is important because it tells us

something about who is asking for this to happen and why he wants it. In this particular case, the

SOA top down approach leads to the conclusion that a business process needs a valid customer

registration, which should be accomplished by amending an existing registration if there is one,

rather than automatically creating a new one. In SOA, this is the responsibility of the customer

registration service, whereas a databased approach pushes this responsibility on to the service

requestor. Similarly, a SOA registerCustomer service will itself decide what the customer-ID is going

to be, whereas a databased service may just as well allow the service requestor to decide this.

Convergence to a single solution

In general, SOA top-down thinking results in many design decisions having only one valid option,

whilst a databased thinker will see that option as just one of the possible choices. That’s an

important SOA advantage, given that SOA is oriented toward interoperability, and interoperability

demands that we all drive on the same side of the road rather than negotiate this with each car we

meet. Those who ignore this – for example by maintaining that web services are just one of the

many ways to implement SOA across systems boundaries – have about as much chance of durable

success as a chess player who thinks only one move ahead. True, there will often be simpler ways

than web services to connect system A to system B, but what are you going to do in order to make

the same data available to the call centre or to an output management facility? What are you going

to do in order to ensure that the data warehouse is informed of the events which you transmit from

one system to the other, or to inform your customers of such events as soon as they happen? In the

SOA world, data and events must be made available to many systems, and web services are the most

effective way of ensuring that this can be done with low investments and low maintenance costs.

One domain in which this difference is evident is EDI. Classical EDI is aimed at determining which

information may need to be communicated between organisations, and defines a vocabulary for

that information. That is not the same as defining a particular information exchange. For example,

you can use the same EDI-message to place an order, to query its progress and to amend it. Two

organisations who want to use these specifications to communicate must sit down and agree exactly

how they are going to use this vocabulary. SOA separates these concerns: vocabulary is handled

using namespaces, and these namespaces may be used in multiple services, each of which has a

specific purpose.

Ten SOA commandments

Author: Hans Wierenga Page 12 of 24 23-02-2010

Another domain in which the SOA top-down approach leads to more specific conclusions is the

question as to what constitutes the boundaries of an orchestrated process. Typically, this is a

question which leads to endless debates. For example, is the delivery of a confirmation message to

the customer using an output management service something that should be orchestrated as part of

a customer process, and if so, should it be a fire-and-forget affair or should the output management

service report the successful completion of the action? In SOA terms, all of these things are part of

the purpose of the customer, and should therefore be orchestrated. Other actions, such as updating

the data warehouse to take account of the new customer order or updating the general ledger, are

clearly not part of that purpose and should not be included in the orchestration of the process, even

if they are carried out by the same party.

4. Don’t do your own chores

Generic functions

A business service should have inbuilt logic only for those functions which are specific to the service.

It should delegate all other functions. That way, the service can itself be kept as simple as possible.

That makes it more easy to design and test, and, if need be, to replace. It is a matter of elementary

mathematics: the greater the number of factors on which a new software components must match,

the less likely it is that there is some common of the shelf software (COTS) that fits the bill, and the

more expensive such a solution is likely to be if it happens to be available. If you can delegate the

non-specific functions using a standard service, that reduces the number of factors on which a match

is necessary.

The primary tasks which the service can delegate are chores: tasks which are concerned with the

housekeeping rather than the actual business functions. These chores are generic by nature, in other

words the way in which they are done is not specific to the business context of the services which

they support.

Generic user interfaces

It may surprise you to learn that the biggest chore that an information system should not do itself is

managing the user experience. This is generic functionality, and should be handled as much as

possible using a standard tool. The user experience includes the work lists from which he may select

work items to carry out, the facilitation of work item execution – for example by launching custom

user interfaces, if we still have any – and signing off completed work items. It includes selection of

transactions which the user may perform, the presentation of input screens – normally generated

from XSD’s – and their validation using the standard validation services corresponding to the

transaction. It includes maintaining the current user context so that he doesn’t need to re-enter the

current customer, product, case, process instance or whatever but can use these values by default

and override them when necessary. It includes the presentation of all documents which are relevant

in the current user context. It includes the presentation of alerts on which the user may need to

react.

All of this can be implemented using a tool that requires no inbuilt knowledge of the business. It is in

general much better for the user to have a uniform environment in which all of this is covered than

to optimise a user interface for a specific type of action. Where there is a clear business case to

Ten SOA commandments

Author: Hans Wierenga Page 13 of 24 23-02-2010

deviate from this principle, remember at least this: don’t optimise the user interface before you

have stabilised it.

Typical generic functions

Other chores include, but are not limited to, the following:

Security: Chores that are concerned with establishing the identity and access rights of

service requestors.

Notification: Chores that are concerned with identifying which recipients should be informed

of a business event. This includes maintaining the subscriptions on which this is based.

Output management: Chores that are concerned with communicating information offline,

rather than as a service response. This is typically the case when a customer request must be

formally confirmed, for example as an email that confirms the web purchase that you just

made via your browser. It is also necessary for unsolicited messages, such as monthly bills.

Output management must determine via which channel the information must be sent, and

which address should be used. It should transform the message into a format which the

recipient can take in, send the message, and add it to the document archive. Output

management includes the maintenance of the templates which are used to transform data

into user-intelligible messages and of the addresses and channel preferences of possible

recipients.

Data transformation: Chores that are concerned with translating data from one format to

another, bundling separate services into one service — in McDonalds terms, happy meals —

and de-bundling, slicing service requests into separate requests for different populations

and de-slicing the responses, queuing and de-queuing, or protocol conversion.

Process orchestration: Chores that are concerned with orchestrating a process, so as to

ensure that the services of which it is comprised are executed in the appropriate sequence

and only when relevant, that alerts are issued when deadlines threaten to be exceeded, and

that interrupts caused by ancillary messages or elapsed deadlines result in new process

paths being taken.

Dossier management: Chores that are concerned with maintaining and accessing dossiers of

related information. These dossiers may be virtual, in the sense that the information which

is presented to the user when he asks for a dossier may be retrieved using queries. For

content which is extracted from databases this is regarded as normal, but there is no

particular reason not to apply the same approach to documents. In some cases this should

be augmented with facilities to assign documents to dossiers specifically.

Record management: Chores that are concerned with maintaining information that is not

allowed to be changed.

Bottom-up delineation

The services that implement these chores do not form part of the business service hierarchy. They

cannot be designed them in a top-down manner, because there is no “top”. For such services a

Ten SOA commandments

Author: Hans Wierenga Page 14 of 24 23-02-2010

bottom-up approach aimed at achieving maximum re-use is more appropriate. This allows the best

service implementations to succeed and inferior ones to be removed from the stage.

With databased approaches it is seldom practical to replace an inferior generic function with a

better one, because that requires changes in all applications which make use of the function. With

SOA, on the other hand, such a replacement is simple, provided that the rule for not knowing what

you don’t need to know has been applied, including the corollaries that a service request should

contain no more information than is necessary to specify the request and that the service itself

should take the initiative to ask for more information when necessary. Take, for example, an

authorisation service which is called by an application in order to determine whether a particular

user should be allowed to perform some function on the data of some customer – for example: “Is

our employee Donald Jones entitled to access financial data regarding the Acme Widgets

company?”. A simple version of the service might be capable of handling only situations in which

these questions can be answered using a table of employees versus functions. A slightly more

complicated version might recognise that Donald Jones is a member of one or more groups and use

the group authorisations in addition to the individual ones. Taking things a step further, the

authorisation service may use the credentials of the user in order to differentiate between

employees and customers, and allow the latter access to only their own data. A sophisticated

version might use a standardised service call to the Business Process Management System or a case

management system, asking whether Donald Jones has been assigned any responsibilities in our

dealings with Acme Widgets. Preferably, the authorisation service will log the requests and the

answers, but simple ones won’t. The organisation can switch from one version of the service to

another without any change to the applications which invoke it.

It is also possible to design generic services to automatically configure themselves for the situation in

which they must operate. An authorisation service could, for example, check whether there is a

service that will tell it about the responsibilities that employees have been given for particular

customers, and decide to use only individual and group access rights if such a service is not available.

In this way services can be made re-usable across many organisations.

5. Don’t bore yourself with testing

Why SOA is easier to test

One of the perceived disadvantages of SOA is the supposed difficulty of testing. This perception is

entirely inappropriate, for many reasons.

Firstly, the use of metadata in SOA makes it possible to prevent errors from being incorporated into

the system. It is possible to validate the system at the metadata level, for example to ensure that all

data which is required for processing is collected and checked before it needs to be used. This can be

done across the breadth of a business process. Don’t test the system when you can validate the

design instead.

Secondly, almost all the testing, and all the systems integration testing, can be done automatically

once a test base has been established. There are, however, some prerequisites to this. There must

be a rigorous separation between presentation and processing. Fortunately, this is a natural way of

working with SOA. For all input, there must be an XSD. Using this XSD, test records are made. In the

Ten SOA commandments

Author: Hans Wierenga Page 15 of 24 23-02-2010

same way, test records are made for all expected output. Where processes do not produce all the

outputs which are needed to prove that the system is working correctly, the test script must include

queries which produce additional output for this purpose. When the test is run, the input records

are presented to the system one by one, and then the actual output is automatically compared with

the expected output. This results in a list of exceptions, each of which must be accounted for. The

test can be run as often as needed. Naturally, the results may depend on the data which has

accumulated in the database, so this needs to be compensated for. Also, the system must not exhibit

time-dependent behaviour. Rather than waiting, say, a week for a time-based trigger to go off, the

system must be capable of reacting to events which can be set off at time intervals which are more

appropriate to an automatic testing sequence. The user interfaces should be tested on a one-by-one

basis, but should never be used in the integration tests.

Thirdly, SOA designs tend to result in more robust systems: there is simply less that can go wrong.

SOA reduces the number of factors on which information systems must agree in order to cooperate,

so there is less scope for a design error which results in a disagreement on some essential factor.

And if things do go wrong, they can be detected before they can do any damage. With SOA, before a

message is processed, it is validated in order to determine whether it is well-formed and conforms

to the relevant XSD.

Realistic testing

Lastly, the rigorous separation between test and production environments that is characteristic of

the databased era is no longer necessary, and is sometimes not appropriate. This is possible because

we are not really testing systems anymore, but the passage and processing of messages. SOA offers

a triply safe, effective manner of distinguishing between test and production messages. Each

message contains version numbers of the message itself and of each namespace it incorporates,

excepting those which are encapsulated. And each message contains a tag which indicates whether

it is intended as test or production. All that is required is a SOA gateway just inside the firewall which

carries out the following processing on each incoming message:

1. The message is checked in order to determine whether it corresponds to a known version of

a known XSD (excepting the encapsulated messages).

2. The message is checked to determine whether it is valid, using our copy of the relevant XSD.

3. If the message is intended for production, we check whether the version numbers have been

approved for production use. Only if this is the case is the message passed on to the

production systems. All other “production” messages are rejected.

4. If the message is intended for test, it may be passed on to the designated test versions of

systems. In special cases, where the message is used only for data retrieval, this may be a

production system.

5. Only after a message has been adequately tested is the register of production versions and

XSD’s updated.

This manner of working is not only triply safe, but also enables the routing of messages to be tested

in realistic manner. It also greatly reduces the need for reconfiguration in order to make the

Ten SOA commandments

Author: Hans Wierenga Page 16 of 24 23-02-2010

transition from test to production. Because this reconfiguration is by its very nature untestable, it is

a frequent source of error. Release managers compensate for this by releasing new software only in

the middle of the night or at weekends, so that if something goes wrong the previous situation can

be restored before anybody much has encountered a problem. But that’s no way of working if the

change will affect other organisations. SOA release management is so much easier!

It is time that the prevailing wisdom about SOA testing is revised. SOA is a way to minimise the need

for testing and the effort to set tests up. It enables crucial tests to be done more automatically, with

better result.

Make it better

Rationale

SOA enables the development and deployment of information systems that support a far richer user

experience than is generally achieved using databased approaches. The systems encompass more

forms of information, have a broader repertoire of behaviour, achieve a higher level of uniformity

and consistency in their behaviour and are more reliable, as measured from the points of view both

of the customers and of those people within the organisation who are concerned with compliance.

But achieving these benefits requires us to say goodbye to databased practices.

6. Always keep your promises

Why databases don’t always keep promises

The act of accepting a service request is by definition a promise to the requestor that the service

request will be executed. This execution is by definition a process, involving at least one but often

very many steps.

Databased thinking and processes don’t sit well together. By their very nature, databases are islands.

And islands promote parochial thinking: anything outside the island can’t be that important. This is

illustrated most vividly by the databased concept of a transaction: a unit of work, which takes the

database from one consistent state to another. In some special cases this concept may be extended

to multiple databases, using 2-phase commit technology, but that is the limit. It is entirely foreign to

the databased view of the world that logical consistency might need to be maintained across a

business process, rather than just a moment in time, and across all places where information is

changed, including not just databases but also process management systems, messages and the

human agents that send and receive them.

The SOA transaction concept

What’s foreign to the databased world sits naturally with SOA. A business transaction — and

therefore the process by means of which it is implemented – is a service. To understand how well

SOA supports the logical consistency requirements, it is important to understand what a business

transaction entails. A business transaction consists of the following elements:

The vendor provides the customer with information on which the customer bases his

purchase decision. Typically, the information is concerned with the properties of the goods

Ten SOA commandments

Author: Hans Wierenga Page 17 of 24 23-02-2010

and services on offer, the terms under which they may be acquired — including, of course,

the prices – and their availability. In legal terms, the vendor represents this information to

be true.

The customer places a purchase order with the vendor, based on the representations made

by the vendor.

The vendor verifies that the information on which the purchase decision was based is still

applicable, and if so, commits to the order. If there has been a change – perhaps because

the goods or services are no longer available, perhaps because the prices have increased,

perhaps because the specifications of the goods or services have changed – then some

processing may be required in order to decide what to do: commit to the order anyway,

negotiate amendments to the order, or perhaps just cancel it.

The vendor and the customer each perform their side of the purchase agreement.

How SOA maintains consistency

SOA maintains the logical consistency of a business transaction in a number of ways. Firstly, all

communication which implies a change to a previous situation can be done using a protocol that

guarantees secure message delivery. Whenever a commitment is made by either the vendor or the

customer, the act of making the commitment involves such a change. In this way, it is not possible

for customer and vendor to have different perceptions as to the current state of the transaction.

Secondly, the logical consistency of databases and the record of process progress in the process

management system can be maintained using a 2-phase commit protocol. Logical consistency across

multiple databases is maintained by a sequence of such 2-phase commits. First database A is

synchronised with the process management system, then this system synchronises with database B,

and so on.

Thirdly, any changes in the vendor’s reality between the time that representations were made to the

customer and the placing of the order are dealt with using an optimistic concurrency control

mechanism. This way of working is natural to SOA: the process of determining whether there has

been a relevant change can be completely automated. And because SOA makes it feasible to access

data at the moment it is used to make a decision, it is only rarely that the optimism is unfounded

and manual intervention is necessary.

Lastly, an abort of the business transaction itself – for example that the customer withdraws the

order, turns out not to be able to pay, or dies – can be handled relatively easily using SOA. Because

the documents which have been used or produced in the context of the business transaction are

clearly identifiable, it is possible to determine which countervailing messages are required in order

to correct the perceptions of customers and vendors. Because it is clear which database updates are

directly related to the transaction, it is clear which changes in databases need to be rolled back using

compensating transactions. Given that aborts occur relatively infrequently, it is normally not cost-

effective to completely automate these actions, but your design must allow for them.

Note that the scope of a business transaction is limited to that which is done directly in order to

handle a service request. In SOA, the rule is to orchestrate your own processes and provide event

Ten SOA commandments

Author: Hans Wierenga Page 18 of 24 23-02-2010

notification to others. If the service is aborted, it may be necessary to notify them of the change.

What they do with this information is entirely up to them.

Note also that creating the content of a customer’s service request is, strictly speaking, something

that precedes the process, rather than part of it. It’s not really your task: in principle it should be

enough to supply your customer with an XSD and a validation service, and let him decide whether he

is going to key data in or generate it from his own information systems in some way. For consumers

this is not – yet! – a feasible approach, but it remains valid that collecting data and processing it are

two separate things, performed in separate environments using separate tools. For the same data

collection there may be many implementations, depending on the wishes of the customers and the

channels that the use to communicate with you, but there should only be one service to process it.

7. Don’t limit yourself to models

Databases are models, SOA is more

A model of a domain is a representation of it which is easier to work with than the domain itself. The

vast majority of databases are models. They represent some administrative, physical or social

domain in such a manner that questions concerning this domain can be answered by a query to the

database, and the need to act in the domain is signalled from within the database. It is, for example,

easier to look in a database to find a customer’s address than to follow him home and note the

address of the house he enters. And it is easier to count records in a database than persons in a city

or products in a warehouse. The functions that need to be supported are in principle sufficient to

determine the data structure of the database. However, because it is not normally possible to

specify all such functions in advance, we use data modelling techniques to analyse the objects of

interest within the domain, including their relationships and the items of information which tell us

what we may need to know about them. For example, if the database is concerned with customers,

there will typically be a database record for each customer, containing the information concerning

the customer that the users of the database are interested in knowing. Because the data model is

based on more fundamental considerations than just some particular uses of the database, basing

the data structure on this model results in a database that is easier to adapt to unforeseen

requirements. The database is semantically structured; in other words, it is structured according to

the meanings of the things it represents.

Databased technology is very uncomfortable with any data structure that does not implement a data

model. It simply can’t do very much with descriptions, documents, photos and the like. SOA, on the

other hand, is just as good with documents and dossiers as it is with data that is structured according

to a data model. SOA allows for a single design which handles both semantically structured

information and documents. However, because databased systems dominate our thought patterns,

we think it only natural to maintain their limitations when applying SOA. But in the SOA world there

is no particular reason to apply such restrictions, and every reason not to.

The benefits of going beyond models

The primary reason to combine semantically structured data with such things as hyperlinks,

documents, photos and sound fragments is that it creates a richer user experience. For consumers,

this is not an option but an imperative. For the knowledge workers you employ, it makes them more

Ten SOA commandments

Author: Hans Wierenga Page 19 of 24 23-02-2010

effective. It is only for the staff performing routine administration that it gets in the way. But the

more you apply SOA, the less you will need them. Their task of keying in data from forms can be

outsourced to anywhere: all you need to do is scan each form and send the image to an agent who

uses essentially the same web form as the customers to key it in. To the extent that the other tasks

of these employees are genuinely routine, they can be automated.

A second major reason to not limit yourself to databased data is that a synthesis of databased and

document-based data is by far the easiest way to maintain the records and audit trails which are

necessary for compliancy purposes. Databases are by their very nature unsuited to such ends.

Because a database is typically built to act as a model of some administrative reality, it should

therefore be easy to change it when that administrative reality changes. Database management

systems are designed to facilitate such changes. When they are used to maintain records which are

not allowed to be changed – bookkeeping entries, for example – the designers have to build all sorts

of defences into the system to prevent malicious manipulation of the records. Even then, no judge in

his right mind would trust a database. The records and audit trails which are needed for compliance

purposes must be entrusted to record management applications, and linked to the databased

systems using SOA.

With SOA, connecting semantically structured data to other forms of information representation is

easy. This makes it ideal for content management and content presentation. A database can contain

hyperlinks to other information. For example, we can store the URL of the input document that gave

rise to a transaction with the database record of the transaction. When the transaction record is

displayed, the link is presented along with it, and may be activated by the user via a separate service.

It is no more difficult to make a service to access a document from a document archive than it is to

make one to access data from a database. Unless, of course, your document archive is not SOA-

capable, in which case one of your first actions must be to replace it.

But it works the other way around too. Not only is the semantically structured data enriched with

the links to records, but the records are enriched with links to the semantically structured data.

When you store a record in a document archive, the semantically structured data that is associated

with it will normally contain all the information which is necessary to index it. There is little to no

need to index the records manually. Better yet, the index can be progressively enriched by updating

it automatically, every time that the transaction is progressed in the database.

Integrating databased and document-based data has the further advantage that it is easy to support

multiple versions of data. That doesn’t make sense in the database world, because a database is a

model, and to be useful as a model it must give at least a best guess in answer to any question about

the administrative reality of which it is a model. A document, however, is a statement made by a

specific party at a specific time in a specific context, and it is quite possible – and often useful to

know – that conflicting statements exist.

Culture shock

Note that databased thinkers are not the only people who will have trouble with the SOA unification

of the worlds of documents and of databases. People whose thinking has been formed by the

current generation of document and record management systems have an even bigger problem.

Suddenly, archived documents are part of business processes, not things which come into existence

Ten SOA commandments

Author: Hans Wierenga Page 20 of 24 23-02-2010

after process completion. And indexing is not something done manually, once per document, but

something automatic and ongoing. Nor is the index part of the archive anymore: an archive only

contains documents, and the indexes are maintained separately. And it is therefore not necessary to

limit yourself to one archive, or even to only your own archive. These changes are so profound that

it is hard to imagine that anybody who was suited to the old world could also be suited for the new

one.

8. Don’t use models as photos

Material and formal history

The biggest mistake of my career was to design a database in which all service requests could be

supplied with a time stamp, so that if it was filled, the service could reply according to the data

contents that the database had at the moment indicated, instead of the current contents. It was

enormously expensive, slowed the database down to a crawl, and served no useful purpose. When

users wanted to see how a particular result in the past had been calculated, they invariably looked in

the document archives, because the archives always contained the result that had been

communicated to the customer. And that’s all they wanted to know.

Note that it is not the use of databases to register past situations that is the problem. Databases are

models of some part of the real world, and the state of that part of the real world at some period in

the past – its material history – may very well be relevant to the purpose of the model. But a

database takes itself too seriously if it wants to recreate its own state at some point in the past, in

other words to register its formal history.

How SOA supports formal history

The SOA approach offers a far more effective solution to the problem of recreating formal history,

not just of a single database but of all the information which an organisation uses and creates in its

administrative processes. The SOA separation between the collection of the data required to update

the organisation’s information and the actual execution of the update allows that data to be

archived in a record management application. Once SOA has been applied, this can be done

automatically, with almost no additional effort in order to incorporate a new data set. In this way, all

incoming messages and all updates made by your own employees are captured.

The situation for outgoing offline messages is, if anything, even simpler. The SOA aversion to mixing

chores and added value for customers leads quite naturally to the use of an output management

service to ensure that offline messages reach the intended recipients via an appropriate channel, at

an address which they really use and in a form which they can adequately take in. The output

management service can take care of recording the outgoing information automatically.

There is, however, one problem. The ease with which organisations can disseminate information

using SOA tends to result in a substantial increase in the number of service responses to with

external parties. These service responses typically imply a commitment of the organisation to their

propositional content. The organisation could get into legal problems if they are incorrect. The

standard SOA means of handling this involves the following guidelines.

Implement business transactions in conformance to the SOA transaction concept.

Ten SOA commandments

Author: Hans Wierenga Page 21 of 24 23-02-2010

Include disclaimers in the service level agreements for all information retrieval services.

Don’t access data until you need to use it. That way it will be as up-to-date as possible.

Don’t lie. If you always tell the truth, you will encounter fewer claims that your information

was incorrect.

If the use of disclaimers cannot prevent you from having to respond to claims that your

information was incorrect, log each service request and response, using the standard

organisation-wide service.

This manner of working is much simpler to organise than the databased solutions, because it does

not require additional logic per message type, but only a call to the logging service, in which the

entire incoming or outgoing message is treated as a single data item.

9. Don’t trust systems

SOA hygiene

If your body applied the same rules of defence against external agents as most web sites, you would

be dead in a day. We live in a hostile world, where just about the only way to avoid your information

systems being attacked is to make them inaccessible. In this world, SOA is a marvellously hygienic

concept. In principle, the only messages we have to let through our firewalls are robustly enveloped

XML-messages. And they can be passed through the firewall in such a way that they are checked for

well-formedness, XSD-conformance and the validity of their authorisations before any further

processing. The potential for them to commit mayhem is very small. Buffer overflows can explode

only in bomb-proof areas, and SQL injections are nothing more than curses in a language to which

we are entirely deaf. Corrupt messages are recognised as such before they can do any harm.

It is important to note that the strength of this approach is based on the authentication and

verification of messages, not of systems. That is necessary, because encapsulated XML is often

passed on by systems that bear no responsibility for their content, so that checking that a message

has come from some authenticated system doesn’t guarantee much. It is more robust, because a

successful hack of one information system or computer does not open the gates to hacks of others.

SOA security contrasts markedly with the systems-based security of the databased era. With

systems-based security, it is the task of each system to avoid buffer overflows, for each message

type and each data item. There are thousands of points where it could go wrong. The chances of a

weak spot are therefore considerable. A SOA gatekeeper, on the other hand, does all of this at a

single point. Systems-based access security tends to result in authorisation controls being built in to

systems, which not only makes it almost impossible to keep track of them, but also enormously

expensive to introduce structural changes, such as the addition of users who are external to the

organisation.

A further advantage of using a SOA gatekeeper is that it can ensure that all outgoing messages are

appropriately encrypted and signed, so that nobody can eavesdrop on them or change them en-

route. In principle, most of the WS-*hell can be handled entirely by the SOA gatekeeper.

Ten SOA commandments

Author: Hans Wierenga Page 22 of 24 23-02-2010

Beyond systems-based authorisations

Systems-based security also promotes mindsets which are much more determined by systems-based

thinking than most of us would care to acknowledge. Because systems are structured around

functions and data structures, authorisations are expressed in these terms. For example, an

employee is either authorised to change product prices or he is not, based on his role within the

organisation. For back office employees that need not be such a problem, but for front office

employees and customers this is inappropriate. For them, the primary determinant of what they

should be permitted to see and do is the slice across all systems and databases of the information

concerning the customer at hand. A brief scan of information security literature is sufficient to

establish that information security consultancies haven’t got this point quite yet: for every reference

to slice-based access security there are thousands to role-based access security. On the positive side,

there are emerging concepts which go a long way toward implementing the kind of access security

that makes full use of the advantages that SOA has to offer: claims based access control, federated

identities and de-perimeterisation.

Make it manageable

Rationale

To get maximal benefits from SOA, it is insufficient to merely adopt new rules of behaviour. The way

the organisation thinks about its self and its IT needs to change too. Otherwise the current

organisation will revert to the old behaviour, not because it wants to but because it sees that as the

only way to maintain control over its own agenda. To counter this, you must start with the way

projects are organised.

10. Don’t leave infrastructure until last

Why worry about infrastructure?

SOA is all about separating infrastructure from business, not just in the functional sense, but also in

the project management sense: a project manager should be able to concentrate on the business

added value of the services that he is responsible for developing, without having to worry about the

infrastructure. It is no less absurd that a business project manager should have to organise a new

authorisation service as part of his project, than that he should have to install a new telephone

exchange. To burden the project manager with such things results in an increase in the number of

things that he has to coordinate, which results in turn in a disproportionate increase in the

complexity of his task. Even letting him wait until some other project manager delivers the

infrastructure is something that should be avoided where at all possible.

The above remarks apply equally well to the hardware infrastructure as to the infrastructural

services. The computing power and network capacity should always exceed the requirements, so

that project managers who implement new services shouldn’t have to worry whether the

infrastructure can support a thousand service requests more per day than originally envisaged.

Given that software development is typically an order of magnitude more expensive than IT

infrastructure, the last thing we want is that systems development projects get bogged down by

bureaucratic processes concerned with ensuring the availability of IT infrastructure.

Ten SOA commandments

Author: Hans Wierenga Page 23 of 24 23-02-2010

Infrastructure: how SOA is different

The good news is that the SOA approach of saying what you want and then shutting up results in

simple, stable interfaces to infrastructural services. That is why it is not necessary to implement the

world’s most advanced infrastructural services before you can start any business projects. The

complexities of these services lie not in the interfaces which are used to invoke them, but in the

information which they request from other sources – or maintain themselves – in order to produce

an optimal result. It is therefore perfectly feasible to start off with rudimentary infrastructural

services, and to make these available for test and production use in plenty of time for them to be

used by business projects.

The bad news is that SOA results in much more functionality being classified as infrastructure. In

particular, each chore results in a generic service that must be implemented as part of the

infrastructure instead of being incorporated into each of the business functions for which it is

required. That is not so bad by itself, but it does result in a new problem: how to organise things so

that, for each part of the infrastructure, there is somebody with both the mandate and the resources

to ensure that it is available on time with the appropriate functionality and capacity. An organisation

that is not geared to addressing this problem is not geared to implementing SOA – it’s as simple as

that.

Where do we go from here? Although the ingredients of SOA have been around for a while, SOA is radically new. Like the

shipping container, it is more than just a new way to do the same as we have always done, because

it enables us to cooperate on a new scale. And it is not primarily a new technology, but a new way of

thinking. It is so different to the current way of thinking that you should not be embarrassed if you

have to read through this article several times before it starts to sink in. It may take a while before

your natural tendency to think that the suggestions it makes are impractical or unnecessary or both

subsides sufficiently for you to be able to take stock of it all.

To be fair, there will be situations in which the SOA solution is not directly practical for you. For

example, you may acquire all your systems as packages from vendors who have no particular

interest in applying SOA in the manner described in this white paper. In that case, treat the

suggestions in this paper as a compass: use them to decide whether the choices you make are

leading you in the right direction.

Because SOA is so radically new, not all the concepts, tools and standards which we need in order to

make full use of it are available just yet. Current gatekeepers do not do everything we should expect

of a SOA gatekeeper, and current user interface facilities have difficulties with encapsulated XML.

Worse, as this white paper has argued, there is currently much in the SOA world that is confusing,

unnecessary or wrong. Current applications of SOA concepts and technologies do not get anywhere

near achieving the level of interoperability that it is capable of. Too often we are confronted with

interoperability problems that are due to somebody else having chosen to implement a slightly

different subset of the many WS-* protocols that are on offer. The millions of possible solutions as

to how parties communicate have been reduced to a few handfuls, but that is still far too many.

Ten SOA commandments

Author: Hans Wierenga Page 24 of 24 23-02-2010

But we have made a good start. The rest we can work out along the way. That, too, is SOA. We don’t

have to know what we can’t know yet in order to make progress. An enormous potential to make

things simpler, more reliable, more predictable, and functionally richer awaits us. It is time to start.