SOA Quality Assurance

Distributed Environment Testing Strategies, Issues and Best Practices

SOA – Services Oriented Architecture

SOA is the journey to an elusive destination.

Service orientation is all about agility, reuse and business focus

The meaning of the term is dualistic: It is both abstract business and technical at the same time. A continuum between philosophy and code.

It is not a specific technology or product, but a collection of enabling technologies, standards and products put together as a SOA

SOA is the journey to an elusive destination.

Service orientation is all about agility, reuse and business focus

The meaning of the term is dualistic: It is both abstract business and technical at the same time. A continuum between philosophy and code.

It is not a specific technology or product, but a collection of enabling technologies, standards and products put together as a SOA

Semantic layer provides business representation of data

User Lifecycle Administration

User Administration

Resource Access Control

Event Management

A Solution – Service Oriented Architectures

SOA is an approach to organizing and using

IT to match and combine needs with

capabilities in support of the overall mission

of an enterprise

Capabilities performed by one for another to achieve a desired outcome

Functionally aligning architecture to enable a collection of independent services to be linked together to

solve a business problem

The fundamental organization of a system embodied in its capabilities, their interactions, and the environment

Architecture

Oriented

Service

SOA - A paradigm that encourages organizations to re-think how their IT capabilities are organized

Difficult decisions

Model-Based Concurrent Engineering Processes

Vision

it’s about

Of course strategy is hard

making tough choices.

SOA Quality Assurance Testing Problem

Distributed Environment Redefines QADistributed = Evolving Distributed Computing SystemsNew software relies on old running systemsTesting environment cannot be controlled

Scale testing a special problem

Requires new relationships and responsibilities for Service Producers and Service ConsumersRequires a SOA Core Service Broker

Service Level Agreements

Core Enterprise Services, Registry and Metadata Repository

Accreditation and Certification Services

What is SOA?

The nexus between IT and Business – allow for a common dialog

An new approach to both IT and Business gets done

Reuse

Agility

Efficiency

Loosely-Coupled

Sharing

SOA is NOT ...WebServices, Enterprise Service Bus, etc.

SOA tools and software are NOT a panacea

SOA Infrastructure software is NOT a replacement for sound distributed systems engineering

SOA tools will NOT address required Social Engineering

SOA tools and Infrastructure software will NOT be universally and consistently understood

Get help from your vendors and understand that most SOA tools can be used cooperatively

Message-Oriented Middlewareasynchronous != loosely coupled

Understand the differences between Broker-based ESB and MOM-based ESB

New

A Methodology

What is SOA not?

SOA is not a Thing

It’s not even an Architecture … it’s just an approach toward an architecture (of which distributed systems architectures are the most applicable).

It’s a way of thinking and working such that the product of the work results in a systems that is:

Agile and Adaptable

Engenders reuse

Encourages and Enables Sharing of Existing Systems and Data

The SOA Products offerings only provide a toolbox of technical solutions, not a “silver bullet.”

SOA is not a Methodology

SOA nor SOA tools DO NOT do Distributed Engineering for you

ESB does not solve ALL design problems

MOM or EAI solutions are engineering solutions for specific Distributed Engineering problems

The only thing harder than designing and building a complex distributed system is TESTING and DEBUGGING them

SOA Governance

I’m not sure what this is really it. Nor do I know anyone else who has a really good answer.

“The Nexus between politics and operations”

Comprised of Policies, Procedures and MetricsService Lifecycle

Governance Tools:UDDI RegistryMetadata RepositoryWebServices Management (SOA Software [BlueTitan], AmberPoint, etc.)Traditional Enterprise Management Systems (Tivoli, HP OpenView, BPM Patrol, etc.)

Governance usually optimized for one of several outcomes:Reuse – The Cornerstone of Reuse is CommunicationAgilityPositive Financial OutcomeSharing

SOA Governance is just beginning to mature“Draconian”, “Autocratic”, “Oligarchy” or “Facist” Governance does not work well in large organizationsUnderstanding how to govern large

Usual System Testing

Development EnvironmentContinuous Integration

Unit and Integration Testing

QA EnvironmentProtected environment

Scale Testing

Hardware identical to Production Environment

Separate network

Production EnvironmentSame as QA Environment

Handles multiple Security Enclaves (NIPR, SIPR, JWICS)

Issues with Distributed SOA Testing

It impossible to perform traditional QA testing a Distributed Environment

Multiple Producer supported environments DevelopmentUnit-TestScale-TestingProduction

Secure enclave testing required standalone serviceProducers must provide packaged service for SCIF based developmentLike to use VMWare or other Hypervisor technology to reduce technology burden on consumer.What happens if the service is a composite service?Even with SCIF based development, final scale and functional testing wants to use the provider-hosted service

Issues with Distributed SOA Testing

There is no “Global” synchronized clockAll event correlation required use of cause-event based clocks – otherwise known as Lamport Clocks.

Unclear how to do this is a large-scale coordinated way

Auditing and LoggingConsistently available common logging and auditing services are required.

Should be provided by a shared service infrastructure (NCES?)NCES does not list this as a common service nor a segment of Enterprise Service Management

Adding Friction

Producer FrictionProducer Accreditation and Certification

Producing “Composite” Services

Consumer FrictionUsage friction – adding hurdles to utilizing shared data and services

Vetting consumers in various degrees

Root-case analysis problems with provided composite services

Responsibility of Service Providers

Provide the service using a “Community Standard” interfaceWebServices: XML, XSD, HTTP, SOAP, etc.

JMS, FTP, SMTP

NOT: MQ Series, .NET, Proprietary “extensions” to Standards

Provide an Accredited ServiceAssert that the services lives up to “Total Assurance or Service Safety” policies and standards

Register the Service with the Shared Service Infrastructure provider

Handle the Unintended User?

Responsibility of Service Consumers

Use the service the way the producers intended it to be usedAccess service via brokered channel

Access via Shared Service Infrastructure as opposed to direct access

No “unintended” Denial-of-Service attacks

No oversized payloads

No unreasonable SLA expectations

Open QA resultsReport errors promptly

Utilize auditing, logging, security, etc. where possible

Role of the Broker-based Infrastructure

Provide the Authoritative Registry and Metadata RepositoryRegistry for Run-time

Repository for Design-time

Provide the Accreditation and Certification ServiceFunctional

Security / IA (Information Architecture)

Service Level Agreement Ranges

Provide SLA Adjudication ServicesConcept: SLA in the “Eye of the Consumer”

Provide Metrics publishing for GovernanceProvide Scalable Service NetworkProvide automatic “Testing Mode” switching

Conclusion

SOA is a dangerously overused acronym that required specific definition in each organization Your Organization with SOA done correctly will display the emergent properties of Agility, Reuse, Efficiency, Loosely-Coupling, etc.Functional Testing (QA) is brutally difficult to do in an open Distributed Computing Environment

Governance from Broker, Provider and Consumer is not a solved problem

Most challenges in Your Organization service creations and usage will depend greatly on how Social Engineering and Communication is accomplished