copyright © 2008 texas education agency. making the move to soa in texas government rick goldgar,...

Copyright © 2008 Texas Education Copyright © 2008 Texas Education Agency. Agency.

Making the Move to SOA in Texas Government

Rick Goldgar, Ph.D.Rick Goldgar, Ph.D.Chief Technology Officer and Deputy CIOChief Technology Officer and Deputy CIO

Texas Education AgencyTexas Education Agency

David E. ButlerDavid E. ButlerProject ManagerProject Manager

Texas Education AgencyTexas Education AgencyAugust 12, 2008August 12, 2008

Copyright © 2008 Texas Education Agency.

Who is this Rick Goldgar? 20+ years in commercial software product development, 20+ years in commercial software product development,

management, marketing:management, marketing:

Systems Simulation / Code generationSystems Simulation / Code generation• 1982-1984 – screen based automated code generation multiple 1982-1984 – screen based automated code generation multiple

languageslanguages• 1990 – 1995 – systems simulation and OO analysis/design/code 1990 – 1995 – systems simulation and OO analysis/design/code

generationgeneration• Worked with key industry OO methodologists (Shlaer/Mellor)Worked with key industry OO methodologists (Shlaer/Mellor)

State Government State Government • 2002 – Current Texas Education Agency2002 – Current Texas Education Agency

AerospaceAerospace• 1984-1990 – Aerospace, embedded systems (Tracor, now 1984-1990 – Aerospace, embedded systems (Tracor, now

British Aerospace)British Aerospace) Systems Administration / Monitoring toolsSystems Administration / Monitoring tools

• 1995-2000 DB / Application management (BMC Software)1995-2000 DB / Application management (BMC Software)• 2000 – 2001 Application monitoring / management2000 – 2001 Application monitoring / management


Who is David Butler

18 years in commercial/public utility/government software 18 years in commercial/public utility/government software development/ management:development/ management: Commercial Systems DevelopmentCommercial Systems Development

• 1988-1993 – Software Developer for AT&T Bell Laboratories 1988-1993 – Software Developer for AT&T Bell Laboratories (now Alcatel/Lucent)(now Alcatel/Lucent)

• 1993 – 1998 – Project Manager for IBM Global Services & 1993 – 1998 – Project Manager for IBM Global Services & AdvantisAdvantis

Public UtilityPublic Utility• 1998 – 2002 – I.T. Program Manager for Entergy Services1998 – 2002 – I.T. Program Manager for Entergy Services

Nuclear Generation and Distribution Business Units Nuclear Generation and Distribution Business Units SoftwareSoftware

State Government State Government • 2005 – Current I.T. Program Manager Texas Education 2005 – Current I.T. Program Manager Texas Education

AgencyAgency


Texas Education Agency - IT Statistics IT development staff of about 150 FTEs and contractorsIT development staff of about 150 FTEs and contractors

80+ legacy and new applications, supporting:80+ legacy and new applications, supporting: Texas K-12 Public Schools Funding, Grants, Entitlements Texas K-12 Public Schools Funding, Grants, Entitlements

(> $20B /yr in funds management / distribution) (> $20B /yr in funds management / distribution) Textbook management (> $150M / yr)Textbook management (> $150M / yr) Data gathering / analysis / reporting / complianceData gathering / analysis / reporting / compliance

(>4M schoolchildren, > 1200 school districts)(>4M schoolchildren, > 1200 school districts) Teacher certification tracking ( > 400K active teachers)Teacher certification tracking ( > 400K active teachers)

Constant changes due to Federal and State LegislationConstant changes due to Federal and State Legislation

Infrastructure under consolidationInfrastructure under consolidation

Over three hundred servers (Windows and AIX)Over three hundred servers (Windows and AIX)

Very little mainframe legacyVery little mainframe legacy

New development is 80% C#, 20% JavaNew development is 80% C#, 20% Java

Virtually all web based applicationsVirtually all web based applications

> 90% of our projects are successful. > 90% of our projects are successful. (The 2006 Standish report states the industry average is now 35%).(The 2006 Standish report states the industry average is now 35%).http://www.sdtimes.com/article/LatestNews-20070201-40.html


We all want to do more with less: More functionalityMore functionality More flexibilityMore flexibility More qualityMore quality

Less resourcesLess resources Less timeLess time Less moneyLess money

Why are we here today?


MAKING Software Development more Efficient

Software development has some of the qualities Software development has some of the qualities of factory work, except that we in the of factory work, except that we in the software world always seem to have to build software world always seem to have to build the car factory the car factory andand the cars. the cars.

To build an efficient, flexible To build an efficient, flexible factoryfactory takes takes careful, creative thought. It is an investment careful, creative thought. It is an investment that will take more than one project to pay off that will take more than one project to pay off and mistakes can be very costly.and mistakes can be very costly.

Building our “software” cars (applications) Building our “software” cars (applications) correctly and efficiently using that factory correctly and efficiently using that factory requires good, integrated processes, good requires good, integrated processes, good tools, and discipline.tools, and discipline.


Why do SOA?Someday (Someday (somedaysomeday) we want to lower the cost and ) we want to lower the cost and

shorten the cycle of application development, shorten the cycle of application development, through:through:

Reusability (wherever reasonable)Reusability (wherever reasonable)

Shared components (wherever reasonable)Shared components (wherever reasonable)

Standards (e.g. XML)Standards (e.g. XML)

A direct relationship between the business A direct relationship between the business process and the resulting applicationprocess and the resulting application

Feedback loops and monitoring to improve Feedback loops and monitoring to improve business and application processes.business and application processes.

SOA claims to help do these things.SOA claims to help do these things.


How will SOA do it?The Grail:The Grail:Business Business ProcessProcess Models Modelsthat that drivedrive Systems Models Systems Modelsthat both that both representrepresent Workflows Workflowsthat that automatically translateautomatically translate to executable code to executable code that uses predefined shared servicesthat uses predefined shared servicesin a scalable and fault tolerant architecture.in a scalable and fault tolerant architecture.

This would ideally be coupled with This would ideally be coupled with the simulation of the business models the simulation of the business models integrated with the monitoring of production systemsintegrated with the monitoring of production systemsto allow validation and performance analysis and to allow validation and performance analysis and explore opportunities for both business and systems explore opportunities for both business and systems

improvement.improvement.


A Model of Model Driven DevelopmentBusiness Model

Developed by Business Analysts and CustomersContaining Workflow, People and Automated Tasks, References to Artifacts

Validated in Walkthroughs and Through Simulation

Implementation ModelDeveloped by System Integrator / Lead Developer

Containing Workflow, Graphical Specifications for Implementation, Validated in Walkthroughs and Through Simulation

Translated by Systems Integrator creating

Code (e.g BPEL, Java web services, other code and SQL)

Automatically Generated from the Implementation Model

Automatically Generating

Operational Integration Integration of code into SOA architecture

Support for Workflow Execution, Service Calls, and Task Management


The Value of this Model Driven Development Model

Business Model

Imp.Model

WorkflowCode (BPEL)

Enterprise Service Bus

Web Service

Web Service

Graphical model of the business that Graphical model of the business that customers can understand, validate customers can understand, validate and see simulateand see simulate

Mappable Constructs from Business Mappable Constructs from Business level to implementation level (e.g. level to implementation level (e.g. artifacts > data structures, artifacts > data structures, business level workflow > system business level workflow > system workflow, etc.)workflow, etc.)

Automatically generated from Automatically generated from Implementation Model, exact Implementation Model, exact representation of actual workflow, representation of actual workflow, directly deployable, easily modified directly deployable, easily modified and validated.and validated.

A standard mechanism for A standard mechanism for integrating workflow and servicesintegrating workflow and services

Reusable Components available to Reusable Components available to other applicationsother applications


How do you get there? In the words of the band, REM:In the words of the band, REM:

““Stand in the place where you liveStand in the place where you liveNow face NorthNow face NorthThink about directionThink about directionWonder why you haven't before.”Wonder why you haven't before.”

More specifically: More specifically: Determine your level of SOA maturityDetermine your level of SOA maturity Start smallStart small Get organizedGet organized Do initiatives that work Do initiatives that work for your maturity levelfor your maturity level Evolve and build on successesEvolve and build on successes Expect to do things twice (or more) before you Expect to do things twice (or more) before you

get thereget there Don’t be afraid to ask for help.Don’t be afraid to ask for help.


SOA Maturity“Stand in the place that you Live”

Coders R UsCoders R Us Pretty PicturesPretty Pictures Analysts R UsAnalysts R Us Totally Business Totally Business

Driven DudesDriven Dudes

Buried in Legacy / Buried in Legacy / Not much OONot much OO

We do OOWe do OO Business / systems Business / systems requirementsrequirements

Model driven Model driven developmentdevelopment

No Requirements No Requirements documents documents

UML diagrams / Do UML diagrams / Do UML and SOA go UML and SOA go together?together?

Traceability of Traceability of requirements to requirements to implementationimplementation

Business Model Business Model drives design / drives design / developmentdevelopment

No standard No standard processesprocesses

Some processesSome processes Wrote and use some Wrote and use some servicesservices

Integrated models / Integrated models / servicesservices

Wrap some legacyWrap some legacy Find and convert Find and convert some code to some code to servicesservices

Start doing business Start doing business process modelingprocess modeling

Define business to Define business to systems to code systems to code translationtranslation

Start business / Start business / requirements requirements analysis analysis

Investigate mapping Investigate mapping business processes business processes to servicesto services

Investigate mapping Investigate mapping business models to business models to systems modelssystems models

Exploit reusability Exploit reusability at all levels.at all levels.

Start understanding Start understanding business process business process modelingmodeling

Investigate Investigate simulation and simulation and monitoringmonitoring


Choosing a Project Depends on your Maturity Level

For Coders R Us and Pretty Pictures – For Coders R Us and Pretty Pictures – Start Start smallsmall

Look for candidate legacy code to wrap or repackage as Look for candidate legacy code to wrap or repackage as services and reuseservices and reuse

Try some new small services that are clearly things you can Try some new small services that are clearly things you can reuse (e.g. login, DB lookup)reuse (e.g. login, DB lookup)

Do a project that can afford to fund the infrastructure Do a project that can afford to fund the infrastructure required for true SOArequired for true SOA

Work on the fringes of a well designed OO application to re-Work on the fringes of a well designed OO application to re-factor useful pieces into services factor useful pieces into services

For Requirements are our Business and For Requirements are our Business and Totally Business Driven Dudes – Totally Business Driven Dudes – Get Focused and AbstractGet Focused and Abstract Focus on business processes and how to map them to Focus on business processes and how to map them to

systemssystems Work on rules for automatically translating models to codeWork on rules for automatically translating models to code Do a project that can afford to fund the infrastructure Do a project that can afford to fund the infrastructure

required for true SOA and that can provide clearly reusable required for true SOA and that can provide clearly reusable service for other future or existing applications.service for other future or existing applications.


Staffing Recommendations

Systems Architecture supportSystems Architecture support Business analysts / requirements Business analysts / requirements

analystsanalysts Systems Integrator(s) / Team LeadSystems Integrator(s) / Team Lead Service DevelopersService Developers Someone(s) with strong Websphere Someone(s) with strong Websphere

production knowledgeproduction knowledge Build PersonBuild Person


What Skills Your Team Needs

Business process modelingBusiness process modeling Systems modelingSystems modeling Service development coding Service development coding

(preferably Java, not too hard to find or (preferably Java, not too hard to find or teach) teach)

Abstraction (hard to find, harder to Abstraction (hard to find, harder to teach)teach)

BPEL knowledge if possibleBPEL knowledge if possible Websphere knowledge or help from Websphere knowledge or help from

outsideoutside


Common Issues STAFFING ISSUES STAFFING ISSUES – The transition to SOA includes a transition in – The transition to SOA includes a transition in

staff roles. New roles may include:staff roles. New roles may include: Business / Requirements AnalystsBusiness / Requirements Analysts Systems IntegratorsSystems Integrators Services DevelopersServices Developers

BUSINESSBUSINESS LIFECYCLE ISSUES LIFECYCLE ISSUES – The greatest value to a business – The greatest value to a business may be in may be in business process re-engineeringbusiness process re-engineering and this should occur prior and this should occur prior to or at least early in the lifecycle, not halfway into the design. Rick’s to or at least early in the lifecycle, not halfway into the design. Rick’s rule #432 – Automating a bad business process just makes it run bad rule #432 – Automating a bad business process just makes it run bad faster.faster.

OLD THINK ISSUESOLD THINK ISSUES – Just because someone can make a piece of – Just because someone can make a piece of C# or Java into a “service” does not make it a useful, well defined or C# or Java into a “service” does not make it a useful, well defined or reusable component for your business. Be very aware of your staff and reusable component for your business. Be very aware of your staff and their interests or you may just get more regular OO code disguised as their interests or you may just get more regular OO code disguised as services, rather than services that are directly tied to business services, rather than services that are directly tied to business process needs and invoked in flexible workflows.process needs and invoked in flexible workflows.

ABSTRACTION ISSUES ABSTRACTION ISSUES – Do not underestimate how important it is – Do not underestimate how important it is to get the right level of abstraction for your models and services. This to get the right level of abstraction for your models and services. This is key to their reusability and finding folks who understand how to do is key to their reusability and finding folks who understand how to do this is hard.this is hard.


How TEA is doing it

Starting with a specific project, with solid Starting with a specific project, with solid funding and a strong needfunding and a strong need

Hiring the best and brightest we can find / Hiring the best and brightest we can find / affordafford(including the best and brightest contractors)(including the best and brightest contractors)

Getting infrastructure in place Getting infrastructure in place in advance ofin advance of the projectthe project

Focus on model driven development and Focus on model driven development and making that making that workwork, instead of making excuses , instead of making excuses why it won’t work.why it won’t work.

Getting help from IBM and othersGetting help from IBM and others


An IBM SOA Technology MapIBM Products IBM Products Likely UsersLikely Users TEA Owns TEA Owns TEA TEA

Uses Uses Someday Someday

May May NeedNeed

W Business W Business ModelerModeler

Analysts / Analysts / DudesDudes

XX XX

W Integration DevW Integration Dev AllAll XX XX

W Rational App DevW Rational App Dev AllAll XX XX

W Process ServerW Process Server AllAll XX XX

W Application W Application ServerServer

AllAll XX XX

W BM PublisherW BM Publisher Analysts / Analysts / DudesDudes

X X XX

W Service Registry W Service Registry & Repository& Repository

Analysts / Analysts / DudesDudes

XX XX

W MonitorW Monitor Post DudesPost Dudes XX

AdaptorsAdaptors Coders R UsCoders R Us XX


Focus on model driven development

If you start with a clear vision/picture, If you start with a clear vision/picture, then you have a better chance of then you have a better chance of realizing it.realizing it.

Caveat: Caveat: You need pictures that each of You need pictures that each of your stakeholder participants your stakeholder participants understand:understand: Business models for CustomersBusiness models for Customers Implementation models for System Integrator Implementation models for System Integrator Operational models or specifications for Operational models or specifications for

ProgrammersProgrammers


Why multiple pictures?

Customer/Business Customer/Business UserUser Sell vouchersSell vouchers Collect vouchersCollect vouchers Store luggageStore luggage Transport up to Transport up to

40 passengers40 passengers Refuel transportRefuel transport

Process OrchestratorProcess Orchestrator 40 passenger bus40 passenger bus Transmission Transmission

system domainsystem domain Seating configSeating config Heat/AC system Heat/AC system

domaindomain Breaking system Breaking system

domaindomain Service DeveloperService Developer

TireTire WheelWheel Disc Break Disc Break

AssemblyAssembly


What sort of pictures will you need?

Business ModelBusiness Model Describes the end-users processes/activities Describes the end-users processes/activities

(User View)(User View) System Implementation ModelSystem Implementation Model

Describes the system needed by user to support Describes the system needed by user to support above process (System Integrator view)above process (System Integrator view)

Operational ModelOperational Model Describes the atomic services that will be Describes the atomic services that will be

developed to implement system (Service developed to implement system (Service Developer View)Developer View)

Business Process Exec. Language (BPEL) Business Process Exec. Language (BPEL) Programming CodeProgramming Code


Business Model

Reflects the overall business processReflects the overall business process Includes all relevant enterprise and external Includes all relevant enterprise and external

processesprocesses Independent of project scope or implementationIndependent of project scope or implementation Business model workflow may be simulatedBusiness model workflow may be simulated Developed in Websphere Business ModelerDeveloped in Websphere Business Modeler


Implementation Model

Includes just the workflow in scope of the systemIncludes just the workflow in scope of the system Use of existing services identified; new services definedUse of existing services identified; new services defined Allocation of tasks to the technology of implementationAllocation of tasks to the technology of implementation Addition of implementation detailsAddition of implementation details Developed in Websphere Business ModelerDeveloped in Websphere Business Modeler


Operational Model

Generated from Generated from Implementation ModelImplementation Model

Workflow, transaction, and Workflow, transaction, and fault tolerance details fault tolerance details addedadded

Developed in Websphere Developed in Websphere Integration DeveloperIntegration Developer

Generated BPEL (Business Generated BPEL (Business Process Execution Process Execution Language) orchestrates Language) orchestrates workflow using Websphere workflow using Websphere Process ServerProcess Server


Things TEA MIGHT HAVE DONE BETTER

Websphere Systems Administration More aggressive search for model

driven development experts (Early Abstraction is KEY)

Picked a project that did not get caught in the middle of mandated legislated changes


Conclusions Business and Requirements analysis are Business and Requirements analysis are

important to truly successful SOAimportant to truly successful SOA

The Grail is model The Grail is model drivendriven development and it development and it is pretty hot stuff. It takes talent and work. is pretty hot stuff. It takes talent and work. Still, you can certainly start the SOA path by Still, you can certainly start the SOA path by just creating and using some reusable servicesjust creating and using some reusable services

It will typically take a few years to realize the It will typically take a few years to realize the bigger gains of SOA, even if you do it right.bigger gains of SOA, even if you do it right.

If you do not keep your team focused on the If you do not keep your team focused on the goal and don’t foster good abstraction skills, goal and don’t foster good abstraction skills, SOA is no more or less useful or valuable than SOA is no more or less useful or valuable than traditional OO or other types of development.traditional OO or other types of development.


QUESTIONS &

CONVERSATIONS


Making the Move to SOA in Texas Government

Rick Goldgar, Ph.D.Rick Goldgar, [email protected]@tea.state.tx.us

Chief Technology Officer and Deputy CIOChief Technology Officer and Deputy CIOTexas Education AgencyTexas Education Agency

David E. ButlerDavid E. [email protected]@tea.state.tx.us

Project ManagerProject ManagerTexas Education AgencyTexas Education Agency

August 12, 2008August 12, 2008

copyright © 2008 texas education agency. making the move to soa in texas government rick goldgar,...

Documents

texas government rick

software cars applications

software developer

efficientsoftware development

software world

statisticsit development

flexible factory

car factory