rules processing and bpm an architectural perspective

© Solidsoft Limited 2005

Copyright Solidsoft 2006

Rules ProcessingRules Processingand BPMand BPM

An Architectural PerspectiveAn Architectural Perspective

Charles YoungCharles YoungPrincipal ConsultantPrincipal Consultant

http://portal/C5/C14/Images0/Solidsoft%20Logos/Forms/DispForm.aspx?ID=7


AgendaAgenda

Rule processingRule processing• What are rules, what do they do and how do you use them in business What are rules, what do they do and how do you use them in business

processes?processes?• Why use rule engines? The architectural driversWhy use rule engines? The architectural drivers

Rule patterns and practicesRule patterns and practices• Rules-driven process governanceRules-driven process governance• Common inter- and intra-process rule patternsCommon inter- and intra-process rule patterns

Rules at the edgeRules at the edge• Event-driven scenariosEvent-driven scenarios• Interaction with the physical worldInteraction with the physical world• What about the user interface?What about the user interface?



RULE PROCESSINGRULE PROCESSING



What are Rules?What are Rules?

““A law or regulation that governs behaviour, actions, or operations.”A law or regulation that governs behaviour, actions, or operations.” AMS GlossaryAMS Glossary

““A clause that defines the relationship or relationships between facts.”A clause that defines the relationship or relationships between facts.” PC AI GlossaryPC AI Glossary

““A data-driven programming construct…consisting of a condition and A data-driven programming construct…consisting of a condition and action. A rule’s action is executed (‘fired’) when its condition is action. A rule’s action is executed (‘fired’) when its condition is satisfied by objects in memory. Sometimes called a ‘production’ or satisfied by objects in memory. Sometimes called a ‘production’ or

‘production rule’.”‘production rule’.” AT&T R++ User ManualAT&T R++ User Manual



What Rules DoWhat Rules Do

Define scope and meaning of termsDefine scope and meaning of terms Define structure of systems and organisationsDefine structure of systems and organisations Constrain actions and bind activitiesConstrain actions and bind activities Derive information and knowledge from dataDerive information and knowledge from data

Rules are everywhere!Rules are everywhere!• Intrinsic to any organisation or processIntrinsic to any organisation or process• Intrinsic to any software system or serviceIntrinsic to any software system or service



Rule CategoriesRule Categories

RuleML categoriesRuleML categories• Integrity constraintsIntegrity constraints

Assertions, SQL foreign key (RI) constraints, etc.Assertions, SQL foreign key (RI) constraints, etc.

• Derivation rulesDerivation rules StaticStatic Condition Condition Conclusion Conclusion

• Reaction rulesReaction rules TemporalTemporal On event, if condition then actionOn event, if condition then action SQL triggers, event handlers, Subscription-based routingSQL triggers, event handlers, Subscription-based routing

• Production rulesProduction rules If condition then actionIf condition then action Widely supported in rules enginesWidely supported in rules engines

• Transformation rulesTransformation rules If condition then transform A into BIf condition then transform A into B

• Deontic rules (future support)Deontic rules (future support) Rights, obligations and empowermentsRights, obligations and empowerments



Implementing RulesImplementing Rules

No single approachNo single approach• Rules are everywhere!Rules are everywhere!• Custom codeCustom code• Message routing and subscriptionMessage routing and subscription• Workflows and processesWorkflows and processes• Database systems Database systems • Rules enginesRules engines• XSLTXSLT• etc.etc.



Rules EnginesRules Engines Executable rulesExecutable rules

• Rule setsRule sets• Policies: govern decision making and interactionPolicies: govern decision making and interaction• Production rulesProduction rules

Map to other rule categoriesMap to other rule categories Sequential enginesSequential engines

• Less expressive, powerful and scalableLess expressive, powerful and scalable• Procedural: familiar, easier to debugProcedural: familiar, easier to debug• Excellent for simple decision logicExcellent for simple decision logic• Microsoft: WF rulesMicrosoft: WF rules

Set-based pattern matching enginesSet-based pattern matching engines• Production systemsProduction systems• More expressive, powerful and scalable (esp. large rule sets)More expressive, powerful and scalable (esp. large rule sets)• Good support for highly combinatorial matchingGood support for highly combinatorial matching• Declarative: unfamiliar, harder to debugDeclarative: unfamiliar, harder to debug• Inferencing and reasoningInferencing and reasoning• Microsoft: MS BREMicrosoft: MS BRE



IF State == POReceivedTHEN PO = GetPurchaseOrder() State == ValidatingCustomer

IF AND State == ProcessingPO OR PO.Accepted == false PO.Total < 100000THEN State = CompletingPOProc

IF AND State == ValidatingCustomer IsApprovedCustomer(PO.CustomerID) NOT OR AccountSuspended(PO.CustomerID) OutstandingArrears(PO.CustomerID, 8000)THEN PO.Accepted = true State = ProcessingPO

IF AND State == ApprovingPO AP.Approved == trueTHEN State = CompletingPOProc

IF AND State == ProcessingPO PO.Accepted == true PO.Total >= 100000THEN SendApprovalRequest(PO.RefId) State = ApprovingPO

IF AND State == CompletingPOProc PO.Accepted == trueTHEN EnterPODetails(PO) State = POProcCompleted

IF AND State == CompletingPOProc PO.Accepted == falseTHEN SendDeclineEmail(PO) State = POProcCompleted

IF AND State == ApprovingPO AP.Approved == falseTHEN PO.Accepted = false State = CompletingPOProc



Why use Rules Engines?Why use Rules Engines?

Better separation of concernsBetter separation of concerns• Separate process definition from governanceSeparate process definition from governance

Improved traceabilityImproved traceability• Align process logic to business goalsAlign process logic to business goals• Validate completeness and correctnessValidate completeness and correctness

Externalisation of rulesExternalisation of rules• Direct definition and maintenance by business usersDirect definition and maintenance by business users• Orthogonal versioning of rules and processesOrthogonal versioning of rules and processes

InferencingInferencing• Sophisticated and efficient framework for reasoning over factsSophisticated and efficient framework for reasoning over facts



MS Business Rules FrameworkMS Business Rules FrameworkApplication, Orchestration, etc.

Rule Policy Objects (executable rule sets)

Rule EngineObject

Executor

Rule Set Translator

TrackingInterceptor Fact Retriever

(Fact Remover)Rule Set

Update Service

Pub/SubAdapter

Pub/Sub Service

Rule StoreObject

Rule Sets &Vocabularies

Fact StoresApplication Data Rule Set

Deployment Driver

Rule LanguageConverter

PolicyComposer

DeploymentTool

Assert Facts, Execute Rules

Assert Facts, Execute Rules

Get FactsGet Rule Set

Fact Creator

Policy Tester

Get Rule SetCache

Versioning

“Long Term”Facts

Pub/Sub Agent

Event Service

“Short Term” Facts

SpecifiedIn Rule Set



RULE PATTERNS & PRACTICESRULE PATTERNS & PRACTICES



Mapping Business Rules to Executable Mapping Business Rules to Executable RulesRules Business rulesBusiness rules

• Business abstractionsBusiness abstractions• Not driven by technologyNot driven by technology• Highly declarativeHighly declarative• No run-time contextNo run-time context

Map at two levelsMap at two levels• Business rule Business rule Executable rule Executable rule• Business rule Business rule Policy (rule-sets) Policy (rule-sets)

Policy ‘chaining’Policy ‘chaining’



Process DefinitionProcess Definition

A process may…A process may…• Decompose to several sub-processesDecompose to several sub-processes

Multiple orchestrations or workflowsMultiple orchestrations or workflows

• Include both automated and manual processingInclude both automated and manual processing• Span multiple platforms and servicesSpan multiple platforms and services

Enterprise BPM servers (e.g. BizTalk Server)Enterprise BPM servers (e.g. BizTalk Server) Human Workflow toolsHuman Workflow tools Web services and browser / desktop applicationsWeb services and browser / desktop applications Line of Business and back office applicationsLine of Business and back office applications

• Cross multiple organisational boundariesCross multiple organisational boundaries



Process GovernanceProcess Governance

Governance involves…Governance involves…• Enforcing Enforcing organisationalorganisational policies within processes policies within processes• Managing the alignment of processes to corporate requirements and Managing the alignment of processes to corporate requirements and

strategiesstrategies• Ensuring traceability, accountability and openness within processesEnsuring traceability, accountability and openness within processes• Ensuring optimal reuse of services across process and Ensuring optimal reuse of services across process and organisationalorganisational

boundariesboundaries Policy-drivenPolicy-driven ExternalisedExternalised ManagedManaged



Technology AlignmentTechnology Alignment

RuleRepositories

BAM & MI

ProcessDefinition Governance

Messaging

Custom Code

WebServices

RulesEnginesOrchestration

Workflow

Procedural Declarative



Intra-Process Rule PatternsIntra-Process Rule Patterns

Used within a single executable processUsed within a single executable process• e.g. within a single orchestratione.g. within a single orchestration

Basic rule-based patterns for BPMBasic rule-based patterns for BPM

Rule-based derivationsRule-based derivations Rule-based assertionsRule-based assertions Rule-driven process flowRule-driven process flow



Rule-Based DerivationsRule-Based Derivations

IntentIntent• Derive and infer new facts from existing factsDerive and infer new facts from existing facts

ApplicabilityApplicability• Data and message generationData and message generation• Data and message transformationData and message transformation

UsesUses• Decision supportDecision support• RatingRating• AnalysisAnalysis• PredictionPrediction• Scheduling and process Scheduling and process optimisationoptimisation



Rule-Based AssertionsRule-Based Assertions IntentIntent

• Enforce validity and integrity constraints within a processEnforce validity and integrity constraints within a process ApplicabilityApplicability

• Data and message validationData and message validation UsesUses

• ValidationValidation• Policy enforcementPolicy enforcement

Enforcement of obligations and permissions (deontic rules)Enforcement of obligations and permissions (deontic rules)

• Alerting and notificationAlerting and notification



Rule-Driven Process FlowRule-Driven Process Flow

IntentIntent• Control sequence of state transitions and process activitiesControl sequence of state transitions and process activities

ApplicabilityApplicability• Process decision pointsProcess decision points• Conditional process logicConditional process logic• Processes with branches and loopsProcesses with branches and loops

UsesUses• Policy-driven processesPolicy-driven processes

Process automationProcess automation Human interactions and workflowHuman interactions and workflow



Intra-Process Patterns: RelationshipsIntra-Process Patterns: Relationships

Rule-BasedDerivations

Rule-BasedAssertions

Rule-DrivenProcess Flow

Estimate anticipated profit margin for new product based on monthly sales of similar products, market conditions, etc.

Estimated profit margin for new product is > 10%

If estimated profit margin is not > 10%, notify product manager

Rules Engine

Rules Engine

Orchestration



An Intra-Process Anti-PatternAn Intra-Process Anti-Pattern

Flow-based derivationFlow-based derivation• Controlling cyclic derivation logic through process flowControlling cyclic derivation logic through process flow

Using process flow inappropriately to implement activity logicUsing process flow inappropriately to implement activity logic

• SymptomsSymptoms Excessively nested loop structuresExcessively nested loop structures

– Iterating data collectionsIterating data collections– Constructing new data collectionsConstructing new data collections

‘‘Meta-loops’ to repeat work when date collections changeMeta-loops’ to repeat work when date collections change– (Forward chaining)(Forward chaining)

• ConsequencesConsequences Inefficient, resource-intensive codeInefficient, resource-intensive code Hard to verify, stabilise and maintainHard to verify, stabilise and maintain

• ResolutionResolution Replace with set-based pattern matching approach using a Rules EngineReplace with set-based pattern matching approach using a Rules Engine Use declarative rules rather than complex procedural codeUse declarative rules rather than complex procedural code



Inter-Process Rule PatternsInter-Process Rule Patterns

Used across multiple executable process definitionsUsed across multiple executable process definitions• e.g. within across multiple orchestrationse.g. within across multiple orchestrations

Advanced rule-driven patterns for BPMAdvanced rule-driven patterns for BPM

Rule-driven process routingRule-driven process routing Policy-driven process versioningPolicy-driven process versioning Policy-driven process compositionPolicy-driven process composition



Rule-Driven Process RoutingRule-Driven Process Routing

IntentIntent• Implement dynamic patterns of process invocationImplement dynamic patterns of process invocation

ApplicabilityApplicability• Message exchange between different processesMessage exchange between different processes

UsesUses• Subscription-based routingSubscription-based routing• Dynamic policy-driven routingDynamic policy-driven routing

Foundational for advanced patternsFoundational for advanced patterns– Fine-grained orthogonal versioning of individual processesFine-grained orthogonal versioning of individual processes– Dynamic composition of discrete processes and servicesDynamic composition of discrete processes and services



Example: Example: BizTalk ServerBizTalk ServerEnhanced SubscriptionEnhanced Subscription

MessageBox

SubscriptionRules

Context

Message

Filter

Orchestration A

Orchestration B

Event-Condition-Action

Set context properties and promote using correlation set

Subscribe on promotedmessage properties

RulesEngine

Policy

Dynamic Routing



Policy-Driven Process VersioningPolicy-Driven Process Versioning

IntentIntent• Fine-grained versioning control of individual processesFine-grained versioning control of individual processes

ApplicabilityApplicability• Discrete sub processes that can be re-used across multiple business Discrete sub processes that can be re-used across multiple business

processesprocesses• Processes that must constantly adapt to rapidly changing business Processes that must constantly adapt to rapidly changing business

environmentsenvironments UsesUses

• Anywhere there is a need to regularly re-deploy versioned processes with Anywhere there is a need to regularly re-deploy versioned processes with maximum agility and minimum impactmaximum agility and minimum impact



Invocation and Worker ProcessesInvocation and Worker Processes

VersionedWorkerProcess



InvocationProcess

RulesEngine

Policy

Correlation

Worker process version[Process version]

Generic invocation process. Should not require versioning. Uses correlation to ‘invoke’ a versioned worked orchestration. The correlation tokens are obtained from the rules engine and policy

The Invocation process may need to provide a version specifier for the business process as a whole. This should flow to the worker process. This may be required when performing long-lived transactions, or in complex scenarios involving many sub-processes.

Process Definition Governance



Policy-Driven Process CompositionPolicy-Driven Process Composition IntentIntent

• Dynamic composition of discrete services and processesDynamic composition of discrete services and processes ApplicabilityApplicability

• Discrete sub processes that can be re-used across multiple business Discrete sub processes that can be re-used across multiple business processesprocesses

• Processes that must constantly adapt to rapidly changing business Processes that must constantly adapt to rapidly changing business environmentsenvironments

UsesUses• Anywhere there is a need to regularly change the composition of overall Anywhere there is a need to regularly change the composition of overall

processes with maximum agility and minimum impactprocesses with maximum agility and minimum impact



Process Controllers and StepsProcess Controllers and Steps

InvocationProcess

WorkerProcess

WorkerProcess

WorkerProcess

WorkerProcess

WorkerProcess

Process Controller

RulesEngine

Messages flow into “top-level” process

Each process follows a common design pattern in which it invokes the process controller that, in turn, uses rules to determine which ‘next step’ worker process, if any, should be invoke.

The Process Controller rules may not specify any ‘next steps’, in which case the current worker process forms the leaf of a process flow branch.

At any point a worker process may interact with other processes or systems outside the control of the Process Controller

The Process Controller is a rule-driven step manager. It executes a versioned policy in order to decide the next steps and then invokes those next steps through correlation techniques

Process Definition Governance

Policy



RULES AT THE EDGERULES AT THE EDGE



Rules in Edge ProcessingRules in Edge Processing

Edge processesEdge processes• ContextualisedContextualised

Customer / service-focusedCustomer / service-focused ‘‘Real world’ interactionsReal world’ interactions

• Event-drivenEvent-driven Non-deterministicNon-deterministic

• TransformationalTransformational Map between events / interactions and managed process definitionsMap between events / interactions and managed process definitions

Filtering and cleansingFiltering and cleansing



Event-Condition-Action RulesEvent-Condition-Action Rules

Processes

Context

EventSource

Transform to

Event FactsRules Engine

Contextual Facts

Policy

Change

ProcessesProcesses

Inferred Facts

Event

Condition

Action

Edge

Centre



Example: RFID ServicesExample: RFID ServicesBusiness Applications

RFID Services

Devices

Physical Layer

DistributionInventoryManufacturing Packaging Warehouse ReceptionQuality Assurance

· Own master data· Drive business process· Apply business logic to data· Provide visibility and analytics· Push down rules and alerts

· Transform events to facts· Add context · Filter and cleanse events· Infer facts · Provide alerts· Discover, configure and manage devices· Provide a hardware abstraction layer

· Fixed and mobile· RFID readers and printers· Bar code scanners· PLCs, etc.

· RFID tags & bar codes· Biometrical systems· Environmental sensors etc.

Rule Processing

Rule Processing



Rules in the Presentation TierRules in the Presentation Tier

““The system architecture … is the user agent”The system architecture … is the user agent” Fred Fred Brooks - The Mythical Man-MonthBrooks - The Mythical Man-Month

Rule-driven interfacesRule-driven interfaces• Validation and derivationValidation and derivation• Composition and navigationComposition and navigation• Analysis, inference and predictionAnalysis, inference and prediction• Guidance and learningGuidance and learning

Shared rule-base with central processesShared rule-base with central processes



SUMMARYSUMMARY



Call to ActionCall to Action

Investigate rule technologiesInvestigate rule technologies• Many vendors, many approachesMany vendors, many approaches• Built-in vs. external tool setsBuilt-in vs. external tool sets

Distinguish between process definition and process governanceDistinguish between process definition and process governance Apply rule patterns and practices to process designApply rule patterns and practices to process design Assess, measure and improve!Assess, measure and improve!


rules processing and bpm an architectural perspective

Documents