rethinking bpm in a cognitive world: transforming how we ... · © 2016 ibm corporation cognitive...

1Copyright © IBM 2016

Rethinking BPM in a Cognitive World:Transforming How We Learn and Perform Business Processes

Richard Hull Hamid R. Motahari NezhadIBM Research IBM ResearchYorktown Heights Almaden

Hamid R. Motahari Nezhad, Rama Akkiraju:

Towards Cognitive BPM as the Next Generation BPM

Platform for Analytics-Driven Business Processes.

Business Process Management Workshops 2014: 158-164

© 2016 IBM Corporation

COGNITIVE

The Future of Computing is …..

2

… and Cognitive Computing will dramatically

impact BPM in the coming decade


Cognitive is emerging as a new computing paradigm

Tabulating Systems Era

Programmable Systems Era

CognitiveSystems Era


Towards Computing-At-Scale as the Shared Characteristic of Recent Advances

4

Scalable Computing over

Massive Commodity Hardware

Building Stronger

Super Computers

Cloud Computing

Crowd Computing

Advanced individual

algorithms

Mass computing applied to AI Complex array of algorithms applied

to make sense of data, and offer

cognitive assistance

Big

Data

Individual

ML Algorithm

Cognitive Computing


Cognitive Era

5

Discovery & Recommendation

Probabilistic

Big Data

Natural Language as the Interface

Intelligent Options


Understands

Conversational

Interface

Adapts and learns

Generates and

Synthesizes

learning techniques

Cognitive System

1

2

3 Cognitive Systems

actively discover, learn and act

A Cognitive System offers computational capabilities typically based on

• Natural Language Processing (NLP),

• Machine Learning (ML), and

• Reasoning chains,

on large amount of data, which provides cognition capabilities that augment and scale human expertise

Watson


Agenda

Emergence of the Cognitive Computing Era

Cognitive and BPM: Introduction

Cognitive Learning Processes

From Process Learning to Executables

Cognitive Enablement of Processes

Challenges, Open questions, Implications


Kinds of Business Processes

Transaction-Intensive Processes

Judgment-Intensive Processes

Decision, Design & Strategy

Processes

Business ProcessHierarchy

Use Cases

Many “ancillary” processes

are performed in ad hoc ways,

spreadsheets, etc.

Case

Mgmt

Rules

Intensive

Knowledge-

intensive

Processes (KiP)

???

• Sales of ComplexIT Services

• Project mgmt.• E.g., Complex client

on-boarding• Commercial Financial

Services (e.g., Loans)

• (Mgmt of) Back-office processing, e.g., • Order-to-Cash

reconciliation• Payroll• . . .

• Enterprise Optimization• New Business Model• New Markets, Geo’s

• Merger/Acquisition• Build vs. Buy

Relevant ModelingApproaches

Challenges

(process-

centric)

BPM

Many “Judgement-Intensive”

processes are fairly simple,

but too expensive to automate

Due to rich flexibility needed,

KiP’s not supported

systematically


Source of these challenges -- “Dark Data”:Digital footprint of people, systems, apps and IoT devices

Handling and managing work (processes) involves interaction among employees, systems and devices

Interactions are happening over email, chat, messaging apps, …

Many text-based descriptions of:

Processes, procedures, policies, laws, rules, regulations, plans, external entities such as customers, partners, government agencies, surrounding world, news, social networks, etc.

Citizens

Assistant

Business

Employees/

agents

Plans

Rules

Policies

Regulations

TemplatesInstructions/

Procedures

ApplicationsSchedules

Communications such as

email, chat, social media,

etc.

Organization

Dark Data: Unstructured Linked InformationIoT Devices and Sensors


A conceptual framework for Cognitive BPM

Unstructured Data, IoT, Smart Devices, Sensors, etc.

Cognitive

Decision

Support

for

Processes

Cognitive

Interaction

with

Processes

Cognitive

Process

Learning

Cognitive

Process

Enablement

Human-Human, and Human-Machine Collaboration

Traditional BPM

and Case Management Abstractions

Cognitive Process

Abstractions


IdentifySales

Owner

ProposeOfferings

Cognitive Decision Support for Processes

Cognitive Decision support is about the ability to digest a myriad of unstructured

information to assist in decision making at a given step in the process

Sales triage in a large corporation with broad array of offerings

Financial and

Technology related

Public data

Similar clients’ past

performance

Social media

data

Sellers’

conversations

Offer Owner

historic win

rates

Seller notes

on client

Seller rankings

according to

qualifications and

performance


Cognitive Interaction with Processes

Today’s Interaction Paradigms

• UI/Screens

• Click

• Multiple apps

Cognitive BPM World

• Conversational

• Talk / Notification

• Integrated agent interface

Additional aspects:• Ability to guide a person through a process

• Ability to construct conversations for a process interaction dynamically

• Using multi-modal interaction models to complement user interface

• Ability to communicate cognitive/analytics results, in a human consumable manner

• Ability to explain learned process optimizations


Agenda








Learning Process: A Myriad of Data Sources

Representative research work

Process Mining literature, e.g.,

[Process Mining Manifesto, 2011]

[Witschel, Nguyen, Hinkelmann, 2012]

[Friedrich, Mendling, Puhlmann 2011]

[Ehrig, Koschmider, Oberweis 2012]

[Aa, Leopold, Reijers 2015]

MailOfMine [Di Ciccio et al, 2011, 2012]

e-Mail Mining [Soares, Santoro, Baiao 2013]

eAssistant [Motahari et. al., in submission]

IT ticket mining

Digital Exhaust, e.g.,• Emails• Chat transcripts• Call-center

transcripts

Structured data• Event logs• System logs• Enterprise meta-data• …

Purpose-built documents• Process Descriptions• Training Manuals• Corp. Policies• Govt. Regulations


Applied on Email, and Activities

eAssistant: Extracting process steps from emails [Motahari Nezhad et al, in submission]

Tool to extract process steps from emails and proactively advise knowledge workers

Extracts actionable statements, commitments, temporal aspects from emails

Includes user feedback, reinforcement learning, and customization to org’s and people


eAssistant Architecture: Machine Learning over SystemT and Watson APIs

SystemT Librabies and Feature Extractors

POS tagging

Conversation and Calendar Sources

Messaging Data

Repositories

Document

collections

Dependency

Analysis

Actionable

Verb Learning

Action Pattern

Learning

Co-reference

resolution

Named Entity

Recognition

Process

Fragment

Discovery

Adaptive and

Personalized

Learning

Action Type Identification

and Template-based

Metadata Extraction

SystemT’s

Action API

(“Watson parser”)

Actionable Insight Components

eAssistant APIs

Conversation

Analytics


Learning of Actionable Statements Identification Bootstrapping

Start with rules-and annotation-based training data generation

Learn from user feedback

Validation

>6K emails from Enron data set

>16K candidate actionable mentions

Hand-annotated for ground truth

After training,Precision: 87%Recall: 83%

Note: User feedback enables continued improvements

Model Learning – Training Phase: Classified Statements

Action Verb

Learning

Adaptive & Online Pattern

Learning

Actionable Statement

Identification Results + Type

Pattern-Based

Action

Prediction

is action verb

yes no

No

Pattern

ConstructionFiltering

Feedback

Prediction: Statement

Action

verbs

Ontology

Feature ExtractionPOS, NLP Tags, Verbs And Dependency Extraction

Feature ExtractionPOS, NLP Tags, Verbs And Dependency Extraction

Model <Adaptive Patterns>

verbs All features All features

is verb tagged

as actionable?

yes no

enclosed verb

yes no

verb can be

independent

verb is

dependent

Verb Independent ?

send true

prepare true

like false

have false

- - - - - -


Agenda





Human Resources example

Commercial Financial Processing example




A key value from automatic learning of process

Today, people focus their attention & investments on high-volume processes

But in many industries, half or more of the expense is on a large number of low volume processes.

Cognitive Computing can bring the ability to automate, optimize & transform “long tail” processes

Long Tail of

Low-Volume Processes

Volu

me

… …

Automating the long tail

of Business Processes

Low volume, but

potentially high cost


Human Resources Example:Based on a Business Process Outsourcing provider

WHY ARE THESE MANUAL ??

Long tail

Many such processes

Many variations by BPO client

Many variations by country

Each individual process is low volume

Employee & HR

Data Entry

•ERP system (e.g., SAP, PeopleSoft) Authoritative or “Golden” copy of employee and payroll data

•Govt. Systems •Payment

Systems

“Ancillary Processes”• New hire• Termination• Garnishment• …

Managed on spreadsheets

Described using MS Word


Extending eAssistant to read Process Descriptions(Preliminary results)

eAssistant+

Process Fragments

Conditions (with Actions)

Purpose-built document about processing of terminations (across multiple countries)


(Exploratory)Human-in-the-Loop

Enable users to examine, refine outputs at all stages

Mapping from process descriptions to executables:

DiscoverProcess

Fragments

ExtractBusiness Entities

AdaptiveBPM Engine

Entities,

Value phrases,

…

ProcessKnowledge

Graph Builder

TaskFlows

ReasoningLogic

ExecutableProcessBuilder

Actions, Rules,

Conditions, …

Chunks

DomainModelSpec.

KG RulesUnstructured

Input

DocumentProcessing

Mappings: Document ref’s Domain Models Database ref’s

Representative Pipeline


(Exploratory)Human-in-the-Loop

Enable users to examine, refine outputs at all stages

Mapping from process descriptions to executables:

DiscoverProcess

Fragments


AdaptiveBPM Engine

Entities,

Value phrases,

…

ProcessKnowledge

Graph Builder

TaskFlows

ReasoningLogic


Actions, Rules,

Conditions, …

Chunks

DomainModelSpec.


Input

DocumentProcessing


Extract Process constructs & fragments• Conditions• Actionable statements• Sequences• Scoping• . . .

Construct all-inclusive knowledge graph• Parse-tree for process

fragments

Targeted Domain Model• Employee attributes,

as occurring in different data sources

• Kinds of updates

Human-consumable abstract representation of executables, • E.g., handful of templates

corresponding to• Conditions• Actions• Conditional actions

Targeted Processing Model• E.g., for HR processing the

focus is on individual employees, and • Validate input data values• Series of updates if valid• Manual treatment of

exceptions

Mapping construction algorithm (self-tuning)

Research/Engineering ChallengesRepresentative Pipeline


Commercial Financial Processing (e.g., loans, insurance, …)

Submission Analysis

Gathermore Data

Final Analysis & Pricing

In the long tail, typically manual and multi-day

Processing logic:• ~70% in manuals,

guidelines, …

• ~30% from experience

Learning the logic enables:• Substantial speed-up

• Improved consistency

• Improved customer satisfaction

• Opportunity for

continuous improvement


Typical insurance rules (commercial sector)

Learned logic

forms hybrid

Decision Tree

http://calmutual.com/ComlUW%20Manual1212.pdf


Human-in-the-LoopEnable users to examine, refine outputs at all stages

Mapping from process descriptions to executables:In the Commercial Financial Modeling case

DiscoverProcess

Fragments & Rules


AdaptiveBPM Engine

Entities,

Value phrases,

…

ProcessKnowledge

Graph Builder

TaskFlows

ReasoningLogic


Actions, Rules,

Conditions, …

Chunks

DomainModelSpec.


Input

DocumentProcessing


Most business logic in form of• If-then rules• Exclusions

Specialized “meta-rules”, e.g., about treatment of exclusions

Domain Ontologies available for Financial Industries

Processing model centered around small process & large rule set

(Exploratory)Opportunity for Continuous Improvement


Agenda








Typical sales engagement process for complex IT services deals

Multiple stakeholders in a “spiral model” to build RFP response with pricing

Many competing requirements; multi-objective optimization

Requirements & prioritizations evolve as new information received from client

Many text-based artifacts throughout the process

Multiple threads of activity, with new threads emerging throughout the process

Traditional BPM approaches, and even Case Mgmt, not flexible enough

Key process logic & best practices are hidden – challenging to apply statistical analytics approaches

RFP

Receipt

Requirements

ingested into

tool (Week 2)

Solutioning, Costing/Pricing,

Executive Reviews and

Approval Milestones

Negotiation, Refinements

On-boarding

RFPDeal Pursuit, Discovery,

Due DiligenceContract . . .

RFP

Response

Deadline

Final

Proposal

Customer Review,

Modifications


Cognitively-Enabled Processes: Shifting process lifecyclefrom Define-Execute-Analyze-Improve to Plan-Act-Learn

For each enactment of the overall process, many iterations around this loop

At a given time, multiple goals & sub-goals may be active

Numerous threads of activity

Each thread modeled essentially as a “case” as in Case Mgmt

Cf. [Vaculin et al, 2013]

As new information arrives the cycle might re-start for some or all threads

Planning based on new info

• New goal formulation

• Planning to achieve those goals

Act on next steps of plan

Optionally perform Learning steps

Cf. KiF’s [Di Ciccio, Marrella, Russo 2015]

Also [Bucchiarone et al 2013], [Marrella, Mecella, Sardina 2014]

“Cognitive Agent” helps by

Perform the planning

Learn from large volumes of structured/unstructured data

Over time, learn best practices and incorporate into planning

Plan / Decide

Act<<World Effect>>

Learn


Key Abstractions for Cognitively Enabled Processes Knowledge, including constraints Knowledge at scale, including from unstructured data, is the fundamentally new element that

Cognitive Computing brings to BPM.

Goals/Subgoals Initial top-level goals may be specified in advance, and additional goals and sub-goals can be

formulated dynamically

Agents (Human & Machine) These agents will have varying intentions, roles, and specialties

Communication between agents may be captured and included into knowledge base

Decisions Based on information & knowledge acquires so far

The decision may lead to new (sub)goals and plans

Actions Atomic unit of work performed by an Agent; including ingestion/analysis of large data sets

Plans These may be partial, and may be revised as the process progresses

May be created and/or re-formulated frequently

Events These may arise from completed actions, new information acquisition, pro-active agents


Planning research in BPM: Selected examples

Goal-driven Business Process Derivation [Ghose et al 2011]

Goals expressed as Boolean combinations of propositional variables

Tasks contribute towards achieving (sub-)goals

Single plan made (no iterations)

“SAP speaks PDDL” [Hoffmann, Weber, Kraft 2012]

Addresses challenge of how to map business objectives into PDDL framework

Focus is on SAP biz-level model for “Status & Application Mgmt (SAM)”

Key observation: SAM describes Business Objects that naturally map to cross-products of FSM’s

On-the-Fly Adaptation of Dynamic Service-Based Systems [Bucchiarone et al 2013]

Context is “service-based systems” – a continuously evolving “context”

Iterative re-planning as new events/data arrive

Interleaving of planning and solution execution

SmartPM [Marrella, Mecella, Sardina 2014]

Goals expressed using first-order logic predicates

Plans are formulated to achieve a next goal

Exceptional situations may occur, in which further plans attempt to remedy

External stimuli also provide new information and/or goals


Plan and Act Cycle in SmartPM: an illustrative example Situation Calculus [Reiter 2001]

“Situations”: First-order logic term describing a state in terms of sequence of actions already performed

“Actions”: typically performed by services

“do(a,s)”: situation after action a done on situation s

“Fluents”: predicates describing state of the world in different situations

IndiGolog [De Giacomo et al 2009]

A logic-based programming language

Familiar constructs: conditionals, sequencing, loops, …

Includes a “lookahead search” operator (), which will find a plan that achieves

PDDL Planner [Edelkamp, Hoffmann 2004]

One of many planning systems

Given an initial state I and goal G, build a plan that will move you from I to a state that satisfies G

In practice, the outcome of the plan may have exceptional conditions, and not reach G after all

s22 = do( Move(John, loc166), s21)

Actions on Situations Fluents

s23 = do( Move(Robot1, loc166), s22)

s21 = do( TakePhoto(John, loc150), s20)Avail_Photos(loc120, s21)

Location(John, loc166, s22)

Location(Robot, loc277, s23)

Program fragments (simplified)

Framework & Axioms

Processing cycle (responses to new info)

If not Location(Robot1, loc166, s)

Then ( Location(Robot1, loc166) )

EndIf

• Constructs for services, service calls, locations, etc.

• Axioms for when can service be invoked, when released, …

• Top priority: exceptional condition Create/execute new plan

• Second priority: Continue to achieve specified goals

Create/execute new plan

• Third priority: Respond to new incoming event (service completion

or from external cause)

If Location(Robot1, loc164)

Then Move(Robot1, loc163);

Move(Robot1, loc164)

EndIf



Mapping from process descriptions and artifacts into SmartPM:Illustration using IT Services Sales

DiscoverNew Goals,

Indicators ofProgress, …

ExtractEntities

SmartPMEngine

Entities, Roles,

Constraints, …

Domain ModelKnowledge

Graph Builder

FluentDefinitions

Actions/ Services:

Pre-cond’s,Post-cond’s

Logical Expressions

Builder

Goals, Change-of-

state, Priorities

Chunks

KG

Logic

SpecUnstructuredInput

(including on-the-fly)

DocumentProcessing


Generic target framework provided by SmartPM

. . .

Domain StatusKnowledge

Graph BuilderFluent

UpdatesKG

update

As before, many opportunities to narrow the scope & fill out framework

Starting point: RFPCog tool [Motahari et al 2016] available to extract “goals” from Requirements docs

Key sub-domains include• IT Services• Pricing• Sales Mgmt• . . .

(Hypothesis)

Again,opportunity for continuous improvement


Agenda








Cognitive will enable the next transformation in BPM

Focus on data relaxes rigidity of flows

Rules naturally refer to the data in cases

Activity-centric process models(BPMN)

Data-centric process models(CMMN)

Knowledge-intensive process models

Cognitively-enabled process models

Paradigm fits with the most tangible aspects of business operations

Focus on knowledge gives even richer flexibility

Analytics on unstructured data enables continuous monitoring & optimization

Tasks &

Sequencing

Data &

Decisions

Goals &

Plans


Disruptions in Computer Science:From Web Services to Cognitive Assistants

From Waterfall Software Development to Agile Programming

From Classical BPM to Cognitive BPM

To

Cognitive BPM

Analyze

Monitor

Act

PlanNext Steps,

Adapt

Side-effect,

Interact

Probe,

Sense

Learn,

Discover

CognitiveBPM

Conventional BPM Lifecycle

ExecuteMonitor

Optimize

Define

Model


Opportunities for several kinds of researchon top of generic framework process learning framework

DiscoverProcess

Fragments


AdaptiveBPM or

Planning Engine

Entities,

Value phrases,

…

ProcessKnowledge

Graph Builder


Actions, Rules,

Conditions, …

ChunksKG Logic

Unstructured

Input

DocumentProcessing


Info Extraction & Machine Learning

Planning

Ontologies


User Experience

Conceptual Modeling


Cognitive BPM: Selected research challenges

Cognitive process learning:

Knowledge acquisition methods from unstructured information (text, image, etc.)

Combine with traditional process mining on logs

Building actionable knowledge graphs & executable code

Cognitively enabled processes: Plan-Act-Learn

Blending of “model” and “instance”

Recognizing goals from digital exhaust and process history

Advances in planning research – incremental, multi-threaded activity, richer goal languages, prioritized and soft goals, …

Enough uniformity to support reporting, identification of best practices

Cognitive Assistants for business processes

Assist workers across numerous tasks, including process management & optimization

Interactive learning where cognitive agents ask process questions

Gradual learning through experience, and process improvement

rethinking bpm in a cognitive world: transforming how we ... · © 2016 ibm corporation cognitive...

Documents