fbkr 2012 - montali - conformance verification when dealing with computerized and human-enhanced...

31
Stefano Bragaglia 1 , Federico Chesani 1 , Paola Mello 1 , Marco Montali 2 1 DISI, University of Bologna 2 KRDB, Free University of Bozen - Bolzano Workshop on Foundations of Biomedical Knowledge Representation 01/11/2012 Lorentz Centre, Leiden

Upload: marco-montali

Post on 09-Jan-2017

34 views

Category:

Presentations & Public Speaking


0 download

TRANSCRIPT

Page 1: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

Stefano Bragaglia1, Federico Chesani1, Paola Mello1, Marco Montali2

1DISI, University of Bologna2KRDB, Free University of Bozen-Bolzano

Workshop on Foundations of Biomedical Knowledge Representation01/11/2012 Lorentz Centre, Leiden

Page 2: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

��CGs developed by applying evidence-based

medicine to large classes of abstract patients�Assumptions

� Ideal patients� statistically relevant� with only the disease targeted by the CG

� Ideal physicians� Ideal resources

�∞ resources

Ideal world

Page 3: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

��Context and patients are not ideal

� Resources may be missing� Each patient has her own story, condition, preferencesà Unforeseen situations are common

�CGs routinely adapted on a per-patient basis, using the Basic Medical Knowledge (BMK)

�CGs enacted together with many additional (local) rules and processes

�Physicians are not ideal(maybe, they would need computerized support J )

Real World

Page 4: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Compliance The act of conforming as requested by the CG� Flexibility The ability of accommodating and promptly

adapting to change and unforeseen situations

Compliance vs Flexibility

Universe of Traces

Compliant

Traces

Compliance

Flexibility

Page 5: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

��CGs propose a recommended behavior�Many factors could lead healthcare professionals

in taking a different behavior�We need to sort this discrepancy out!�Goal of conformance checking: detect deviations

between the expected and the actual behavior� I.e., provide to domain experts all useful information

to understand and explain these deviations

Conformance

Page 6: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

��Not to be intended as a normative component�“Global” usage (totality of cases): CGs

understanding and improvement� Improvement of the organization� Improvement of the CG model

�“Local” usage (single patient): decision support� Track the state of affairs

(where is the patient located wrt the CG?)� Report deviations� Run-time and offline perspective

Usages of Conformance

Page 7: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�Conformance Checking

Conformance module

Events

Deviations

CG model

� Nature of deviations depends on when conformance is checked� Run-time à open-time window� Offline à closed-time window

Page 8: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Definitions and terminology, to describe terms and

applicability conditions of the CG� Workflows, characterizing the allowed courses of execution� Rules, to handle particular cases and exceptions, and

declarative fragments � Linguistic labels to explain features, conditions, criteria

� “Low”, “high”, “risky”, …� Temporal constraints (metric, repetitions, …)

� In addition to the ones imposed by workflows

What is a CG Model?

Page 9: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Interplay between CGs and BMK

� Complex interaction: they can defeat each other depending on the specific situation

� “Closed” vs “open” fragments of the CG� Doctors always have the last word!

� Interplay between workflows and rules� Workflows: procedural knowledge� Rules: declarative knowledge

� Humans in the loop� They are not web services!� Missing a deadline for 50 ms is actually a deviation?à “Grades” of conformance

Criticalities

Page 10: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�Conformance: overview

Conformance module

Events

Deviations

CG model

Page 11: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�Conformance: overview

Conformance module

Events

Deviations

CG model

Conformance module

State of affairs

(fluents)

ExpectationsEvents

Deviations

CG model

Eventsemantics Constraints

Page 12: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�Procedural Knowledge

Page 13: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Activities are

connected to an expected lifecycle� Internal states of

activities� Transitions

triggered by atomic events

Intra-Activity Perspective

active

completed

startend

candidate

Page 14: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Correlation of

events to the corresponding lifecycle

� “Next-transition” expectation

� Generation of corresponding “activity state” fluent

Intra-Activity Conformance

active

completed

startend

candidate

Page 15: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�Inter-Activity Perspective11

Table 1. Basic workflow patterns in GLARE, and their corresponding enabling con-ditions

Pattern Representation Enabling conditions

SequenceA B

When A is completed, B becomes candidate

Exclusive choiceA

B

C

cond

else

When A is completed and cond holds, B becomescandidateWhen A is completed and cond does not hold, Cbecomes candidate

Simple merge B

C

D

When B is completed, D becomes candidateWhen C is completed, D becomes candidate

Parallel split

A B

C When A is completed, B and C become candidate

Synchronization

DB

CWhen B and C are completed, D becomes candi-date

– Properly handle the computation of candidate activities, applying the control-flow patterns semantics to the current state of a�airs (which includes infor-mation about the currently completed activities).

– Impose and verify the negative expectation about non-candidate activities:only candidate activities can be activated by means of a start event.

– Ensure the proper termination of the CG execution when the trace of eventsis finished; the proper termination is in turn formalized by the followingexpectations:• every active activity instance is expected to be completed before the

termination;• when the execution terminates, no activity can be candidate for execu-

tion.

We observe that, considering the conformance characterization provided sofar, the CG procedural knowledge gives raise to a “closed” notion of confor-mance, where every event that is not explicitly expected is considered as forbid-den, and no unforeseen activity can be executed. To partially open the workflowspecification, more sophisticated forms of activity lifecycle can be introduced.For example, Figure 2 (right) presents an improved version of the basic lifecycle.The new version contains an additional state and transition, to explicitly accountfor exceptional situations that require the prompt interruption of the runningactivity instance. This exceptional transition is associated to a failure eventthat leads the instance to an aborted terminal state. In this way, it is possibleto “cancel” the execution of an activity instance under critical and exceptional

Page 16: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Generation of “candidate” activity instances

� Todo list� Progression principle

� Every candidate activity is expected to be started� Enforcement of “closed” procedural knowledge

� Every non-candidate activity is expected not to be started

� What about exceptions? (see next slide)� Closed time-window: every executed activity must

be completed before the end of the trace

Inter-Activity Conformance

Page 17: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�Semi-openness

active

completed

aborted

startend

failurecandidate

� Failure situations allow to skip activities

� Exceptional flows can be managed with rules/workflows� By “enabling”

additional activities� By default:

robustness principle

Page 18: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

Formalize the refined model towards conformance checking

Refine CGs (GLARE) to accommodate BMK

Understand how CGs are interpreted by healthcare professionals

Collecting real examples about BMK+CGs

Research agenda[with Terenziani’s group]

Page 19: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Both BMK and CG may involve declarative and

procedural knowledge� Procedural knowledge fixes the sequencing of

actions to be done� Declarative knowledge captures constraints and

properties to be satisfied, without saying “how”

CG+BMK: Example

CG in GLARE [Terenziani et al.] BMK

Threats to patient’s life must be addressed immediately.An hearth failure is a life threat.Diuretic therapy is a possible immediate response for acute heart failure.

Electrocardiographicstudy

Echocardiographicstudy

Coronaryangiorgraphy

Page 20: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� The interplay between the two kinds of knowledge

occurs at execution time � Brainstorming with physicians led to a specialized

activity life cycle� Capturing the semantics of “executing activities” from

the viewpoint of domain experts� Pointing out where BMK-driven decision making

comes into play� Showing that data are as much as important as the

process

Binding CG with BMK

Page 21: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� BMK

� Eligibility checks (preconditions)

� Abnormality checks to identify exceptional cases� Before the

activity execution � During the

activity execution

Revised Lifecycle

ready

candidate

active

completed

discarded aborted

preconditions∧ ¬abnormal

else

start

end

failure∨ abnormal

Page 22: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�Conformance with CG+BMK

� Ready and candidate states collapsed� Expected life cycle à triggered by logical conditions� Real life cycle à triggered by event occurrences� Conformance: detect and show deviations

expected real

candidate

active

completed

discarded aborted

start end

failure∨ abort

abort

ready

candidate

active

completed

discarded aborted

preconditions∧ ¬abnormal

else

start

end

failure∨ abnormal

Page 23: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Proposed in 1986 by Kowalsky and Sergot� Events� Fluents, i.e. properties whose truth value can change

along time� Domain axioms: link the happening of events with

the change of truth value of fluents

Representing the current state: Event Calculus

Page 24: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�The Simple EC Ontology

1 2 3 4 5 6 7 8 9 10 11 12 13 14

initiates(a,f,3) terminates(b,f,12)

happens(a,3)

holds_at(f,7)

declip clip

0

f f holds in (3,12]

a b

Page 25: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�An example…

17

Fig. 4. EC-based conformance evaluation of a CG execution.

timestamps, we use natural numbers (N0) to represent time values. The mappingof a real timestamp to a corresponding number depends on the chosen time gran-ularity (such as ms or day). E.g., by choosing ms as the time granularity, eachtimestamp t could be mapped to the number of milliseconds between 1/1/197000:00 and t.

The combination of the domain knowledge and a concrete execution traceleads to infer “what is true when”, i.e., the intervals during which fluents hold.The holds at(f , t) predicate of the EC ontology is specifically used to test whetherf holds at time t.

Continuation of this section: formalization of the concepts described in Section3 using the Event Calculus. The formalization is ongoing and will be included in thefinal version, discussing at the workshop to which level of details. Figure 4 shows ascreenshot of conformance checking using our EC reasoner.

5 Generating and Matching Expectations with ECERules

In several previous works we have explored the notion of expectations, and wehave defined several di�erent languages for defining rules that support the defi-nition of expectation. Here we will refer only to two recent works, that providea good starting point for presenting our approach.

In [4] we have introduced the Event-Condition-Expectations (ECE-) rules.Based on the rule framework Drools6, ECE-Rules allow to link current systemstatus (properties, and also Event Calculus fluents) and the dynamic happeningof events to the generation of expectations.

6 http://www.jboss.org/drools/

• Reification of deviations as special fluents• Expectations not explicitly represented

Page 26: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Events� Something happened (what)� At a time point (when)

� Fluents� Properties/status of the system� Affected by events

� Expectations� About events� About fluents

� Achievement properties (existentially quantified)� Maintenance properties (universally quantified)

� Only positive vs. positive/negative expectations

Declarative Conformance:few concepts

Page 27: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Matching function: return a score if an observed event

matches any (positive/negative) expectation� Should support different semantics

� Ontologies� Fuzzy concepts� Temporal reasoning

� Fulfillment� an event matching a positive expectation has happened� No event matching negative expectation has happened� Achievement/maintenance properties are treated almost

similarly…

Events, fluents, expectations and…

Page 28: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Violation

� an event matching a positive expectation did not happen

� An event matching negative expectation has happened� Fluents: strong negation vs. weak negation, in case of

maintenance properties

Events, fluents, expectations and…

Page 29: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�� Work in progress!!!� Based on Drools/Java and Drools Chance

CG representation and expectations: ECE rules

18

rule "Risk factor evaluation"when

$pat : Patient( ... ) // patient identifier

// evaluation of risk factor and confidence degree

$risk : EvaluatedRisk( $phys , $pat , $disease , $factor , $conf )$factor == "high"$conf >= "medium"

thenexpect InitiateTreatment( $pat , $disease , this after[0,1hour] $risk )

on fulfillment { // if the treatment is initiated

/* some increase in patient health */

}on violation { // if the treatment is not initiated

alert( ... );}

end

Fig. 5. An example of ECE-Rule [4].

An example of a rule is shown in Figure 5. When a patient $pat is evaluatedto be at risk of a disease $disease, with a factor judged as to be “high” and witha confidence equal or greater to the “medium” grade, then it is expected that aproper treatment is initiated within one hour from the evaluation.

This simple rule already shows the power of the ECE-Rules: this rule in par-ticular triggers when the evaluation of the disease risk is inserted as a event andconditions are met. As a consequence, dynamically an expectation is generated.The expectation then can be satisfied by an event representing the start of propertreatment. Note that the formalism allows to defines also proper actions in caseof satisfaction of the expectations (rewards) and in case of violations (possiblyexpected countermeasures).

In [5] instead we focuses on the possibility of extending Drools rules withfuzzy ontologies: in particular, the work present a first attempt to extend thenative matching mechanism supported by Drools with reasoning capabilitiesderived by fuzzy ontologies.

rule "Fuzzy evaluation of conformance"when

Order ($e: expectedInDays )DeliveryLog(

$d: delay ~InTime $e, @imperfect(kind=="userOp")$p: packaging nec ~isA "GoodPackaging")

thenprintln("Degree of Delivery Conformance: " +

Drools.degree);end

Fig. 6. A rule that checks the conformance of a delivery with respect to a fuzzy ontol-ogy. Taken from [5].

Page 30: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

�ECE rules…

18

rule "Risk factor evaluation"when

$pat : Patient( ... ) // patient identifier

// evaluation of risk factor and confidence degree

$risk : EvaluatedRisk( $phys , $pat , $disease , $factor , $conf )$factor == "high"$conf >= "medium"

thenexpect InitiateTreatment( $pat , $disease , this after[0,1hour] $risk )

on fulfillment { // if the treatment is initiated

/* some increase in patient health */

}on violation { // if the treatment is not initiated

alert( ... );}

end

Fig. 5. An example of ECE-Rule [4].

An example of a rule is shown in Figure 5. When a patient $pat is evaluatedto be at risk of a disease $disease, with a factor judged as to be “high” and witha confidence equal or greater to the “medium” grade, then it is expected that aproper treatment is initiated within one hour from the evaluation.

This simple rule already shows the power of the ECE-Rules: this rule in par-ticular triggers when the evaluation of the disease risk is inserted as a event andconditions are met. As a consequence, dynamically an expectation is generated.The expectation then can be satisfied by an event representing the start of propertreatment. Note that the formalism allows to defines also proper actions in caseof satisfaction of the expectations (rewards) and in case of violations (possiblyexpected countermeasures).

In [5] instead we focuses on the possibility of extending Drools rules withfuzzy ontologies: in particular, the work present a first attempt to extend thenative matching mechanism supported by Drools with reasoning capabilitiesderived by fuzzy ontologies.

rule "Fuzzy evaluation of conformance"when

Order ($e: expectedInDays )DeliveryLog(

$d: delay ~InTime $e, @imperfect(kind=="userOp")$p: packaging nec ~isA "GoodPackaging")

thenprintln("Degree of Delivery Conformance: " +

Drools.degree);end

Fig. 6. A rule that checks the conformance of a delivery with respect to a fuzzy ontol-ogy. Taken from [5].

Page 31: FBKR 2012 - Montali - Conformance Verification when Dealing with Computerized and Human-Enhanced Processes

candidate

now

Answer questions