phd pre-defense september 2015

1/43www.janclaes.info

INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION

Jan ClaesSupervisors UGent : Geert Poels & Frederik Gailly

Supervisors TU/e : Paul Grefen & Irene Vanderfeesten

Investigating the process of process modeling and its relation to modeling quality

The Role of Structure Serialization

Structure of the PhD

CHAPTER 5CONCLUSION

CHAPTER 4THEORIZATION

CHAPTER 3EXPLORATION

CHAPTER 2VISUALIZATION

CHAPTER 1INTRODUCTION

CHAPTER 1 – INTRODUCTIONResearch objectives

Context

Business Process Management

Conceptual ModelingPhD

Context

Article availableOrder received

Check availability

late deliveryundeliverable

Payment received

Procurement

Remove articleFrom catalogue

Inform customer

Financial settlement

Ship article

Customer informed

Article removed

Inform customer

(From BPMN Quick Guide, OMG, 2015)

Definitions Definition 1: Business process“A business process consists of a set of activities that are performed in coordination in an organizational and technical environment. These activities jointly realize a business goal.” (Weske, 2007, p. 5)

Definition 2: Business process model“A business process model is a mostly graphical representation that documents the different steps that are or that have to be performed in the execution of a particular business process under study, together with their execution constraints such as the allowed sequence or the potential responsible actors for these steps.”

Definition 3: Process of process modeling“the sequence of steps a modeler performs in order to translate his mental image of the process into a formal, explicit and mostly graphical process specification: the process model.”

Study 1: visualization

Research Objective 1Build knowledge about

how people create models

Overall objectiveCuriosity-driven

Build knowledge about PPM

relation with quality

structured modeling

Study 3: theorizationStudy 2: exploration

Research Objectives

CHAPTER 2 – VISUALIZATIONPPMChart

Data collection

Pnina SofferMatthias Weidlich

Barbara WeberJakob Pinggera

Stefan ZugalJan Mendling

Hajo ReijersIrene Vanderfeesten

Dirk Fahland

Observational data

Cheetah Experimental

Platform

‘Experiment’ design

CREATE_ACTIVITYCREATE_START_EVENTCREATE_END_EVENTCREATE_ANDCREATE_XORCREATE_EDGE

Data collection

MOVE_ACTIVITYMOVE_START_EVENTMOVE_END_EVENTMOVE_ANDMOVE_XOR

DELETE_ACTIVITYDELETE_START_EVENTDELETE-END_EVENTDELETE_ANDDELETE_XORDELETE_EDGE

NAME_ACTIVITYRENAME_ACTIVITYNAME_EDGERENAME_EDGE

Study 1 – Visualization

(From Pinggera et al., 2014)

Not enough detail

Study 1 – VisualizationPoor visual expressivenessPoor perceptual discriminabilityPoor graphic economyPoor semiotic clarityPoor semantic transparencyPoor complexity managementPoor cognitive integration

Study 1 – VisualizationPPMChart

CREATE_ACTIVITY CREATE_START_EVENT CREATE_END_EVENT CREATE_AND CREATE_XOR CREATE_EDGE MOVE_ACTIVITY MOVE_START_EVENT MOVE_END_EVENT MOVE_AND MOVE_XOR DELETE_ACTIVITYDELETE_START_EVENT DELETE-END_EVENT DELETE_AND DELETE_XOR DELETE_EDGE NAME_ACTIVITY RENAME_ACTIVITY NAME_EDGE RENAME_EDGE

Start event Edge Activity

Gateway

Activity

Edge Gateway

FastmodelingSlow

modelingInitialdelayMany

pauzesFewelementsMany

elements No(separate)lay-outing

Quicklay-outingDedicated

lay-outingphase

Continuouslay-outingUnpaired

eventcreation

Pairedevent

creationNo pauzes

SerializationPaired

gatewaycreation

Delayededge

creationChunked

modeling

Study 1 – Visualization

Based on dataset of 357 unique modeling executions

Study 1 – VisualizationEvaluation

Sample of intended users (6 academic researchers) Five extreme examples in PPMChart or Dotted Chart Observe and measure amount, quality, and timing of

insights gained through the visualization Observe and ask about perceived usefulness

Results Perceived as useful More cognitive effective than Dotted Chart

CHAPTER 3 – EXPLORATIONRelation with quality

Study 2 – Exploration

Fastmodeling

Slowmodeling

Initialdelay

Manypauzes

Fewelements

Manyelements

No(separate)lay-outing

Quicklay-outing

Dedicatedlay-outing

Continuouslay-outing

Unpairedevent

creation

Pairedevent

creation

No pauzesSerializationPairedgatewaycreation

Delayededge

creation

Chunkedmodeling

Study 2 – ExplorationFast

modelingSlow

modelingInitialdelay

Manypauzes

Fewelements

Manyelements

Quicklay-outing

Dedicatedlay-outing

Unpairedevent

creation

Pairedevent

creation

No pauzes Pairedgatewaycreation

Delayededge

creation

Chunkedmodeling

Serialization

modelingSlow

Manypauzes

Fewelements

Manyelements

Quicklay-outingContinuous

lay-outingUnpaired

eventcreation

Pairedevent

creation

Delayededge

creation

Chunkedmodeling

Dedicatedlay-outing

modelingSlow

Manypauzes

Fewelements

Manyelements

Quicklay-outing

Dedicatedlay-outing

Unpairedevent

creation

Pairedevent

creation

Delayededge

creation

Chunkedmodeling

Study 2 – ExplorationSlow

Manypauzes

Fewelements

Manyelements

Quicklay-outing

Dedicatedlay-outing

Unpairedevent

creation

Pairedevent

creation

Delayededge

creation

Chunkedmodeling

Fastmodeling

Manypauzes

Fewelements

Manyelements

Quicklay-outing

Dedicatedlay-outing

Unpairedevent

creation

Pairedevent

creation

Delayededge

creation

Chunkedmodeling

Slowmodeling

Study 2 – ExplorationStructuredness Movement Speed

Fastmodeling

Slowmodeling

Quicklay-outing

Dedicatedlay-outing

Serialization

Chunkedmodeling

CONJECTURE 1

CONJECTURE 2

CONJECTURE 3

Study 2 – ExplorationConjecture 1: Structured modeling

results in understandable models

Conjecture 2: A high number of move operations results in less understandable models

Conjecture 3: Slow modeling results in less understandable models

Structuredness• MaxSimulBlock• PercNumBlockAsAWhole

Speed• TotTime• TotCreateTime

Movement• AvgMoveOnMovedElements• PercNumElementsWithMoves

Model quality• Perspicuity

a model that is unambiguously interpretable and can be made sound with only small adaptations based on minimal assumptions on the modeler’s intentions with the model

CONJECTURE 2

CONJECTURE 3

T-testt=-2,231 (p=0,028)

T-testt=2,199 (p=0,030)

T-testt=-1,984 (p=0,049)

T-testt=0,457 (p=0,648)

T-testt=-2,183 (p=0,031)

T-testt=2,505 (p=0,014)

CHAPTER 4 – THEORISATIONStructured Process Modeling Theory

(SPMT)

Study 3 – Theorization

T-testt=-2,231

(p=0,028)

T-testt=2,199

(p=0,030)

CONJECTURE 1structuredness

T-testt=-1,984

(p=0,049)

T-testt=0,457

(p=0,648)

T-testt=-2,183

(p=0,031)

T-testt=2,505

(p=0,014)

CONJECTURE 2movement

CONJECTURE 3speed

Combined

Flow-oriented Aspect-oriented

Undirected

Study 3 – Theorization Observation 1. Almost all modelers paused frequently

during the modeling process Observation 2. A large group can be categorized as

“flow-oriented process modeling” Observation 3. A smaller group can be categorized as

“aspect-oriented process modeling” Observation 4. Another large group used a combination

of both former styles Observation 5. Another small group can be categorized

as “undirected process modeling” Observation 6. The “undirected” sessions lasted longer

than the other approaches

Study 3 – Theorization Impression 1. Modelers need serialization of the modeling

process to deal with its complexity Impression 2. Structured serializing of the modeling

process helps avoiding ‘mistakes’

Impression 3. Structured serializing does not support every

modeler to avoid ‘mistakes’ to the same extent

Study 3 – TheorizationCognitive Load Theory

Working memory capacity is limited Working memory overload causes decrease in

• Effectiveness (i.e., more mistakes)• Efficiency (i.e., more time and effort)• Learning

Cognitive Fit Theory Load is lower when there is a fit

• Between representation, tool or strategy on the one hand• And task or modeler on the other hand

A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B

learning style

degree of serialization

adopted serialization style

field-dependency need for structure

course of intrinsic cognitive load for process modeling phases

course of intrinsic cognitive load for aggregation phases

course of cognitive overload

course of intrinsic cognitive load for strategy building phases

serialization style fitstructuredness of serialization– – – –

Evaluation of utilityNovelty

(uses existing theories in fundamental new way)Parsimony

(11 constructs, 15 associations)Consistency

(can explain additional observations)Plausibility

(accurate and profound explanation)Credibility

(building blocks are established theories)Transferability

(problem solving in general)

Consistency based on dataset of 143 unique modeling executions

CHAPTER 5 – CONCLUSIONSummary & Future work

FUTURE WORK

Knowledge gaps

Knowledge contribution Research instrument

Knowledge gap 4How to change one’s modeling strategy?

Knowledge gap 3How should one model

in a specific context?

Study 1: visualizationContributions

A – PPMChartB – 22 patternsC – 13 observations

Study 2: explorationContributionsD – 8 patternsE – 3 conjecturesF – 1 metric

Knowledge gap 1How do people

currently model?

Knowledge gap 2Which strategy is

intrinsically better?

H HG I

Study 3: theorizationContributions

G – 5 stylesH1 – 6 observationsH2 – 3 impressionsI – SPMT

Future Work

Use developed knowledge toDevelop prescriptive theory

Cognitive profile best modeling approachDevelop method (SPMM)

(1) Determine (2) learn (3) apply best approachDevelop tool support

Cognitive tests Interactive digital approach tutorial

Reflection – Deduction, induction, abduction

Rule Case

Result

Deduction

Reveals effectResults in certainty

InductionCase Result

Reveals mechanismResults in probability

AbductionResult Case

Reveals causeResults in possibility

Process modeling as application domain> ordering of data facilitates pattern recognition <

Reflection – Student observationAssumption: overload has same causes

for students and practitionersAssumption: overload has same consequences

for students and practitioners

STUDENTS…… are representative participants

… don’t suffer from Expert-Reversal Effect… form a homogeneous group

… provide heterogeneous set of observations… reach point of overload faster than practitioners

Reflection – Empirical behavioral researchProcess modeling = complex and dynamic taskIdentify/measure/control confounding variables Ignore/assume constant/assume minimal effect

Include multiple variables in modelMix techniques (de/in/abduction, quanti/qualitative)

Open-world assumption> conclusions are incomplete unless proven otherwise <

> no conclusions from insignificant results <> more accurate, but slower progress <

Thanks for your attention! Do you have any questions?

Jan Claesjan.claes@ugent.be

http://www.janclaes.infoTwitter: @janclaesbelgium

phd pre-defense september 2015

Business

phd dutangc-defense

sheila kinsella phd defense

(phd students) - en.sharif.edu · pre-defense of phd...

phd thesis defense

phd defense -- ashish mangalampalli

phd defense presentation

mina deng phd defense

phd dissertation defense

jay's phd dissertation defense

bertrand dumenieu phd defense

public phd defense

phd defense

phd defense november 2015

acher phd thesis defense

raford phd defense final

phd defense cyh

adrien gomar, phd defense

phd defense - aurélien martinet

phd defense - luiz ricardo

phd defense script