superiority of an error mastery approach further investigated: findings from both the lab and the...
TRANSCRIPT
Superiority of an error mastery
approach further investigated:
Findings from both the lab and the
field
Cathy van Dyck
The Cat’s Eye / VU University
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 2-
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 3-
Cafeteria Mind Your Head
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 4-
Cafeteria Mind Your Head
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 5-
“Employees have to put a stamp with their identification code on their work[…]. But they’re even smarter, they just don’t put down their identification code, so we don’t know who has made the mistake.” …“Checking, checking, checking. We are now thinking about installing video cameras.”
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 6-
“[…] Learning from errors. I see them as positive, as free feedback. I always see them as positive. If people report their errors to me, I thank them for that. I don’t complain, I regard it as an opportunity to improve things. That how I see errors.”
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 7-
Error aversion culture Error mastery culture
Fear of errorsRigid focus on error prevention
Realistic focus on error preventionCombined with error management
Deepest layer: examples of beliefs
Errors are a sign of incompetenceWe don't make mistakes
To err is humanIt’s all about good error handling
Middle layer: examples of reinforcements
Punishment of error occurrence Rewarding constructive error handling
Superficial layer: examples of error behavior
Strain caused by errorsCovering up
Detection, correction, learningOpen communication
Illustrating quotes from our organizational research:
"But I don't want to discus errors at great length. [...] I indicated that it
shouldn't happen again and that was the end of it."
"I have spoken to the responsible manager, and have asked him to use
this incident as a learning opportunity in his department."
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 8-
Errors: What do we know? Organizations can be differentiated on
mastery and aversion aspects of error culture. – Eta2 = .33, p < .01 for Mastery– Eta2 = .38, p < .01 for Aversion
Mastery is related to company performance– positive correlation with both subjective (r = .33**) &
objective (r = .46*) performance measures – relationship is upheld when controlling for age, size &
line of industry– replication of results in Germany
Van Dyck et al, JAP, 2005
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 9-
Errors: What do we know?
More is learned from errors with severe consequences– Homsma et al. (in press), JBR
More constructive error handling in organizations that have a tolerant, yet decisive ‘basic view’ (assumptions)– Homsma (2007)
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 10
-
Overview
What are the underlying processes that explain superiority of mastery over aversion?– Experiments:
• mediators of the error culture-performance relationship
What is the effect of setting (degree of risk)?– Field study:
• Actual errors followed over time
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 11
-
Paradigm
Manipulation error approach Error inducement:
– Task paricipants are unfamiliar with
– Too little instructions given
Practice task (relatively easy) Performance task (difficult) Questionnaire in between tasks Task behaviour coded (video)
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 12
-
Experiments: manipulations
Mastery: “to err is human”A lot can be learned from errors. Errors point you at things you don’t know. So errors are really not bad. Try to learn as much as possible from your errors!
Aversion: “better safe than sorry”When working on a task, it’s important to have the right approach from the start. Errors have a tendency of escalation. If you strive to work without errors you can learn a lot!
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 13
-
Exp1: error approach & reinforcement
Manipulation error approach X reinforcement– reward/penalty points
N = 83 Task: programming in BASIC DV: task performance
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 14
-
Performance task: two main effects
1
2
3
4
5
Mastery Aversion
RewardPunishment
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 15
-
Experiment 2: control & responsibility
Manipulation: error approach
N = 58 (dyads, 118 participants)
Task: programming in BASIC
All task behaviour video recorded & coded
DVs:
– succesful correction of error (control)
– blaming the partner (taking responsibility (R))
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 16
-
Error handling practice task
1
2
3
4
5
6
7
8
9
10
Mastery Aversion
errors (n.s)corrected (n.s)
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 17
-
Error handling performance task
1
2
3
4
5
6
7
8
9
10
Mastery Aversion
errors (n.s)corrected (n.s)
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 18
-
Percentage corrected
0102030405060708090
100
Mastery Aversion
practice task (n.s.)
performance taskp<.05
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 19
-
Taking responsibility: blaming the partner (rev.)
1
2
3
4
5
Mastery Aversion
blaming p< .05
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 20
-
Experiment 3: attribution further explored
Manipulation: Locus X Stability N = 80 Task: Lemmings (computer game) Experienced control over and
responsibility for error causes. DV: Quality strategy
Homsma et al. (2007), JB&P
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 21
-
Perceived control cause: interaction
1
2
3
4
5
Internal External
UnstableStable
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 22
-
Quality strategy: interaction
0
5
10
15
20
25
30
Internal External
UnstableStable
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 23
-
Experiment 4: self-regulation
Manipulation: error approach (+ control group)
N= 68 Task: Tower of London, with error
inducement to minimize variation in number of errors, and avoid learning benefits from mastery approach.
DV: Self-focussed attention (Bagozzi & Verbeke, 2003).
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 24
-
Self focussed attention
1
2
3
4
5
Mastery Aversion Control
Threat core self
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 25
-
Experiment 5: self-focussed attention
Manipulation: SFA vs noSFA (+ control group)
• Be filmed (SFA)
• Filler info (no SFA)
• Write short piece (control)
N= 68 Task: Boloball (computer game) DVs: Strategy use
Performance improvements
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 26
-
Competitive strategy (blocking opponent)
01
23
456
78
910
SFA no-SFA Control
Blocking opponent
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 27
-
Performance increase
-100
-75
-50
-25
0
25
50
75
100
SFA no-SFA Control
performance gain(game 2 - game 1)
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 28
-
Results: Summary
Pure effect error approach (Exp.1) Less control of consequences & taking
responsibility under Aversion instructions (Exp. 2)
Better strategies with controllable causal attribution (Exp. 3)
Error approach affects SFA (Exp. 4) SFA negatively affects strategy and
performance gains (Exp. 5)
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 29
-
Does error management work every where?
“[…] I know the mountain [Everest] is an environment so extreme there is
no room for mistakes.”
(Cahill, 1997, p.245-246).
•The influence of the environment
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 30
-
Errors & environment
Severe errors? .. do not exist.
– Krakauer (Adventure Consultants exp. 1996)
The environment affects the likelihood of
severe consequences (risk).
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 31
-
Field study: Error tracking over time
Testing the effects of error approach – in real error incidents (T1)– with T2 follow-up
… on control – reduction of negative error consequences
and learning– single-loop: improvements– double-loop: integration new insights & ideas
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 32
-
Error Tracking Instrument (ETI) Description of a recent error Error handling
– communication (T1)– clarity cause (T1)
Control– (possible) consequences (T1)– reoccurrence (T2)
Learning– trouble shooting/LT solution (T1)– improvements (T2)– new insights (T2)
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 33
-
Error tracking: Research settings
Innovative e-business company– Low risk
Nurses of emergency unit– High risk
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 34
-
Examples of errors
e-business: Overlooking a big lay-out error in an
expensive advertisement. The sales department selling a service for
which there is no knowledge of implementation nor capacity to support it.
Erroneous billing of hours to a client. Installing an excellent, but for this client,
totally useless application.
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 35
-
Examples of errors (2)
EU nurses: A patient falling of a hospital trolley. A blood sample mix up. Using an expired drug.
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 36
-
Error incidents (1)
• Wrong assessment of situation. Illness of patient in the emergency room deteriorated. The patient got into a shock.
• Error was made upon arrival of patient. The error was discovered and corrected three hours later. The patient would otherwise have died.
• The supervisor was not informed, but a colleague was. Colleagues reacted uninterested.
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 37
-
Error incidents (1 cont’d)
• The error was not further discussed within the department, because people feared for their position. This was especially salient as there was rumor of a merger.
• At the measurement three weeks later, the respondent reported that no measures had been undertaken since, nor had new insights emerged. The error had not reoccurred in the weeks after its first occurrence.
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 38
-
Error incidents (2)• Taking up and starting work on a new
project for an important client. There was a verbal agreement, but no contract had been signed by the client yet. Due to financial difficulties, top management of the client company was unwilling to sign, or to pay for the work that had already been done. There was a strong (legal) case to make the client pay, but playing "hardball" might be undesirable as it would probably result in losing this client all together.
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 39
-
Error incidents (2 cont’d)• The problem was recognized two weeks into
the project, but no action was taken for four more weeks.
• For the IT company, this problem meant a loss of income, for the employees involved it meant not attaining a bonus.
• Supervisors were informed and responded emphatic. Without laying blame, the problem was further discussed with all people involved, which resulted in a decision to cut the losses, while making sure that this problem could not reoccur.
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 40
-
Error incidents (2 cont’d)
• One of the reasons that this could have happened, they concluded, was a flawed communication process, lack of cooperation and consulting each other.
• Contract managers, who served as a central contact for clients were appointed and check ups were implemented for contract statuses. Further, the sales process was improved. It turned out to be an important lesson for every one.
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 41
-ETI: Error Handling
Communication – 1 (e.g., “I have not discussed this error with
anyone.”) through 5 (e.g., “We thoroughly discussed it in order to handle matters in the best possible way.”; kappa = .91).
Clarity cause – 1 (e.g., “We did not analyze it. It hasn’t become
clear what happened.”) through 3 (e.g., “Yes, boundaries are clear now.”; kappa = .89).
Correction – 1 (e.g., “The error was not corrected, maybe it will
go better next time.”) through 3 (e.g., “Yes, we corrected through alternations in the system.”; kappa = .97).
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 42
-
ETI: Control Severity of error consequences (T1)
– 1 (e.g., “There were hardly any consequences”; “The patient just had to wait a bit longer.”) through 5 (e.g., “Serious dispute with client”; “Patient got into shock”; kappa = .93).
Containment of consequences (T1)– 1 (e.g., “No, the consequences could not have been
more serious”) through 3 (e.g., “Consequences could certainly have been much more serious, if the error had been detected later”; kappa = .82).
Error reoccurrence at T2 was measured by a dichotomous (no/yes) scale.
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 43
-
Learning from error
Long-term solution (T1) ranged from 1
(trouble shooting only) to 3 (long term
solution and improvement of processes;
kappa = 1.00).
Improvements at T2 ranged from 1 to 5
New ideas at T2 ranged from 1 to 5
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 44
-
Error tracking: Control & learning
IT EU
Control: Severity consequences
2.90 2.29 F (1, 76) = 5.34, p = .02
Control: Containment consequences
1.91 2.32 F (1, 74) = 3.49, p = .07
Learning: Long-term solutions
2.00 1.50 F (1, 71) = 4.38, p = .04
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 45
-
Results: T1 => T2
Error handling at T1 predicts learning at T2:
– Improvents predicted by Communication (r = .35*) and Clarity cause (r = .29#),
– Integration new insights predicted by Communication (r = .36*) and Clarity cause (r = .27#)
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 46
-
“When I first started as a supervisor, I used to get angry at people when they made a mistake. That’s very easy and seems forceful. But you have to learn that it simply doesn’t work. People will get frustrated, fearful, they will be less open about their mistakes and therefore errors will be discovered later.”
…
“I try to create an open atmosphere and tell people that they should tell me when they have made a mistake so that we can do something about it. We try to be open and discuss errors, because we believe that is the only way we can control damage.”
Sup
erio
rity
of
erro
r m
aste
ry a
ppr
oach 48
-Error
- goal not attained- not atributable to chance- could have been avoidedin short: - "good" intentions - failed action
Violation- conscious decision to disobey rulesin short: - "wrong" intentions - not necessarily failed action
Consequenceseither - negative (e.g. time loss, costs, accident etc)or- positive (e.g. new information, innovation, future error prevention)
versus versus