hungarian association of executives evidence-based management an introduction hungarian association...

Hungarian Association of Executives

Evidence-Based Management

An introduction


October 26, Budapest


Where does it come from?

What is it?

Why do we need it?

How does it look like in practice?

(An example)


Evidence based management:

Where does it come from?


“there is a large research-user gap”

“practitioners do not read academic journals”

“the findings of research into what is an effective intervention

are not being translated into actual practice”

“academics not practitioners are driving the research agenda”

“the relevance, quality and applicability of research is

questionable”

“practice is being driven more by fads and fashions than

research”

“many practices are doing more harm than good”

What field is this?


McMaster University Medical School, Canada

Medicine: Founding fathers

David Sackett Gordon Guyatt


How it all started


More than 1 million articles in 40,000 medical journals per

year (= 1995; now probably more than 2 million). For a

specialist to keep up this means reading 25 articles every

day (for a GP more than 100!)

Most of the new insights and treatment

methods don’t reach the target group

Problem I: too much information


Problem I: too much information

HRM: 1,350 articles in 2010 (ABI/INFORM). For an HR

manager to keep up this means reading 3 to 4 articles

every day (for a ‘general’ manager more than 50!)


Problem II: persistent convictions

if you’re hyperventilating

breathe into a bag


elderly people who have an irregular heartbeat are much more likely to die of

coronary disease

give them a drug that reduces the number of

irregular beats

Problem II: persistent convictions


How 40,000 cardiologists can be wrong

In the early 1980s newly introduced anti-arrhythmic drugs were found to be highly successful at suppressing arrhythmias.

Not until a RCT was performed was it realized that, although these drugs suppressed arrhythmias, they actually increased mortality.

The CAST trial revealed Excess mortality of 56/1000.

By the time the results of this trial were published, at least 100,000 such patients had been taking these drugs.


David Sackett

Half of what you learn in medical school will be

shown to be either dead wrong or out-of-date

within 5 years of your graduation; the trouble is that

nobody can tell you which half.

The most important thing to learn is how to learn

on your own: search for the evidence!

(Remember that your teachers are as full of bullshit

as your parents)


Evidence-Based Practice

1991Medicine

1998Education

1999Social care

2000Nursing

2000Criminal justice

????Management?



What is it?


Definition

Evidence-based management means making decisions

about the management of employees, teams or

organizations through the conscientious, explicit and

judicious use of four sources of information:

1. The best available scientific evidence

2. Organizational facts, metrics and characteristics

3. Stakeholders’ values and concerns

4. Decision supports and practitioner expertise


Four sources


Definition

When making an important decision,

an evidence-based manager knows

whether there is scientific evidence

available to support this decision

(and how ‘strong’ the evidence is).


Evidence-based management:

Why do we need it?


Four sources


Trust me: 20 years of management experience!


Errors and Biases of Human Judgment


Seeing order in randomness Mental corner cutting Misinterpretation of incomplete data Halo effect False consensus effect Attribution error Group think Self serving bias Sunk cost fallacy Cognitive dissonance reduction

Confirmation bias Authority bias Small numbers fallacy In-group bias Recall bias Anchoring bias Inaccurate covariation detection Distortions due to plausibility



“I’ve been studying intuition for 45 years, and I’m no better than when I started. I make extreme predictions. I’m over-confident. I fall for every one of the biases.”



Doctors and managers hold many erroneous beliefs,

not because they are ignorant or stupid, but because

they seem to be the most sensible conclusion

consistent with their own professional experience!



stress & lifestyle peptic ulcer

Peptic ulcer – an infectious disease!

This year's Nobel Prize in Physiology or Medicine goes to Barry Marshall and Robin Warren, who with tenacity and a prepared mind challenged prevailing dogmas. By using technologies generally available (fibre endoscopy, silver staining of histological sections and culture techniques for microaerophilic bacteria), they made an irrefutable case that the bacterium Helicobacter pylori is causing disease. By culturing the bacteria they made them amenable to scientific study.

In 1982, when this bacterium was discovered by Marshall and Warren, stress and lifestyle were considered the major causes of peptic ulcer disease. It is now

firmly established that Helicobacter pylori causes more then 90% of duodenal ulcers. The link between Helicobacter pylori infection and peptic ulcer disease has been established through studies of human volunteers, antibiotic treatment studies and epidemiological studies.

Oct 2005


Beliefs vs Evidence

“What gets us into trouble is not what we

don't know,

it's what we know for sure that just isn't so.”

Mark Twain


1. Incompetent people benefit more from feedback than

highly competent people.

2. Task conflict improves work group performance while

relational conflict harms it.

3. Encouraging employees to participate in decision

making is more effective for improving organizational

performance than setting performance goals.

True or false?


How evidence-based are we?

959 (US) + 626 (Dutch) HR professionals

35 statements, based on an extensive body of

evidence

true / false / uncertain

On average: 35% - 57% correct

HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008)


Evidence-based?

Competence management

Excellent care

Total Quality Management

Flexible workspace / The New World of Work

Knowledge Management

Investors in People, Great Place To Work

Balanced Score Card / INK

Lean / Six Sigma / TOC


How ‘new’ is this question or problem?

Is there evidence from scientific research that

can help us find the most effective approach?

How ‘strong’ is this evidence?

Is the evidence applicable to my situation?

Evidence-based approach



How does it look like in practice?


The 5 steps of EBP

1. Formulate a focused question (Ask)

2. Search for the best available evidence (Acquire)

3. Critically appraise the evidence (Appraise)

4. Integrate the evidence with your managerial

expertise and organisational concerns and apply

(Apply)

5. Monitor the outcome (Assess)


1. Formulate a focused question


Asking the right question?

Does team-building work?

Does the introduction of self-steering teams work?

Does management development improve the performance

of managers?

Does employee participation prevent resistance to change?

Is 360 degree feedback effective?


What is a ‘team’?

What kind of team?

In what contexts/ settings?

What counts as ‘team-building’?

What does ‘work’ mean?

What outcomes are relevant?

Over what time periods?

Focused question?

Does team-building work?


P = Population

I = Intervention or success factor

C = Comparison

O = Outcome

C = Context

Answerable question: PICOC


P = Population

I = Intervention or successfactor

C = Comparison

O = Outcome

C = Context

Focused question: PICOC

Employee productivity?

Job satisfaction?

Return on investment?

Market share?

Organizational commitment?


2. Finding the best available evidence


Where do we search?


Hands on instruction


3. Critical appraisal of studies

Making sense of evidence


Best available evidence?


Research designs

What is the BEST car?


Which design for which question?

Research designs


Explanation

Which design for which question?


Best research design?


Best available?


Evidence is not the same as ‘proof’ or ‘hard facts’

Evidence can be

- so strong that no one doubts its correctness, or

- so weak that it is hardly convincing at all

What is evidence?


Step 4: Turning evidence into practice


Applicable / Feasible?

organizational facts and characteristics

cultural aspects

stakeholders’ values and concerns

political aspects

financial aspects /cost-effectiveness / ROI

priorities

change readiness / resistance to change

implementation capacity

timing


Evidence-based practice:

Focuses on the decision making process

Uses research findings to increase the likelihood of a positive outcome

hungarian association of executives evidence-based management an introduction hungarian association...

Documents

information slide

bag slide

budapest slide

applicability of research

research agenda

findings of research

medical journals

questionable practice