back to school day, sept. 2010“facilitating high-quality conversations about decision...

Back to School Day, Sept. 2010 “Facilitating High-quality Conversations about Decision Situations”

Opening session byProfessor Patrick S. Noonan

[email protected]

Back to School Day, Sept. 24, 2010

Understanding Decision MakingFacilitating High-Quality Conversations

about Decision Situations


Deepwater Horizon, Gulf of Mexico, April 2010

© 2003-2010 Patrick S. Noonan - Emory University 2


Where am I coming from?The Decision-Centric Worldview

In a few hundred years, when the history of our time will be written from a long-term perspective, it is likely that the most important event historians will see is not technology, not the Internet, not e-commerce.

It is an unprecedented change in the human condition. For the first time -- literally -- substantial and rapidly growing numbers of people have choices. For the first time, they will have to manage themselves.

And society is totally unprepared for it.-- Peter Drucker

The only way that individuals can purposely exercise any control over their lives, their careers, or their surroundings it through their decision making.

-- Ralph Keeney

3© 2003-2010 Patrick S. Noonan - Emory University


Decisions are “Fundamental Particles”



Mastering decision-making is easy…

…isn’t it?



Discussion Question

• What are the biggest obstacles to high-quality decision making…– … to you as a consumer?– … to you as a manager & leader?– … to your organization?



Decisions are Inescapable

• US President Herbert Hoover on vacation:– “I’m tired of making decisions -- one after

another all day long.”– “My view of Heaven is of a place where no

one ever has to make a decision.”



… or Are They?• Decision-making researcher Gary Klein to an NYC

Fire Department battalion commander:– “Tell me about some difficult decisions you’ve made.”

• Commander:– “I don’t make decisions.”– “I don’t remember when I’ve ever made a decision.”– “Commanders never make decisions.”

– Source: Gary Klein, Sources of Power, 1999



Telling us to obey instinct is like telling us to obey "people." People say different things: so do instincts.

Our instincts are at war… Each instinct, if you listen to it, will claim to be gratified at the expense of the rest.

-- C. S. Lewis



Going with the Gut



This Slide is Brought to you by... the Letter K

• In a typical sample of text in the English language, is it more likely that a word (of three or more letters):

– starts with the letter K? or

– has K is its third letter?



The Beijing Questions



The Pentagon Questions



Risk Estimation Questions

• Which cause of death is the more common in the US?

– Diabetes or homicide– Tornado or lightning– Car accident or stomach cancer– Shark or items falling from airplanes



Level of Confidence

• Example question: Tourmaline is...– A) a kind of cloth, or– B) a precious stone

• My answer is:– A) or B)

• The probability that my answer is correct is:– .50 .55 .60 .65 .70 .75 .80 .85 .90 .95 1.00



Quantity Estimation

• Suppose you could fold a piece of paper in half, again and again and again.

• After 100 folds, how thick will it be?– My best guess is that the paper will be

_________ thick.– I am 90 percent sure that the correct answer

lies between _________ and __________ .



Some Heuristics… and Resulting Biases• ease of recall, imaginability• presumed associations

• status quo bias: insufficient adjustment of anchor

• overconfidence• excessive influence of regret

• stereotyping (pattern matching)– insensitivity to base rates– insensitivity to sample size

• Availability

• Anchoring & Adjustment

• Representativeness



St. Louis’ Gateway Arch: Wider or Taller?



The Arch Fits Perfectly into a Square



Doh! Being Smart Doesn’t Help… … and It Might Even Hurt

• Edward de Bono:(“Thinking in America… the Lost Art,” Critical Intelligence, 1994)

– The “intelligence trap”– A highly intelligent person will take a point of view...

and then use that intelligence to defend it.– Many excellent minds are trapped in poor ideas

because they can defend them well.• See also: “The Dunning-Kruger Effect”



High Points in the Art of Prediction“This ‘telephone' has too many shortcomings to be seriously

considered as a means of communication. The device is inherently of no value to us.”

-- Western Union internal memo, 1876

“The wireless music box [radio] has no imaginable commercial value. Who would pay for a message sent to nobody in particular?”

-- David Sarnoff, US radio pioneer, 1921

“But what… is it good for?” -- IBM engineer, 1968, on the microchip

“There is no reason anyone would want a computer in their home.”

-- Ken Olson, President, Chairman & Founder of DEC, 1977

“Stocks have reached what looks like a permanently high plateau.”

-- Irving Fisher, Yale Prof. of Economics, 1921

“A severe depression…is outside the range of probability.” -- Harvard Economic Society, Nov. 1929



High Points… continued“Who the hell wants to hear actors talk?” -- H.M.Warner, Warner Brothers, 1927

“Everything that can be invented has been invented.” -- Charles H. Duell, Commissioner, US Patent Office, 1899 [disputed]

“Pasteur’s theory of germs is ridiculous fiction.” -- Pierre Pachet, Professor of Physiology, , 1872

“Heavier than air flying machines are impossible.” -- Lord Kelvin, c. 1895

“No flying machine will ever fly from New York to Paris.”

“Man will not fly for 50 years.” [1901]

-- Orville Wright, co-inventor of airplane.

“Prof. Goddard does not know the relation between action and reaction and the need to have something better than a vacuum against which to react. He seems to lack the basic knowledge ladled out daily in high schools.”

-- 1921 New York Times editorial about rocket pioneer Robert Goddard’s work



High Points… continued

“Airplanes are interesting toys but of no military value.” -- Marechal Ferdinand Foch, Professor, École Superieure de Guerre, c. 1904

“That idea is so damned nonsensical & impossible that I’m willing to stand on the bridge of a battle-ship while that nitwit tries to hit it from the air!”

-- Newton Baker, US Sec. of War, 1921 (on Gen. Billy Mitchell’s claim that airplanes could sink battleships by dropping bombs on them)

“They couldn’t hit an elephant at this dist- ...” -- Last words, Union Army Gen. John Sedgwick (Battle of Spotsylvania, 1864)

“The radio craze will die out in time.” -- Thomas Edison, 1922

"640k of memory ought to be enough for anybody.” -- Bill Gates



January 28, 1986 – Challenger’s Last Flight


Before

After


“What are some limits on my intuition?”• Experiments reveal important insights about the mental

operating system (OS) of humans• Intuition & expertise have important roles…

– …but also some limitations.

• The human OS displays systematic weaknesses…– … but that’s actually good news!

The brain is a machine assembled to survive.-- Edward O. Wilson

This view of mind suggests that the liberal arts are those modes of reasoning needed in civilized society that are not innate because they had no survival value on the savanna.

-- David S. Moore



The Central Questions of Decision & Information Analysis

To think is easy.

To act is hard. But the hardest thing in the world is to act in accordance with your thinking.

-- Goethe



What DIA seems to be about



What DIA actually is about

“The objective of decision analysis is to facilitate a high-quality conversation about a decision situation.” [Prof. Ron Howard, Stanford University]

“The art of management is making meaningful generalizations from inadequate facts.”[Stanley Teele, former Dean of the Harvard Business School]



Most Decision Makers need some Therapy



The Decision Quality Chain“What do I need for a great decision?”

Source: Matheson &

Matheson



Russo & Schoemaker’s “Decision Traps”

Source: Decision Traps, Russo & Schoemaker (1989)

• Most common organizational decision-making dysfunctions:1. Plunging In2. Frame Blindness3. Lack of Frame Control4. Overconfidence5. Shortsighted Shortcuts6. Shooting from the Hip7. Group Failure8. Fooling oneself about Feedback9. Not Keeping Track10. Failure to Audit the Decision Process



Decision Trap #1: Plunging In

• Behavior: – Starting the analysis & decision making before thinking

about the objectives, the context and the most appropriate decision process

• Consequence: – Inefficient decision making, often sloppy and

misdirected

• Solution: – Devote careful thought to the metadecisions first.




The Metadecisions: “How Should I Decide?”

• There are very important decisions you can make about the decision processes you use

• Higher-order decisions make you think about the nature of what you’re trying to decide

• Examples:– What’s the main difficulty in this issue? – In general, how should decisions like this one be made (participants, style,

time allotted, etc.)?– Is this decision linked to others, i.e. does it materially affect other decisions,

or is it affected by others?




More Metadecision Questions• Must this decision actually be made? • Should I make the decision along, with

others, or delegate it?• How much time should it take?• Must the decision be made now?

If not, when should it be made?• Are the constraints and deadlines real?• Should I move sequentially in the

process, or back and forth?


The most basic of all decisions is who shall decide. This is easily lost sight of in discussions that proceed directly to the merits of particular issues, as if they could be judged from a unitary, of God's eye, viewpoint.A more human perspective must recognize the respective advantages and disadvantages of different decision making processes, include their widely varying costs of knowledge, which is a central consideration often overlooked in analyses which proceed as if knowledge were either complete, costless, or of a 'given' quantity.

-- Thomas Sowell



Still More Metadecision Questions• What’s a good balance of effort, i.e. where should

I concentrate my time & resources?• Is there some feedback or history from related

experiences to draw upon?• What are my own capabilities, biases &

limitations in dealing with an issue like this?• Do I need to bring in others’ points of view?

What other points of view might be helpful?• How would an admired, more experienced

decision maker handle this issue? Source: Decision Traps, Russo & Schoemaker (1989)



Discussion Questions

• How much thinking or discussion does your organization apply to the meta-decisions, the decisions about how to decide?

• How would you describe your organization’s typical “balance of effort” in tackling important decisions?




“What’s the appropriate balance of effort?”


“Analysis Paralysis” vs. “Extinction by Instinct”



Decision Tree: An Analytical Journey



The Road Ahead



“What are the elements of the decision?”


Decisions & Alternatives

Consequences

Uncertainties & Outcomes


From Insight to Action“How do I move from issues to implementation?”



The Generic Modeling Process“How should I proceed? What are the steps?”

Success doesn't "happen." It is organized, preempted, captured, by consecrated common sense.

-- Frances E. Willard

Industry is a better horse to ride than genius.

-- Walter Lippman



The Process must be Robust


When you’re going through Hell… keep going.

-- Sir Winston Churchill


Always Look for Good Shortcuts!“Have we done enough to make a good choice?”



Real World vs. Model World

All models are wrong,

but some are useful.-- George Box



Step One: Defining the Problem“What are the central issues and boundaries?”

The greatest challenge to any thinker is stating the problem in a way that will allow a solution.

-- Bertrand Russell



Consulting Best Practices: Problem Solving



Typical: Alternative-Focused Thinking

Adapted from Ralph Keeney, Value Focused Thinking



Better: Value-Focused Thinking




The Value of Value-Focused ThinkingReasonable men adapt themselves to the world. Unreasonable men adapt the world to themselves.That’s why all progress depends on unreasonable men.

--George Bernard Shaw

A wise man will make more opportunities than he finds.

-- Sir Francis Bacon

The people who get on in this world are the people who get up and look for the circumstances they want, and if they can't find them, make them.

-- George Bernard Shaw




Starting with Endpoints



Identifying & Structuring Objectives“What do we want? Why do we want it?”



“What metrics should we use?”



“What drives level of preference?”



“How can we value and compare uncertainties?”



“What is our appetite for – or aversion to – risk?”



Risk is Variability



Step Two: Structuring the Problem“What are the elements? And how do they fit together?”



Step Three: Analyze… and “Choose”“What does a preliminary analysis suggest?”



Here’s Where this Stuff Lives!



Step Four: Testing the Analysis“Which of my assumptions are most crucial?”

It is fashionable today to assume that any figures about the future are better than none.

To produce figures about the unknown, the current method is to make a guess about something or other -- called an "assumption" -- and to derive an estimate from it by subtle calculation. The estimate is then presented as the result of scientific reasoning, something far superior to mere guesswork.

This is a pernicious practice that can only lead to the most colossal planning errors, because it offers a bogus answer where, in fact, an entrepreneurial judgment is required.

-- E F. Schumacher66© 2003-2010 Patrick S. Noonan - Emory University


Everything You Know is Wrong…… Fortunately, not Everything Matters

If a man hasn't discovered something that he would die for, he isn't fit to live.

-- Dr. Martin Luther King, Jr.

I would never die for my beliefs because I might be wrong.

-- Bertrand Russell

Round numbers are always false-- Samuel Johnson

Point estimates are for suckers.-- Patrick Noonan



“What can I learn from sensitivity analysis?”• How confident are we about our preliminary recommendation?• If we’re a little off in our estimates, what difference does it make

to our decision strategy?• How far wrong could we be on our assumptions before our

optimal strategy is no longer the best?• What other decision strategies might be optimal, under slightly

different values of the problem parameters, or different structural assumptions?

• Within the members of our team, we can’t get complete agreement on some assumptions. Does it matter?

• We have limited time and money available for refining our assumptions. Which ones should have top priority for getting our managerial attention?



Muddling Thinking about Quantification



Increasing Depth of Sensitivity Analysis



“How can we sharpen our beliefs using data?”



The Process of Data Analysis



The Problem with Numbers



Numbers are Very Powerful!



Closing the Deal: From Insight to Action“What should I do, how do I get it done?”



Selling a Recommendation = “What do the Gate-Keepers Need?”



Trap #10: Failure to Audit the Decision Process“What Worked, What Didn’t Work?”

• Behavior: – No organized approach to understanding your decision

making

• Consequence: – Risk of falling into any or all of the aforementioned

decision traps

• Solution: – Routine, explicit decision evaluation process.




Review: What should we do with “decisions worth thinking about”?

“The objective of decision analysis is to facilitate a high-quality conversation about a decision situation.” [Prof. Ron Howard, Stanford University]



Coming March 2011 from McGraw-Hill/Irwin


back to school day, sept. 2010“facilitating high-quality conversations about decision...

Documents

school day

decision situations

high quality decision

decisioncentric worldview

professor patrick

difficult decisions

department battalion

biggest obstacles