using forecast advice: role of optimism vs pessimism in ... · Ȍnkal, sayım, gönül - tfsc, 2013...

Using Forecast Advice:

Role of Optimism vs Pessimism

in Scenarios

Dilek Önkal Faculty of Business Administration, Bilkent University Paul Goodwin School of Management, University of Bath M. Sinan Gönül Department of Business Administration, METU

K. Zeynep Sayım Hotelschool The Hague

Scenarios

• Powerful tools for constructing alternative futures and exploring the pathways leading to these futures (Godet, 1982)

• Emphasize how future might evolve (Goodwin & Wright, 2010)

• Support planning for the range of plausible uncertainties and challenge managerial thinking (Schnaars & Topol, 1987)

• Used in corporate strategy and planning since early 1960s (Godet, 1982; Schoemaker, 1991; Goodwin & Wright, 2010)

▫ Anderson (1983) - use of scenarios may have more impact on users’ judgments than that of statistical predictions

▫ Wright & Goodwin (2009) - scenarios may help with problems due to cognitive biases

▫ Schnaars & Topol (1987), Taylor & Thompson (1982) - scenarios may be perceived to be more credible by users in comparison to dull & dry statistical numbers

Scenarios and Forecasting

Through structured storylines of plausible futures, scenarios:

▫ offer open platforms for information sharing

▫ facilitate communication

future-focused thinking channels of forecast advice

Ȍnkal, Sayım, Gönül - TFSC, 2013

• Optimistic & pessimistic scenarios may be utilized effectively as channels of forecasting advice in individual and group prediction tasks

• Scenario availability appears to:

• reduce the size of adjustments

• increase confidence

• Group discussions seem to be equally effective in communicating the

distinct scenario information – no significant differences between dyads where each member receives a different scenario vs where each member receives both scenarios

Scenarios used were optimistic and pessimistic:

• in tone and content

• with indicative labelling

Optimistic scenario labelled as “best-case scenario”

Pessimistic scenario labelled as “worst-case scenario”

Research Goals

Content vs frame ??

Investigating potential effects of incorporating

optimistic vs pessimistic scenarios into forecast advice

Comparing individuals’ vs small groups’ judgmental adjustments to given forecast advice

Research Design

• Phase 1: Individual forecasts

• Phase 2: Consensus forecasts (two-person teams)

• 78 business students 39 dyads

Phase 1– Individual Forecasts

• Participants were given 18 time-series plots showing past demand for mobile phones + model-based point forecasts

• They were asked to make their individual predictions using the following formats:

Point forecasts

Best-case/worst-case forecasts

Surprise probability (probability that the actual demand will be higher than their best-case forecast, or lower than their worst-case forecast)

Four experimental groups:

G1 – Optimistic content without labels (N = 15) Time-series plots + model-based forecasts + optimistic scenarios (entitled “scenario”)

G2 – Optimistic content with labels (N = 24) Time-series plots + model-based forecasts + optimistic scenarios (entitled “best-case scenario”)

G3 – Pessimistic content without labels (N = 15) Time-series plots + model-based forecasts + pessimistic scenarios (entitled “scenario”)

G4 - Pessimistic content with labels (N = 24) Time-series plots + model-based forecasts + pessimistic scenarios (entitled “worst-case scenario”)

Sample Form (G1)

Model-based forecast for period 21: 237

Scenario:

This is one of our best performing mobile phones with complex features. Its built-in camera is a premium one, plus it had got all of the other usual features (e.g. GPS, music player, Bluetooth, etc.). It was a hit product when first introduced, and kept its upward trend ever since. With new features added from time to time (for instance, new colours, improved camera, etc.) its popularity has been kept above a certain level. With a price very reasonably positioned among that of its major competitors, Product E is reported to be a desirability object particularly for the teenager consumer group. The future looks very bright for this product. It is expected to carry on its upward trend, even to the levels that look overly optimistic at the moment.

105100

134145

167

157

126

180

163153

161

184

212

174 177

246

215225

232

210

237

8090

100110120130140150160170180190200210220230240250260

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Dem

and

(uni

ts)

Time (periods)

YOUR FORECASTS: Please provide your point forecast for period 21 : ………………

Please provide your best-case forecast(highest value you predict) for period 21 : ………………

Please provide your worst-case forecast(lowest value you predict) for period 21 : ……………… What is the probability that the actual demand for period 21 will be a surprise to you?

(What is the probability that the actual demand for period 21 will be higher than your best-case forecast or

lower than your worst-case forecast?):

…………….%

In constructing my forecasts, the given Scenario was

Not informative at all

(provided no information

at all)

Extremely informative

(provided very

significant information)

1 2 3 4 5

Not useful at all

(was not helpful at all in

making my forecasts)

Extremely useful (was

extremely helpful in

making my forecasts)

1 2 3 4 5

Not influential at all

(had no effect whatsoever

on my forecasts)

Extremely influential

(had a very significant

effect on my forecasts)

1 2 3 4 5

Phase 2– Consensus Forecasts Participants were assigned to two-person teams.

Each participant was provided with a "Consensus Forecasts Form" that included the same 18 time-series plots, the same model-based forecasts and the corresponding scenarios with/without labels.

Participants were requested to:

▫ discuss the given model-based forecasts, past demands and scenarios as a dyad,

▫ arrive at consensus forecasts in the form of point, best-case, and worst-case predictions, as well as their dyad’s surprise probability assessments (i.e., assessments of predicted surprise index) for each of the 18 products.

Upon completing the consensus forecasts for each product, they were asked to individually convey their level of agreement with these consensus predictions

Two experimental groups:

• X1 – Unlabelled scenarios (15 dyads) : G1+G3

▫ One member received optimistic scenario without labels

▫ Other member received pessimistic scenario without labels

• X2 – Labelled scenarios (24 dyads) : G2+G4

▫ One member received optimistic scenario with labels

▫ Other member received pessimistic scenario with labels

Sample Form (X2)

Model-based forecast for period 21: 237

Worst-case scenario:

Product E has shown a good performance since its initial presentation to the market. However, the sales stumbled at certain points, sometimes experiencing declines in two or three successive periods. It needed to be smartened up to recover the upward trend. But these revamping efforts have been quite costly, reducing the profit margin considerably. And there is a limit to how much ‘redecoration’ can be done. Instead of spending large sums on revamping efforts, this model can be left to its own devices until it is making no more profit. Such a strategy will probably be less costly in the long run, although the sales volume can be expected to decline rather quickly.

105100

134145

167

157

126

180

163153

161

184

212

174 177

246

215225

232

210

237

8090

100110120130140150160170180190200210220230240250260

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Dem

and

(uni

ts)

Time (periods)

CONSENSUS FORECASTS FOR YOUR GROUP Please provide your group's consensus point forecast for period 21 : ……………

Please provide your group's consensus best-case forecast(highest value predicted) for period 21 : ……………

Please provide your group's consensus worst-case forecast(lowest value predicted) for period 21 : ……………

What is the probability that the actual demand for period 21 will be a surprise to your group?

(What is the probability that the actual demand for period 21 will be higher than your group's consensus best-case

forecast or lower than your group's consensus worst-case forecast?):

…………….%

Please provide your level of agreement with your group's consensus POINT forecast

Totally disagree Totally agree 1 2 3 4 5

Please provide your level of agreement with your group's consensus BEST-CASE forecast

Totally disagree Totally agree 1 2 3 4 5

Please provide your level of agreement with your group's consensus WORST-CASE forecast

Totally disagree Totally agree 1 2 3 4 5

Performance Measures Five performance measures used for both individual and dyad forecasts:

1. %CPoint : Percentage change of final point forecasts from the given model-based point forecasts

2. %CBest : Percentage change of final best-case forecasts from the given model-based point forecasts

3. %CWorst : Percentage change of final worst-case forecasts from the given model-based point forecasts

4. SI : Surprise Index Average value across all surprise probabilities that the realized values will fall outside the prediction limits (i.e., mean probability of actual values turning out to be higher than final best-case forecasts or lower than final worst-case forecasts)

5. ASR : Asymmetry Ratio (O’Connor, Remus & Griggs, 2001)

Findings: Perception of Scenarios

3.71

3.63

3.73

3.85

3.77

3.88

3.67

3.59

3.65

3.88

3.73

3.71

1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00

Influence (Pessimistic sce.)

Usefulness (Pessimistic sce.)

Informativeness (Pessimistic sce.)

Influence (Optimistic sce.)

Usefulness (Optimistic sce)

Informativeness (Optimistic sce.)

Unlabelled scenarios Labelled scenarios

• Overall, the provided scenarios were perceived to be informative, useful and influential

• Presence/absence of labels did not lead to any significant differences in these perceptions

Individual Forecasts

%CPoint %CBest %CWorst SI ASR

Unlabelled

scenarios

(G1)

9.54%

(15)

39.30%

(15)

-19.73%

(15)

8.19%

(15)

52.64%

(15)

Labelled

scenarios

(G2)

10.65%

(24)

33.41%

(24)

-10.05%

(24)

9.42%

(24)

50.89%

(24)

Optimistic Scenarios

%CPoint %CBest %CWorst SI ASR

Unlabelled

scenarios

(G3)

2.44%

(15)

34.80%

(15)

-23.89%

(15)

10.74%

(15)

51.54%

(15)

Labelled

scenarios

(G4)

-6.80%

(24)

12.77%

(24)

-25.93%

(24)

15.60%

(24)

54.68%

(24)

Pessimistic Scenarios

Individual forecasts

Significant difference between the unlabelled and labelled scenarios on the forecasts at the other end

When optimistic scenarios are labelled, final worst-case forecasts are significantly closer to the given model-based (point) predictions

When pessimistic scenarios are labelled, final best-case forecasts are significantly closer to the given model-based (point) predictions

Presence of labels appears to create a moderating/balancing effect – imposing an adjustment limit on the opposite prediction

Consensus Forecasts

%CPoint %CBest %CWorst SI ASR

Unlabelled

scenarios

(X1)

5.00%

(15)

31.69%

(15)

-18.94%

(15)

6.85%

(15)

54.19%

(15)

Labelled

scenarios

(X2)

-0.35%

(24)

16.97%

(24)

-19.33%

(24)

8.01%

(24)

53.99%

(24)

[NOTE: One member received optimistic scenario, other member received pessimistic scenario]

Results – Agreement with Consensus

4.36 4.24 4.22

3.84 3.80 3.79

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

Point Forecasts Best-case Forecasts Worst-case Forecasts

Labelled scenarios Unlabelled scenarios

Group forecasts • Unlabelled forecasts are adjusted more

(for point and best-case predictions)

• When scenarios are labelled, dyad members seem to agree more with their consensus forecasts

Framing through labels • has an influence on how the scenarios are used

• affects adjustments on the forecasting advice given

Repercussions for Forecast Management

• Scenarios may be used as effective forecast advice to

▫ Challenge mental frames and tunnel vision in decision making

▫ Debias against confirmation bias, hindsight bias, overconfidence

▫ Counteract future myopia and retrospective sensemaking

• Optimism – pessimism balance

NEXT?

Forecasting performance of practitioners in diverse organizational contexts

▫ Role of group size?

▫ Role of group process?

Scenarios constructed by individual forecasters vs groups?

Scenarios constructed by ▫ forecasters

▫ domain experts

▫ collaboratively by both

…….

THANK YOU