an experimental sentiment indicator for the euro area ... · european commission directorate...

European Commission Directorate General Economic and Financial Affairs

An experimental sentiment indicator

for the euro area – tracking and

nowcasting q-o-q GDP growth

Andreas Reuter

Business and consumer surveys and

short-term forecast (ECFIN A4.2)

2

Outline

2. Strengths / Weaknesses of the Economic Sentiment Indicator (ESI)

4. Improving the ESI's tracking performance of q-o-q GDP growth

step a: re-constructing the ESI based on

"best-performing" survey questions

step b: an ESI with amplified changes

3. Refresher on ESI Construction Method

1. Introduction: the Economic Sentiment Indicator (ESI)

5. The nowcasting performance of the experimental ESI

step a: Model selection (based on regression analyses)

step b: out-of-sample properties (sim. real-time scenario)

3

1. Introduction: the Economic Sentiment Indicator (ESI)

Added value: timeliness (complementing delayed quantitative

statistics)

high frequency

Purpose of the ESI:

summarising developments in all 5 sectors covered by DG ECFIN's

Business and Consumer Surveys (BCS):

services

tracking GDP growth at Member State, EU and euro-area level

industry

construction

retail trade

consumers

ingredient for bridge models now-/forecasting GDP growth

-6

-4

-2

0

2

4

6

60

70

80

90

100

110

120

96-q

2

97-q

2

98-q

2

99-q

2

00-q

2

01-q

2

02-q

2

03-q

2

04-q

2

05-q

2

06-q

2

07-q

2

08-q

2

09-q

2

10-q

2

11-q

2

12-q

2

13-q

2

y-o-y (%)

quarterly averages

ESI and GDP growth; euro-area (1996q2-2013q2)

ESI GDP growth (rhs)

Note: monthly BCS data are converted into quarterly by

averaging the balances over 3 months. GDP figures refer

to y-o-y changes (%). Source: European Commission.

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

60

70

80

90

100

110

120

96-q

2

97-q

2

98-q

2

99-q

2

00-q

2

01-q

2

02-q

2

03-q

2

04-q

2

05-q

2

06-q

2

07-q

2

08-q

2

09-q

2

10-q

2

11-q

2

12-q

2

13-q

2

q-o-q (%)

quarterly averages

ESI and GDP growth; euro-area (1996q2-2013q2)

ESI GDP growth (rhs)

Note: monthly BCS data are converted into quarterly by

averaging the balances over 3 months. GDP figures refer

to q-o-q changes (%). Source: European Commission.

4

2. Strengths / Weaknesses of the Economic Sentiment

Indicator (ESI)

0.93

0.88

Correlations (1996q2-2013q2):

coincident

leading 1

0.72

0.49

Correlations (1996q2-2013q2):

coincident

leading 1

excellent sub-optimal

see-sawing

movement

absent in ESI

pace of

recovery under-

estimated by

ESI

addition of 100

multiplication by 10

5

ingredients: balance series of 15 survey questions

effect:

values >100 indicate above-

average economic sentiment

2/3 of observations will be in the

interval [90 ; 110] if norm. distr.

3. Refresher on ESI Construction Method

The questions are:

seasonally adjusted

standardised

allocating weights per sector:

Industry: 40% ; Services: 30% ; Consumers: 20% ; Construction: 5% ; Retail Trade: 5%

calculation of arithmetic mean of weighted balances

standardisation of the ESI and:

effect:

comparability of balance series in

terms of mean and volatility

no series dominates development

of ESI due to a higher amplitude

individual indu question has weight of 13,3% (=40% weight / 3 questions)

% of positive answers minus % of negative answers

6

4. Improving the ESI's tracking performance of q-o-q GDP

growth

step a: re-constructing the ESI based on "best-performing" survey questions

correlation of all individual EU BCS survey questions (quarterly averages) with:

i) the respective reference series (e.g. Gross Value added in

Manufacturing for the industry questions, etc.) ii) q-o-q GDP growth (EA)

construction of 3 new Confidence Indicators (CIs) for every sector observed: arithmetic mean of the respective (s.a.) balance series

i) based on 2/3 best performing questions of respective sector

iii) based on all forward-looking questions of respective sector

for each sector: selection of best sectoral CI (reg. correlation with ref. series &

q-o-q GDP growth (EA))

questions contained in the best CIs make up the new ESI

7

Intermediate Results:

4. Improving the ESI's tracking performance of q-o-q GDP growth – step a: reconstructing the ESI based on "best-performing" survey questions

Industry Services Retail Trade Consumers

order books

- currently

business

- last 3 months

business activity

(sales)

- last 3 months

household's fin.

position

- next 12 months

stock of (finished)

products

- currently

demand for firm's

services

- last 3 months

volume of stock

- currently

econ. situa-tion in

MS

- next 12 months

production

- next 3 months

demand for firm's

services

- next 3 months

business activity

(sales)

- next 3 months

unemploy-ment in

MS

- next 12 months

likelihood of

saving money

- next 12 months

Construction

order books

- currently

firm's employment

- next 3 months

production

- last 3 months

expected level of

major purchases

- next 12 months

expected orders

with suppliers

- next 3 months

building activity

- last 3 months

current ESI improvement ESI (new questions)

coincident 0.72 7% 0.77

correlations with GDP q-o-q:

leading 1 0.49 10% 0.54

8

step b: an ESI with amplified changes

Intuition of the approach:

4. Improving the ESI's tracking performance of q-o-q GDP growth

Comparable changes in the ESI should be taken more "seriously", when reflected

by many survey questions.

Example:

-2

-1.5

-1

-0.5

0

0.5

change in 1999Q1 (compared to previous quarter)

change in 2000Q4 (compared to previous quarter)

Change in modified ESI: -1.8

Change in modified ESI: -2

Instead:

change in ESI

should be

-2*x (with x > 1)

standard deviation of balance series

>>>> change in ESI should be multiplied, if a critical amount of questions changes

in the same direction we propose:

8 (out of 11) questions we propose:

multiplication by 3

9

Calculation method for the experimental ESI

Ingredients: balance series from 11 EU BCS questions see slide 7:

i) make sum of all balance series

v) re-calculate series A:

questions expressed as:

quarterly averages

standardised

weighted (remember: industry questions get

40%, services questions 30%, etc.)

ii) calculate absolute q-o-q change in series A (series B)

(series A)

iii) make dummy-variable taking value 1 if, in a given quarter, >=8 balance series

increase/decrease (series C)

iv) re-calculate q-o-q changes: if series C = 1: series B * 3 if series C = 0: series B

new series

shows q-o-q

changes, which

are, if applicable,

amplified by 3

(series D)

for quarter q: series A (q-1) + series D (q)

"classical" sum of balance

series (previous quarter)

amplified q-o-q change of current

quarter (unamplified if not enough

questions move up/down)

vi) standardise & re-scale the new series >>>>experimental ESI

4. Improving the ESI's tracking performance of q-o-q GDP growth – step b: an ESI with amplified changes

60

70

80

90

100

110

120

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

96

-q2

97

-q2

98

-q2

99

-q2

00

-q2

01

-q2

02

-q2

03

-q2

04

-q2

05

-q2

06

-q2

07

-q2

08

-q2

09

-q2

10

-q2

11

-q2

12

-q2

13

-q2

q-o-q (%) quarterly averages ESI, PMI, experimental ESI and GDP growth;

euro-area (1996q2-2013q2)

GDP growth (rhs)

Composite PMI

experimental ESI

10

4. Improving the ESI's tracking performance of q-o-q GDP growth – step b: an ESI with amplified changes

Results:

60

70

80

90

100

110

120

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

96

-q2

97

-q2

98

-q2

99

-q2

00

-q2

01

-q2

02

-q2

03

-q2

04

-q2

05

-q2

06

-q2

07

-q2

08

-q2

09

-q2

10

-q2

11

-q2

12

-q2

13

-q2



GDP growth (rhs)

ESI

Composite PMI

experimental ESI

60

70

80

90

100

110

120

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

96

-q2

97

-q2

98

-q2

99

-q2

00

-q2

01

-q2

02

-q2

03

-q2

04

-q2

05

-q2

06

-q2

07

-q2

08

-q2

09

-q2

10

-q2

11

-q2

12

-q2

13

-q2



GDP growth (rhs)

ESI

experimental ESI

"micro"-volatility of

GDP better captured

2008-downturn announced

with steeper slope

2009-upswing reflected

with steeper slope

"micro"-volatility of GDP better

captured

Better leading

properties

60

70

80

90

100

110

120

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

96

-q2

97

-q2

98

-q2

99

-q2

00

-q2

01

-q2

02

-q2

03

-q2

04

-q2

05

-q2

06

-q2

07

-q2

08

-q2

09

-q2

10

-q2

11

-q2

12

-q2

13

-q2



GDP growth (rhs)

ESI

Composite PMI

experimental ESI

time-period current ESI PMI ESINEW (ampl.)

increase

compared to

current ESI

increase

compared to

PMI

98Q3 – 08Q1 0.61 (0.38) 0.77 (0.55) 0.76 (0.56) 0% (2%)

08Q2 – 13Q2 0.69 (0.31) 0.86 (0.56) 0.89 (0.71) 28% (131%) 3% (28%)

98Q3 – 13Q2 0.75 (0.50) 0.86 (0.65) 0.89 (0.75) 19% (48%) 3% (14%)

Correlation with GDP growth q-o-q (in brackets: leading 1 correlations)

26% (47%)

11


step a: Model selection (based on regression analyses):

exp. ESI is based on quarterly averages of BCS questions

>>>>any nowcast using exp. ESI can only be produced at the end of a

given quarter (when quarterly average of BCS questions is known)

several hard-data are available at the end of a given quarter and can thus be

included in the now-casting model:

Δ index of industrial production

still 45 days

before first

GDP release!

Δ index of production in construction

Δ index of turnover in retail trade

(except of motor

vehicles/motorcycles) Δ value of euro-area exports

(extra- and intra-euro-area) unemployment rate Δ new passenger car registrations

VStoxx (measure of volatility on European stock markets)

Calculation of quarterly changes (Δ):

i) in-sample:

for a quarter q:

(monthly average over quarter q)

(monthly average over quarter q-1)

ii) out-of-sample:

for a quarter q:

(month 1 value of quarter q)

(month1 value of quarter q-1)

12

constant constant constant constant constant

Benchm'k

Model(s)

constant constant

GDP(-1) exp. ESI exp. ESI exp. ESI exp. ESI exp. ESI exp. ESI

GDP(-2) ΔIPI ΔIPI ΔIPI ΔIPI

GDP(-3) ΔIPC ΔIPC ΔIPC

VSt'x(-1) VSt'x(-1) VSt'x(-1)

ΔReta

Δexports

Unempl.

New Model(s)

Δcar reg.

R2: 0.49 R2: 0.93

R2: 0.48 R2: 0.80 R2: 0.91 R2: 0.84 R2: 0.90 R2: 0.90 R2: 0.88

5. The nowcasting performance of the experimental ESI – step a: Model selection (based on regression analyses)

13

time-period GDP(-1) exp. ESI

exp. ESI

ΔIPI

ΔIPC

exp. ESI

ΔIPI

VSt'x(-1)

exp. ESI

ΔIPI

MAE: 0.39 0.27 0.21 0.20 0.21

step b: out-of-sample properties (simulated real-time scenario):

04q1 - 08q1 0.18 0.16 0.15 0.16 0.15

08q2 - 09q4 1.10 0.49 0.42 0.41 0.42

10q1 - 13q2 0.28 0.28 0.16 0.15 0.17

RMSE: 0.67 0.36 0.29 0.27 0.26

04q1 – 08q1 0.22 0.19 0.20 0.20 0.18

08q2 – 09q4 1.45 0.61 0.51 0.43 0.44

10q1 – 13q2 0.34 0.34 0.22 0.23 0.23

hit ratio 1: 32/38 33/38 35/38 36/38 34/38

04q1 – 08q1 17/17 17/17 17/17 17/17 17/17

08q2 – 09q4 5/7 6/7 7/7 6/7 6/7

10q1 – 13q2 10/14 10/14 11/14 13/14 11/14

Hit ratio 2: 14/37 26/37 28/37 30/37 29/37

04q2 – 08q1 5/16 11/16 9/16 11/16 11/16

08q2 – 09q4 5/7 7/7 7/7 7/7 6/7

10q1 – 13q2 4/14 8/14 12/14 12/14 12/14

Observations: AR-model's nowcasts on

average 0.40 pp off actual

GDP (>>bad)

AR-model: MAE/RMSE

highest in crisis-period,

lowest in pre-crisis period

exp. ESI model: clearly

outperforming AR-model;

added value of exp. ESI lies

in crisis period (logical:

amplification technique

captures "momentum")

addition of hard-data helps in

terms of:

MAE/RMSE;

hit ratios

improvements mainly in

calm times (post-crisis)

0.39 0.27 0.20

0.67 0.36 0.26

33/38 36/38

30/37

0.49 1.10

0.15

1.45 0.61

10/14

8/14

13/14

12/14

0.22

26/37

0.34

+3

+4

+3

+4

0.28

-46%

-55%

-26%

-46%

-31%

-58%

-35%

-28%

best model is "exp. ESI, IPI,

Vstoxx(-1)":

lowest MAE

highest hit-ratio 1

highest hit-ratio 2


14

Comparative advantage of new model most obvious in:

last 2 years (correct sign of GDP just once missed, although GDP

hovered around 0)

-4.0

-3.5

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

08-q

1

08-q

2

08-q

3

08-q

4

09-q

1

09-q

2

09-q

3

09-q

4

10-q

1

10-q

2

10-q

3

10-q

4

11-q

1

11-q

2

11-q

3

11-q

4

12-q

1

12-q

2

12-q

3

12-q

4

13-q

1

13-q

2

q-o-q (%)

Nowcasts of GDP growth produced by bridge models; euro area (2008q1-2013q2)

observed GDP growth model using GDP(-1) model using exp ESI, IPI, Vstoxx(-1)

Note: The nowcasts are produced by a pseudo out-of-sample exercise using soft- and hard data downloaded on 30/08/13. The in-sample period for the nowcasts is 1999q1 to (incl.) the quarter preceding the quarter to be nowcast. Source: European Commission.

crisis-period

5. The nowcasting performance of the experimental ESI – step b: out-of-sample properties (simulated real-time scenario)

15

Conclusion:

BCS data can be used to construct indicator tracking q-o-q GDP growth

satisfactorily

Step 1: choose the BCS questions best correlated with q-o-q GDP growth

Experimental sentiment indicator produces better GDP nowcasts than

simple AR-model

Step 2: (artificially) amplify q-o-q changes of the sentiment indicator, in

case a vast majority of questions displays q-o-q changes in the same

direction

In combination with hard-data, the experimental sentiment indicator allows

producing GDP nowcasts of excellent quality

16

4. Improving the ESI's tracking performance of q-o-q GDP growth – step b: an ESI with amplified growth

Shortcomings of the approach

Is it possible to apply the same construction method to monthly

data?

Calculations are done with quarterly averages of BCS questions

>>>> indicator could only be published once a quarter

technically yes !

Will quarterly averages of the resulting monthly ESI-series

remain well-correlated with q-o-q GDP growth?

yes: even slightly higher

correlations with GDP q-o-q

correlation of the two

quarterly ESI series is at

0.97

However…

17

4. Improving the ESI's tracking performance of q-o-q GDP growth – step b: an ESI with amplified growth – Shortcomings of the approach

Problem of constructing ESI with amplified changes for

monthly data:

too high volatility

55

65

75

85

95

105

115

125

ESINEW - amplified changes (monthly levels) current ESI (monthly levels)

Sep 2009: in upswing-period, ESINEW with

amplified changes drops by 10 points

(=1 standard deviation)

18


If amplification is applied in t-1, but not in t, ESI will usually suggest a

drop in sentiment in t (also in case the underlying data continues the

upward/downward trend of t-1)

Source of volatility (note: amplifying changes does not only increase the

amplitude of the series, but also its volatility):

9596979899

100101102103104105

Jan Feb Mar Apr

unamplified ESI

ESI (amplified change): assuming8 questions go up in Feb+2 +2 +2

+2*3 = +6

This additional volatility improves the fit of our quarterly series, but

renders the monthly series TOO volatile.

Main reason for this difference: criterion for amplification is more

restrictive in case of quarterly set-up:

for quarterly: >= 8 questions must have gone up/down over 3

months-period (amplification in 63% of quarters)

for monthly: >= 8 questions must have gone up just one month

(amplification in 73% of the months)

19


Solution: When multiplying change of t (compared to t-1), the

resulting amplified change should be added to ESI for

month t, but also ESI of t+1 and t+2 (1/3 of the change

respectively should be added). Approach smoothens the monthly curve substantially…

55

65

75

85

95

105

115

125

ESINEW - amplified changes (monthly levels)

ESINEW - amplified changes distributed over 3 months (monthly levels)

When constructing quarterly averages, correlations

with q-o-q GDP growth remain high.

an experimental sentiment indicator for the euro area ... · european commission directorate...

Documents