an assessment of the the ber's manufacturing survey in south africa
DESCRIPTION
Firm weights are applied to the qualitative responses of participants to calculate business tendency survey (BTS) results. Sector weights are employed to produce higher levels of aggregation. What impact does weighting have on the accuracy of the BTS results?TRANSCRIPT
![Page 1: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/1.jpg)
The impact of weight adjustment on the
accuracy of business tendency surveys
An assessment of the manufacturing survey of South Africa
George Kershoff, BER
![Page 2: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/2.jpg)
What are business tendency surveys?
Use the same questionnaire
to collect qualitative information
from the same selection (panel) of firms.
![Page 3: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/3.jpg)
Use weights to calculate the survey results
Sample weights
Seize weights
Firm weights
Sector weights
![Page 4: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/4.jpg)
Firm weights
No. of
employees
Size class Firm
weight
Small
1 – 19 1 1
20 – 49 2 4
50 – 99 3 10
Large
100 – 199 4 19
200 – 299 5 34
300 – 399 6 48
400 – 499 7 62
500 – 999 8 94
1 000+ 9 286
![Page 5: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/5.jpg)
Firm weights
No. of
employees
Size class Firm
weight
Small
1 – 19 1 1
20 – 49 2 4
50 – 99 3 10
Large
100 – 199 4 19
200 – 299 5 34
300 – 399 6 48
400 – 499 7 62
500 – 999 8 94
1 000+ 9 286
![Page 6: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/6.jpg)
Sector weightsSector Weight %
Food 15.1
Beverages 4.7
Textiles 2.4
Clothing 2.3
Wood 2.3
Paper 4.6
Printing 4.0
Chemicals 8.7
Plastic 3.3
Non-metal minerals 3.3
Basic metals 7.6
Metal products 5.4
Machinery 6.0
Electrical machinery 4.5
Transport equipment 18.1
Furniture & other 5.3
Not published 2.2
Total excl petroleum 100.0
![Page 7: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/7.jpg)
Calculating the results in practiceSize
class
Firm
weight
Sector
weight
Total
weight
Up Same Down Total
M
e
a
t
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
C. Large 6 48 0.051 2.448 2.448
D.
Large8 94 0.051 4.794 4.794
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V
e
h
i
c
l
e
s
E. Small 3 10 0.106 1.060 1.060
F. Large 8 94 0.106 9.964 9.964
G. Large 8 94 0.106 9.964 9.964
Sum 11.024 0.000 9.964 20.988
% 53 0 47 100
T
o
t
Sum 11.228 2.958 14.758 28.944
% 39 10 51 100
![Page 8: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/8.jpg)
Calculating the results in practiceSize
class
Firm
weight
Sector
weight
Total
weight
Up Same Down Total
M
e
a
t
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
C. Large 6 48 0.051 2.448 2.448
D.
Large8 94 0.051 4.794 4.794
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V
e
h
i
c
l
e
s
E. Small 3 10 0.106 1.060 1.060
F. Large 8 94 0.106 9.964 9.964
G. Large 8 94 0.106 9.964 9.964
Sum 11.024 0.000 9.964 20.988
% 53 0 47 100
T
o
t
Sum 11.228 2.958 14.758 28.944
% 39 10 51 100
![Page 9: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/9.jpg)
Calculating the results in practiceSize
class
Firm
weight
Sector
weight
Total
weight
Up Same Down Total
M
e
a
t
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
C. Large 6 48 0.051 2.448 2.448
D.
Large8 94 0.051 4.794 4.794
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V
e
h
i
c
l
e
s
E. Small 3 10 0.106 1.060 1.060
F. Large 8 94 0.106 9.964 9.964
G. Large 8 94 0.106 9.964 9.964
Sum 11.024 0.000 9.964 20.988
% 53 0 47 100
T
o
t
Sum 11.228 2.958 14.758 28.944
% 39 10 51 100
![Page 10: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/10.jpg)
Calculating the results in practiceSize
class
Firm
weight
Sector
weight
Total
weight
Up Same Down Total
M
e
a
t
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
C. Large 6 48 0.051 2.448 2.448
D.
Large8 94 0.051 4.794 4.794
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V
e
h
i
c
l
e
s
E. Small 3 10 0.106 1.060 1.060
F. Large 8 94 0.106 9.964 9.964
G. Large 8 94 0.106 9.964 9.964
Sum 11.024 0.000 9.964 20.988
% 53 0 47 100
T
o
t
Sum 11.228 2.958 14.758 28.944
% 39 10 51 100
![Page 11: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/11.jpg)
Calculating the results in practiceSize
class
Firm
weight
Sector
weight
Total
weight
Up Same Down Total
M
e
a
t
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
C. Large 6 48 0.051 2.448 2.448
D.
Large8 94 0.051 4.794 4.794
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V
e
h
i
c
l
e
s
E. Small 3 10 0.106 1.060 1.060
F. Large 8 94 0.106 9.964 9.964
G. Large 8 94 0.106 9.964 9.964
Sum 11.024 0.000 9.964 20.988
% 53 0 47 100
T
o
t
Sum 11.228 2.958 14.758 28.944
% 39 10 51 100
![Page 12: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/12.jpg)
Average number of response per quarter
0
50
100
150
200
250
300
350
400
2001 2002 2003 2004 2005 2006 2007 2008 2009
Sector Total - Small (<100) Sector Total - Large
![Page 13: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/13.jpg)
Impact of non-responsesSize
class
Firm
weight
Sector
weight
Total
weight
Up Same Down Total
M
e
a
t
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
C. Large 6 48 0.051 2.448 2.448
D.
Large8 94 0.051 4.794 4.794
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V
e
h
i
c
l
e
s
E. Small 3 10 0.106 1.060 1.060
F. Large 8 94 0.106 9.964 9.964
G. Large 8 94 0.106 9.964 9.964
Sum 11.024 0.000 9.964 20.988
% 53 0 47 100
T
o
t
Sum 11.228 2.958 14.758 28.944
% 39 10 51 100
![Page 14: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/14.jpg)
Impact of non-responsesSize
class
Firm
weight
Sector
weight
Total
weight
Up Same Down Total
M
e
a
t
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
C. Large 6 48 0.051 2.448 2.448
D.
Large8 94 0.051 4.794 4.794
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V
e
h
i
c
l
e
s
E. Small 3 10 0.106 1.060 1.060
F. Large 8 94 0.106 9.964 9.964
G. Large 8 94 0.106 9.964 9.964
Sum 11.024 0.000 9.964 20.988
% 53 0 47 100
T
o
t
Sum 11.228 2.958 14.758 28.944
% 39 10 51 100
![Page 15: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/15.jpg)
Impact of non-responsesSize
class
Firm
weight
Sector
weight
Total
weight
Up Same Down Total
M
e
a
t
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
C. Large 6 48 0.051 2.448 2.448
D.
Large8 94 0.051 4.794 4.794
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V
e
h
i
c
l
e
s
E. Small 3 10 0.106 1.060 1.060
F. Large 8 94 0.106 9.964 9.964
G. Large 8 94 0.106 9.964 9.964
Sum 11.024 0.000 9.964 20.988
% 53 0 47 100
T
o
t
Sum 11.228 2.958 14.758 28.944
% 39 10 51 100
3.162
24.150
![Page 16: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/16.jpg)
Composition of the survey results vis-à-vis
the benchmark
2001 2005 2009 Average
Transport equipment
% of total
Benchmark
(domestic sales)15.3 18.1 15.3 17.5
Survey (sum of
responses)9.9 8.6 4.7 8.1
![Page 17: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/17.jpg)
Sector composition of survey results using
fixed weights
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%0
1Q
20
1Q
30
1Q
40
2Q
10
2Q
20
2Q
30
2Q
40
3Q
10
3Q
20
3Q
30
3Q
40
4Q
10
4Q
20
4q3
04
Q4
05
Q1
05
Q2
05
Q3
06
Q1
06
Q2
06
Q3
06
Q4
07
Q1
07
Q2
07
Q3
07
Q4
08
Q1
08
q2
08
Q3
08
Q4
09
Q1
09
Q2
09
Q3
09
Q4
Not publ -
Transp equip -
Elec machin -
Machinery -
Metal products -
Basic metals -
Non-metal min -
Chemicals -
Wood -
Clothing -
Textiles -
Beverages -
Food -
Plastics -
Printing -
Paper -
Furn & other -
![Page 18: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/18.jpg)
Weight adjustment
Adjust individual weights ex post to align the survey’s
sector composition with that of the reference series
Provide for
Over and under representation
Non-responses
![Page 19: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/19.jpg)
Weight adjustment in practiceSize
class
Firm
weight
Sector
weight
Total
weight
Up Same Down Total
M
e
a
t
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
C. Large 6 48 0.051 2.448 2.448
D.
Large8 94 0.051 4.794 4.794
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V
e
h
i
c
l
e
s
E. Small 3 10 0.106 1.060 1.060
F. Large 8 94 0.106 9.964 9.964
G. Large 8 94 0.106 9.964 9.964
Sum 11.024 0.000 9.964 20.988
% 53 0 47 100
T
o
t
Sum 11.228 2.958 14.758 28.944
% 39 10 51 100
3.162
24.150
![Page 20: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/20.jpg)
Sector composition after weight adjustment
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%0
1Q
20
1Q
30
1Q
40
2Q
10
2Q
20
2Q
30
2Q
40
3Q
10
3Q
20
3Q
30
3Q
40
4Q
10
4Q
20
4q3
04
Q4
05
Q1
05
Q2
05
Q3
06
Q1
06
Q2
06
Q3
06
Q4
07
Q1
07
Q2
07
Q3
07
Q4
08
Q1
08
q2
08
Q3
08
Q4
09
Q1
09
Q2
09
Q3
09
Q4
Not publ -
Transp equip -
Elec machin -
Machinery -
Metal prod -
Basic metals -
Non-metal min -
Chemicals -
Wood -
Clothing -
Textiles -
Beverages -
Food -
Plastics -
Printing -
Paper -
Furn & other -
![Page 21: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/21.jpg)
Survey results with and without weight
adjustment
-80
-60
-40
-20
0
20
40
60
80
100
Ma
r-01
Sep-0
1
Ma
r-02
Sep-0
2
Ma
r-03
Sep-0
3
Ma
r-04
Sep-0
4
Ma
r-05
Sep-0
5
Ma
r-06
Sep-0
6
Ma
r-07
Sep-0
7
Ma
r-08
Sep-0
8
Mar-
09
Sep-0
9
Ne
t %
= %
up
le
ss
% d
ow
n
Without weight adjustment (lhs) With weight adjustment (lhs)
![Page 22: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/22.jpg)
Adjusted survey results vs. reference series
-30
-20
-10
0
10
20
30
-100
-80
-60
-40
-20
0
20
40
60
80
100
Ma
r-01
Sep-0
1
Ma
r-02
Sep-0
2
Ma
r-03
Sep-0
3
Ma
r-04
Sep-0
4
Ma
r-05
Sep-0
5
Ma
r-06
Sep-0
6
Ma
r-07
Sep-0
7
Ma
r-08
Sep-0
8
Ma
r-09
Sep-0
9
Ye
ar
on
ye
ar
% c
ha
ng
e
Ne
t %
= %
up
le
ss
% d
ow
n
With weight adjustment (lhs) Reference series (rhs)
![Page 23: An assessment of the the BER's manufacturing survey in South Africa](https://reader034.vdocuments.net/reader034/viewer/2022051516/5595a71e1a28ab90168b47ea/html5/thumbnails/23.jpg)
Conclusion
Weight adjustment did not improve accuracy in the
case of the manufacturing sector between 2001 and
2009
Accuracy = closeness between survey results and
reference series
Survey results are therefore robust