sharing best practices for the redesign of three business surveys
DESCRIPTION
Sharing best practices for the redesign of three business surveys. Charles Tardif, Business Survey Methods Division,Statistics Canada presented at the ICES-III, session 75 June 21 st , 2007. Alternative title. Can we design a monthly Unified Enterprise Survey?. Outline. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/1.jpg)
1
Sharing best practices for the Sharing best practices for the redesign of three business redesign of three business
surveyssurveys
Charles Tardif, Business Survey Methods Division,Statistics Canada
presented at the ICES-III, session 75
June 21st, 2007
![Page 2: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/2.jpg)
2
Alternative titleAlternative title
Can we design a monthly Unified Enterprise Survey?
![Page 3: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/3.jpg)
3
Outline
What are monthly business surveys? – a simplified view
Common objectives of monthly business surveys
Statistics Canada (STC) monthly business surveys:
Monthly Survey of Manufacturers (MSM)
Monthly Survey of the Food Services and Drinking Places (MFS)
Monthly Wholesale and Retail Trade Survey (MWRTS)
![Page 4: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/4.jpg)
4
Outline (cont’d)
Commonalities between the surveys
Comparing survey elements
Summary – A monthly unified enterprise survey?
Conclusion
![Page 5: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/5.jpg)
5
What are monthly business surveys? – a simplified view
Production of monthly estimates mainly on financial information (sales, revenues, expenses, etc):
By geographic level
By industrial level, classified by the North American Industrial Classification System (NAICS)
With a targetted quality, expressed in terms of Coefficients of Variation (CV).
![Page 6: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/6.jpg)
6
Common objectives of monthly business surveys
Measure trends and levels for key financial variablesEstablish best possible survey design to meet the surveys’ objectivesMeet the 6 elements of the STC Quality Framework:
Relevance, accuracy, timeliness, accessibility, interpretability, coherence
Maximise use of administrative data, mainly tax data, to reduce response burden
![Page 7: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/7.jpg)
7
STC monthly business surveys
MSMManufacturers
MFSRestaurants
MWRTS Retail
MWRTS Wholesale
Variables of interest
Shipments, inventories and orders
Sales,
Number of locations
Sales,
Number of locations
Sales, Inventories
Geo level Provinces, territories
Provinces, territories
Provinces, territories
Provinces, territories
Industrial level (NAICS)
311 to 339, at the 3rd to 6th digit
722, at the 4th digit
44 and 45, for 19 trade groups (TG)
41, for 15 trade groups (TG)
![Page 8: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/8.jpg)
8
STC monthly business surveys (cont’d)
MSMManufacturers
MFSRestaurants
MWRTS Retail
MWRTS Wholesale
Population size(establishments or clusters of est.)
100,000 90,000 180,000 100,000
Collection units 11,000 2,100 12,000 8,000
Number of domains
More than 1,000
52 247 possible domains (19 TGs)
195 possible domains (15 TGs)
Yearly revenues (in billions)
550 36 350 450
![Page 9: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/9.jpg)
9
Commonalities between the Commonalities between the surveyssurveys
All surveys:All surveys:share common objectivesshare common objectivesundertook or completed a redesign or a undertook or completed a redesign or a restratification in the last 2 yearsrestratification in the last 2 yearsare facing pressure to make a more extensive are facing pressure to make a more extensive use of Tax datause of Tax datahave skewed populations have skewed populations have an annual counterpart, all integrated in have an annual counterpart, all integrated in the Unified Enterprise Survey (UES)the Unified Enterprise Survey (UES)are managed by different subject matter are managed by different subject matter divisions, although centralized methodological divisions, although centralized methodological support.support.
![Page 10: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/10.jpg)
10
Commonalities between the Commonalities between the surveyssurveys
One monthly unified surveyOne monthly unified survey??We’ll see what has been done so far to We’ll see what has been done so far to harmonize the different surveys.harmonize the different surveys.
![Page 11: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/11.jpg)
11
Comparing survey elements Comparing survey elements
MSM Manufacturers
MFS Restaurants
MWRTS Wholesale, Retail
Frame
Before BR, no exclusions
Business Register (BR), with the exclusions of the non-employing establishments
Now BR, no exclusions, using a Survey specific Universe Frame
Sampling
Unit
Before Establishment level
Company level
Now Establishment level or cluster of establishments
![Page 12: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/12.jpg)
12
Comparing survey elementsComparing survey elements
MSM Manufacturers
MFS Restaurants
MWRTS Wholesale,
Retail
Stratification variables
Before Province/territory, NAICS and a size measure: Gross Business Income (GBI) from the BR
Now Province/territory, NAICS and a size measure: annualized monthly data, data from annual
survey, tax data, GBI
Take-none stratum
Before Bottom 2% by province
No take-none stratum
5% by geo and TGs
Now Bottom 10% (MSM, MFS) or 5% (MWRTS) by province and stratification NAICS, subject to a
cap.
![Page 13: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/13.jpg)
13
Comparing survey elementsComparing survey elements
Stratification – Now:Stratification – Now: Differences between surveys, but all:Differences between surveys, but all:
are using the Lavallée-Hidiroglou algorithm are using the Lavallée-Hidiroglou algorithm (efficient with skewed population) to stratify (efficient with skewed population) to stratify their population with the following their population with the following characteristics:characteristics:
1 Take-all / must-take stratum by prov/NAICS1 Take-all / must-take stratum by prov/NAICS1 or a few take-some strata by prov/NAICS1 or a few take-some strata by prov/NAICSMinimum sample size per stratumMinimum sample size per stratumCapped design weightCapped design weightOversampling for out-of-scopes, deaths and non-Oversampling for out-of-scopes, deaths and non-responseresponse
![Page 14: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/14.jpg)
14
Comparing survey elementsComparing survey elements
Use of GST - Now:Use of GST - Now:All surveys are making use of GST data, All surveys are making use of GST data, although in a different way:although in a different way:
MSM and MWRTS: MSM and MWRTS: Micro approach, with selected units replaced by Micro approach, with selected units replaced by GST data through modeling. Units have a design GST data through modeling. Units have a design weight > 1.weight > 1.
Use of GST data varies by NAICS for MSMUse of GST data varies by NAICS for MSM
Standard stratification design explained in the Standard stratification design explained in the previous slide is usedprevious slide is used
![Page 15: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/15.jpg)
15
Comparing survey elements
GST micro approach, selected units
Take-somes 1
Take-somes 2
Take-nones
Take-alls
S1 S2
![Page 16: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/16.jpg)
16
Comparing survey elementsComparing survey elements
Use of GST - Now:Use of GST - Now:All surveys are making use of GST data, All surveys are making use of GST data, although in a different way:although in a different way:
MFS: MFS: Micro approach, where a sample of units known as Micro approach, where a sample of units known as simples are selected to model the value of all the simples are selected to model the value of all the other simple units. All units have a weight of 1.other simple units. All units have a weight of 1.
Independent stratification design for these simple Independent stratification design for these simple units.units.
Applied for selected combinations of provinces and Applied for selected combinations of provinces and NAICS.NAICS.
![Page 17: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/17.jpg)
17
Comparing survey elements
GST micro approach, all units modelledTake-alls
Take-nones
Simples
Com
plex
uni
ts
![Page 18: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/18.jpg)
18
Comparing survey elementsComparing survey elements
MSM Manufacturers
MFS Restaurants
MWRTS Wholesale, Retail
Sample selection
STC Generalized Sampling (GSAM) system for sample selection
Random sampling within each stratum
Random sampling within each stratum and for the GST modelling
Systematic sampling within each stratum
Same sample, sampling births every month
![Page 19: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/19.jpg)
19
Comparing survey elementsComparing survey elements
MSM Manufacturers
MFS Restaurants
MWRTS Wholesale,
Retail
Edit and imputation
BeforeMSM in-house E&I program
MFS in-house E&I program
MWRTS in-house E&I program
Now
BANFF, a STC generalized system for E&I
MWRTS in-house E&I program
All are doing outlier detection, historical edits, trend and mean imputation
![Page 20: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/20.jpg)
20
Comparing survey elements
MSM Manufacturers
MFS Restaurants
MWRTS Wholesale, Retail
Estimation STC Generalized Estimation System (GES) used to produce the estimates.
Variance -Sampling variance computed using GES-Computing variability from other sources (imputation, use of admin data) being considered.
Publication -Estimates published in The Daily, a STC publication.-Estimates available in CANSIM.
![Page 21: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/21.jpg)
21
Summary - A monthly unified Summary - A monthly unified enterprise survey? enterprise survey?
AdvantagesAdvantagesTaking advantage of the best practices of Taking advantage of the best practices of each survey to integrate all monthly each survey to integrate all monthly surveyssurveys
Easier to ensure coherence between the Easier to ensure coherence between the monthly surveys – and their annual monthly surveys – and their annual counterpart, for comparison purposescounterpart, for comparison purposes
Annual surveys are already integratedAnnual surveys are already integrated
![Page 22: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/22.jpg)
22
Summary - A monthly unified Summary - A monthly unified enterprise survey? enterprise survey?
Advantages (cont’d)Advantages (cont’d)Would facilitate the implementation of Would facilitate the implementation of changes to the surveys (no need for a changes to the surveys (no need for a guinea pig):guinea pig):
For example, introduction of the GST was For example, introduction of the GST was done at different times for the 3 surveysdone at different times for the 3 surveys
Implementation of better measures of Implementation of better measures of variability could be done more efficiently:variability could be done more efficiently:
Integrating more components of variabilityIntegrating more components of variability
Variance of the differences between the estimatesVariance of the differences between the estimates
![Page 23: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/23.jpg)
23
Summary - A monthly unified Summary - A monthly unified enterprise survey? enterprise survey?
Issues - MethodologicalIssues - MethodologicalAs seen, there are conceptual differences As seen, there are conceptual differences between the surveys:between the surveys:
Different data elements collectedDifferent data elements collected
Level at which the information is collected is Level at which the information is collected is differentdifferent
Use of administrative data is differentUse of administrative data is different
Although these differences could be Although these differences could be factored in the sample designfactored in the sample design
![Page 24: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/24.jpg)
24
Summary - A monthly unified Summary - A monthly unified enterprise survey? enterprise survey?
Issues - OperationalIssues - OperationalCurrently managed by three different Currently managed by three different subject matter divisions, but centralized subject matter divisions, but centralized methodological supportmethodological support
Impact on the field workImpact on the field work
![Page 25: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/25.jpg)
25
ConclusionConclusion
Past:Past:Used to be important differences between the Used to be important differences between the monthly surveysmonthly surveys
Present:Present:Important steps have been made to share best Important steps have been made to share best practices between the monthly surveys to practices between the monthly surveys to harmonize themharmonize them
Future:Future:Should we go one step further and integrate Should we go one step further and integrate them in a Monthly Unified Enterprise Survey?them in a Monthly Unified Enterprise Survey?
![Page 26: Sharing best practices for the redesign of three business surveys](https://reader035.vdocuments.net/reader035/viewer/2022081517/5681591e550346895dc64809/html5/thumbnails/26.jpg)
26
For more information, please contact
Pour plus d’information, veuillez svp contacter