2_sources of errors in measurement
TRANSCRIPT
-
8/22/2019 2_Sources of Errors in Measurement
1/29
1
Types and Sources of Errors in Statistical
Data
-
8/22/2019 2_Sources of Errors in Measurement
2/29
2
Types of Errors
In general, there are two types of errors:
a. Non-sampling errors and
b. Sampling errors.
It is important for a researcher to be aware of these
errors, in particular non-sampling errors, so that they
can be either minimised or eliminated from the data
collected.
-
8/22/2019 2_Sources of Errors in Measurement
3/29
3
Non-sampling errors
These are errors that arise during the course of all
data collection activities.
In summary, they have the following
characteristics: exist in both sample surveys and censuses
data.
difficult to measure.
-
8/22/2019 2_Sources of Errors in Measurement
4/29
4
Sources of non-sampling errors
Non-sampling errors arise from:
defects in the sampling frame.
failure to identify the target population.
non response.
responses given by respondents.
data processing and
reporting, among others.
-
8/22/2019 2_Sources of Errors in Measurement
5/29
5
Defects in the sampling frame
This result in coverage errors.
These occur when there is an omission, duplicationor wrongful inclusion of units in the sampling frame.
Omissions are referred to as under coverage whileduplications and wrongful inclusions are called overcoverage.
These errors are caused by defects such asinaccuracy, incompleteness, duplication, inadequacy
and out of date sampling frames.
Coverage errors may also occur in field operations,that is, when an enumerator misses severalhouseholds or persons during the interviewing
process.
-
8/22/2019 2_Sources of Errors in Measurement
6/29
6
Failure to Identify Target Population
This occurs when the target population is not clearly
defined through the use of imprecise definitions or
concepts or when the survey population does not
reflect the target population due to an inadequatesampling frame and poor coverage rules.
-
8/22/2019 2_Sources of Errors in Measurement
7/29
7
Response
They result from the data that have been requested,
provided, received or recorded incorrectly.
They may occur as a result of inefficiencies with
the questionnaire, the interviewer,
the respondent or
the survey process.
-
8/22/2019 2_Sources of Errors in Measurement
8/29
8
a. Poor questionnaire design
The content and wording of the questionnaire may
be misleading and the layout of the questionnaire
may make it difficult to accurately record responses.
As a rule, questions in questionnaire should not beloaded, double-barrelled, misleading or ambiguous,
and should be directly relevant to the objectives of
the survey.
It is essential to pilot test questionnaires to identifyquestionnaire flow and question wording problems,
and allow sufficient time for improvements to be
made to the questionnaire.
-
8/22/2019 2_Sources of Errors in Measurement
9/29
9
Poor questionnaire design contd
The questionnaire should then be re-tested to
ensure changes made do not introduce other
problems.
-
8/22/2019 2_Sources of Errors in Measurement
10/29
10
b. Interviewer bias
An interviewer may influence the way a respondent
answers survey questions.
To prevent this, interviewers must be trained to
remain neutral throughout the interviewing processand must pay close attention to the way they ask
each question.
-
8/22/2019 2_Sources of Errors in Measurement
11/29
11
c. Respondent errors
These arise through the respondent providing
inaccurate or wrong information.
They occur because of memory biases or
respondents giving inaccurate or false informationwhen they believe that they are protecting their
personal interests or integrity.
They can also arise from the way the respondent
interprets the questionnaire and the wording of theanswer that the respondent gives.
Careful questionnaire design and effective
questionnaire testing can overcome these problems
to some extent.
-
8/22/2019 2_Sources of Errors in Measurement
12/29
12
d. Problems with the survey process
Errors can also occur because of problems with the
actual survey process such as using proxy
responses, that is, taking answers from someone
other than the respondent or lacking control over the
survey procedure.
-
8/22/2019 2_Sources of Errors in Measurement
13/29
13
Non-Response Non-response results when data is not collected from
respondents.
The proportion of these non-respondents in the sample is calledthe non-response rate.
Non-response can be eithertotal orpartial.
Total non-response or unit non-response can arise if arespondent cannot be contacted (because the sampling frame isincomplete or out-of-dated) or the respondent is not at home or is
unable to respond because of language difficulties or illness orout rightly refuses to answer any questions or the dwelling unit isvacant.
Other respondents may indicate that they simply don't have the
time to complete the interview or survey form.
-
8/22/2019 2_Sources of Errors in Measurement
14/29
14
Non-response - contd
When conducting surveys it is important to document information
on why a respondent has not responded.
Partial non-response or item non-response can occur when a
respondent replies to some but not all questions of the survey.
This can arise due to memory problems, inadequate information
or an inability to answer a particular question/section of the
questionnaire.
A respondent may refuse to answer if;
a. they find questions particularly sensitive, or if
b. they have been asked too many questions.
-
8/22/2019 2_Sources of Errors in Measurement
15/29
15
Non-response - contd
To reduce non-response, the following approaches can beused:
care should be taken in questionnaire design through the use
of simple questions.
pilot testing of the questionnaire.
explaining survey purposes and uses.
assuring confidentiality of responses.
public awareness activities including discussions with key
organisations and interest groups, news releases, mediainterview and articles.
-
8/22/2019 2_Sources of Errors in Measurement
16/29
16
Processing
These occur at various stages of data processing such as
data cleaning, data capture and editing.
Data cleaning involves taking preliminary checks before
entering the data onto the processing system.
Coder bias is usually a result of poor training or incomplete
instructions, variability in coder performance and data entry
errors.
-
8/22/2019 2_Sources of Errors in Measurement
17/29
17
Processing contd
Inadequate checking and quality management at this stage
can introduce data loss (where data is not entered into the
system) and data duplication (where the same data is entered
into the system more than once) thus introducing errors in
data.
To minimise these errors, processing staff should be given
adequate training, instructions and realistic workloads.
-
8/22/2019 2_Sources of Errors in Measurement
18/29
18
Time Period Bias
This occurs when a survey is conducted during an
unrepresentative time period.
Survey timing is thus important and failure torecognise this introduces errors in data.
-
8/22/2019 2_Sources of Errors in Measurement
19/29
19
Analysis and Estimation
Analysis errors include any errors that occur whenusing wrong analytical tools or when preliminaryresults are used instead of the final ones.
Errors that occur during the publication of the dataresults are also considered as analysis errors.
Estimation errors occur when inappropriate orinaccurate weights are used in the estimationprocedure thus introducing errors to the data.
They also occur when wrong estimators areselected by the analyst.
-
8/22/2019 2_Sources of Errors in Measurement
20/29
20
Reducing non-sampling errors
Can be minimised by adopting any of the following
approaches:
using an up-to-date and accurate sampling frame.
careful selection of the time the survey isconducted.
planning for follow up of non-respondents.
careful questionnaire design.
providing thorough training and periodic retraining
of interviewers and processing staff.
-
8/22/2019 2_Sources of Errors in Measurement
21/29
21
Reducing non-sampling errors contd
- designing good systems to capture errors that occurduring the process of collecting data, sometimes
called Data Quality Assurance Systems.
-
8/22/2019 2_Sources of Errors in Measurement
22/29
22
Sampling error
Refers to the difference between the estimate derived from asample survey and the 'true' value that would result if a
census of the whole population were taken under the same
conditions.
These are errors that arise because data has been collected
from a part, rather than the whole of the population.
Because of the above, sampling errors are restricted to
sample surveys only unlike non-sampling errors that canoccur in both sample surveys and censuses data.
-
8/22/2019 2_Sources of Errors in Measurement
23/29
23
Sampling errors contd
There are no sampling errors in a census because
the calculations are based on the entire population.
They are measurable from the sample data in thecase of probability sampling.
-
8/22/2019 2_Sources of Errors in Measurement
24/29
24
Factors Affecting Sampling Error
It is affected by a number of factors including:
a. sample size
In general, larger sample sizes decrease the sampling error,however this decrease is not directly proportional.
As a rough rule of the thumb, you need to increase the samplesize fourfold to halve the sampling error but bear in mind thatnon sampling errors are likely to increase with large samples.
b. the sampling fraction.
this is of lesser influence but as the sample size increases as afraction of the population, the sampling error should decrease.
-
8/22/2019 2_Sources of Errors in Measurement
25/29
25
Factors Affecting Sampling Error contd
c. the variability within the population.
More variable populations give rise to larger errors as the
samples or the estimates calculated from different samples
are more likely to have greater variation.
The effect of variability within the population can be reduced
by the use of stratification that allows explaining some of thevariability in the population.
d. sample design.
An efficient sampling design will help in reducing samplingerror.
-
8/22/2019 2_Sources of Errors in Measurement
26/29
26
Characteristics of the sampling error
generally decreases in magnitude as the sample size
increases (but not proportionately).
depends on the variability of the characteristic of interest in the
population.
can be accounted for and reduced by an appropriate sample
plan.
can be measured and controlled in probability sample surveys.
-
8/22/2019 2_Sources of Errors in Measurement
27/29
27
Reducing sampling error
If sampling principles are applied carefully within the
constraints of available resources, sampling error
can be kept to a minimum.
-
8/22/2019 2_Sources of Errors in Measurement
28/29
28
Potential Sources of Error in Research Designs
Surrogate Information Error
Measurement Error
Population Definition Error
Sampling Frame Error
Data Analysis Error
Respondent Selection Error
Questioning Error
Recording Error
Cheating Error
Inability Error
Unwillingness Error
Total Error
Non-sampling
Error
Random
Sampling Error
Non-responseError
ResponseError
Interviewer
Error
Respondent
Error
Researcher
Error
-
8/22/2019 2_Sources of Errors in Measurement
29/29
29
End of Topic