I International Course of I International Course of NeuroepidemiologyNeuroepidemiologyin Eastern Europein Eastern Europe

Chisinau, Moldova, September 23-28Chisinau, Moldova, September 23-28

Population Surveys in Moldova: Population Surveys in Moldova:

source of datasource of data

Maurizio A. LeoneMaurizio A. Leone..

Clinica Neurologica, Clinica Neurologica,

““Maggiore della Carità” Hospital, Maggiore della Carità” Hospital,

Novara, Italy.Novara, Italy.

Objectives of the presentationObjectives of the presentation

Define the Define the sources of datasources of data for different types of for different types of epidemiological studiesepidemiological studies

Understand the advantages and disadvantages Understand the advantages and disadvantages of different types of data sourcesof different types of data sources

Understand the importance of Understand the importance of representativenessrepresentativeness

Understand the importance of evaluating the Understand the importance of evaluating the quality of data quality of data

RELATIVE RISK

Cases

Controls

RATIO / RATE

Cases (time)

______________

Population

ODDS RATIO

Cases / Controls

OUTCOME

Sources of data in Epidemiology: Sources of data in Epidemiology: current datacurrent data

Death certificatesDeath certificatesNotification of infectious diseasesNotification of infectious diseasesHospital admission/discharge archivesHospital admission/discharge archivesExemptions codes for specific diseasesExemptions codes for specific diseasesDrug prescriptions archivesDrug prescriptions archivesReports of accidents at work (Ministry of Reports of accidents at work (Ministry of

Welfare)Welfare)Reports of professional diseasesReports of professional diseases

Temporal trend of cancer incidence in USA 1973-1999: annual Temporal trend of cancer incidence in USA 1973-1999: annual rates, age-standardizedrates, age-standardized

FemalesMales

World Incidence of melanoma World Incidence of melanoma (Age-adjusted rates/100.000)(Age-adjusted rates/100.000)

John Graunt (1662)John Graunt (1662) - mortality and - mortality and birth rates in Londonbirth rates in LondonWilliam Farr (1800)William Farr (1800) - collecting death - collecting death certificates with indication of the certificates with indication of the causecauseJohn Snow (1854)John Snow (1854) - causes of the - causes of the cholera outbreak in Londoncholera outbreak in London

Are routinely collected, by area of residenceAre routinely collected, by area of residence Can be used for descriptive and analytical epidemiological studiesCan be used for descriptive and analytical epidemiological studies Allow for epidemiological assessments in a short time and with Allow for epidemiological assessments in a short time and with

limited resourceslimited resources Are an acceptable substitute of incidence for several diseasesAre an acceptable substitute of incidence for several diseases Constitute an additional source of data source for disease registriesConstitute an additional source of data source for disease registries Used together with incidence data allow to estimate the prevalenceUsed together with incidence data allow to estimate the prevalence Have a good level of intra-and supra-national comparabilityHave a good level of intra-and supra-national comparability Allow to assess survival by disease or by medical intervention and Allow to assess survival by disease or by medical intervention and

then to carry out assessments of the quality of carethen to carry out assessments of the quality of care For treatable or preventable diseases allow to estimate the For treatable or preventable diseases allow to estimate the

"avoidable deaths" due to a lack of action on the NHS"avoidable deaths" due to a lack of action on the NHS

Mortality data: advantagesMortality data: advantages

Are based on the most significant morbid condition that caused the deathAre based on the most significant morbid condition that caused the death Cannot be used for diseases with low lethalityCannot be used for diseases with low lethality Great variability in the certification or coding for some causes with low lethality, but Great variability in the certification or coding for some causes with low lethality, but

possible high lethality complications (diabetes mellitus, supraventricular possible high lethality complications (diabetes mellitus, supraventricular arrhythmias, hypertension, etc.). arrhythmias, hypertension, etc.).

Not very accurate for highly specific diseases (astrocytoma vs. glioblastoma, ....)Not very accurate for highly specific diseases (astrocytoma vs. glioblastoma, ....) Possible discrepancies in interpretation between local and cental codifiersPossible discrepancies in interpretation between local and cental codifiers The interpretation of differences in mortality (areas, periods) must be cautious for The interpretation of differences in mortality (areas, periods) must be cautious for

possible systematic errors (differences in completeness or accuracy), random possible systematic errors (differences in completeness or accuracy), random errors (due to small numbers), survival differences, differences in the incidenceerrors (due to small numbers), survival differences, differences in the incidence

Chronic disease mortality refers to events that have their diagnosis at a unknown Chronic disease mortality refers to events that have their diagnosis at a unknown time in the past and therefore, when used as a surrogate for incidence, relate to time in the past and therefore, when used as a surrogate for incidence, relate to events more or less remote.events more or less remote.

Mortality data: limitsMortality data: limits

Sources of data in Epidemiology: Sources of data in Epidemiology: ad hoc dataad hoc data

Registers (tumors, specific disease or group Registers (tumors, specific disease or group of diseases)of diseases) Clinical data banksClinical data banks Ad hoc epidemiological studiesAd hoc epidemiological studies

REGISTERSREGISTERS

Population-basedPopulation-based Multiple sources of casesMultiple sources of cases Known denominatorKnown denominator Possible to calculate Possible to calculate

rates/ratiosrates/ratios RepresentativenessRepresentativeness Few clinical dataFew clinical data

DATA-BASE (BANKS)DATA-BASE (BANKS)

Hospital-basedHospital-based Single source of casesSingle source of cases Unknown denominatorUnknown denominator Impossible to calculate Impossible to calculate

rates/ratiosrates/ratios UnrepresentativenessUnrepresentativeness Large amount of clinical dataLarge amount of clinical data

All incident /prevalent

cases

HospitalOut-patients

Archives

Exemption files

Admission/Discharge Archives

Drug Prescription

Archives

Death certificates

Retirement Archives

Others

EEG/EMG/EP

CT/MRI

HospitalIn-patients Archives

General Practitioners

STRENGHTS:STRENGHTS: Large numbers ------ high statistical power.Large numbers ------ high statistical power. Population-based.Population-based. “ “Real-world” practice patterns.Real-world” practice patterns. Versatile information (sociodemographic and Versatile information (sociodemographic and economic).economic). Data readily available and inexpensive to be Data readily available and inexpensive to be acquired. acquired.

WEAKNESSES:WEAKNESSES: Data are collected for reasons other than research.Data are collected for reasons other than research. The data-base might not contain critical data The data-base might not contain critical data elements.elements. Quality of data collected often is not validated.Quality of data collected often is not validated.

It is possible to use It is possible to use administrative administrative data-bases to evaluate the quality data-bases to evaluate the quality

of care for stroke patients ?of care for stroke patients ?

The feasibility of studies relying on The feasibility of studies relying on administrative data-bases, using ICD administrative data-bases, using ICD codes depends on:codes depends on:

CompletenessCompleteness Accuracy (sensitivity and positive Accuracy (sensitivity and positive predictivepredictive

value)value) Variability (inter-hospital, inter-Variability (inter-hospital, inter-rater, ..) rater, ..)

……. of data recorded.. of data recorded.

Sistema Sistema Informativo Informativo Sanitario Sanitario

Regionale (SISR).Regionale (SISR).

ICD-9 (International Classification of

Diseases-9th Revision)Codes for cerebrovascular

diseases (430-438).

Search in the SISR of discharged records (dead or alive) with codes 430-438

either at primary or secondary positions (1017 patients in 5

departments).

Manual search in the same 5 Departments of any possible diagnosis of stroke or TIA, after review of the medical

record (1005 patients).

LinkageLinkage

Falsepositives

121

False negatives

109

Automatedlinkage679

Manuallinkage217

DIAGNOSIS of TIA-STROKE

430-438 as principal or secondary diagnosis

YES NO

YES

NO

Tot.

896 121

109

1017

Tot. 1005

Sensitivity (896/1005) = 89%Sensitivity (896/1005) = 89%

Positive predictive value (896/1017) = 88%Positive predictive value (896/1017) = 88%

Accuracy of ICD-9 codes.Accuracy of ICD-9 codes.

DIAGNOSIS of TIA-STROKE

430-438 as principal or secondary diagnosis

YES NO

YES

NO

Tot.

True + False +

False -

1017

Tot. 1005

Accuracy of ICD-9 codes.Accuracy of ICD-9 codes.

Evaluation of variability includes:Evaluation of variability includes:

Intra-rater agreement : variability of Intra-rater agreement : variability of single raters during timesingle raters during time

Inter-rater agreement : variability Inter-rater agreement : variability between different coders, including the between different coders, including the effect of their experience, their role and effect of their experience, their role and the coding practice of each hospital.the coding practice of each hospital.

Clinical records, including discharge letter with Clinical records, including discharge letter with diagnosis, of 53 consecutive patients admitted diagnosis, of 53 consecutive patients admitted to our Department. (27 stroke and 26 other to our Department. (27 stroke and 26 other diseases) were coded with the ICD-9-CM by 12 diseases) were coded with the ICD-9-CM by 12 physicians from 7 different Italian hospitals (4 physicians from 7 different Italian hospitals (4 residents,4 general neurologists and 4 residents,4 general neurologists and 4 experts).experts).

Each coder was blind to the original code and Each coder was blind to the original code and to the other codifiersto the other codifiers

For each pair we calculated the observed For each pair we calculated the observed agreement; inter-rater agreement was agreement; inter-rater agreement was evaluated by the kappa statistics.evaluated by the kappa statistics.

ICD-9 430-438 ICD-9 430-438 vs. any other vs. any other code), primary position only.code), primary position only.

R1 R2 R3 R4 N1 N2 N3 N4 E1 E2 E3 E4

R1 R2 0,7 R3 0,85 0,54 R4 0,81 0,66 0,81 N1 0,77 0,7 0,77 0,96 N2 0,89 0,73 0,74 0,85 0,89

N3 0,77 0,69 0,62 0,81 0,85 0,89 N4 0,89 0,73 0,74 0,85 0,89 1 0,89 E1 0,74 0,53 0,74 0,85 0,81 0,67 0,67 0,77 E2 0,92 0,77 0,77 0,81 0,7 0,96 0,85 0,96 0,74 E3 0,77 0,61 0,62 0,66 0,85 0,81 0,7 0,7 0,67 0,85 E4 0,73 0,96 0,57 0,62 0,66 0,7 0,65 0,7 0,5 0,73 0,57

< 0,00 POOR AGREEMENT< 0,00 POOR AGREEMENT

0,00 - 0,20 SLIGHT AGREEMENT0,00 - 0,20 SLIGHT AGREEMENT

0,21 - 0,40 FAIR AGREEMENT0,21 - 0,40 FAIR AGREEMENT

0,41 - 0,60 MODERATE AGREEMENT0,41 - 0,60 MODERATE AGREEMENT

0,61 - 0,80 SUBSTANTIAL 0,61 - 0,80 SUBSTANTIAL

AGREEMENTAGREEMENT

Above 0,80 EXCELLENT AGREEMENTAbove 0,80 EXCELLENT AGREEMENT

““Epidemiologist are investigators in Epidemiologist are investigators in search of a denominator”search of a denominator”

Area Population Time

Sources of cases: Sources of cases:

Cases should be a representative sample of all individuals with that particular disease in the study population.

Cases must be chosen without knowing the exposure of interest Cases can be selected a priori on the basis of the hypothesis

formulated

Purpose of the comparison group Purpose of the comparison group

Case control study:

“The comparison group serves to provide an estimate

of the exposure distribution in the source population

from which the cases originate.”Rothman KJ, 1986

Who are the right controls? Who are the right controls?

Conceptual questionsConceptual questions Comparable to the Comparable to the

cases ?cases ? Representative of all non Representative of all non

diseased peoplediseased people

Practical QuestionsPractical QuestionsResourcesResourcesTimeTimeSample sizeSample size

CasesExposed

Unexposed

Source population

Controls:• Sample of the denominator• Representative with regard to exposure

Controls

Sample

Sources of controlsSources of controls

a) Population of defined areaa) Population of defined area b) Hospital patientsb) Hospital patients c) Probability sample of total populationc) Probability sample of total population d) Neighborsd) Neighbors

(i) walk (door to door)(i) walk (door to door) (ii) phone (random digit dialing)(ii) phone (random digit dialing)

(iii) letter carrier routes(iii) letter carrier routes e) Friends or associates of casese) Friends or associates of cases f) Siblings, spouses or other relativesf) Siblings, spouses or other relatives g) Otherg) Other

Purpose of the comparison group

Cohort study:

“The comparison group serves to provide an estimate

of the expected disease incidence in the exposed

group if the exposure had been absent.”Rothman KJ, 1986

DA Grimes and KF SchulzCohort studies: marching towards outcomes The Lancet; 2002;359:341-345

Representativeness.? Has Representativeness.? Has "referral pattern""referral pattern" been described ?been described ?

Many prognostic studies in the literature come from Many prognostic studies in the literature come from large hospitals and University Centres (Level III) and large hospitals and University Centres (Level III) and patients followed in these centers are not a patients followed in these centers are not a representative sample of those that are followed in the representative sample of those that are followed in the community. For example, they can be referred to these community. For example, they can be referred to these centers because non-responders to a therapy or centers because non-responders to a therapy or because they have complications of the disease.because they have complications of the disease.The The selection biasselection bias is due to systematic error introduced is due to systematic error introduced in a study depending on the selection of the population in a study depending on the selection of the population investigated. investigated.

Copyright ©2006 BMJ Publishing Group Ltd.

Chio, A et al. J Neurol Neurosurg Psychiatry 2006;77:948-950

Figure 1 Survival curves of patients with amyotrophic lateral sclerosis (ALS) followed up by tertiary ALS centres (continuous line) and general neurology clinics (dotted line; p = 0.0008).

Selection of comparison groupSelection of comparison group Internal comparisonInternal comparison

Only one cohort identifiedOnly one cohort identified Later on, classified into study and comparison cohort based on Later on, classified into study and comparison cohort based on

exposureexposure

External comparisonExternal comparison More than one cohort identified More than one cohort identified e.g. Cohort of blue collars compared with white collarse.g. Cohort of blue collars compared with white collars

Comparison with general population ratesComparison with general population rates If no comparison group is available we can compare the rates If no comparison group is available we can compare the rates

of study cohort with general populationof study cohort with general population e.g. stroke rate of miners with stroke rate in the general e.g. stroke rate of miners with stroke rate in the general

populationpopulation

DeathDeath Clinical response (percentage improving)Clinical response (percentage improving) Complete recoveryComplete recovery Recurrence/relapseRecurrence/relapse Disease-freeDisease-free First occurrence of a disease or health-related First occurrence of a disease or health-related

outcomeoutcome Transition between states of health/diseaseTransition between states of health/disease Transitions between functional statesTransitions between functional states Surrogate outcomes (MRI, lab test, etc..)Surrogate outcomes (MRI, lab test, etc..)

MEASURES OF OUTCOME MEASURES OF OUTCOME (end points)(end points)

Sources of informationSources of information

Out-patient recordsOut-patient records Automated archives (admission/discharge)Automated archives (admission/discharge) Planned ad hoc medical visitsPlanned ad hoc medical visits Planned ad hoc laboratory/imaging testsPlanned ad hoc laboratory/imaging tests Cohort members: self-administered Cohort members: self-administered

questionnaires, interviews, telephone interviews, questionnaires, interviews, telephone interviews, mailed questionnairesmailed questionnaires

Multiple methodsMultiple methods

0

2

4

6

8

10

12

14

16

18

20

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

Barthel Index Score

Frequency

Telephone Face-to-face