evaluating data quality in the cancer registry
DESCRIPTION
Evaluating data quality in the Cancer Registry, Freddie Bray - Deputy Head, Section of Cancer Information IARCTRANSCRIPT
![Page 1: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/1.jpg)
Evaluating data quality in the Cancer Registry
Freddie Bray
Deputy Head, Section of Cancer Information
IARC
Dharmais Cancer Hospital · Jakarta · November 2010
![Page 2: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/2.jpg)
Cancer Incidence in Five Continents: Vol 1 (1966) Introduction
Reliable cancer registries:
• Those able to amass information (diagnostic and personal) on virtually all cases of cancer among patients genuinely resident within a defined catchment area during a prescribed period of time;
• able to supplement this with death certificate data for patients not seen in hospital
• having an adequate system for eliminating duplicate entries for the same person
• and good population data - by sex and by 5-year age groups and, if relevant, by race/language
![Page 3: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/3.jpg)
Data quality and its evaluation
• Evaluation of data quality in the cancer registry: Principles and methods.
• Part I: Comparability, validity and timeliness (Bray & Parkin)• Part II: Completeness (Parkin & Bray)• Eur J Cancer (2009) 45: 747-77, 756-64• Update of 1994 IARC Technical Report• Application to Cancer Registry of Norway:• Larsen et al (2009) Eur J Cancer 45:1218-31
• Standards and guidelines for cancer registration in Europe: The ENCR recommendations, vol 1. Lyon: IARC (2003).
![Page 4: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/4.jpg)
Data quality and its evaluation
Conclusion:
“This review indicated that the routines in place at the Cancer Registry of Norway yield comparable data that can be considered reasonably accurate, close-to-completion and timely, and serves as a justification for our policy of reporting annual incidence one year after the close of registration.”
![Page 5: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/5.jpg)
Data quality and its evaluation
Four “classical” dimensions of quality:
• Comparability
• Validity
• Completeness
• Timeliness
![Page 6: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/6.jpg)
1. Comparability2. Completeness3. Validity4. Timeliness
Special Issue: Data Quality at the Cancer Registry of Norway
http://www.kreftregisteret.no
![Page 7: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/7.jpg)
![Page 8: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/8.jpg)
![Page 9: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/9.jpg)
![Page 10: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/10.jpg)
Data quality and its evaluation
Four “classical” dimensions of quality:
• Comparability
• Validity
• Completeness
• Timeliness
![Page 11: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/11.jpg)
Data quality and its evaluation
Comparability
• Ensuring comparable standards of reporting and classification across registries and within registries over time;
• Reporting of routines, standards and practices in place and, especially, dates in changes of practice;
• Where standards within a registry differ from “accepted” practice, requirement to provide means of conversion from one to other.
![Page 12: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/12.jpg)
Data quality and its evaluation
Comparability
• Classification and coding systems
• Definition of incidence date
• Handling of multiple primaries
• Incidental diagnosis (basis)
• Screening and testing
• Imaging
• Autopsy diagnosis (basis)
• Handling of death certificate information
![Page 13: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/13.jpg)
Data quality and its evaluation
Bray and Parkin (2009) EJC 45:747
![Page 14: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/14.jpg)
Data quality and its evaluation
Larsen et al (2009) EJC 45:1218
![Page 15: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/15.jpg)
![Page 16: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/16.jpg)
Note: Rates are age-adjusted to the 1970 U.S. standard. Rates from 1973-1987 are
based on data from the 9 standard registries. Data from San Jose and Los
Angeles are included in the rate calculation for 1988-1995.
![Page 17: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/17.jpg)
Data quality and its evaluation
Four “classical” dimensions of quality:
• Comparability
• Validity
• Completeness
• Timeliness
![Page 18: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/18.jpg)
Data quality and its evaluation
Validity
• Accuracy of reporting
• Do cases reported to have a specific characteristic truly have that characteristic
• Depends on
• Accuracy of source information
• Registry “skill” in abstracting, coding and reporting
![Page 19: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/19.jpg)
Data quality and its evaluation
Validity – assessment procedures:
• Diagnostic criteria methods
• Histologic/microscopic verification (% HV/MV)
• Death certificate only (% DCO)
• Missing information (e.g. % PSU)
• Internal consistency checks (QC)
• Re-abstracting and recoding (QA)
![Page 20: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/20.jpg)
Data quality and its evaluation
Microscopic verification (% MV)
• Varies by cancer site (and age);
• Depends on pathology/cytology service
• 100% not always best;
• Are statistical tests to compare % MV of a registry against standard, other registries or itself at different time.
![Page 21: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/21.jpg)
Data quality and its evaluation
Larsen et al (2009) EJC 45:1218
![Page 22: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/22.jpg)
Data quality and its evaluation
Death certificate only (% DCO)• Varies by cancer site (and age);
• Depends on clinical service;
• Associated with reduction in validity (especially site and diagnosis date) and increase in missing information;
• Other validity issues around “Death certificate notified (DCN)” or “Death certificate initiatied (DCI)”.
![Page 23: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/23.jpg)
Data quality and its evaluation
Larsen et al (2009) EJC 45:1218
![Page 24: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/24.jpg)
Data quality and its evaluation
Bray and Parkin (2009) EJC 45:747
![Page 25: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/25.jpg)
Data quality and its evaluation
Missing information (e.g. % PSU)• Varies by cancer site and age;
• Varies by data item (e.g. stage);
• Depends on both registry and clinical record practice;
• Care required in codes used to define “primary site uncertain” (not just “Unknown primary site ICD-10 C80);
• Low % MV associated with high “PSU”.
![Page 26: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/26.jpg)
Data quality and its evaluation
Larsen et al (2009) EJC 45:1218
![Page 27: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/27.jpg)
Data quality and its evaluation
Internal consistency checks (QC)• Invalid (or unlikely) codes or combinations of codes
or sequences of dates;
• Can be operationalised within software (including during data entry);
• IARC has developed such tools (IARC-CHECK) within IARCcrgTools which can be downloaded from IACR website: www.iacr.com.fr
• Checks applied should be reported along with results.
![Page 28: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/28.jpg)
Data quality and its evaluation
Re-abstracting and recoding (QA)
• Expensive and time consuming;
• Can be operationalised on sample basis;
• Can make use of other ad-hoc studies;
• Requires approaches to correct identified problems prospectively and retrospectively.
![Page 29: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/29.jpg)
Data quality and its evaluation
Four “classical” dimensions of quality:
• Comparability
• Validity
• Completeness
• Timeliness
![Page 30: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/30.jpg)
Data quality and its evaluation
Completeness:
• The extent to which all of the incident cancers occurring in a target population are included in the registry database;
• Key defining criterion for population basis to registration;
• No perfect assessment tool
![Page 31: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/31.jpg)
Data quality and its evaluation
Completeness assessment:
• Methods based on comparisons and inspection;
• Methods based on independent assessment.
• Ad-hoc planned or incidental studies
• Use of multiple (independent) sources of notification especially death certificates
![Page 32: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/32.jpg)
Data quality and its evaluation
Completeness assessment:
• Methods based on comparisons and inspection;
• Compare rates over time and/or with similar populations;
• Inspect age-incidence curves;
• Stability of childhood cancer rates.
![Page 33: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/33.jpg)
Data quality and its evaluation
Completeness assessment:• Methods based on independent assessment
• Ad-hoc planned or incidental studies(comments as for validity)
• M/I ratios
• Capture-recapture methods
• The DC and M/I methodAjiki et al (1998) Nippon KEZ 45:1011
• The Flow method (also measures timeliness)Bullard et al (2000) B.J.Cancer 82:111
Read Parkin & Bray
(2009) for details
![Page 34: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/34.jpg)
Data quality and its evaluation
Completeness assessment:• M/I ratios;• Number of incident cases during defined time period;• Number of deaths during the same time period;• Assumption that mortality data from a source
independent of cancer registration;• Should analyse by cancer site and by age group;• Absolute values depend on survival rates and quality
of both registration and death certification;• Not robust to (usually rare) short-term changes in
incidence or survival.
![Page 35: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/35.jpg)
Data quality and its evaluation
Larsen et al (2009) EJC 45:1218
![Page 36: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/36.jpg)
Data quality and its evaluation
Four “classical” dimensions of quality:
• Comparability
• Validity
• Completeness
• Timeliness
![Page 37: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/37.jpg)
Data quality and its evaluation
Timeliness:
• Speed with which registry can collect, process and make available data at a given standard of completeness and quality;
• Often pressure to increase timeliness at expense of other quality indicators;
• Some registries (e.g. SEER) publish at a given time point and make estimates of under reporting;
• 12-24 months after year end represents current “standard”.
![Page 38: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/38.jpg)
![Page 39: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/39.jpg)
Data quality indicators CI5 vol. 9Breast cancer (f)
Registry No. MV% DCO% M/I%
Brazil
Sao Paulo
22598 82.2 4.6 22.8
SEER (14) 237378 98.5 0.6 21.3
Norway 12521 98.4 0.3 29.4
UK
Scotland
17137 96.4 0.3 32.9
Japan
Osaka
11103 90.1 2.5 30.5
![Page 40: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/40.jpg)
Data quality indicators CI5 vol. 9Lung cancer (m)
Registry No. MV% DCO% M/I%
Brazil
Sao Paulo
6525 66.9 13.8 72.8
SEER (14) 123409 89.8 1.8 80.7
Norway 6516 87.4 1.0 88.6
UK
Scotland
12969 74.9 0.9 88.3
Japan
Osaka
16759 73.1 19.3 83.7
![Page 41: Evaluating Data Quality in the Cancer Registry](https://reader031.vdocuments.net/reader031/viewer/2022022403/5475c2e3b4af9fb40a8b5ddb/html5/thumbnails/41.jpg)
Data quality and its evaluation
• Cancer registration is a worldwide activity and leads the way in global surveillance for non communicable diseases;
• The benefit of population based registration to cancer control programs and to epidemiological research can be realised only to the extent that data are of a comparable, high quality standard;
• Reporting on data quality in a registry is as important as reporting analyses of the data.