Computer Assisted Evaluation of Clinical Data Quality

Nordic Casemix Conference 4.6.2010

Olafr Steinum, Sequelae ABSeppo Ranta, Datawell Oy

2

Introduction

Reported health care information is widely used by authorities

• For health care planning• For quality analyses• For reimbursement• For science and research

It is of paramount importance that the reported health data are correct and valid

Quality assurance is needed• Coding audits• Automatized controls

3

Datawell DRG QA for data quality assurance

DRG QA is a Datawell product

Uses three different logics for data quality evaluation1. Rules and reference databases (e.g. diagnosis codes) used for NordDRG

grouping2. Evaluation of the order of diagnoses (which one is the primary diagnosis, which

are secondary diagnoses) inferrred from large statistical database (Normalization)

3. Clinical Validation Rulebase (CVRB) created and maintained by Sequelae AB

The software can be used as• part of encoder system for immediate feedback of coding quality• standalone system for evaluation of organization data quality• benchmarking system comparing several peer organizations

4

Datawell DRG QA

DRG QA is a Datawell product ****

Uses three different logics for data quality evaluation1. Rules and reference databases (e.g. diagnosis codes) used for NordDRG

grouping2. Evaluation of the order of diagnoses (which one is the primary diagnosis, which

are secondary diagnoses) inferrred from large statistical database (Normalization)

3. Clinical Validation Rulebase (CVRB) created and maintained by Sequelae AB

The software can be used as• part of encoder system for immediate feedback of coding quality• standalone system for evaluation of organization data quality• benchmarking system comparing several peer organizations

Sequelae AB

Is a joint colloboration between

Emendor Consulting AB, (Staffan Bryngelsson)

and Olafr Steinum (diaQualos AB)

Gunnar Henriksson (DRG Henriksson AB)

Sequelae AB

Is a joint colloboration between

Emendor Consulting AB, (Staffan Bryngelsson)

and Olafr Steinum (diaQualos AB)

Gunnar Henriksson (DRG Henriksson AB)

5

Datawell DRG QA

DRG QA is a Datawell product

Uses three different logics for data quality evaluation1. Rules and reference databases (e.g. diagnosis codes) used for NordDRG

grouping2. Evaluation of the order of diagnoses (which one is the primary diagnosis, which

are secondary diagnoses) inferrred from large statistical database (Normalization)

3. Clinical Validation Rulebase (CVRB) created and maintained by Sequelae AB

The software can be used as• part of encoder system for immediate feedback of coding quality• standalone system for evaluation of organization data quality• benchmarking system comparing several peer organizations

6

PatId Dg-a Dg-d Pr LOS Age Dischg Sex

13213 3 134 HOME M

43242 H10.1 J80 WX101 1 54 HOME N

43242 F0289 E756 GD1BD 6 3 HOSP N

64243 V02.0 2 41 HOME N

34212 O75.7 MAF00 4 34 HOME M

DRG QA – An Example of Indicator Calculation Logic

Age not within acceptable limits

External cause code as principal diagnosis

Input data set

ErroneousICD-10 code

Missing principal diagnosis

PatId Dg-a Dg-d Pr LOS Age Dischg Sex

13213 3 134 HOME M

43242 H10.1 J80 WX101 1 54 HOME N

43242 F0289 E756 GD1BD 6 3 HOSP N

64243 V02.0 2 41 HOME N

34212 O75.7 MAF00 4 34 HOME M

Local procedure code

Mismatch of diagnosis and gender

Validations

7

PatId Dg-a Dg-d Pr LOS Age Dischg Sex FDG FPR SEXDG FAGE

13213 3 134 HOME M 110 0 2

43242 H10.1 J80 WX101 1 54 HOME N 100 110 0 0

43242 F0289 E756 GD1BD 6 3 HOSP N 100 100 0 0

64243 V02.0 2 41 HOME N 120 0 0

34212 O75.7 MAF00 4 34 HOME M 100 100 3 0

DRG QA – An Example of Indicators in DRG QA Database

Errors in Diagnosis coding

Errors in Procedure coding

Mismatch of diagnosis and gender

Errors in Age coding

8

DRG QA Pilot Benchmark Database

Seven Hospital Districts in Finland• DRG QA Database contains patient cases from the Ecomed KPP databases from

2008• Data source: Ecomed KPP used in the 7 hospitals

Three County Councils in Sweden• DRG QA Database contains all patient visits and stays from 2008• Data source: Patient Administrative Systems in corresponding county councils

Number of patient cases• Finland n = 4.928.113• Sweden n = 4.332.206

9

Hospital Districts’Ecomed KPP databases (FI), orsimilar data retrieval from Patient Administrative Systems (SE)

etc.

County Council C

DRG QA Database Formation Process

Datawell DRG QA

ETL

EcomedDRG QADatabase

Ecomed Analyzer

Analysis ofData Quality

ReportingCounty Council B

County Council A

District C

District B

District A

Datawell DRG QAIndicator

Calculation

• Data format transformations: hospital code common code mappings

• Calculation of DRG grouping indicators• DRG normalization• Calculation of CVRB matching

• Includes refence population data (1-year intervals) for standardization

10

Data Analysis for the Current Presentation

For this presentation purporses we produced a sample of the benchmarking database

Rules for formation of the present sample• Included hospital stays (coded as ward stays in the source data)• Length of stay (LOS) over zero days• Excludes psychiatry

After applying above mentioned filters the analysis sample consists of• 493 689 ward stays in seven Finnish Hospital Distrcits’ hospitals, and• 444 255 ward stays in three Swedish County Councils’ hospitals

11

0

20

40

60

80

100

120

140

160

180

200

1 2 3 4 5 6 7 8 9 10

Number of Hospital Stays (LOS > 0 d) per 1.000 standardized population

Female

Male

Background Information: Number of Hospital Stays

Finnish University Hospitals

Non-university (Fi)

Swedish County Councils

n = 493 674 (FI); n = 444 255 (SE)

12

Background Information: Number of Stays in Different Age Groups

n = 493 674 (FI); n = 444 255 (SE)

0,000

100,000

200,000

300,000

400,000

500,000

600,000

Num

ber o

f Sta

ys p

er 1

.000

pop

ulati

on

Ward Stays (LOS > 0 d) per 1.000 standardized population in one-year age groups

Finland

Sweden

13

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10

Day

sAverage LOS

Background information: Average Length of Hospital Stays

Finnish University Hospitals

Non-university (Fi)

Swedish County Councils

n = 493 689 (FI); n = 444 255 (SE)

14

0,00

0,50

1,00

1,50

2,00

2,50

3,00

3,50

1 2 3 4 5 6 7 8 9 10

Avg number of diagnosis codes per ward stay (LOS>0 d)

Background information: Average Number of Diagnosis Codes per Stay

Finnish University Hospitals

Non-university (Fi) Swedish County Councils

n = 493 689 (FI); n = 444 255 (SE)

15

0,00

0,50

1,00

1,50

2,00

2,50

3,00

3,50

1 2 3 4 5 6 7 8 9 10

Avg number of procedure codes per ward stay (LOS>0 d)

Background information: Average Number of Procedure Codes per Stay

Finnish University Hospitals

Non-university (Fi)

Swedish County Councils

n = 493 689 (FI); n = 444 255 (SE)

17

0,000

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

50,000

1 2 3 4 5 6 7 8 9 10

Number of DRG CC cases per 1.000 population (age and gender standardized)

DRG CC Cases

Finnish University Hospitals

Non-university (Fi)

Swedish County Councils

n = 120 060 (FI); n = 147 740 (SE)

18

0,000

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

1 2 3 4 5 6 7 8 9 10

Number of cases with diabetes code (E10-E14) in any diagnosis position per 1.000 standardized population

Diabetes as Principal or Secondary Diagnosis

Finnish University Hospitals

Non-university (Fi)

Swedish County Councils

n = 20 713 (FI); n = 40 085 (SE)

19

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

1 2 3 4 5 6 7 8 9 10

Average Number of Surgical Procedure Codes per Ward Stay

Number of Surgical Procedure Codes (OR property in NordDRG)

Finnish University Hospitals

Non-university (Fi) Swedish County Councils

n = 493 689 (FI); n = 444 255 (SE)

20

Surgical Procedure Codes after Stays in Operative Wards

n = 283 633 (FI); n = 203 684 (SE)

0

0,2

0,4

0,6

0,8

1

1,2

1,4

1,6

Surgery Gynecology and Obstetrics

Eye Disease ENT

Avg

Num

ber o

f Su

rgic

al P

roce

dure

Cod

es p

er W

ard

Stay

Average Number of Surgical Procedure Codes per Ward Stay (LOS > 0d)

FI 1

FI 2

FI 3

FI 4

FI 5

FI 6

FI 7

SE 3

SE 1

SE 2

21

The classification of diagnosis (ICD-10)

A complex system for collecting data for statistics

• Many axes• Many rules

• Explicit rules• Rules expressed in the Tabular volume in connection to code categories• Rules assumed, but not explicitely expressed

Clinical validation rule base - CVRB• A collection of identified rules

22

Rate of Z51.1 Chemotherapy session as Principal or Secondary diagnosis.Swedish county councils 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Huvuddiagnos

Bidiagnos

Data from Swedish National Patient Registry

Principal dxSecondary dx

23

1 Not to be used2 Ought not to be used

3 Not to be used for children < 15 years4 Ought not to be used for children

< 15 yrs

5 Not to be used in inpatient care6 Ought not to be used in inpatient care

7 Not to be combined with code28 Ought not to be combined with code2

9 Not to be used as principal dx10 Ought not to be used as principal dx

11 Not to be used as secondary dx12 Ought not to be used as secondary dx

13 Not to be used as principal dx in combination with code2

14 Ought not to be used as principal dx in combination with code2

15 Not to be used as secondary dx in combination with principal dx code2

16 Ought not to be used as secondary dx In combination with principal dx code2

17 Not to be used as single code18 Ought not to be used as single code

19 Not to be used in combination with other dagger code than code2

20 Ought not to be used in combination with other dagger code than code2

21 Must be combined with code222 Ought to be combined with code2

23 Must be combined with an external cause code (chapter XX) for toxic agent

24 Ought to be combined with external cause code (chapter XX) for toxic agent

25 Must be used as secondary dx only in combination with principal dx = code2

26 Ought to be used as secondary dx only in combination with principal dx = code2

27 Must be secondary dx when combined with code228 Ought to be secondary dx when in

combination with code2

29 Must be principal dx in combination with code230 Ought to be principal dx in combination with code2

31 Rare code32 Rare code inpatient care33 Rare code outpatient care© Sequelae AB

CVRB Rules 2009

24

Frequency of CVRB violation in a test database from 10 provinces

0,00 %

2,00 %

4,00 %

6,00 %

8,00 %

10,00 %

12,00 %

1 2 3 4 5 6 7 8 9 10

% o

f all

war

d pe

riod

s

Any CVRB Rule violation

n = 493 689 (FI); n = 444 255 (SE)

25

0,000 %

0,001 %

0,010 %

0,100 %

1,000 %

10,000 %

100,000 %

CVRB1 Violations

CVRB2 Violations

CVRB3 Violations

CVRB5 Violations

CVRB7 Violations

CVRB8 Violations

CVRB9 Violations

CVRB10 Violations

CVRB11 ViolationsCVRB13 Violations

CVRB14 Violations

CVRB15 Violations

CVRB16 Violations

CVRB18 Violations

CVRB19 Violations

CVRB21 Violations

CVRB22 Violations

CVRB Violations (Hospital Stays, LOS > 0 d)

Finland

Sweden

CVRB Violation Profile in Finland and Sweden, Ward Stays, LOS > 0 days

n = 493 689 (FI); n = 444 255 (SE)

26

02 0004 0006 0008 000

10 00012 00014 000

CV

RB

1

CV

RB

2

CV

RB

3

CV

RB

4

CV

RB

5

CV

RB

6

CV

RB

7

CV

RB

8

CV

RB

9

CV

RB

10

CV

RB

11

CV

RB

12

CV

RB

13

CV

RB

14

CV

RB

15

CV

RB

16

CV

RB

17

CV

RB

18

CV

RB

19

CV

RB

20

CV

RB

21

CV

RB

22

CV

RB

23

CV

RB

24

CV

RB

25

CV

RB

26

CV

RB

27

CV

RB

28

CV

RB

29

CV

RB

30

CV

RB

31

CV

RB

32

CV

RB

33

Violation Rule

Cas

es w

ith

vio

lati

on

Finland

Sw eden

Distribution of CVRB violation in test database (10 provinces)

CVRB violation rule

27

CVRB Violation rules

Not to be used

Ought not to be used

Not to be used in inpatient care

Not to be combined with code2

Ought not to be combined with code2

Not to be used as principal dx

Ought not to be used as principal dx

Not to be used as secondary dx

Not to be used as principal dx in combination with code2

Ought not to be used as secondary dx in combination with principal dx code2

Ought not to be used as single code

Not to be used in combination with other dagger code than code2

Must be combined with code2

Ought to be combined with code2

Ought to be combined with external cause code (chapter XX) for toxic agent

Ought to be used as secondary dx only in combination with code2

Must be secondary dx when in combination with code2

Rare code

Rare code inpatient care

0 5 000 10 000 15 000

CVRB 1

CVRB 2

CVRB 5

CVRB 7

CVRB 8

CVRB 9

CVRB10

CVRB11

CVRB13

CVRB16

CVRB18

CVRB19

CVRB21

CVRB22

CVRB24

CVRB26

CVRB27

CVRB31

CVRB32

28

CVRB11 Not to be used as secondary dx 273 codes from Ch 21 (Z-codes) only to be used as Reason for admission

CVRB10 Ought not to be used as principal dx 16 code categories, mainly unspecified codes and codes marking sequelae

Example of CVRB Violation rules result in percent of inpatient stays

0,0%

1,0%

2,0%

3,0%

4,0%

5,0%

6,0%

7,0%

8,0%

Finland Sweden

CVRB10 CVRB11

1 2 3 4 5 6 7 8 9 10

29

Information Process and the Identified Sources of Quality Failure

Human-Computer interface

Feeding of structuredinformation into the PAS

Processing rules andlogics of the informationsystems

Usability and maintenance ofnational code systems(ICD, NCSP, DRG etc.)

Current transversal studyof the information process

Entry of data Processing of data

Utilization of dataand information

Code systems

30

Information Process and Benefits of Datawell DRG QA

Immediate feedback ofcoding results to codingpersonnel

Information on organizationdata quality for focusingeducation and other correctiveactions.

Benchmarking data quality withpeer organizations.

Entry of data Processing of data

Utilization of dataand information

Code systems

Reports of data qualityincorporated with otherreporting

31

Data which nobody is using has a quality that nobody

wants

Thank you!