07-qualitative semi quant

8/3/2019 07-Qualitative Semi Quant

1/20

Copyright 2002-2010, Data Innovations, LLC. All rights reserved world-wide.This document may not be reproduced without express written permission.

Qualitative and Semi-quantitativeModule (QMC)

By Carol Lee


2/20

2

Method Comparison Qual/SemiQuant

Purpose To evaluate the degree of concordance between two

methods for qualitative or semi-quantitative methods.

Principle

Qualitative methods and semi-quantitative methods lacksufficient analytical sensitivity to demonstrate accuracyacross a wide range of observed results. Therefore:

The results are categorized in states of diagnostic

description. i.e.: postive / negative, equivocal, gray zone,

Qualitative 2 states (i.e. positive and negative)

Semi-quantitative a limited number of states (up to 6are allowed in EP Evaluator).


3/20

3

QMC

Evaluate the degree of concordance between two qualitative orsemi-quantitative methods.

colorful bubble chart size of bubbles proportional to number of points

concordance or truth tables

Qualitative 2 states (i.e. positive and negative)

% pos agreement, % neg agreement False pos and false neg in red

If one method is gold standard

Sensitivity and specificity is calculated

Semi-quantitative up to 6 states allowed in EP Evaluator Click custom codes to set up states

Alphanumeric i.e., Equivocal, gray zone

Numeric cutoff values

< cutoff, cutoff

2002-2008 David G. Rhoads Associates, Inc


4/20

4

Experimental Design

Qualitative: At least 20 specimens.

Equal numbers of positives and negatives

Semi quantitative Same number of states for both methods

User defined results or cutoffs and labels

0 to 10, few,

Try for similar N in each state.


5/20

5

Method Comparison Qualitative

Enter values as 2 states User defined labels

Calculations

If One method is specified as a gold standard (i.e.

absolutely correct). calculates specificity and sensitivity

If Neither method is a gold standard.

Only calculates relationship of the two methods, namely

the degree of agreement and whether one method ismore or less sensitive relative to the other.


6/20

Qualitative

Default is 2 states

P = positive

N = Negative

Allowed results showin the drop down boxin the data entry area


7/20

Gold Standard Reference


8/20

Enter 2state

resultsGold

standard


9/20

Not Gold Standard


10/20

Report Excerpt

Qualitative Method ComparisonRef. Method: Immunochromatic Test Method: NCCLSEx2

neg pos1 2

1

2

Kappa is the proportion of agreement above what's expected bychance. Rule of thumb is Kappa>75% indicates "high" agreement.We would like to see VERY high (close to 100%) agreement.

Statistical Analysis

Agreement 94.6% (92.3 to 96.2%)

Positive Agreement 95.3%Negative Agreement 93.7%95% confidence intervals calculated by the "Score" method.

McNemar Test for Symmetry:Test < Reference 14 (2.6%)Test > Reference 15 (2.8%)Symmetry test PASSES p = 0.853 (ChiSq=0.034, 1 df)A value of p


11/20

Concordance table


12/20

Statistical tests

Cohen's Kappa: Similar to Agreement, but adjusted for theprobability that the two methods agree by chance. Kapparanges from -100% to 100%. A value of zero indicatesrandom agreement. A value of 100% indicates perfectagreement. It is desirable for Kappa to be well above 75%.

McNemar Test for Symmetry: A test for bias -- whether onemethod is consistently larger than the other. If the number ofcases where X>Y is equal (within random error) to thenumber of cases XY (or when X


13/20

Semi quantitative

Ref. Method: Chem Assay Test Method: Analyzer

neg pos1 2 3 4 5 6

1

2

3

4

5

6



Agreement 71.9% (61.8 to 80.2%)95% confidence intervals calculated by the "Score" method.

McNemar Test for Symmetry:Test < Reference 23 (25.8%)Test > Reference 2 (2.2%)Symmetry test FAILS p < 0.001 (ChiSq=17.640, 1 df)A value of p


14/20

Allow 1 step difference


15/20

Allow 1 step

Ref. Method: Chem Assay Test Method: Analyzer

neg pos1 2 3 4 5 6

1

2

3

4

5

6



Agreement 71.9% (61.8 to 80.2%)

Agreement within two 98.9% (93.9 to 99.8%)95% confidence intervals calculated by the "Score" method.

McNemar Test for Symmetry:Test < Reference 23 (25.8%)Test > Reference 2 (2.2%)

Symmetry test FAILS p < 0.001 (ChiSq=17.640, 1 df)A value of p


16/20

Custom results codesCutoff values Cutoff value > than

the number entered


17/20

Policy Definitions: Editing Analytesettings Your choices in Modules

and options will define whatheaders are visible in theanalyte settings

If you dont select an sdgoal in SP, the column forrandom err budget will notbe present.

If you check QMC, a

separate tab appears forQMC

17Jan 6, 2010


18/20

18

QMC entering analyte parameters

edit paste headers into excel

Make changes

Copy / paste from excel back to EE9

18Jan 6, 2010

Analyt

e

Custom

?

Result

sType

LvlN

ame

1

LvlVal

ue1

LvlNa

me2

LvlVal

ue2

LvlNa

me3

LvlVal

ue3

LvlNa

me4

LvlVal

ue4

LvlNa

me5

LvlVal

ue5

LvlNa

me6

LvlVal

ue6

BACT y 1

Nor

m Abn 1200

fuzzy

cells y 1

Nor

m Abn 1


19/20

QMC analyte settingspolicy definitions

19Jan 6, 2010

0 = alpha

1 = numeric large is pos2 = numeric large is neg


20/20

20

Questions and Discussion

2002-2008 David G. Rhoads Associates, Inc 20

07-qualitative semi quant

Documents