sensitivity and specificity part i – introduction and definitions

Sensitivity and SpecificityPart I – Introduction and Definitions

This video is designed to accompany

pages 81- 94

in

Making Sense of UncertaintyActivities for Teaching Statistical

ReasoningVan-Griner Publishing Company

Pregnancy Screening

A home pregnancy test (HPT) uses chemistry to bond a particular glycoprotein (hCG) to an antibody and an indicator. hCG is secreted by the placenta just after fertilization.

There are two ways the test can be wrong. It can say the user is pregnant when she isn’t. Or it can say she is not pregnant when she is.

Yes/No Decisions

1. All screening tests are faced with making a similar decision.

2. Either the test indicates you have what the test is designed to look for (a “positive” or “yes” result) or …

3. The test indicates you don’t have what the test is designed to look for (a “negative” or “no” result).

4. Two decision errors are always possible.

.

Naming the Errors

Can say you do have what the test is looking for, when you really don’t - a false positive - or …

Can say you don’t have what the test is looking for, when you really do - a false negative.

.

These two error rates are a common way to assess how well any screening test is working, not just an HPT. Ideally you want them both to be low.

Error

Important Language

Specificity = 1 - false positive rate

In generic terms, this is the ability of the screening test to correctly identify “Negatives” as “Negatives”

Sensitivity = 1 - false negative rate

In generic terms, this is the ability of the screening test to correctly identify “Positives” as “Positives”

Label Claims

So just how accurate is an HPT?

Clearblue Digital Pregnancy Test • No test is more accurate - over 99% accurate from the day of your

expected period2 • Results 5 days sooner • Unmistakably clear results in words • Easy to use with its unique hourglass symbol to show you the test is working • Fast – results in just 3 minutes • Easy-to-use 1-step test

--------------------------------------------------------------------------------299% accurate at detecting typical hormone levels. Note that hormone levels vary. See product insert.

Say Again?

This type of claim relates to a Food and Drug Administration regulation on how these kits have to be screened. 99% means how likely is it that the Clearblue home pregnancy test made by this manufacture shows a positive results when it's testing a non-pregnant woman's urine mixed with a certain amount of commercial hCG.

Theoretical False Negative RateThe presence of hCG means you are pregnant. And if there is hCG present, the HPT will detect it 99% of the time. So the claim is that the Clearblue HPT will fail to say a woman is pregnant when she really is, only 1% of the time.

That’s quite good. Most HPTs make that same claim about the sensitivity and false negative rate of HPTs.

False Negative Rate in Practice?

Kit Type% Time Kit Didn't Say Subject Was Pregnant, When She Actually Was

Ova II 47.4Predictor 3Answer 22Daisy 2 18

e.p.t. 18e.p.t. plus 10Advance 14

Fact 0Answer 2 0

First Response 7.1Advance 8.8

Seems to be a lot harder in practice to get just right. These studies used real women, not lab results.

From an older study in Arch Fam Med Sept/Oct 1998. A meta-analysis by Bastian, et al.

False Positive Rate in Practice?

Kit Type% Time Kit Said Subject Was Pregnant, When She Wasn’t

Ova II 48.1Predictor 4Answer 36Daisy 2 36

e.p.t. 25e.p.t. plus 8Advance 9

Fact 6.5Answer 2 6.5

First Response 0Advance 0

Typically claimed to be an error that is less socially problematic.

Data from study in Arch Fam Med Sept/Oct 1998. A meta-analysis by Bastian, et al.

Head-to-Head Comparison

Kit Type FNR FPROva II 47.4 48.1

Predictor 3 4Answer 22 36Daisy 2 18 36

e.p.t. 18 25e.p.t. plus 10 8Advance 14 9

Fact 0 6.5Answer 2 0 6.5

First Response 7.1 0Advance 8.8 0

Results are mixed regarding which error is more prevalent.

Data from study in Arch Fam Med Sept/Oct 1998. A meta-analysis by Bastian, et al.

One-Sentence Reflection

The false positive rate and false negative rate are common numerical assessments of the risks associated with the results of a screening test.