WWW.ICAD-CISD.COM

New Prevention Technologies

Workshop

Module 7: Media Analysis

INTERPRETING TRIAL RESULTS

Trial Size

One statistical calculation that occurs before a trial begins is the sample size or the number of volunteers that need to be enrolled.

If the overall incidence in the trial population is low, more volunteers are necessary. Therefore need more volunteers if recruiting from general population versus high-risk populations

Some trials are also designed to continue until a pre-determined number of HIV infections or endpoints occur: if the HIV incidence is low, the trial duration is longer.

The precision with which the efficacy of the vaccine is determined is based on the number of HIV infections that occur during the study, not the total number of volunteers involved.

Statistical Variation

Statistical variation is the technical term for chance fluctuations.

Even if disease rates in two populations (say 1 million people each) are identical, they may not appear so in a study.

If we study two identical populations with identical risk factors and exposures, and pick a sample of 1,000 from each population, it likely will turn out that the disease rates measured will be similar but not identical.

As we increase the sample (to 10,000 subjects each), our study-based estimates of the true disease rates in the entire populations will be more accurate.

Statistical Power

"Statistical power" measures the ability of a study to find an association between exposure and disease, when such an association actually exists.

If an intervention does indeed decrease the risk of disease, then a study with high power will be very likely to find an association.

However, if the study has low power, then it has little chance of finding an association even if there is an actual association between the intervention and the disease.

Depends on SAMPLE SIZE

Statistical Significance

Did the vaccine actually work or did the results happen merely by chance?

The smaller the p-value and the closer the confidence interval, the better the chance that less HIV infections in the vaccine arm actually means something

P-values and confidence intervals are based on the same underlying concepts of probability.

PRINCIPLES OF PROBABILITY

Data

Freq

uenc

y

50 heads : 50 tails

95% confidence interval

2 heads : 98 tails 98 heads : 2 tails

The effect of sample size

95% CI

If you toss the coin 10,000 times

95% CI

If you toss the coin 10 times

Efficacy

Efficacy: compare the number of HIV infections that occurred in the vaccine and placebo groups.

If more infections occur in volunteers who received placebo (e.g., Thai vaccine trial), researchers can then estimate the efficacy of the vaccine candidates.

Thai (AIDSVAX/ALVAC) trial: 74 infections occurred among volunteers in

the placebo group 51 in those who received the full prime-boost

regimenTherefore the efficacy of the vaccine candidates was 31.2%

Confidence Intervals

To account for the possibility that the number of HIV infections were higher in the placebo arm just by chance, epidemiologists use "confidence intervals."

CI for the actual efficacy of the vaccine. = range of values around the best estimate of efficacy, all of which are contenders

A "95% confidence interval" means that there is a 5% chance that even this broad range is due to chance. In other words, there is still a 5% chance that the true value of the measurement lies outside of the interval.

Thai Vaccine Trial Results

EFFICACY point estimate = 31.2%

95% CI = 1.2-52.1% efficacy

1.2%

52.1%

P-Value

P-value (probability value) quantifies uncertainty about whether an outcome is due to chance or whether it actually reflects a true difference.

P-value is the percent likelihood of a chance outcome, thus the lower the p-value, the better confidence you can have in the results of the study.

Traditionally (and arbitrarily), a p-value of .05 or less has been accepted as evidence of actual difference.

Common misunderstandings about p-values: The p-value is not the probability that the vaccine has

no effect. The p-value is not the probability that a finding is

"merely a fluke.“ The p-value is not the probability of falsely finding the

vaccine to be effective. The p-value is not the probability that a replicating

experiment would not yield the same conclusion.

Rather, the p-value is the chance of obtaining such results if the vaccine actually had no effect.

Interpreting the P-value

Let’s Practice Interpreting the Results from the Thai HIV Vaccine Trial

31% efficacy (CI = 1.2%-52.1%, P=0.04)

“The vaccine recipients had a 31% lower risk of HIV infection than those who received placebo.”

“The efficacy of the prime-boost regimen could be anywhere in the range of 1.2% to 52.1%, yet the most likely efficacy is at the middle of that range, or 31.2%.”

“If the vaccine had no effect whatsoever, there is a 4% chance that this split in infections, or an even larger one, would have occurred anyway.”

Odds Ratio

used to assess the risk of a particular outcome (or disease) if a certain factor (or exposure) is present.

a relative measure of risk, telling us how much more likely it is that someone who is exposed to the factor under study will develop the outcome as compared to someone who is not exposed.

to calculate the OR, we calculate the odds of exposure among cases and divide it by the odds of exposure among controls.

OR > 1 means outcome is more likely OR < 1 means outcome is less likely

Jaspan et al., 2006. Adolescent HIV Prevalence, Sexual Risk, and Willingness to Participate in HIV Vaccine Trials. Journal of Adolescent Health: 39(5), pp. 642-648.

Purpose: To determine human immunodeficiency virus (HIV) prevalence, sexual risk behaviors, and attitudes toward HIV vaccine trials among 11–19 year-olds in a peri-urban community near Cape Town, South Africa.

Results: Of the 510 adolescents selected, 356 (73%) participated. The HIV prevalence of the group was 10.6% (95% confidence interval [CI] 7.5–14.4). One-third of adolescents had experienced sexual debut, with a mean age of 14.6 years. Number of lifetime sexual partners was independently associated with HIV infection (odds ratio [OR] = 1.62; 95% CI 1.1–2.3). In a multivariate analysis, increasing age, female gender, and attending school were independently associated with having had sex. The majority of adolescents (79%) were willing to participate in an HIV vaccine trial. Increasing age and length of residence in the community were significantly associated with willingness to participate (OR = 1.19; 95% CI 1.01–1.4 and OR = 1.14; 95% CI 1.03–1.26, respectively).

Conclusions: The prevalence of HIV and risk behavior among adolescents in this community is high. HIV vaccines are required that target preadolescents. HIV vaccine trials in adolescents in this setting will be facilitated by their willingness to participate.

SCENARIO PLANNING

A Good Key Message is:

Concise – uses accessible language Simple to say aloud Focused on one idea Easy for people to understand and

remember Persuasive Nonjudgemental Relevant to the targeted audience

Scenario #1: Highly Beneficial

It clearly works! PrEP reduces the risk of HIV among IDUs by 80% There is weak statistical significance (CI = 20-

90%, p=0.05)

Scenario #2: Moderately Beneficial It kind of works PrEP reduces the risk of HIV among IDUs by 30% There is strong statistical significance (CI=28-

32%, p=0.01)

Scenario #3: Flat Result

It doesn’t work The HIV rates are the same in the people

receiving PrEP as those who received the placebo The product caused no harm There is strong statistical significance (CI=-2.0 to

3.0%, p=0.01)

Scenario #4: Evidence of Harm

It makes things worse People who receive PrEP have 30% higher risk of

becoming HIV positive than those who receive the placebo

There is strong statistical significance (CI=-35 to -28%, p=0.01)

Discussion

What are key messages regardless of the trial result scenario?1. Prevention efforts must continue2. Research must continue3. Much has been learned to advance further

research4. [Our country] must prepare for eventual

availability of NPTs5. The safety and well-being of trial

participants remains the top priority for researchers

CRITICAL ANALYSIS OF TRIAL REPORTING

Exercise: CAPRISA microbicide trial

Understanding real trial results…

Prepare one PowerPoint slide or 50-word summary

Key Concepts of Media Literacy1. All media are constructions

2. Each person interprets messages differently

3. The media have commercial interests

4. The media contain ideological and value messages

5. Each medium has its own language, style, techniques, codes, conventions and aesthetics

6. The media have commercial implications

7. The media have social and political implications

8. Form and content are closely related in the media

Forms of Bias in Media

Support or attack a particular political party, candidate, or ideology

Advertising bias Corporate bias Mainstream bias Sensationalism Favour or attack a particular race, religion,

gender, age, sexual orientation, or ethnic group.

Exercise: Media Analysis

1. How accurate is the media coverage of the trial results?

2. What biases (if any) do you detect?

3. What seem to be the sources for the article?

4. What is the overall tone of the article?

5. What do you think the impact of the article might be in the community?

6. How does the article’s content and tone compare to what you think should be conveyed to the community?

7. How does the summary you prepared in the previous exercise compare to the key message from this article?

Exercise: Media Analysis

Community response to media coverage

What do you think of TAC’s response? Did they address the same concerns you

had when you read the Sowetan article? Were any elements missing? Are there any consequences—good or bad

—to responding like this? Would you have responded? In a similar

way? Differently?