washington d.c., usa, 22-27 july 2012 statistical design and analysis for immune correlates...

Washington D.C., USA, 22-27 July 2012www.aids2012.org

Statistical Design and Analysis for Immune Correlates Assessment:

Basic Concepts and RV144 Illustration

Yunda Huang1, Holly Janes1, 2, Peter Gilbert1, 2

1 Vaccine and Infectious Disease Division

Fred Hutchinson Cancer Research Center2Department of Biostatistics, University of Washington

Seattle, Washington, USA

IAS 2012 Correlates Workshop, Washington DC, USA


Reasons to Be a Statistician

• …• …• …• …• No one knows what we do so we are always right.


Outline

• Definitions

• Correlates Study and Sampling Design

• Example: RV144 Immune Correlates Study

• Strategies to Evaluate Immune Correlates

• Summary


In the context of preventive HIV vaccine clinical trials

• Rate of HIV infection: frequency of new HIV infections during a specified time frame – can be measured in vaccine (Rv) and placebo (Rp) groups separately

• Vaccine efficacy: proportion of infections prevented by vaccine relative to placebo (1- Rv/ Rp) – need to know the rate of infection in both vaccine and placebo groups

• Correlates of risk (CoR): Immune markers statistically correlated with the rate of HIV infection in the vaccine group (Qin & Gilbert et al., JID, 2007)

• Correlates of protection (CoP): Immune markers statistically correlated with vaccine efficacy in the vaccine and placebo groups (Qin & Gilbert et al., JID, 2007; Plotkin & Gilbert, CID, 2012)


Correlates of Risk (CoR) and Correlates of Protection (CoP)

• Vital for vaccine development

– Choice of antigens included in vaccines

– Bridge from previously collected protection data

– Surrogate for efficacy evaluation

– Population and individual level immunity measure

• Easily being confused and used inter-changeably

• Three facts:

1. A CoR may not be a CoP, but could be

2. A CoP must be a CoR

3. Not all CoRs or CoPs are created equal


Fact #1a: A CoR may not be a CoP

• It is a CoR – because the levels of the immune response are correlated with the rate of infection in the vaccine group

• Is it a CoP?

Immune Response

Ra

te o

f HIV

Infe

ctio

n



• It is a CoR

• It is not a CoP –

if, in the same way, the rate of infection is correlated with the immune response (had it been measured) in the placebo as well

Ra

te o

f HIV

Infe

ctio

n

Immune Response



• It is a CoR

• It is not a CoP

• The immune responses from this biomarker are not predictive of VE (≠ CoP), although overall VE=40%

Va

ccine E

fficacy (%)

Ra

te o

f HIV

Infe

ctio

n

Immune Response


Fact #1a: A CoR may not be a CoP• It is a CoR • It is not a CoP • Why:

– Those individuals who could mount a strong immune response are better able to ward off infection “on their own” with no impact of the vaccine-induced immune responses of this marker

– “On their own”: It may mark susceptibility to infection independent of Vaccination, e.g., risk behavior or host genetics

Ra

te o

f HIV

Infe

ctio

n Va

ccine E

fficacy (%)

Immune Response


Fact #1b: A CoR could be a (perfect) CoP

• It is a CoR

• Is it a CoP?

Ra

te o

f HIV

Infe

ctio

n

Immune Response


Fact #1b: A CoR could be a (perfect) CoP

• It is a CoR

• It is a CoP: those individuals who could mount a strong immune response are better able to remain uninfected, differently in vaccine and placebo recipients if assigned vaccine

• And, these immune responses (=CoR) are also predictive of vaccine efficacy (=CoP)

Ra

te o

f HIV

Infe

ctio

n

Immune Response

Va

ccine E

fficacy (%)


Fact #2: A CoP must be a CoR

• It’s equivalent to show: If not a CoR, then not a CoP

• Immune responses from vaccinees are not predictive of rate of infection -- not a CoR

Ra

te o

f HIV

Infe

ctio

n

Immune Response




• Immune responses from placebos will not be predictive of rate of infection

Ra

te o

f HIV

Infe

ctio

n

Immune Response




• Immune responses will not be predictive of vaccine efficacy

Ra

te o

f HIV

Infe

ctio

n

Immune Response

Va

ccine E

fficacy (%)


Fact #3a: Not all CoRs are created equal

Immune Response

Ra

te o

f HIV

Infe

ctio

n


Fact #3b: Not all CoPs are created equal

Immune Response

Va

ccin

e E

ffica

cy (

%)


Outline

• Definitions

1. A CoR may not be a CoP, but could be

2. A CoP must be a CoR

3. Not all CoRs or CoPs are created equal

• Correlates Study and Sampling design • Example: RV144 Immune Correlates study • Strategies to Evaluate Immune Correlates • Summary


Time-dependent and Time-independent CoR

• Time-independent immune correlates analysis: discover

correlates at a specific time point

– e.g. immune responses 2 weeks after the last vaccination

– Peak immune response time point close to baseline

– Informative and practical

• Time-dependent immune correlates analysis: discover

correlates whose levels may change over time

– e.g., most recent immune responses before diagnosis of infection

– Immune response near the time of exposure with the acute risk of

infection

– Informative about the mechanism of protection


CoR Study Design

• HIV vaccine-induced immune responses are only assessed in vaccinees

• Statistical power of an immune correlates study is driven by the number of HIV infections among vaccinees

• For a given total, the number of vaccinee infections depend on the vaccine efficacy: the higher the VE, the smaller number of infections from the vaccine group

• The smaller # infections is, the stronger the correlation between the immune response and the rate of infections needs to be, in order to have the same statistical power for CoR detection


CoR Study Sampling design

• With unlimited resources, we could measure the post-vaccination immune responses from every vaccinees

• Several cohort study designs have been developed to save resources with minimal loss of power after adjusting for the sampling design

• Case-cohort (traditional): controls are sampled without regard to infection time as part of a subcohort

• Case-control: controls are sampled after ascertainments of cases


CoR Study Sampling Design: Case-cohort

• Traditionally, controls are sampled without regard to failure times as part of a subcohort– Sampling can be done a priori without regard to case

status or time– All cases are included whether they occur in the

subcohort or not; controls are included only if in the subcohort

– Estimate population level immune responses– Could select controls for multiple outcomes

• Lately, some case-cohort designs are also outcome-dependent


CoR Study Sampling Design: Case-control

• Controls are sampled after ascertainments of cases

• Individual matching, frequency matching or stratification

to sample appropriate controls for cases

• Matching addresses issues of confounding in the

DESIGN stage of a study as opposed to the ANALYSIS

phase, providing a more efficient analysis (reduction in

standard errors of estimates)

• Matching on non-confounders may lose efficiency

compared to the non-matched case-control approach


Analysis Method

• Standard Cox or logistic regression models if data on the full cohort were available

• Modified Cox or logistic regression models if sub-sampling is done to account for the sampling design– Breslow and Holubkov (1997, Biometrika)– Borgan et al. estimator II (2000, Lifetime Data Analysis)


Outline

• Definitions• Correlates Study and sampling design

– Number of infections drives the power of the study– Sampling designs with corresponding analysis

methods

• Example: RV144 immune correlates study • Strategies to evaluate different immune

correlates • Summary


RV144 Thai Trial Primary Results

28


29

RV144 Thai Trial


Impetus for the Correlates Study:Evidence for Partial Vaccine Efficacy

Objective: To carry out an immune correlates analysis to begin to identify how the vaccine might work

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5

1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.0

Years

Prob

abili

ty o

f HIV

Infe

ction

(%) Placebo

Vaccine

C. Modified Intention-to-Treat Analysis*

*N=16,395 assessed; 51 Vaccine, 74 Placebo HIV-1 infected Estimated VE = 31% [95% CI 1−51%], p=0.04

30


RV144 Correlates of Risk Results

31


What the RV144 Correlates Study Assessed

• The analysis sought to discover Correlates of Risk (CoR): Immune response variables that predict whether vaccinees become HIV-1 infected

• Thus, the study is designed to generate hypotheses that certain immune responses are Correlates of Protection (CoP) that would need validation in future research

32


RV144: Two Tiers of Studies

• Pilot immunogenicity studies– Multiple immunology labs to perform assays on

sample-sets from HIV uninfected RV144 participants

– Conducted standardized comparative analyses of all candidate assays, to down-select the best performing assays and to optimize the immune variables to study as correlates

• Case-control study– Assessed the selected immune variables as

correlates of infection risk

33


RV144 Example Pilot Data: gp70-V1V2 Binding Antibodies (ELISA)

34


RV144: Criteria for Advancing Assaysto the Case-Control Study

35

Criterion

1. Represents a niche in immunological space (not highly correlated with other assays) 2. Low false positive rate (judged in placebo recipients and Week 0 responses of vaccinees)

3. Vaccine-induced responses with broad variability 4. Relatively low noise (e.g., high reproducibility on replicate samples)

5. Relatively low specimen volume requirement 6. Previously supported as a correlate of infection in the North American VaxGen trial of AIDSVAX


RV144 Case-Control Analysis: Two Tiers

• Primary Analysis: 6 priority immune response variables

• Secondary Analysis: All other immune response variables that passed pilot study criteria

• This division maximizes statistical power for the priority immune variables while allowing a broader exploratory analysis

36


Down-Selected Primary Immune Variables

Primary Variable Principal Investigator

• Plasma IgA Binding (14 envelope panel) Georgia Tomaras

• IgG avidity score to A244 gp120 Munir Alam

• Antibody-dependent cellular cytotoxicity-AE-92TH023. HIV infected CD4 T cells

David EvansMichael Alpert

• Neutralization of Tier 1 viruses (6 envelope panel) David Montefiori Rungpeung Sutthent Chitraporn Karnasutra

• IgG binding to scaffolded gp70-V1V2 Susan Zolla-Pazner

• CD4 T cell intracytoplasmic cytokines (IFN, IL-2, TNF, CD154) stimulated by AE-92TH023 peptides

Julie McElrathNicole Frahm

37


RV144 Case-Control Study

• Time-independent CoR: What are the immunologic measurements at a fixed time-point (wk26) in vaccinees that predict HIV-1 infection over a 3 year follow-up?

• Sampling design: Balanced stratified random sampling for vaccinees− 5:1 (control:case) ratio within each of the following covariate strata

Gender × Number of vaccinations × Per-protocol status

– 41 infected vaccinees (all available)

– 205 uninfected vaccinees (5:1 stratified random sample)

– 40 placebo recipients (simple random sample)

• Outcome-dependent 2-phase sampling case-cohort study

38


Why 5:1 Sampling? Only a Small Power Loss Moving from 5:1 to 10:1

39


RV144 Immune Correlates Study Main Result*

Variable Relative risk per sd P-value Q-value**

IgA Binding to Envelope Panel 1.54 0.027 0.08

IgG Avidity A244 gp120 0.81 0.37 0.56

ADCC AE.HIV-1 Infected CD4 Cells 0.92 0.68 0.68

Tier 1 Neutralizing Antibodies 1.37 0.22 0.45

IgG Binding to gp70-V1V2 0.57 0.015 0.08

CD4+ T Cell Intracellular Cytokines 1.09 0.61 0.68

*Multivariate logistic regression (quantitative variables) adjusted for gender, baseline behavioral risk (low, medium, high) **1-Qvalue ≈ estimated prob. that the immune variable correlates with infection rate• All 6 variables together in multivariate analysis: p=0.08• The 2 correlates in multivariate analysis: p=0.01

40


V1V2-gp70 Scaffold ELISA

Medium

High

Low

41


Cumulative Infection Rates with V1V2-gp70 Scaffold Assay

Estimated Relative Risk High vs. Low = 0.29

HighV1V2

Low/Medium V1V2

42


Plasma IgA Binding To Envelope Panel

Medium

High

Low

43


Cumulative Infection Rateswith IgA Env Binding Assay

Estimated Relative Risk High vs. Low = 1.89

High Env IgA

Low/Medium Env IgA

44


Sieve Analysis is an Integral Part of Immune Correlates Assessment

• The correlates analysis showed V1V2 Abs predicted infection in the vaccine group only

• Sieve analysis examines evidence for a difference in the sequences of viruses infecting vaccinees versus placebo recipients

– Observed differences can be attributed to the vaccine in a randomized trial

– Detection of a ‘sieve effect’ may suggest that the vaccine blocks infection with some types of exposing HIVs

• If certain epitope-specific Ab responses are protective, then would expect to see a relative absence of these specific epitopes in sequences of infected vaccinees compared to infected placebo recipients

• Found additional evidence for vaccine pressure on the V2 mid-loop region

45


• The gp70-V1V2 antibody CoR would be most useful for vaccine development if it strongly predicted VE (i.e., was a good CoP)

What it Could Mean(Most Useful for Vaccine Development)

46


• The gp70-V1V2 antibody CoR does not predicted VE (≠CoP)

But, It Could Also Mean

47


Outline

• Definitions• Correlates Study and Sampling design • Example: RV144 Immune Correlates Study

– Case-control study

– Evidence for two correlates of infection risk in vaccinees

– IgG antibodies that bind to scaffolded-V1V2 recombinant

protein correlated inversely with infection rate

– Plasma IgA antibodies correlated directly with infection

rate

• Strategies to evaluate different immune correlates • Summary


• Collect the requisite data for correcting the CoR analysis for potential exposure confounding (e.g., risk behavior, host genetics)

• Collect the requisite data for directly assessing the ability of a CoR to predict VE (more on next few slides)

• Conduct sieve analysis of HIV sequences to assess whether the vaccine applied pressure on the HIV Env target(s) specific to the immune correlate

• Collaborate with other groups (e.g, CHAVI, CAVD, VRC) conducting experiments (e.g., in non-human primates) testing hypotheses about the CoRs

49

Strategies to Assess CoRs as VE-Predictors (CoPs) and as Mechanisms of Protection


Once a positive CoR is discovered in vaccinees

• Collect the requisite data for directly assessing the ability of a CoR to predict VE

• To assess the relationship between VE and an immune marker (i.e., CoP), we need to know the level of the immune marker for both vaccine and placebo recipients – fill in all the blanks

Group OutcomeImmune response levels

1 2 3 4

VaccineUninfected

Infected

Placebo Uninfected

Infected


Once a positive CoR is discovered in vaccinees

• Collect the requisite data for directly assessing the ability of a CoR to predict VE

• To assess the relationship between VE and an immune marker (i.e., CoP), we need to know the level of the immune marker for both vaccine and placebo recipients -- fill in all the blanks

Group OutcomeImmune response levels

Total1 2 3 4

Vaccine

Uninfected

Infected

Total

Placebo

Uninfected ? ? ? ?

Infected ? ? ? ?

Total ~ ~ ~ ~

CoR study Clinical Trial


Predicting the potential HIV specific immune response (X) to HIV vaccination for placebo recipients

(Follmann, Biometrics, 2006)• BIP approach: baseline immunogenicity predictor (W)

– W is correlated with X

– At baseline, measure W in both vaccine and placebo recipients, e.g., immune responses to a non-HIV vaccine

– Randomization ensures W same in both vaccine and placebo groups

– Build statistical models between W and X based on vaccinees’ data

– Use placebo subjects’ W to impute X and pretend X is what we would have seen, had the placebo subjects received the HIV vaccine

• CPV approach: close-out placebo vaccination– At the end of the trial, inoculate placebo uninfecteds with HIV vaccine

– Measure immune response on the same schedule as was measured for vaccinees

– Pretend that is what we would have seen, had we inoculated at baseline


Remarks

• Correlates of Risk (CoR) and Correlates of Protection (CoP) are different but both important

• Impact of a positive CoR– Vaccine development (other speakers)

– HVTN go/no go guidelines based on RV144 immune correlates finding

• CoR assessment in vaccine efficacy trials– Future HVTN efficacy trials will use two-stage sequential

designs* with important secondary objectives to examine immune correlates

• BIP and/or CPV approach for the assessment of CoP

* References: • Gilbert P, Grove D, Gabriel E, Huang Y, Gray G, Hammer S, Buchbinder S, Kublin J, Corey L, Self

S (2011, Statistical Communications in Infectious Diseases) • Corey L, Nabel GJ, Dieffenbach C, Gilbert PB, Haynes BF, Johnston M, Kublin J, Lane HC,

Pantaleo G, Picker LJ, Fauci AS (2011, Science Translational Medicine)


Acknowledgement

• Research supported by– NIAID, NIH– USMHRP – Bill and Melinda Gates Foundation– Henry Jackson Foundation

• Statistical Center for HIV/AIDS research and Prevention (SCHARP)– Youyi Fong– Allan DeCamp– Ying Huang– Paul Edlefsen – Steve Self

• RV144 Immune Correlates Study

– Leadership: Bart Haynes, Peter Gilbert, Jerome Kim, Nelson Michael, Julie McElrath

– Host of laboratory scientists from several institutions

• Duke University led the antibody work (Bart Haynes, Georgia Tomaras, David Montefiori)

• FHCRC led the T cell work (Julie McElrath)


Appendix


Statistical Approach to Sieve Analysis

• Local sieve analysis (high-dimensional)– Assess Env amino acid (AA) sites as ‘signature sites’

• Signature = site with different distribution of residues vaccine vs. placebo relative to an insert-residue*

– Assess sets of AA sites as ‘signature sets’ • E.g., 9-mers potentially constituting T cell epitopes• E.g., clusters of sites potentially constituting antibody epitopes

• Global sieve analysis (low-dimensional)– Assess if and how vaccine efficacy depends on the

distance of the exposing virus to an insert-sequence– Global = summarize a breakthrough HIV by one or a few

numbers quantifying distance*3 vaccine-inserts: ALVAC-AE.92TH023, rgp120-AE.CM244, rgp120-B.MN

washington d.c., usa, 22-27 july 2012 statistical design and analysis for immune correlates...

Documents

rate of infection

vaccine rv

immune markers

immune responses

immune correlates assessment

vaccine relative

strong immune response

org4correlates of risk