Guidelines for Guidelines for Appropriate OAI Data UseAppropriate OAI Data Use

December 6, 2007December 6, 2007

Yuqing Zhang, Boston University

OAI Design and Analysis IssuesOAI Design and Analysis Issues

• Many of the variables are clustered by knee.Many of the variables are clustered by knee.

• Many of the variables are collected Many of the variables are collected longitudinally over time on a person and/or longitudinally over time on a person and/or kneeknee

• Expensive MRI measurements may not be Expensive MRI measurements may not be available for all participants – may make available for all participants – may make case-control design (nested within cohort) case-control design (nested within cohort) more attractivemore attractive

ConsequencesConsequences

• Need to weigh the advantages and Need to weigh the advantages and disadvantages of various design strategies.disadvantages of various design strategies.

• Failure to properly analyze clustered or Failure to properly analyze clustered or longitudinal data can cause p-values and longitudinal data can cause p-values and confidence intervals to be seriously confidence intervals to be seriously incorrect.incorrect.

AgendaAgenda

• Yuqing Zhang to discuss design considerations Yuqing Zhang to discuss design considerations with a 15 minute question and answer periodwith a 15 minute question and answer period

• Chuck McCulloch to discuss analysis issues Chuck McCulloch to discuss analysis issues with a 15 minute question and answer period with a 15 minute question and answer period

Study DesignStudy Design

Types of Epidemiologic StudyTypes of Epidemiologic Study

• Cross-sectional StudyCross-sectional Study

• Cohort StudyCohort Study

• Longitudinal cohort studyLongitudinal cohort study

• Case-control StudyCase-control Study

• Matched case-control studyMatched case-control study

Cross-sectional StudyCross-sectional Study

• Assess presence or absence of both risk factor and disease at Assess presence or absence of both risk factor and disease at same point in time among study population and examine their same point in time among study population and examine their association association

• Measure of associationMeasure of association

odds of disease among exposed subjects odds of disease among exposed subjects Prevalence odds ratio = Prevalence odds ratio =

odds of disease among non-exposed subjectsodds of disease among non-exposed subjects

Cross-sectional StudyCross-sectional Study

ROAROA Subjects with Subjects with Knee PainKnee Pain

No YesNo Yes

Total No. Total No. SubjectsSubjects

Prevalence of Prevalence of Knee Pain (%)Knee Pain (%)

Prevalence Prevalence odds ratioodds ratio

NONO 700700 100100 800800 12.512.5 1.01.0

YESYES 120120 8080 200200 40.040.0 4.74.7

In a survey of 1000 subjects, 200 subjects had ROA in at In a survey of 1000 subjects, 200 subjects had ROA in at least one knee. Of those with knee OA, 80 reported knee least one knee. Of those with knee OA, 80 reported knee pain; and of those without knee ROA, 100 reported knee pain; and of those without knee ROA, 100 reported knee painpain

Cohort StudyCohort Study

Subjects are classified on Subjects are classified on the basis of presence or the basis of presence or absence of exposure to a absence of exposure to a particular risk factor and particular risk factor and followed for a specified followed for a specified period of time to period of time to determine development of determine development of disease in each groupdisease in each group

Measure of associationMeasure of association

RR = CIe / CIu

Cohort StudyCohort Study

BMIBMI

(kg/m(kg/m22))

No. subjects No. subjects Developing Knee Developing Knee

ROAROA

Total No. Total No. SubjectsSubjects

Risk of ROA Risk of ROA

Over 2 years (%)Over 2 years (%)

Risk Ratio Risk Ratio

< 25< 25 5050 10001000 5.05.0 1.01.0

>> 25 25 200200 20002000 10.010.0 2.02.0

3000 subjects without knee ROA were followed up for 2 3000 subjects without knee ROA were followed up for 2 years. Among 2000 subjects with BMI years. Among 2000 subjects with BMI >> 25 kg/m 25 kg/m22, 200 , 200 developed knee ROA. Among subjects with BMI < 25 kg/mdeveloped knee ROA. Among subjects with BMI < 25 kg/m22, , 50 developed knee ROA. 50 developed knee ROA.

Longitudinal Cohort StudyLongitudinal Cohort Study

• Studies in which outcome variable is repeatedly Studies in which outcome variable is repeatedly measured over time (i.e., outcome variable is measured measured over time (i.e., outcome variable is measured in same individual on several different occasions)in same individual on several different occasions)

• Observations of one individual over time are not Observations of one individual over time are not independentindependent

• Special statistical techniques are required to take into Special statistical techniques are required to take into account correlation between repeated observationsaccount correlation between repeated observations

• It can assess effect of change of exposure over time on It can assess effect of change of exposure over time on change in outcome variablechange in outcome variable

Longitudinal Cohort StudyLongitudinal Cohort Study

• Both knee pain (WOMAC) and BMI were assessed Both knee pain (WOMAC) and BMI were assessed repeatedly (5 times) among participants repeatedly (5 times) among participants

• Is greater BMI associated with higher WOMAC score Is greater BMI associated with higher WOMAC score (cross-sectional association)(cross-sectional association)

• Is change in BMI associated with change in WOMAC score Is change in BMI associated with change in WOMAC score (longitudinal association)(longitudinal association)

• Is increasing BMI associated with increasing WOMAC Is increasing BMI associated with increasing WOMAC scorescore

• Is decreasing BMI associated with decreasing WOMAC Is decreasing BMI associated with decreasing WOMAC score score

Case-control StudyCase-control Study• Subjects with a disease of Subjects with a disease of

interest are selected as the interest are selected as the cases, and a sample of cases, and a sample of subjects from the source subjects from the source population which gives rise population which gives rise to cases are selected as the to cases are selected as the controlscontrols

• ‘‘Nested case-control’ study: Nested case-control’ study: cases and controls are drawn cases and controls are drawn from the defined sample of a from the defined sample of a cohort studycohort study

• Measure of associationMeasure of association

OR (odds ratio) =(a*d)/(b*c)OR (odds ratio) =(a*d)/(b*c)

K. Rothman, Epidemiology: an Introduction, 2002

Case-control StudyCase-control Study• Of 100 knees with incident knee pain at follow-up visit (cases), Of 100 knees with incident knee pain at follow-up visit (cases),

40 knee had BMLs present at baseline examination; among 100 40 knee had BMLs present at baseline examination; among 100 knees without pain (controls), 10 had BMLs present at baseline knees without pain (controls), 10 had BMLs present at baseline examination.examination.

Frequent Frequent Knee PainKnee Pain

Presence of BMLs at BaselinePresence of BMLs at Baseline

No YesNo Yes

Odds Ratio Odds Ratio

Controls Controls 9090 1010 1.01.0

Cases Cases 6060 4040 6.06.0

Matched Case-control StudyMatched Case-control Study

• Subjects are selected in such a way that some Subjects are selected in such a way that some potential confounders (i.e., matching variables) are potential confounders (i.e., matching variables) are distributed in an identical manner among cases and distributed in an identical manner among cases and controlscontrols• Explicit matching Explicit matching

• e.g., each case and control pair was matched by study e.g., each case and control pair was matched by study center, sex age (center, sex age (++ years), etc. years), etc.

• Implicit matchingImplicit matching• e.g., knee with pain (e.g., knee with pain (casecase) vs. contra-lateral knee ) vs. contra-lateral knee

without pain (without pain (controlcontrol))

Should We Analyze Knees or Persons?Should We Analyze Knees or Persons?

BackgroundBackground

• In a knee OA study, data are often collected at In a knee OA study, data are often collected at both the person-level and the knee-level both the person-level and the knee-level

• Analysis for knee-specific outcome variable Analysis for knee-specific outcome variable presents special issue in statistical inference presents special issue in statistical inference owing to correlation between two knees within owing to correlation between two knees within a persona person

Observation Unit (Knees vs. Person) Observation Unit (Knees vs. Person)

• Research QuestionsResearch Questions• Do subjects with SxOA take longer time to complete 15-m Do subjects with SxOA take longer time to complete 15-m

walk than those without SxOA?walk than those without SxOA?

• Is greater BMI associated with an increased risk of knee Is greater BMI associated with an increased risk of knee ROA? ROA?

• Is presence of synovitis associated with prevalent knee pain?Is presence of synovitis associated with prevalent knee pain?

• Are changes of severity of bone marrow lesions (BMLs) Are changes of severity of bone marrow lesions (BMLs) associated with pain fluctuation? associated with pain fluctuation?

• Cost of Additional Data Collection Cost of Additional Data Collection • Outcome Outcome • Exposure Exposure • ConfoundingConfounding

• Residual ConfoundingResidual Confounding

Research QuestionsResearch Questions

• Do subjects with SxOA take longer time to complete 15-m Do subjects with SxOA take longer time to complete 15-m walk than those without SxOA? walk than those without SxOA? • Outcome is a person-basedOutcome is a person-based

• Is greater BMI associated with an increased risk of knee Is greater BMI associated with an increased risk of knee ROA? ROA? • Outcome can be either person-based or knee-basedOutcome can be either person-based or knee-based

Person-Based ApproachPerson-Based Approach

• Observation unit for the outcome variable is Observation unit for the outcome variable is the personthe person• Worst kneeWorst knee• None, unilateral, or bilateral None, unilateral, or bilateral

• Model outcome as a function of a set of risk Model outcome as a function of a set of risk factors factors

Pros and ConsPros and Cons

• Statistical software is readily available Statistical software is readily available (logistic regression, linear regression) (logistic regression, linear regression)

• Effect estimate for site-specific risk factor might Effect estimate for site-specific risk factor might be biased be biased

• Statistical power might be sacrificed Statistical power might be sacrificed

• Is greater BMI associated with an increased Is greater BMI associated with an increased risk of knee ROA? risk of knee ROA?

• Is presence of synovitis associated with Is presence of synovitis associated with prevalent knee pain?prevalent knee pain?

Research Question

Knee-Based ApproachKnee-Based Approach

• Observation unit for the outcome variable is the Observation unit for the outcome variable is the kneeknee

• Model outcome in each knee as a function of a Model outcome in each knee as a function of a set of risk factors, while accounting for set of risk factors, while accounting for correlation between two knees within a person correlation between two knees within a person

Pros and ConsPros and Cons

• Offer improved statistical powerOffer improved statistical power

• Provide new insights into etiology of risk factorProvide new insights into etiology of risk factor

• Statistical software is readily available Statistical software is readily available (Generalized Estimating Equations (GEE)) (Generalized Estimating Equations (GEE))

• Increase cost of data collection in both kneesIncrease cost of data collection in both knees

• Is presence of synovitis associated with Is presence of synovitis associated with prevalent knee pain?prevalent knee pain?

Research QuestionsResearch Questions

Cost of Data Collection Cost of Data Collection

• All subjects in OAI had knee MRI taken All subjects in OAI had knee MRI taken repeatedlyrepeatedly

• Cost of reading all MRIs is extreme Cost of reading all MRIs is extreme • To reduce cost, read MRIs of cases and a sample of controlsTo reduce cost, read MRIs of cases and a sample of controls

Traditional ApproachTraditional Approach Study design: case-control study Study design: case-control study

• Knees with pain (Knees with pain (casescases))

• Knees without pain (Knees without pain (controlscontrols) )

• Assess presence of synovitis (or its severity)Assess presence of synovitis (or its severity)

• Evaluate association between synovitis and Evaluate association between synovitis and prevalent knee pain using logistic regression prevalent knee pain using logistic regression model adjusting for confounding factors model adjusting for confounding factors

Pros and ConsPros and Cons

• Reduce cost of data collectionReduce cost of data collection

• Statistical model is readily available Statistical model is readily available (logistic regression model)(logistic regression model)

• Residual confounding is a concernResidual confounding is a concern

Which is better control knee? Which is better control knee?

Person-Matched Case-Control DesignPerson-Matched Case-Control Design

• Eligible subjects Eligible subjects

• Knee with pain (Knee with pain (casecase) )

• Contra-lateral knee without pain (Contra-lateral knee without pain (controlcontrol))

• Case and control are implicitly matched by Case and control are implicitly matched by person-level risk factorsperson-level risk factors

Pros and ConsPros and Cons

• Reduce cost of data collectionReduce cost of data collection

• Eliminate effect of person-level confounding Eliminate effect of person-level confounding factorsfactors

• Statistical model is readily available Statistical model is readily available (conditional logistic regression)(conditional logistic regression)

• Unable to examine effect of change in risk factor Unable to examine effect of change in risk factor on pain fluctuationon pain fluctuation

• Findings may not apply to persons with bilateral Findings may not apply to persons with bilateral knee painknee pain

• Are changes of severity of synovitis Are changes of severity of synovitis associated with pain fluctuation?associated with pain fluctuation?

Research QuestionsResearch Questions

Potential ConfoundingPotential Confounding

• Pain is subjective measurement Pain is subjective measurement • TThere is a natural variability in pain sensitivity, here is a natural variability in pain sensitivity,

perception and tolerance to pain stimuliperception and tolerance to pain stimuli• Many factors affecting pain variability are often not Many factors affecting pain variability are often not

collected in the studycollected in the study• genetic predispositiongenetic predisposition• socio-cultural environmentsocio-cultural environment• pprior experiencerior experience• expectationsexpectations• current mood statuscurrent mood status

• Use each knee as its own control to minimize potential Use each knee as its own control to minimize potential confounding between personsconfounding between persons

Traditional ApproachTraditional Approach

Study design: longitudinal study Study design: longitudinal study

• Collect data on presence of pain for each knee Collect data on presence of pain for each knee repeatedly repeatedly

• Collect data on presence (or severity) of Collect data on presence (or severity) of synovitis for each knee repeatedlysynovitis for each knee repeatedly

• Analyze severity of synovitis and its change in Analyze severity of synovitis and its change in relation to pain fluctuation adjusting for relation to pain fluctuation adjusting for confounding factorsconfounding factors

Pros and ConsPros and Cons

• Allow investigators to assess relation of severity Allow investigators to assess relation of severity of synovitis and its change to occurrence of knee of synovitis and its change to occurrence of knee pain pain

• Statistical model is readily available (GEE or Statistical model is readily available (GEE or MIXED model)MIXED model)

• Cost of assessing synovitis from MRI for all Cost of assessing synovitis from MRI for all subjects is too highsubjects is too high

• Residual confounding is still a concernResidual confounding is still a concern

Self-Matched Case-Control DesignSelf-Matched Case-Control Design

• Each knee serves as its own control Each knee serves as its own control (akin to case-crossover study design) (akin to case-crossover study design)

• Included are only knees that have pain at one or more, Included are only knees that have pain at one or more, but but not allnot all, examinations, examinations

• Synovitis, either severity or presence, is assessed for Synovitis, either severity or presence, is assessed for each knee repeatedly each knee repeatedly

Eligible KneesEligible Knees

• Knees which had pain in Knees which had pain in at least oneat least one, , but not allbut not all,, visits (i.e., exams) visits (i.e., exams)

• Case-visit:Case-visit: a knee had pain at a scheduled visit a knee had pain at a scheduled visit

• Control visitControl visit: the : the samesame knee that did not have pain knee that did not have pain at a scheduled visit at a scheduled visit

Knees Baseline 12-month 24-month Eligible Knees Baseline 12-month 24-month Eligible

1

2

3

4

1

2

3

4

pain -> no pain -> pain

no pain -> pain -> no pain

pain -> no pain -> pain

no pain -> pain -> no pain

NO

NO

NO

NO

5

6

7

8

5

6

7

8

pain -> no pain

pain -> no pain

no pain -> pain

no pain -> pain

pain -> no pain

pain -> no pain

no pain -> pain

no pain -> pain

Knees Baseline 12-month 24-month Eligible Knees Baseline 12-month 24-month Eligible

Pros and ConsPros and Cons

• Allow investigators to examine effect of change Allow investigators to examine effect of change in severity of synovitis on pain fluctuationin severity of synovitis on pain fluctuation

• Statistical model is readily available Statistical model is readily available (conditional logistic regression model)(conditional logistic regression model)

• Reduce cost of MRI readingReduce cost of MRI reading

• Minimizing residual confoundingMinimizing residual confounding

• Unable to identify risk factors for persistently Unable to identify risk factors for persistently painful or never painful kneespainful or never painful knees

SummarySummary • No single study design can fit every research question No single study design can fit every research question

in OAI in OAI • When choosing study design, one must considerWhen choosing study design, one must consider

• CostCost• Internal validity Internal validity

• ConfoundingConfounding• BiasBias

• Generalizability or external validity (Generalizability or external validity (??????) ) • Collaboration and group effort are required Collaboration and group effort are required

• Clinician, basic scientist, statistician, epidemiologistClinician, basic scientist, statistician, epidemiologist

It takes a village!!!!It takes a village!!!!

Thank you for your attention.Thank you for your attention.