promis: the right place at the right time? david cella, ph.d. department of medical social sciences...
TRANSCRIPT
PROMIS: The Right Place at the Right Time?
David Cella, Ph.D.Department of Medical Social Sciences
Northwestern UniversityChair, PROMIS Steering Committee
Principal Investigator, Statistical Coordinating Center
What is PROMIS?
• A nine-year commitment of NIH to improve and standardize measurement of patient reported outcomes (PROs)
• 50 million dollar investment (excl. ARRA)
• An effort to harness new psychometric methods to improve the quality and interpretability of PROs
• An attack on the PRO “Tower of Babel”
PROMIS Roadmap Goal:
• Create highly valid and reliable item banks • and associated computerized adaptive• testing• Encourage wide adoption to improve • assessment of
• self-reported symptoms • other health-related quality of life
• domains – …across a wide range of chronic diseases
UNC –Chapel Hill ● UNC –Chapel Hill ●
● Duke University● Duke University
● Stanford University● Stanford University
Stony Brook University
●
Stony Brook University
●● University of Pittsburgh● University of Pittsburgh
● University of Washington● University of Washington
Northwestern ♥Northwestern ♥ ●
NIHNIH●
NIHNIH
PROMIS Network: 2004-2009
“Item Bank”• A large collection of items measuring one
thing
• Items in the same bank are linked on a common metric
• Basis for – Computer Adaptive Testing (CAT)
• Items are selected to maximize precision and retain clinical relevance
– Customized short forms tailored to the population and research purpose
Building a Bank
Testing and Analysis
DepressionDefined and
Calibrated Bank
CAT and Multiple Short Forms
Items Deposited After Review
item
“PROMIS-1” Highlights
• Consensus-driven PRO framework• Qualitative study of > 1,000 people• Quantitative study of > 20,000 people• Eleven Item Banks available to collaborators• Website and functioning Assessment Center• Ongoing clinical validation in chronic disease
populations• Clear path/agenda for “PROMIS-2” and
expanded collaborations
Self-Self-reportedreported HealthHealth
Mental HealthMental Health
PhysicalPhysical HealthHealth
SocialSocial HealthHealth
Physical Function*Physical Function*
SymptomsSymptoms
Emotional DistressEmotional Distress
Cognitive FunctionCognitive Function
Social FunctionSocial Function
Positive Positive Psychological Psychological
FunctionFunction
Social Social RelationshipsRelationships
Upper Extremities: dexterityUpper Extremities: dexterity
Lower Extremities: mobilityLower Extremities: mobility
Central: neck and back Central: neck and back
Activities: Instrumental Activities of Daily Living Activities: Instrumental Activities of Daily Living
PainPain
Fatigue*Fatigue*
OtherOther
QualityQuality
BehaviorBehavior
Impact*Impact*
ExperienceExperience
ImpactImpact
Anxiety*Anxiety*
Depression*Depression*
AngerAnger
Substance AbuseSubstance Abuse
Negative Impact of IllnessNegative Impact of Illness
Alcohol AbuseAlcohol Abuse
Positive Impact of IllnessPositive Impact of Illness
Mastery & Control (self-efficacy)Mastery & Control (self-efficacy)
Subjective Well-Being (positive Subjective Well-Being (positive affect)affect)
Ability to Participate*Ability to Participate*
Satisfaction with Participation*Satisfaction with Participation*
SatisfactionSatisfactionSatisfactionSatisfaction
SatisfactionSatisfaction
SatisfactionSatisfaction
Meaning & SpiritualityMeaning & Spirituality
Stress ResponseStress Response
Self-conceptSelf-concept
Sleep/Wake Sleep/Wake Function*Function*
Sexual FunctionSexual Function
Social RolesSocial Roles
Discretionary Social ActivitiesDiscretionary Social Activities
Social RolesSocial Roles
Discretionary Social ActivitiesDiscretionary Social Activities
Meaning & SpiritualityMeaning & Spirituality
CopingCoping
Self-conceptSelf-concept
Social ConnectionSocial Connection
Social IsolationSocial Isolation
Social IsolationSocial Isolation
Quantity of Social Support Quantity of Social Support (Integration)(Integration)
Quality of Social SupportQuality of Social SupportInstrumental/InformationalInstrumental/Informational
CompanionshipCompanionship
Emotional SupportEmotional Support
Sleep DisturbanceSleep Disturbance
Wake DisturbanceWake Disturbance
CaPS bank in developmentCaPS bank in development
PROMIS v1.0 bankPROMIS v1.0 bank
PROMIS area tested but no bank developed PROMIS area tested but no bank developed for v1.0for v1.0
Area addressed (in part) by bank within Area addressed (in part) by bank within lineagelineage
Area not addressed yetArea not addressed yet
* = * = Additional cancer-specificAdditional cancer-specific PROMIS bank PROMIS bank
PROMIS Domain Framework
Item BankingItem Banking
Item BankItem Bank
Local IndependenceLocal Independence
Measurement RangeMeasurement Range
Target PopulationTarget Population
Static FormsStatic Forms
Multi-StageMulti-Stage
CATCAT
Item banks can support a variety of scale assembly strategiesItem banks can support a variety of scale assembly strategies
Advantages of Short-Forms Developed from Item Banks
• Flexibility in length and content– Select items matched to clinical features and
severity in the target population– Select items known to provide the most
information
• Any form selected or created produces scores on a common metric
PROMIS Profile Short Forms
11* reduced set (the full bank has 112 items) was used for real data simulation purposes
Anxiety29
Anxiety29
Depression28
Depression28
Fatigue95
Fatigue95
Pain Impact41
Pain Impact41
Sleep Disturbance
27
Sleep Disturbance
27
Physical Function86*
Physical Function86*
Social Role14
Social Role14
46
8
Mental
Physical
Social
Anxiety
T-score
rel.=.90
rel.=.95
0
5
10
15
20
25
10 30 50 70 90
Depression
T-score
rel.=.90
rel.=.95
0
10
20
30
10 30 50 70 90
Fatigue
T-score
rel.=.90
rel.=.95
0
10
20
30
10 30 50 70 90
Pain Impact
T-score
rel.=.90
rel.=.95
0
10
20
30
40
50
60
10 30 50 70 90
Physical Function
T-score
rel.=.90
rel.=.95
0
10
20
30
40
10 30 50 70 90
Sleep Disturbance
T-score
rel.=.90
rel.=.95
0
5
10
15
10 30 50 70 90
Social Role
T-score
rel.=.90
rel.=.95
0
10
20
30
40
10 30 50 70 90
SF 8SF 6SF 4
12
PROMIS Profile Subscales: Test Information Functions
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Anxiety
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Depression
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Fatigue
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Pain Impact
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Physical Function
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Sleep Disturbance
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Social Role
Length
Cor
rela
tion
PROMIS Profile Subscales: Correlations with Full Banks
14
Anxiety
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Depression
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Fatigue
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Pain Impact
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Physical Function
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Sleep Disturbance
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Social Role
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
SF 8SF 6SF 4
Estimated Power to detect a small effect (d=0.2) when the study population is centered on the general population (T-Score = 50)
15
Anxiety
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Depression
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Fatigue
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Pain Impact
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Physical Function
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Sleep Disturbance
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Social Role
sample size (n per group)
pow
er
0 50 100 200 300
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
SF 8SF 6SF 4
Estimated Power to detect a small effect (d=0.2) when the study population is centered on 1 SD worse than the general population
Computerized Adaptive Testing (CAT)
• Select questions based on a person’s response to previously administered questions.
• Iteratively estimate a person’s standing on a domain (e.g., depressive symptoms)
• Administer most informative items
• Desired level of precision can be obtained using the minimal possible number of questions.
Beginning of CAT
01
23
45
Item 15
T-ScoreIn
form
atio
n
20 30 40 50 60 70 80
|
Max at T-Score=56
20 40 60 80
T-Score
Po
ste
rio
r D
istr
ibu
tion
20 30 40 50 60 70 80
T-Score = 50 SE = 10
Items Administered
T-S
core
0 1 2 3 4 5 6 7 8
20
30
40
50
60
70
80 Items: 15
T-Score: 50 SEM: 10
Best Item-I felt depressed
0.0
0.2
0.4
0.6
0.8
1.0
Item 15
T-Score
Pro
ba
bili
ty
20 30 40 50 60 70 80
12345
I felt depressed1. Never2. Rarely3. Sometimes4. Often5. Always
T-Score = 52 SE = 4
Next Best Item-I felt like a failure
20 40 60 80
T-Score
Po
ste
rio
r D
istr
ibu
tion
20 30 40 50 60 70 80
Items Administered
T-S
core
0 1 2 3 4 5 6 7 8
20
30
40
50
60
70
80 Items: 15
T-Score: 52 SEM: 4 01
23
45
Item 10
T-ScoreIn
form
atio
n
20 30 40 50 60 70 80
|
Max at T-Score=57
0.0
0.2
0.4
0.6
0.8
1.0
Item 10
T-Score
Pro
ba
bili
ty
20 30 40 50 60 70 80
12345
I felt like a failure1. Never2. Rarely3. Sometimes4. Often5. Always
T-Score = 53 SE = 3
Next Best Item-I felt worthless
Items Administered
T-S
core
0 1 2 3 4 5 6 7 8
20
30
40
50
60
70
80 Items: 15,10
T-Score: 53 SEM: 3 01
23
45
Item 1
T-ScoreIn
form
atio
n
20 30 40 50 60 70 80
|
Max at T-Score=59
0.0
0.2
0.4
0.6
0.8
1.0
Item 1
T-Score
Pro
ba
bili
ty
20 30 40 50 60 70 80
12345
20 40 60 80
T-Score
Po
ste
rio
r D
istr
ibu
tion
20 30 40 50 60 70 80
I felt worthless1. Never2. Rarely3. Sometimes4. Often5. Always
T-Score = 55 SE = 2
Next Best Item-I felt helpless
Items Administered
T-S
core
0 1 2 3 4 5 6 7 8
20
30
40
50
60
70
80 Items: 15,10,1
T-Score: 55 SEM: 2 01
23
45
Item 3
T-ScoreIn
form
atio
n
20 30 40 50 60 70 80
|
Max at T-Score=58
0.0
0.2
0.4
0.6
0.8
1.0
Item 3
T-Score
Pro
ba
bili
ty
20 30 40 50 60 70 80
12345
20 40 60 80
T-Score
Po
ste
rio
r D
istr
ibu
tion
20 30 40 50 60 70 80
I felt that nothing could cheer me up1. Never2. Rarely3. Sometimes4. Often5. Always
T-Score = 55 SE = 2
Items Administered
T-S
core
0 1 2 3 4 5 6 7 8
20
30
40
50
60
70
80 Items: 15,10,1,3,21,2,5
T-Score: 55 SEM: 2
20 40 60 80
T-Score
Po
ste
rio
r D
istr
ibu
tion
20 30 40 50 60 70 80
26
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Anxiety
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Depression
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Fatigue
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Pain Impact
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Physical Function
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Sleep Disturbance
Length
Cor
rela
tion
1 2 3 4 5 6 7 8
0.80
0.85
0.90
0.95
1.00
Social Role
Length
Cor
rela
tion
SFCAT
PROMIS Profile Subscales vs CATs: Correlations with Full Banks
PROMIS Assessment Center and Opportunities for Interaction
A publicly available, adaptable and sustainable Internet-based system that:
1. Hosts 11 item banks for precise, valid, and efficient health status assessment via short forms or CAT
2. Collects and analyze patients’ responses3. Provides instant health status reports to users to:
• Enable and enhance research• Improve clinical decision-making• Facilitate policy-making by health plan and systems and
public programs
4. Supports health services / comparative effectiveness research applications
http://www.nihpromis.org
29
Assessment Center Aims
Assessment Center supports…
• Research and clinical applications
• CAT
• Custom or off-the-shelf short forms
• Non-PROMIS measures welcome
• Study set up, maintenance and storage
• Scoring, reporting and interpretation
DomainsItems
inBank
Items in First ShortForm
Emotional Distress – Anger 29 8Emotional Distress – Anxiety 29 7Emotional Distress – Depression 28 8Fatigue 95 7Pain – Behavior 39 7Pain – Impact 41 6Physical Function 124 10Satisfaction with Discretionary Social
Activities12 7
Satisfaction with Social Roles 14 7Sleep Disturbance 27 8Wake Disturbance 16 8Global Health 10
2009 PROMIS Ver 1.0 Instruments
