building on the measurement of cfir constructs in fqhcs: where do we go from here? maria fernandez,...
TRANSCRIPT
Building on the Measurement of CFIR Constructs in FQHCs: Where Do We Go From Here?
Maria Fernandez, PhDon behalf of the CPCRN FQHC WG
Investigators
CPCRN Steering Committee - Virtual Spring WebinarApril 16, 2014
CPCRN CHC Survey
FQHC Workgroup
Partnership Committee
Qualitative Inquiry
Subgroup
Data Subgroup
CHC Survey
Subgroup
Align with CHCs’ missions
Guided by real world
health policy
&health
care delivery
landscapes Health Care Reform
Meaningful Use of EHRPatient-Centered Medical Home
National Association of Community Health Centers (NACHC)
Primary Care Associations
Community Health Centers (CHCs)
Developing Measures of Constructs Associated with EBI Implementation
Purposes
• Describe the survey development process and rationale
• Present initial psychometric validation of CFIR Inner Setting measures
• Discuss findings and what is next in our work assessing CFIR constructs (and PAR) related to EBI implementation
Value
• One of the first studies to measure several constructs from CFIR
• Developed and tested measures that describe factors influencing implementation of EBAs in FQHCs
• Sets the foundation for establishing causal pathways and developing interventions that target these factors
Consolidated Framework for Implementation Research (CFIR)
Source: Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science 2009; 4:50.
Note: Authors from the VA and University of Michigan, SPH, Department of Health Management and Policy
CFIR Constructs
Patient Needs & Resources,
External Policies & Incentives
Available Resources, Leadership
Engagement, Culture,
Implementation Climate,
Learning Climate
Knowledge & Beliefs
about EBAs
Executing,Reflecting & Evaluating
Relative Advantage, Complexity
Outer Setting
ProcessCharacteristics of Individuals
Inner Setting
Characteristics of Intervention
Multiple Recruitment Strategies
• Sites partnered with their state’s PCAs; PCAs emailed their CHCs
• Sites recruited CHCs via email, telephone calls, or in-person meetings
• Introductory email with online survey link; 4 reminder emails; in-person meeting (one site)
• January - May 2013• IRB approval at each site and coordinating center
Inner Setting Constructs
• Available Resources• Culture Overall• Culture Effort• Culture Stress• Implementation Climate• Learning Climate• Readiness for Change
Analysis plan
• We conducted separate CFAs for each of the seven inner setting constructs.
• Multiple fit indices were used to evaluate CFA model fit (CFI >.90, TLI >.90, SRMR <.08, and RMSEA <.08).
• Cronbach’s alpha to evaluate internal consistency reliability.
• Correlation coefficients of each pair of scales to examine discriminant validity.
• Inter-rater reliability (ICC(1) & ICC(2)) and inter-rater agreement (rWG(J)) statistics were computed to assess the
reliability and validity of computing clinic- and systems-level means from the individual-level data.
Available Resources
A35A
A35B
A35C
C20A
C20B
C20C
C20D
.419
.546
.580
.708
.766
.626
.741
A11
A12
A13
A14
Readiness for change
.824
.896
.847
.872
C11
C12
C13
C05
Implementation climate
.546
.830
.759.462
A01
A06
A02
A15
A19
Learning climate
.694
.646
.768.727.735
A03
A05
A16Rev
A07
A22
A08
A09
A10
A21
A36
A37
A38
A39
A40
A21
A42Rev
A43
A44Rev
Culture
.663
.723.556.642.742.674.719.731.775
.332.438
.387.594
.480
.612.440
.581.387
Inner Setting
0.627
0.945
0.980
0.514
0.968
Combined CFA results and fit statistics
CFA Standardized Factor Loadingsa
1. Available Resources 0.627 (0.068)2. Culture 0.945 (0.022)3. Implementation Climate 0.514 (0.068)
4. Learning Climate 0.980 (0.024)
5. Readiness for Change 0.968 (0.017)
Robust Std. Errors in parenthesis CFI = 0.848 TLI = 0.835; SRMR = 0.079; RMSEA = 0.065
Discriminant validity of the five dimensions of inner setting
Scale Culture Available resources
Learning climate
Implementation climate
Readiness for change
Culture 1.00
Available resources
0.3641 1.00
Learning climate 0.6757 0.4555 1.00
Implementation climate
0.2464 0.6254 0.3201 1.00
Readiness for change
0.6978 0.5203 0.8114 0.3758 1.00
Correlations using average score for each scale*p < .05
Clinic Level Inter-rater reliability and inter-rater agreement statistics
Scale ICC(1) ICC(2) rWG(J)
Culture .182* .479 .929
Available resources .116* .321 .821
Learning climate .178* .476 .785
Implementation climate
.219* .504 .828
Readiness for change .169* .462 .634
Using average score for each scale*p < .05
System level Inter-rater reliability and inter-rater agreement statistics
Scale ICC(1) ICC(2) rWG(J)
Culture .181* .725 .925
Available resources .114* .542 .867
Learning climate .150* .679 .774
Implementation climate
.292* .792 .845
Readiness for change .106* .558 .592
Using average score for each scale*p < .05
Conclusions
• Our measures have structural validity• Our measures have reliability• Our measures have discriminant validity• Our measures can be aggregated to the clinic
level• Clinics within systems have more similar values
for inner-setting domain constructs than do clinics in different systems
• We fit a second order “inner setting” factor
PRACTICE ADAPTIVE RESERVE AND PATIENT CENTERED MEDICAL HOME BEST PRACTICES AT COMMUNITY HEALTH CENTER CLINICS IN SEVEN STATES
Adjusted Logistic Regression Frequency of PCMH Best Practices and PAR
Scores
PAR Frequency of PCMH Best Practices (6-8 vs. 0-5)
OR 95% CI
0.08 – 1.00 5.49 2.31,13.06
0.60 - <0.80 2.23 1.11,4.47
0.00 - <0.60 Referent
PCMH CRC Screening Best Practices Dichotomized Score (6-8 vs. 0-5)Respondent reported performing PCMH best practices “usually” or “always”
Adjusted for state, age, job type, years worked at the clinic, hours worked each week
National Demonstration Project - Highly-motivated practices w/ significant capability for change • Mean baseline PAR score 0.69 (s.d. 0.35)• Post intervention PAR score increased to 0.74
Discussion
• What have we learned about some of the measurement challenges in FQHCs?
• Where do we go from here in terms of measurement?
• Which constructs need more conceptual and measurement work?
• What additional work is needed in terms of reliability and validity assessment?
Discussion Continued
• Where do we go from here in EBI implementation in FHQCs?
• What opportunities exist?
• How can the measurement work be used in EBI implementation in FQHCs?
How FQHC WG can move forward?
• How could this group move forward on what’s next?
• What resources, partners, and preliminary work is needed to pursue what’s next?