sir 101: interpretation and public reporting dana burshell, mph, cph, cic hai epidemiologist...
TRANSCRIPT
SIR 101: Interpretation and public reporting
Dana Burshell, MPH, CPH, CICHAI Epidemiologist
Virginia Department of Health
NHSN’s Guide to the Standardized Infection Ratio (SIR)
http://www.cdc.gov/nhsn/PDFs/Newsletters/NHSN_NL_OCT_2010SE_final.pdf
What is a standardized infection ratio?
The standardized infection ratio (SIR) is a summary measure used to track healthcare-associated infections (HAIs) at a national, state, or local level over time. The SIR adjusts for patients of varying risk within each facility.
- The National Healthcare Safety Network (NHSN)
Benefits of using the SIR
• Single metric– One number that can be used to make comparisons
• Scalable– National, regional, facility-wide, location-specific, by surgeon for SSIs,
etc.– Can combine the SIR values at any level of aggregation – Can perform more detailed comparisons within any individual risk
group• Risk-adjusted
– Adjusts for factors known to be associated with differences in HAI rates
– Risk-adjustment differs between types of HAIs and types of surgical procedures
- HHS HAI Action Plan - http://www.hhs.gov/ash/initiatives/hai/appendices.html#appendix_g_comparison
The SIR calculation• In HAI data analysis, the SIR compares the actual
number of HAIs reported (observed) with the baseline U.S. experience (predicted) adjusting for several risk factors that have been found to be significantly associated with differences in infection incidence.
• SIR is a ratio that is a comparison of two values
SIR = number of observed HAIs number of predicted HAIs
What is the “baseline U.S. experience”?
• 2006-2008 NHSN aggregate data are used as the standard population and considered to be the baseline U.S. experience for the SIR calculations.
• NHSN baseline data used in an SIR are used to calculate the predicted number of HAIs adjusting for the identified risk factors.
What does the SIR number mean?• An SIR greater than 1.0 indicates that more HAIs were
observed than predicted. • An SIR of 1.0 indicates that the number of HAIs observed
was equal to the number predicted.• An SIR less than 1.0 indicates that fewer HAIs were
observed than predicted.
• However, the SIR alone does not imply statistical significance. – The SIR is only a point estimate and needs additional information
to indicate if the finding is significant and not likely due to chance (that is, statistically significantly different from 1).
Statistical significance
• A p-value and 95% confidence interval (CI) are calculated by NHSN for each SIR.
• The p-value identifies if the information is statistically significant. – If the p-value is < 0.05, the SIR is “statistically significant".
• The 95% CI can sometimes be used to approximate statistical significance.– A 95% CI assesses the SIR’s magnitude and stability. – If the SIR 95% CI does not contain the value 1, the SIR is
considered "statistically significant".
What does a significant SIR mean?
• While in many cases, significantly high SIRs may reflect a need for stronger CLABSI prevention efforts and significantly low SIRs may support already existing strong CLABSI prevention efforts, several other factors such as validation of reported data may play a role.
• The real measure of success is following the SIRs over time to indicate if positive progress occurs and is sustained.
• Because the ultimate goal is zero HAIs, prevention efforts are never complete.
Explaining and interpreting the SIR: Virginia data
• During 2011, there were 233 CLABSIs identified and 194,483 central line days observed in Virginia adult intensive care units.
• Based on the NHSN 2006-2008 baseline data and the composition of locations in Virginia facilities, 366 CLABSIs were predicted.
• This results in an SIR of 0.64 (O/P= 233/366), signifying that during this time period, Virginia facilities identified 36% fewer CLABSIs than predicted.
• The p-value (<0.001) and 95% confidence interval (0.56, 0.72) indicate that the number of observed CLABSIs is statistically significantly lower than the number of predicted CLABSIs. (Reminder: If the p-value is less than 0.05 and the 95% CI does not cross 1, the SIR is statistically significantly different than 1.)
Time Period CLABSIs observed (#)
CLABSIs predicted (#)
Central line days (#) SIR p-value 95% CI
2011 233 366 194,483 0.64 <0.001 0.56, 0.72
Why publicly report HAI data?
• Infection data can give healthcare facilities, patients, and public health agencies the knowledge needed to design and implement prevention strategies that protect patients and save lives.
• Research shows that when healthcare facilities are aware of their infection issues and implement concrete strategies to prevent them, rates of certain hospital infections can be decreased by more than 70 percent.
- CDC NHSN 2009 Report Q&A: http://www.cdc.gov/hai/QA_stateSummary.html
Intra-facility data sharing benefits:
• Top benefits identified by SSI pilot study IPs (2011)– Increased awareness of HAIs within the facility– Presented data to those who can make a difference– Provided benchmark data to support improvement
initiatives– Kept HAIs in the spotlight
How is the SIR being used currently?
• NHSN SIR reports– CLABSI, SSI
• CMS Hospital Compare website– CLABSI
• Updated VDH HAI report– CLABSI
• Other states• Within hospitals
NHSN SIR report: CLABSI July-Dec 2009
Virginia
US - all
Note: Includes PICU, but not NICUNote: Data only for states using NHSN to comply with a legislative mandate* to report HAIs to the state health department (reported as of September 2010)
http://www.cdc.gov/HAI/pdfs/stateplans/state-specific-hai-sir-july-dec2009r.pdf
Note: Reference Period = 2006-2008http://www.cdc.gov/HAI/pdfs/stateplans/SIR-2010_JunDec2009.pdf
Hospital Compare
• Go to the Hospital Compare website http://www.hospitalcompare.hhs.gov/• Find hospitals near your location• Choose up to 3 hospitals to compare• Select “Patient Safety Measures”• Scroll down to “Healthcare Associated Infections (HAIs)”• Click “View Graphs”
Hospital Compare displays SIR
• Only includes SIR point estimate (no p-value or 95% CI)• Language
– Better than the US National Benchmark– Same as the US National Benchmark– Worse than the US National Benchmark– Not available – Lower numbers are better. A score of zero – meaning no CLABSIs – is best.
New VDH CLABSI report: Table 1 shows annual SIR of all hospitals
New VDH CLABSI report: Color legend
• In Hospital X’s adult intensive care units during 2011, there were 5 CLABSIs identified and 2,611 central line days observed.
• Based on the NHSN 2006-2008 baseline data and the composition of locations in Hospital X, 5.3 CLABSIs were predicted.
• This results in an SIR of 0.9 (O/P= 5/5.3), signifying that during this time period, Hospital X identified 10% fewer CLABSIs than predicted.
• The 95% confidence interval (0.30, 2.21) indicates that the number of observed CLABSIs is not statistically different than the number of predicted CLABSIs.
SIR comparisons
• Hospital X’s SIR of 0.9 is higher than both the overall SIR for its bedsize category (0.6) and the SIR of all Virginia hospitals with adult ICUs (0.6).
• Interpretation: Hospital X identified more CLABSIs than the average for its bedsize and in the state overall; however, it is not statistically significant.
New VDH CLABSI report: Hospital-specific graph shows annual CLABSI SIR over time
Interpreting VDH CLABSI report graph
SIR point estimate
SIR 95% CI
SIR = 1observed = predicted
New VDH CLABSI report: Table 2 has hospital-specific and comparison data
2009Q1
2009Q2
2009Q3
2009Q4
2010Q1
2010Q2
2010Q3
2010Q4
2011Q1
2011Q2
2011Q3
2011Q4-1.66533453693773E-16
0.2
0.4
0.6
0.8
1
1.2
Year and Quarter
Stan
dard
ized
infe
ction
rati
o (S
IR)
Central line-associated bloodstream infections (CLABSIs) in adult intensive care units standardized infection ratio by quarter, Virginia, 2009-2011
Trendline added using Excel to visually show overall increases or decreasesLine graph of
SIR point estimates
• Since 2009, Virginia facilities have identified fewer CLABSIs than predicted in adult ICUs, reflecting already existing strong infection prevention efforts. • Although there is not a consistent decrease from quarter to quarter, the overall trend since 2009 has been decreasing.• Because the ultimate goal is sustaining zero CLABSIs, individual hospital effective prevention efforts should continue in addition to identifying and acting to address gaps.
Resources
• CDC's National Healthcare Safety Network (NHSN) HAI Summary Data Reports Q and A – http://www.cdc.gov/HAI/surveillance/QA_stateSum
mary. html• NHSN e-News: SIRs Special Edition– http://www.cdc.gov/nhsn/PDFs/Newsletters/NHSN_
NL_OCT_2010SE_final.pdf
• VDH HAI website – surveillance– http://www.vdh.virginia.gov/Epidemiology/Surveillan
ce/HAI/SurveillanceReporting.htm
Henrico Doctors’ Hospital IP Team Shared SIR with Leadership of Committee
• Why you decided to educate your team– Hospital Compare website (1st Quarter Data)– VDH planning to use SIR– Corporate 2011 report using SIR (red, yellow, green)
• Who you thought it was important to educate– Leadership in IP (Chairman, Chief Nursing personnel, Quality Director)
• How did it go?– Explanation of SIR focused on > < 1 compared with NHSN national data– Example calculation – Examples of CLABSI corporate SIRs compared with CLABSI rates– Hospital Compare screenshot– Tables from VDH newsletter comparing CLABSI rates 2009-11 beside SIRs.
• Nice display of confidence intervals.
• Tips– Bring NHSN data summary reports to show where comparative data comes from
• Next steps
- http://www.vdh.virginia.gov/epidemiology/surveillance/hai/communication.htm
Questions?