by copyright

BMJ Open is committed to open peer review. As part of this commitment we make the peer review history of every article we publish publicly available. When an article is published we post the peer reviewers’ comments and the authors’ responses online. We also post the versions of the paper that were used during peer review. These are the versions that the peer review comments apply to. The versions of the paper that follow are the versions that were submitted during the peer review process. They are not the versions of record or the final published versions. They should not be cited or distributed as the published version of this manuscript. BMJ Open is an open access journal and the full, final, typeset and author-corrected version of record of the manuscript is available on our site with no access controls, subscription charges or pay-per-view fees (http://bmjopen.bmj.com). If you have any questions on BMJ Open’s open peer review process please email

[email protected]

on February 2, 2022 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-024514 on 15 F

ebruary 2019. Dow

nloaded from

http://bmjopen.bmj.com/

[email protected]


For peer review only

Impact of Repeated Hospital Accreditation Surveys on Quality and Reliability? An Eight Year Interrupted Time

Series Analysis.

Journal: BMJ Open

Manuscript ID bmjopen-2018-024514

Article Type: Research

Date Submitted by the Author: 04-Jun-2018

Complete List of Authors: Devkaran, Subashnie; Cleveland Clinic AbuDhabi, Quality and Patient Safety; Edinburgh Business School, O'Farrell, Patrick ; Heriot-Watt University Edinburgh Business School,

Emeritus Professor of Economics Ellahham, Samer; Sheikh Khalifa Medical City Arcangel, Randy; Statistical Society-Central Luzon State University

Keywords: Joint Commission International, healthcare quality measures, high reliability, life cycle model, interrupted time series analysis, hospital accreditation

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml

BMJ Open on F

ebruary 2, 2022 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2018-024514 on 15 February 2019. D

ownloaded from



Page 1 of 28

01/06/18

TITLE PAGE:

Impact of Repeated Hospital Accreditation Surveys on Quality and

Reliability? An Eight Year Interrupted Time Series Analysis

1Dr. Subashnie Devkaran, PhD, FACHE, MScHM, CPHQ, FISQUa.

2Prof. Patrick N. O'Farrell, BA, PhD.

3Dr. Samer Ellahham, MD.

4Mr. Randy Arcangel, BSc.

1Corresponding author: Director, Quality and Patient Safety Institute, Cleveland Clinic Abu

Dhabi. United Arab Emirates. P.O. Box 48481, Abu Dhabi, United Arab Emirates,

Telephone:+971 505215866, Fax:+9712 410 8374, Email: [email protected]

2Emeritus Professor of Economics, Edinburgh Business School, Heriot-Watt University,

Riccarton, Edinburgh, UK.

3Quality and Patient Safety Institute, Sheikh Khalifa Medical City, Abu Dhabi, United Arab

Emirates.

4Statistical Society-Central Luzon State University. Philippines

Key words: hospital accreditation, Joint Commission International, healthcare quality

measures, high reliability, life cycle model, interrupted time series analysis

Word count: 3745

Page 1 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 2 of 28

01/06/18

ABSTRACT

Objective. To evaluate whether hospital re-accreditation improves quality, patient safety and

reliability over three accreditation cycles by testing the accreditation life cycle model on

quality measures.

Design. The validity of the life cycle model was tested by calibrating interrupted time series

(ITS) regression equations for 27 quality measures. The change in the variation of quality

over the three accreditation cycles was evaluated using The Levene’s test.

Setting. A 650 bed tertiary academic hospital in Abu Dhabi, UAE.

Participants. Each month (over 96 months) a simple random sample of 10% of patient

records was selected and audited resulting in a total of 388,800 observations from 14,500

records.

Intervention(s). The impact of hospital accreditation on the 27 quality measures was

observed for 96 months, one year pre- accreditation (2007) and three years’ post-

accreditation for each of the three accreditation cycles (2008, 2011 and 2014).

Main Outcome Measure(s). The life cycle model was evaluated by aggregating the data for

27 quality measures to produce a composite score (YC) and to fit an interrupted time series

(ITS) regression equation to the unweighted monthly mean of the series.

Results. The results validate the four phases of the life cycle namely, the initiation phase, the

pre-survey phase, the post-accreditation slump and the stagnation phase. Furthermore, the life

cycle model explains 87 percent of the variation in quality compliance measures (R2 =0.87).

The best fit interrupted time series model contains two significant variables (�1 and �3) (p≤

0.001).The Levene’s test (p ≤ 0.05) demonstrated a significant reduction in variation of the

quality measures (YC) with subsequent accreditation cycles.

Page 2 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 3 of 28

01/06/18

Conclusion: The study demonstrates that accreditation has the capacity to sustain

improvements over the accreditation cycle. The significant reduction in the variation of the

quality measures (YC) with subsequent accreditation cycles indicates that accreditation

supports the goal of high reliability.

ARTICLE FOCUS

• Do hospitals maintain compliance with quality measures over the re-accreditation cycles?

• Do subsequent accreditation surveys reduce variation in quality and increase reliability?

• Is the accreditation life cycle model an appropriate and statistically valid explanation of

the pattern of quality performance over the accreditation cycle?

KEY MESSAGES

• Interrupted time series analysis is an effective method for studying quality interventions

such as accreditation.

• This research has further developed the conceptual framework of hospital accreditation –

the Life Cycle Model - by (1) by studying another hospital over an extended 8 year period

from January 2007 to end December 2014; (2) by conducting an interrupted time series

analysis of 27 quality compliance measures over a period incorporating three separate

accreditation surveys; and (3) by evaluating the impact of subsequent accreditation

surveys on variation in quality and reliability.

STRENGTHS AND LIMITATIONS OF THIS STUDY

• The study uses interrupted time series, an alternative to the randomised control trial which

is a gold standard by which effectiveness is measured in clinical disciplines.

• This is the second interrupted time series analysis of hospital accreditation.

• This is also the first study on hospital accreditation over 3 accreditation cycles and

validates the life cycle model on hospital accreditation.

• The study is limited to one hospital in the UAE.

• The quality measures were dependent on the accuracy of documentation in the patient

record.

Page 3 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 4 of 28

01/06/18

INTRODUCTION

Both the frequency and magnitude of medical errors in hospital settings is a matter of public

concern globally. Consequently, healthcare leaders are seeking rigorous methods for

improving and sustaining quality of healthcare outcomes in hospitals. Hospital accreditation

is the strategy most often selected to improve quality and it has become an integral part of

healthcare systems in more than 90 countries.[1]

A key constraint for hospitals is the cost of accreditation, a process that consumes resources

that could be used for frontline medical services.[2] There are two key questions: (1) Does

accreditation make a difference to the quality of care and hospital performance? and (2) to

what extent is any positive effect, if evident, sustainable over time? The literature, however,

shows inconsistent results over the impact and effectiveness of hospital accreditation.[3-7]

Greenfield and Braithwaite investigated the outcomes across 66 studies and inconsistent

findings were reported for the relationship between quality measures and accreditation.[5]

Furthermore, Devkaran and O’Farrell have argued that rigorous empirical studies that

evaluate whether hospitals sustain compliance with quality and patient safety standards over

the accreditation cycle are lacking.[8] Most previous research has used cross sectional

designs and/or comparative static analysis of data at two points in time.[9-11] In order to

draw causal inferences on the impact of accreditation on quality and patient safety measures,

a dynamic analysis is necessary. This was accomplished by pioneering the use of an

interrupted time series model to analyse the impact of accreditation on quality compliance

measures in a single hospital over a 4 year period.[8,12] We also outlined a new conceptual

framework of hospital accreditation - the Life Cycle model- and presented statistical

evidence to support it.[8]

Page 4 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 5 of 28

01/06/18

In this paper we advance the research in four important ways: (1) by extending the time

period of the analysis from 4 to 8 years; (2) by analysing data for a series of quality

measures at a hospital over 8 years between January 2007 and December 2014, a period

incorporating three separate accreditation cycles; (3) by comparing the results of this

hospital ( Hospital B) accreditation time series with our previous study hospital, a 150 bed ,

multi-speciality, acute care hospital (hospital A) in Abu Dhabi, UAE; and (4) by evaluating

the extent to which subsequent accreditation cycles impacts on the variation in quality.

Conceptual framework: the Life cycle model

Based on the Joint Commission International (JCI) accreditation strategy, most hospitals will

pass through various phases during the process of accreditation.[13] Devkaran and O’Farrell

hypothesised four distinct phases of the accreditation cycle and derived predictions

concerning the time series trend of compliance during each phase.[8] The predictions

constitute the building blocks of the Life Cycle Model. The first Initiation Phase is

characterised by a gradual improvement in compliance to standards with a positive change of

slope for the quality measures. The second – Presurvey Phase - occurs within 3 to 6 months

of the accreditation survey. A marked improvement (ramp up) in compliance occurs during

this phase, because staff are aware of the proximity of the survey and because the

organisation invests resources in preparation. The peak level of compliance performance

occurs during this phase. During the third phase – the Post-accreditation Slump –a drop in

levels of compliance occurs immediately following the accreditation survey followed by a

negative change in slope over time.8 Finally, the Stagnation Phase follows the post-

accreditation slump and there is an undulating plateau of compliance characterised by

sporadic changes, but at an overall level substantially above pre-accreditation values.[8]

Page 5 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 6 of 28

01/06/18

METHODS

Study population

The study was conducted in a publicly funded 650-bed, multi-specialty, acute-care hospital in

Abu Dhabi, UAE. The annual inpatient census is approximately 18,000. The hospital treats

approximately 220,000 ambulatory care patients per year.

Patient involvement

No patients were involved in this study.

Data collection

To test the life cycle model, a total of 27 quality measures were recorded each month at the

hospital, over an eight year period, including three JCI accreditation surveys (Table 1). The

quality measures were selected by an expert panel to ensure the: (1) interpretability, enabling

direct correlation with a specific JCI standard; (2) consistency, with high values indicating

better quality; and (3)systems-based, measures designed to evaluate a system/ domain of

quality rather than a single process. The outcome measures for the time series analysis

incorporated clinical quality measures and were expressed as percentages, proportions or

rates (Table 1). These performance differences were compared across monthly intervals

between four time segments, one year pre- accreditation, three years post-accreditation cycle

one, three years post accreditation cycle two and one year post accreditation cycle 3 for the

selected quality measures. The principal data source was a random sample of 14, 500 patient

records drawn from 10% of the patient population during the study period resulting on

388,800 observations.

Quality measures that displayed an inverse relationship to percentage measures were

transformed. For example, ‘percentage of patients with myocardial infarction within 72

hours after coronary artery bypass graft surgery’ was transformed to ‘percentage of patients

without myocardial infarction within 72 hours after coronary artery bypass graft surgery,’

thus equating higher values to good quality. The measures represent both important

Page 6 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 7 of 28

01/06/18

indicators of quality- including patient assessment, surgical procedures, infection control and

patient safety –and 9 of the 14 chapters of the JCI Hospital Standards manual.[14]

Table 1: Quality measure descriptions for the SKMC ( Hospital B) Time Series Analysis

Measures Value Rationale

Y1 Percentage of patients with

complete Medical History and

Physical Examination done within

24 hours of admission

Percentage To monitor the completion of History

and Physical Examination reports

Y2 Percentage of inpatients who have

allergies assessed and documented

on admission

Percentage To provide appropriate treatment to

patients with allergies

Y3 Hospital acquired pressure ulcer

incidence (transformed)

Percentage

Y4 STAT laboratory orders completed

within 1 hour

Percentage STAT orders are laboratory requests

requiring a TAT of less than 60

minutes usually due to medical

emergency. The indicator provides a

valuable tool for addressing the

medical and logistical necessities

underlying STAT ordering practices

Y5 STAT emergency room Troponin

orders with a turnaround time

(TAT) within 1 hour

Percentage Monitors the efficiency of the total

testing cycle, from order entry to

availability of results, for STAT

Troponin orders from all emergency

locations

Y6 STAT Potassium order with TAT

within 1 hour

Percentage Monitors the processing efficiency

(from specimen receipt to result

verification) for STAT and Routine

orders from all locations. Potassium is

the Chemistry indicator.

Y7 STAT Hemoglobin with TAT

within 1 hour

Percentage Monitors the processing efficiency

(from specimen receipt to result

verification) for STAT and Routine

orders from all locations. Hemoglobin

is the Hematology indicator.


myocardial infarction within 72

hours after coronary artery bypass

graft surgery ( transformed)

Percentage Monitors surgical procedure

complications

Y9 Percentage of Completed Pre-

Anaesthesia Assessments

Percentage Monitors anesthesia compliance with

the standards


completed pre-induction

assessments.

Percentage Monitors whether patient is fit for

anesthesia

Y11 Percentage of patients with post-

dural headache post anesthesia

(transformed)

Percentage Monitors this as a complication within

72 hours of surgery done under

epidural or spinal anesthesia, or after

Page 7 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 8 of 28

01/06/18

delivery under epidural labor analgesia

Y12 Percentage of patients with a

prolonged post anesthesia care unit

(PACU) stay (>2 hours)

(transformed)

Percentage To measure delays in recovery

Y13 Red blood cell (RBC) unit

expiration rate ( transformed)

Percentage Monitors the RBC expiration rate. It

ensures that RBC wastage is kept to a

minimum

Y14 Percentage of STAT Cross matches

done within 1 hour

Percentage Monitors the efficiency (from

specimen receipt in the Blood Bank to

the completion of the cross match to

antihuman globulin phase with the red

cell unit(s) appropriately tagged and

ready for release) of STAT cross

match orders required for immediate

transfusion

Y15 Percentage of correct documents in

the medical record

Percentage Monitors the accuracy of the

documents filed in the medical record.

Y16 Percentage of “Do Not Use

Abbreviations” documented in the

Medical Record ( transformed)

Percentage Monitors the usage of unapproved

abbreviations in the medical records

Y17 Central Line-associated

Bloodstream Infection (CL-BSI)

rate in ICU per 1000 device days

( transformed)

Percentage Monitors Bloodstream Infection rate

related to Central Lines in the ICU

Y18 Indwelling catheter-associated

Urinary Tract Infection (UTI) Rate

in ICU per 1000 device days

(transformed)

Percentage Monitors Indwelling catheter-

associated Urinary Tract Infections

(UTI) in the ICU

Y19 Ventilator Associated Pneumonia

(VAP) rate in per 1000 device

days( transformed)

Percentage Monitors Ventilator Associated

Pneumonia (VAP) in the ICU

Y20 Overall Health Care Associated

Infection Rate / 1000 patients bed

days ( transformed)

Percentage Rate of the main health care associated

infections that are being monitored in

the hospital per 1000 patients days

Y21 Percentage of Supply Wastage

value in the Consumable Store (

transformed)

Percentage Monitors capital due of Expired items

in Consumable Store

Y22 Pulmonary TB cases reported to the

Health Authority within 24 hours of

diagnosis

Percentage Ensures that newly diagnosed TB

cases are reported as per the law

Y23 Percentage of adverse events

reported per 1000 patient days

Percentage Monitors the culture of safety in the

organization

Y24 Readmissions within 48 hours per

1000 discharges ( transformed)

Percentage Rate of readmitted patients is an

important balancing measure to

indicate if changes to patient flow

through the system are negatively

affecting care.

Y25 Unplanned Readmission rate within Percentage Monitors unplanned readmission rates

Page 8 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 9 of 28

01/06/18

one month per 1000 discharges

(transformed)

to hospital within one month

following discharge. Readmissions

may be indications of quality issues

related to shortened length of stay and

premature discharge.

Y26 Hand Hygiene Observation Rate Percentage Compliant hand hygiene patient care

practices per 100 patient care practices

Y27 Inpatient fall rate per 1000 patients

days ( transformed)

Percentage Patient falls occurring during

hospitalization can result in serious

harm.

Source: author

Study design

Interrupted time series analysis is the most powerful quasi-experimental design for evaluating

the longitudinal effects of an intervention (e.g. accreditation) on an outcome of interest where

the trend before the accreditation intervention is used as a control period. The advantage of

this method over a simple before -and- after study is due to the repeated monthly measures of

variables, while controlling for seasonality, secular trends and changes in the environment.

Shifts in level (intercept) or slope, with P<0.05, were defined as statistically significant.

Segmented regression models fit a least squares regression line to each segment of the

independent variable, time, and thus assume a linear relationship between time and the

outcome within each segment.[15] The following linear regression equation is specified to

estimate the levels and trends in the dependent variable before each of three accreditations,

and the changes in levels and trends after each accreditation:

Yt = β0 + β1×t1 + β2×I1 + β3×t2 + β4×I2 + β5×t3 + β6×I3 + β7×t4 + et .........(1)

Where Yt is the outcome, for example, the inpatient fall rate per 1000 patient days; time t1,

t2, t3 and t4 indicates time in months from the start of each observation period to the end of

the period; interventions I1, I2, and I3 are dummy variables taking the value 0 before the

Page 9 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 10 of 28

01/06/18

intervention and 1 after the intervention. In this model β0 is the baseline level of the

outcome at the beginning of the series; β1 the slope prior to accreditation, that is the baseline

trend; β2, β4, and β6 are the changes in level immediately after each accreditation; and β3, β5

and β7 are the changes in slopes from pre-accreditation to post the three accreditations,

respectively, and represents the monthly mean of the outcome variable; and et is the random

error term.

Data analysis

First, a plot of observations against time was completed in order to reveal key features of the

data, including trend, seasonality, outliers, turning points and any discontinuities. Second,

segmented regression models were fitted using ordinary least squares regression analysis; and

the results reported as level and trend changes. Third, the Durbin-Watson (DW) statistic was

used to test for the presence of two types of autocorrelation: (1) the autoregressive process

and (2) the moving average process. If the DW was significant, the model was adjusted by

estimating the autocorrelation parameter and including it in the segmented regression model.

Fourth, the Dickey- Fuller statistic was used to test for stationarity and seasonality. A series

displaying seasonality or some other non-stationary pattern was controlled by taking the

difference of the series from one period to the next and then analysing this differenced series.

Since seasonality induces autoregressive and moving average processes, the detection and

inclusion of a seasonal component was implemented in the time series models using the

Autoregressive Moving Average (ARIMA), ARMA and dynamic regression. Fifth, since no

major hospital changes occurred during the period, it may be assumed that the accreditation

interventions were the key events to impact the time series. The analysis was conducted

using EViews 7.0. In order to verify whether the accreditation process exhibits the life cycle

effect, the statistical predictions specified for the 27 measures were tested.

Page 10 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 11 of 28

01/06/18

The ultimate confirmatory test of the Life Cycle model and of the impact of three separate

accreditations is to aggregate the data for all 27 quality compliance measures to produce a

composite score (YC) and to fit an interrupted time series regression equation to an

unweighted mean monthly value of the series.

RESULTS

Table 2 outlines the interrupted time series equations for the 27 quality compliance measures.

Several equations display autocorrelation, in which cases the autoregressive (AR) or moving

average (MA) variable was included to correct for it. First, 78% of the β1 coefficients ( the

slope prior to the first accreditation) are positive, as predicted and half are statistically

significant (Table 2). Conversely, 26% of the coefficients are negative, but only three are

significant. Second, the β2 coefficients – the change in level following the first accreditation

– are negative and significant in five cases and positive and significant in six. Hence, in 60%

of cases the first intervention effect is not significant. The β3 slope coefficient results are

more mixed following the first accreditation: in five cases, coefficients are both negative and

significant, and also five are positive and significant. Conversely, for 63% of cases there is

no significant effect. Fourth, in the case of the second intervention, β4, seven coefficients are

both negative and significant, whereas only four positive coefficients are significant. For β5,

the second post-accreditation slope, 59% of the coefficients are not significant but 8 of the 11

significant slopes are negative. Similarly, some 85% of the coefficients on (β6) – the third

intervention – are not statistically significant ; and, finally, 86% of the post-accreditation

slopes (β7) are not significant (Table 2).

The results of the overall test, using a composite score (YC), are summarised in Table 3.

Diagnostic assumption tests show that there is autocorrelation. Hence, the model was

Page 11 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 12 of 28

01/06/18

adjusted by estimating the autocorrelation parameter AR (1) and incorporating it in the

segmented regression model; inclusion of it eliminates the autocorrelation problem (Table 3).

The slope prior to the first accreditation (β1) is positive and highly significant, as predicted by

the Life Cycle model of Devkaran and O’Farrell.[8,12] The change in level following the

first accreditation survey (β2) is unexpectedly positive, but is not significant. The post-

accreditation slope (β3), however, is negative and statistically significant, as postulated by the

model. The changes in level following the second and third accreditation surveys are both

negative, but are not significant (Table 3). Similarly, the post-accreditation slopes following

these two later surveys are both negative, as hypothesised, but are not significant. The R2

value for the composite model with the AR(1) function indicates that over 87% of the

variation in quality compliance outcomes is explained (Table 3). There is, however, a

problem with multicollinearity. Inspection of the three post-accreditation slopes in Figure 1

shows a long gently undulating plateau of compliance which is consistent with the non-

significance of the second and third accreditations; and is substantiated by the evidence that

the mean compliance level before the first accreditation was 89.2% and, following the three

accreditations, the mean levels were 95.2%, 96.3% and 97.4%, respectively. Given that our

model has a high R2 value of 0.87 , it is a useful predictive tool, although it results in

somewhat unstable parameter an estimate which makes it more difficult to assess the effect of

individual independent variables.

Clearly, we cannot forecast precisely what would have occurred if the one accreditation in

2008 had not been followed by subsequent survey visits in 2011 and 2014, i.e. the

counterfactual position. However, if compliance had been allowed to slip following the first

survey, it would be expected that improvements in quality would occur both before the

Page 12 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 13 of 28

01/06/18

second and third surveys in 2011 and 2014; and that there would also be falls in levels of

compliance immediately following these surveys. None of these outcomes occurred. This

implies that once a high level of compliance has been achieved after the initial accreditation

survey, it is highly likely to be maintained ( Figure. 1).

Finally, we compare the results of the composite model (YC) for the 27 measures (hospital

B) with that of the 23 quality measures for the 150 bed hospital A ( Figure.2) [8,12]. A

number of interesting patterns are apparent. First, the slopes prior to accreditation ( β2) are

both positive and highly significant, as hypothesised. Second, the change in level following

the first accreditation survey (β3) signals a significant decline in compliance , as predicted,

in the case of hospital A; while for the current study, hospital B, the effect is not significant.

Third, as postulated, the post-accreditation slope (β3) is both negative and statistically

significant for each hospital. Fourth, there is a striking similarity in the shape of the two

graphs with a marked improvement in compliance during the first Pre-Survey Phase; a drop

in the level of compliance following the accreditation survey at hospital A, while similar falls

in level were recorded after two of the three accreditations at hospital B, followed in both

hospitals by undulating plateaus of compliance, at a level substantially greater than those

recorded prior to the first accreditation survey. Fifth, a notable feature of the results is that,

although the pattern of compliance change is very similar, the level of compliance at hospital

B is slightly higher: for the pre-survey phase the average level of compliance at hospital B is

87.40% compared with 79.5% at hospital A; while for the post-accreditation period, the

hospital B compliance average of 96% also exceeds that of hospital A (93%).

Finally, having demonstrated that there is a significant difference between group means of the

composite measure (YC), we tested the null hypothesis that there is no significant difference

Page 13 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 14 of 28

01/06/18

between the group variances. The results of Levene's test show that the hypothesis of

homogeneity of variances is rejected at P<0.05 (Table 4). Therefore, there is a significant

difference between the four group variances and Figure 3 (the box plot) shows that the

variances decrease after each successive accreditation. Hence, with the exception of the

means for groups C and D, successive accreditations lead to an increase in the mean and

decrease in the variance of the composite compliance measure(YC). The results of the

confirmatory test of the proposed Life Cycle model, using a composite score (YC) of the 27

quality measures, provide proof of the life cycle model.

Page 14 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 15 of 28

01/06/18

Table 2: Time series analysis for the 27 quality measures

Response

Variable

Model

Model Validation and Parameter Estimation Diagnostic tests

Intercept(

βo) Time(β1)

Interventi

on (β2)

Tim_Aft_I

nt (β3)

interventio

n2(β4)

t_aft_int_

2(β5)

interventio

n3(β6)

t_aft_int_

3(β7) AR(1) MA(1)

F-

stati

stics

R2

Autocorrelation (AC)

Check Durbin

Watson

Test for

Seasonality/

Stationarity:

(Dickey

Fuller Unit

Root Test)

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

p-

valu

e

Result of Durbin

Watson

p-value

Result

(Y1) % patients

with complete

Medical

History and

Physical

Examination

within 24 hrs

Full Model

25.59

0.00

4.51

0.00

4.04

0.29

-4.47

0.00

0.52

0.86

0.00

0.99

-0.10

0.98

-0.03

0.96 0.00

87.

0%

2.

74

1.

26

No

Autocorr

elation

0.0

0

Series

is

statio

nary

(Y2) % of

inpatients who

have an

allergies

documented

on admission

Full Model

86.00

0.00

0.40

0.06

3.94

0.07

-0.29

0.19

-2.18

0.20

-0.03

0.68

-0.18

0.94

-0.13

0.71 0.00

53.

3%

1.

61

2.

39

No

Autocorr

elation

0.0

0

Series

is

statio

nary

(Y3) Hospital

acquired

pressure ulcer

incidence

(transformed)

Full Model

99.20

0.00

-0.01

0.58

-0.18

0.24

0.03

0.05

-0.04

0.75

-0.02

0.00

-0.01

0.95

0.00

0.91 0.00

52.

3%

1.

64

2.

36

No

Autocorr

elation

0.0

0

Series

is

statio

nary

Y4) STAT

(within 1 hour)

order rate (first

differencing)

75.20

0.00

0.32

0.03

0.92

0.50

-0.17

0.26

2.45

0.02

-0.26

0.00

2.51

0.13

-0.24

0.27

0.85

0.00

0.00 78.

0%

1.

85

2.

15

No

autocorr

elation

0.0

0

Series

is

statio

nary

after

first

differ

Page 15 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on February 2, 2022 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2018-024514 on 15 February 2019. Downloaded from



Page 16 of 28

01/06/18

encin

g

(Y5) STAT ER

Troponin with

TAT within 1

hour (with

MA1)

94.93

0.00

-0.16

0.27

4.05

0.00

0.18

0.25

-1.61

0.17

0.09

0.12

-0.41

0.82

-0.17

0.47

0.52

0.00

0.00 45.

7%

1.

91

2.

09

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y6) STAT

Potassium with

TAT within 1

hour(with

MA1)

82.80

0.00

1.21

0.00

-4.18

0.00

-1.16

0.00

-2.32

0.01

0.01

0.91

0.91

-0.09

-0.03

0.86

0.74

0.00

0.00 79.

7%

1.

73

2.

27

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y7) STAT

Hemoglobin

with TAT

within 1 hour

(with MA1)

96.28

0.00

0.04

0.45

-0.03

0.95

0.00

0.98

0.95

0.04

-0.07

0.00

-0.62

0.36

0.18

0.36

0.33

0.00

0.00 44.

0%

1.

88

2.

12

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y8) Percentage

of patients

without

myocardial

infarction

within 72

hours after

coronary

artery bypass

graft surgery

99.94

0.00

-0.04

0.77

-1.15

0.44

0.09

0.54

-1.41

0.25

0.01

0.89

0.00

1.00

0.00

1.00

0.50

5.3

5%

0.

00

2.

19

No

autocorr

elation

0.5

0

Series

is

statio

nary

(Y9)

Completion of

pre-

anaesthesia

assessments

59.72

0.00

3.10

0.00

-6.68

0.05

-3.38

0.00

5.80

0.03

0.47

0.00

-1.41

0.73

-0.42

0.45

0.00

65.

95

%

0.

00

1.

57

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y10)

Completion of

Immediate

pre-induction

assessments

55.22

0.00

3.13

0.00

-8.60

0.00

-3.03

0.00

0.23

0.91

-0.03

0.81

-0.22

0.95

-0.08

0.85

0.00

80

.11

%

0.

00

1.

83

No

autocorr

elation

0.0

0

Serie

s is

statio

nary

(Y11) Percent of

patients

without post

dural puncture

headaches

100.00

0.00

0.00

1.00

-1.61

0.04

0.06

0.44

-0.59

0.33

-0.06

0.04

0.00

1.00

0.00

1.00

0.02

12.

37

%

0.

00

2.

10

No

autocorr

elation

0.0

0

Series

is

statio

nary

Page 16 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 17 of 28

01/06/18

(Y12) Prolonged

recovery more

than 2 hours

(transformed)

(First

differencing)

96.85

0.00

0.04

0.00

-3.53

0.00

-0.05

0.72

-2.18

0.04

0.04

0.50

0.07

0.96

-0.07

0.74

0.37

0.00 0.00

28.

20

%

0.

00

1.

83

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y13)

percentage of

red blood cell

units expired

(transformed)

99.74

0.00

0.02

0.82

-0.17

0.87

-0.03

0.79

-2.62

0.00

-0.09

0.03

-0.40

0.74

-0.12

0.47

0.01

18

.96

%

0.

00

2.

07

No

autocorr

elation

0.0

0

Serie

s is

statio

nary

(Y14) Stat

crossmatch

within 1 hour

90.11

0.00

0.05

0.00

2.05

0.06

0.06

0.44

-0.82

0.00

-0.03

0.66

0.18

0.93

-0.05

0.85

0.02

47.

69

%

0.

00

1.

49

No

autocorr

elation

0.0

0

Serie

s is

statio

nary

(Y15)

percentage of

correct

documents

filed in the

record

96.81

0.00

0.07

0.00

2.20

0.13

0.70

0.00

0.72

0.70

-0.13

0.16

-0.14

0.96

0.01

0.97

0.00

40.

90

%

0.

00

1.

85

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y16)

Percentage of

unapproved

abbreviations

used

( transformed)

15.23

0.00

-0.26

0.00

2.72

0.62

2.33

0.30

-0.04

0.99

0.53

0.05

0.00

1.00

0.00

1.00 0.00

23.

82

%

0.

00

1.

69

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y17) central

line associated

bloodstream

infections

(transformed)

99.51

0.00

0.04

0.00

0.21

0.01

0.00

0.10

-2.18

0.04

0.04

0.50

0.07

0.96

-0.07

0.74

0.00

28.

20

%

0.

00

1.

83

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y18) Indwelling

catheter-

associated

Urinary Tract

Infection (UTI)

Rate

(transformed)

99.76

0.00

0.00

0.07

0.16

0.01

0.01

0.53

-0.03

0.74

-0.01

0.00

0.24

0.05

-0.03

0.05

0.00

23.

21

%

0.

00

1.

96

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y19) Ventilator

Associated

Pneumonia

(VAP) rate

(transformed)

First

99.51

0.00

0.00

0.03

0.21

0.05

0.09

0.00

0.06

0.58

-0.03

0.00

0.07

0.68

0.00

0.85

0.48

0.00

0.00

40.

22

%

0.

00

2.

05

No

autocorr

elation

0.0

0

Series

is

statio

nary

Page 17 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 18 of 28

01/06/18

differencing

(Y20) Overall

Health Care

Associated

Infection Rate

( transformed)

99.92

0.00

0.00

0.09

-0.00

0.91

0.01

0.03

0.02

0.11

-0.00

0.38

-0.02

0.35

0.01

0.26 0.03

15

.38

%

0.

00

1.

80

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y21)

Percentage of

Supply

Wastage value

( transformed)

first

differencing

99.97

0.00

-0.01

0.01

0.01

0.78

0.02

0.00

-0.03

0.29

-0.01

0.00

0.00

1.00

0.00

1.00

0.60

0.00 0.00

55.

61

%

0.

00

2.

05

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y22)

Pulmonary TB

cases reported

to PMD within

24 hours of

diagnosis

first

differencing

29.05

0.01

0.25

0.21

25.26

0.08

-0.93

0.58

-24.29

0.06

-0.24

0.71

-26.04

0.19

4.44

0.09

0.26

0.00

0.00

27.

39

%

0.

00

2.

05

No

autocorr

elation

0.0

2

Series

is

statio

nary

(Y23) Adverse

event reports

per 1000

patient days

( transformed)

First

differencing

99.76

0.00

-0.02

0.00

0.32

0.24

-0.03

0.32

0.72

0.00

0.01

0.61

-0.06

0.85

-0.01

0.87

0.68

0.00

0.00

75.

67

%

0.

00

2.

07

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y24)

Readmission

within 48

hours per 1000

discharges

( transformed)

First

differencing

98.43

0.00

0.05

0.22

-0.37

0.35

-0.05

0.22

-0.70

0.03

0.02

0.14

-2.25

0.00

0.37

0.00

0.70

0.00 0.00

48.

36

%

0.

00

1.

91

No

autocorr

elation

0.0

0

Series

is

statio

nary

(Y25)

Unplanned

Readmission

91.76

0.00

0.13

0.43

-0.51

0.73

-0.15

0.36

-3.36

0.00

0.22

0.00

-9.58

0.00

1.20

0.00

0.74

0.00 0.00

45.

10

%

0.

00

1.

79

No

autocorr

elation

0.0

4

Series

is

statio

Page 18 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 19 of 28

01/06/18

rate within one

month per

1000

discharges

( transformed)

First

differencing

nary

(Y26) Hand

Hygiene

Observation

Rate

First

differencing

67.05

0.00

-0.02

0.96

8.19

0.09

0.36

0.50

-5.23

0.17

-0.33

0.08

4.36

0.45

-0.61

0.44

0.69

0.00 0.00

59.

03

%

0.

00

1.

93

No

autocorr

elation

0.0

1

Series

is

statio

nary

(Y33) Inpatient

fall rate

( transformed)

First

differencing

99.9

0.0

0.0

0.1

0.0

0.0

0.00

0.10

0.02

0.11

0.00

0.79

-0.01

0.51

0.00

0.73

0.26

0.00 0.00

20.

01

%

0.

00

1.

76

No

autocorr

elation

0.0

2

Series

is

statio

nary

* p ≤0.05; **p ≤0.001; AR: Autoregressive variable: MA: Moving average variable;¶No autocorrelation; ⱡNo seasonality-series is stationary;

Page 19 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 20 of 28

01/06/18

Table 3: Interrupted Time Series Composite Model for the 27 Quality Measures

Response

Variable

Mode

l


Intercept

(βo)

Time

(β1)

Interventio

n (β2)

Tim_Aft_In

t (β3)

intervention

2

(β4)

t_aft_int_

2

(β5)

intervention

3

(β6)

t_aft_int_

3

(β7)

AR

(1)

MA

(1)

F-

statistic

s

R2

Autocorrelation

(AC) Check

Durbin Watson

Statistic

Test for

Seasonality/

Stationarity:

(Dickey

Fuller Unit

Root Test)

Coeff.

p-value

Coeff

.

p-

value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff.

p-value

Coeff

.

p-

value

Coeff

.

p-

value

p-value Result of Durbin

Watson

p-value

Result

YC

Composit

e Model

Full

Model

87.20

0.00

0.49

0.00

0.79

0.17

-0.45

0.00

-0.03

0.94

-0.01

0.71

-0.44

0.53

-0.01

0.95 0.00

86.60

%

1.46

2.54

Positive

autocorrelation

0.00

Series is

stationary

Full

Model

with

AR (1)

86.89

0.00

0.52

0.00

0.60

0.25

-0.48

0.00

-0.07

0.95

-0.01

0.87

-0.40

0.91

-0.01

0.99

0.25

0.00 0.00

87.37

%

1.91

2.09

No

autocorrelation

0.00

Series is

stationary

* p ≤0.05; **p ≤0.001; AR: Autoregressive variable: MA: Moving average variable;¶No autocorrelation; ⱡNo seasonality-data is stationary;

*Composite quality measure (Yc) is the mean of the 27 quality measures

Page 20 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 21 of 28

01/06/18

Table 4. Anova and Levene’s test for variances between accreditation cycles

SUMMARY

Groups Count Sum Average Variance

Presurvey Phase 16 40.28 2.52 3.00

Post Survey Phase after accreditation 1 36 28.62 0.80 0.41



A Levene's test for the homogeneity of variance showed that we must reject the null hypothesis of the variances being the same in four groups

since p-value is ˂0.05. Therefore, there is significant difference between the four groups based on the variance.

Page 21 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 22 of 28

01/06/18

DISCUSSION

Empirical evidence to support the effectiveness of accreditation is still lacking, which creates

a legitimacy problem for healthcare policymakers and hospital management.[3] Is achieving

and, above all, maintaining accreditation worth the time and money if there is uncertainty

about whether it results in measurable improvements in healthcare delivery and

outcomes?[16] While accreditation enhances quality performance, its major benefit lies in

organizations integrating standards into their routine workflows. Integration ensures that the

ramp up to surveys is avoided and that organizations reliably apply the evidence based

practices for each patient during each encounter.

Announced triennial surveys have been criticised for permitting healthcare organizations to

perform for the ‘test’; and , when the accreditation survey is completed, facilities may return

to their pre-survey reality. Therefore, unannounced surveys have been proposed to mitigate

this effect and to encourage a continuous improvement culture. However, there are only two

published (Australian and Danish) studies comparing announced and unannounced surveys.

Both studies show no evidence of increased citations of non-compliance in unannounced

surveys compared with announced surveys.[17, 18]

Rather than assign the accountability to accreditation bodies for the associated life cycle,

organizations need to review their own continual survey readiness strategies. The components

of an effective continual survey readiness programmes remains unexplored. Therefore the

authors propose a survey readiness cycle that is grounded on the four phases of the

accreditation life cycle model, supported by the literature and influenced by the Institute for

Healthcare Improvement Model for Improvement.[16-19] The proposed survey readiness

cycle consists for four components; (1) a gap analysis; (2) a mock survey; (3) post survey

action plans that occur after the actual survey; and (4) intra-cycle internal reviews and

Page 22 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 23 of 28

01/06/18

improvement (Figure. 4). For the cycle to be effective a leadership oversight body needs to

be created with the objective of conducting regular reviews of compliance using associated

metrics to ensure that the process is sustained. If an accredited organization has integrated the

standards into routine practice with a foundation that is built on fundamental patient safety

principles, they are likely to minimize errors. Furthermore, when hospitals consistently

perform according to standard, they attain the status of a high reliability organization.[20]

We pioneered the first study of hospital accreditation to conduct a dynamic analysis of the

impact of accreditation on quality compliance measures using interrupted time series

analysis.[8,12] This paper has advanced the research in several important ways: (1) by

studying another hospital over an extended 8 year period; (2) by conducting an interrupted

time series analysis of 27 quality compliance measures over a period incorporating three

separate accreditation evaluations; and (3) by demonstrating that subsequent accreditation

surveys significantly reduces variation in quality performance which correlates with higher

reliability.

The evidence from both hospital studies suggests that the tangible impact of accreditation has

the capacity to sustain improvements over the accreditation cycle. Our results suggest that

once a high level of quality compliance has been achieved - following the first accreditation

visit - it is highly likely to be sustained. In addition, repeated surveys reduce variations in

quality performance therefore supporting the organization’s journey to high reliability.

Our results justify the need for a continuous approach of being ready for the next patient, not

just the next survey. The literature endorses regular self- assessments, ongoing monitoring,

intra-cycle mock surveys and benchmarking of quality measures to a data library for

sustaining quality improvement.[16-18]

Page 23 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 24 of 28

01/06/18

The following limitations should be acknowledged. First, the accuracy of measures is

dependent on the quality of documentation in the patient record. Second, the choice of

quality measures is defined by the availability of evidence in patient records. Third, this

study is set in the UAE and may not be generisable to hospitals in other settings. Finally,

more studies are required to evaluate methodologies for achieving continuous survey

readiness.

Page 24 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 25 of 28

01/06/18

Acknowledgements:

Many thanks to the SKMC Quality Team along with all SKMC caregivers, whose

commitment supported the outcomes achieved.

Contributors

SD conceived and designed the experiments. SD analysed the data: SD and PO’F interpreted

the data and wrote the manuscript. SD and PO’F jointly developed the model and arguments

for the paper. SD and PO’F revisited the article for important intellectual content. SE

provided access to the data as well as review of the article. RA provided technical assistance

with the statistical analysis. All authors reviewed and approved of the final manuscript.

Funding

This research received no funding from any agency in the public, commercial or not-for-

profit sectors.

Competing interests

None declared.

Provenance and peer review

Not commissioned; externally peer reviewed.

Data sharing statement

No additional data available.

Page 25 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 26 of 28

01/06/18

REFERENCES

1. https://www.jointcommissioninternational.org/about-jci/jci-accredited-

organizations/ (accessed May 2018).

2. Ovretveit J, Gustafson D. Evaluation of quality improvement programmes.

Quall Health Care 2002;11: 270-5

3. Greenfield D, and Braithwaite J. Developing the evidence base for accreditation

of health care organisations: A call for transparency and innovation. Qual Saf

Health Care 2009; 18: 162–163.

4. Shaw C D. Evaluating accreditation. Int J Qual Health Care, 2003;15: 455-

5. Greenfield D, Travaglia J, Braithwaite J, et al. An analysis of the health sector

accreditation literature. A report for the Australian accreditation research

network: examining future healthcare accreditation research. Sydney: Centre for

Clinical Governance Research, The University of New South Wales, 2007.

6. Griffith JR, Knutzen ST, Alexander JA. Structural versus outcome measures in

hospitals: a comparison of Joint Commission and Medicare outcome scores in

hospitals. Qual Manag Health Care 2002; 10:29-38

7. Overtreit J, Gustafson D. Improving the quality of healthcare: using research to

inform quality programmes. BMJ 2003; 326:759-61.

8. Devkaran S, O’Farrell PN. The impact of hospital accreditation on clinical

documentation compliance: a life cycle explanation using interrupted time series

analysis. BMJ Open 2014; 4: 1-9; e005240.

9. El-Jardali F, Jamal D, Dimassi H, et al. The impact of hospital accreditation on

quality of care: perception of Lebanese nurses. Int J Qual Health Care 2008;

20:363-71.

Page 26 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 27 of 28

01/06/18

10. Chandra A, Glickman SW, Ou FS, et al . An analysis of the association of

society of chest pain centres accreditation to American College of

cardiology/American heart association non-ST segment elevation myocardial

infarction guideline adherence. Ann Emerg Med 2009; 54:17-25.

11. Sack C, Lutkes P, et al. Challenging the Holy Grail of hospital accreditation: a

cross- sectional study of inpatient satisfaction in the field of cardiology. BMC

Health Serv Res 2009; 10:120-7.

12. Devkaran S, O’Farrell PN. The impact of hospital accreditation on quality

measures: an interrupted time series analysis. BMC Health Serv Res 2015;

15:137 DOI 10.1186/s 12913-015-0784-5.

13. Joint Commission international. Joint Commission International Accreditation:

Getting Started. 2nd

edition. Oakbrook Terrace, IL: Joint Commission Resources,

2010.

14. Joint Commission international. Joint Commission International Accreditation

Standards for Hospitals: 5nd

edition. Oakbrook Terrace, IL: Joint Commission

Resources, 2014.

15. Wagner AK, ,Soumerai SB, Zhang F, et al. Segmented regression analysis of

interrupted time series studies in medication use research. J Clin Pharm Ther

2002; 27:299-309.

16. Nicklin W, Dickson S The value and impact of accreditation in healthcare: a

review of the literature. Accreditation Canada, 2015.

17. Greenfield D, Moldovan M, Westbrook M et al. An empirical test of short notice

surveys in two accreditation programmes. Int J Qual Healthcare 2012;24:65–71.

Page 27 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 28 of 28

01/06/18

18. Ehlers LH, Simonsen KB, Jensen MB, Rasmussen GS, Olesen AV. Unannounced

versus announced hospital surveys: a nationwide cluster-randomized controlled

trial. Int J Qual Health Care. 2017;1:6.

19. Langley GL, Moen R, Nolan KM, et al. The Improvement Guide: A Practical

Approach to Enhancing Organizational Performance. 2nd

edition. San Francisco:

Jossey-Bass Publishers; 2009.

20. Chassin MR, Loeb JM. High-Reliability Health Care: Getting There from Here.

Milbank Q 2013;91:459–90.

Page 28 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Figure 1. Phases of the accreditation life cycle: empirical evidence over eight years

297x209mm (300 x 300 DPI)

Page 29 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Figure 2. Life cycle model comparison between Hospital A (previous study) and Hospital B (current study)

297x209mm (300 x 300 DPI)

Page 30 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Figure 3. Boxplot comparing variation in performance between the accreditation cycles

297x209mm (300 x 300 DPI)

Page 31 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Figure 4. Devkaran et al. Wheel of continual survey readiness

297x210mm (300 x 300 DPI)

Page 32 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



�

�� !"#�

$��%�� &��%� ��&��

��'�� (��)� �� !��)�

��

��

(��

�� !"#�� #�� $�%�

&�'�� $� (��#��'��)��#��#�� #�� $�%�

*��

(�� %� �� ##�� $)�%��*�

(�� +� ,�� )��)��)��#��)��"#��)��#)��

��

��+�-�

��#�� .� ��/0�� )�� #��#��1�,��

�� #�

�� /0�� )��

��1�0��

�� /0�� )�� #��#��

��+�-�

��/2�� )�� "#��"#��

�� /2�� )�� #��

��

3�� -� 4��)��"#��)�#��)�#��)��1�0��

��)��##��

��-�*�

,�� 56� �2�� )�� 1�,��

��#�� #�

��-�*�

�� *� ,��#�� *��7�

(��8�� 7� !"#�� 8�� )�*��7�

9�� !"#�� :�� 1��##��)�� #��

��

��*��7�

(�� ,�� )�� *��

��,�� "��#�� 7��

��!"#��

��/��##��)��"#�� #��

�� /��##��)��"#��

�� /��##��)�� #��

��

� ��,�� *��7�

Page 33 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




�

��

��#�� $6� ��;�#�� /��#��)��"��)�

��)�� )��#��#)��

��

� � ��0��#��#��

� � ��4��

,��#�� %6� ��0�� #��#��# �)��)��"#��

#��

��

� � ��#��#��

� � ��/(��#��)��

&�� +6� ��/;�#��

� � �� ;�#�� "#��)��"#��

� � �� ;�#�� %�

<�� .� ��0��'��)��##��)��'�� #��)�*+=�

��1�<�� '��

��

� � ��;�#�� 8��

� � ��)��#��

&� �� -� ;�#�� /��#��)��

+��

>�� 5� (�� '�� $��%�

?�� *� ,�� )��#��#��1�,��

��#��

��-��5�

��#�� 7� 0��#��'��)��)��#��)��

��)��

��-��5�

0�� ,�� "�� -��5�

��

2�� 0�� #��)��##��)��

� � �� #��

��-)�5�

�

60��#��)��##��)��"#��"#��#�� 1�

,��-�@��!"#��!�� #�� "�#��#��#��1�A ��(A;&�!�

��'�� B��?�(�<�� #C ��1#��1�� )�@��<��

��#C ��1��1�� )��!#�� #C ��1�#��1�� 1�� (A;&�!��1��1��1�

Page 34 of 34


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



For peer review onlyImpact of Repeated Hospital Accreditation Surveys on

Quality and Reliability? An Eight Year Interrupted Time Series Analysis.

Journal: BMJ Open

Manuscript ID bmjopen-2018-024514.R1


Date Submitted by the Author: 30-Aug-2018

Complete List of Authors: Devkaran, Subashnie; Cleveland Clinic AbuDhabi, Quality and Patient Safety; Edinburgh Business School, O'Farrell, Patrick ; Heriot-Watt University Edinburgh Business School, Emeritus Professor of EconomicsEllahham, Samer; Sheikh Khalifa Medical CityArcangel, Randy; Statistical Society-Central Luzon State University

<b>Primary Subject Heading</b>: Health policy

Secondary Subject Heading: Health services research

Keywords:Joint Commission International, healthcare quality measures, high reliability, life cycle model, interrupted time series analysis, hospital accreditation


BMJ Open on F


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 1 of 30

22/08/18

TITLE PAGE:

Impact of Repeated Hospital Accreditation Surveys on Quality and

Reliability? An Eight Year Interrupted Time Series Analysis











Emirates.




Word count: 3509

Page 1 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 2 of 30

22/08/18

ABSTRACT



quality measures.







records.





27 quality measures to produce a composite score (YC) and to fit an interrupted time series

(ITS) regression equation to the unweighted monthly mean of the series.

Results. The results provide some evidence for the validity of the four phases of the life cycle

namely, the initiation phase, the pre-survey phase, the post-accreditation slump and the

stagnation phase. Furthermore, the life cycle model explains 87 percent of the variation in

quality compliance measures (R2 =0.87). The best fit interrupted time series model contains

two significant variables (�1 and �3) (p≤ 0.001).The Levene’s test (p ≤ 0.05) demonstrated a

Page 2 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 3 of 30

22/08/18

significant reduction in variation of the quality measures (YC) with subsequent accreditation

cycles.






• The study uses segmented regression interrupted time series analysis, an alternative to the

randomised control trial which is a gold standard by which effectiveness is measured in

clinical disciplines.

• This is the second interrupted time series analysis of hospital accreditation.

• This is also the first study on hospital accreditation over 3 accreditation cycles and


• The study is limited to one hospital in the UAE.

• The quality measures were dependent on the accuracy of documentation in the patient

record.

INTRODUCTION








Page 3 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 4 of 30

22/08/18
















The primary objective of this paper is to evaluate whether hospital re-accreditation results in

an improvement in quality and safety standards over three accreditation cycles by testing the

effect of accreditation on 27 quality measures and by comparing the results of this hospital

(Hospital B) accreditation time series with our previous study hospital, a 150 bed, multi-

speciality, acute care hospital (hospital A) in Abu Dhabi, UAE. The secondary objective is to

evaluate the extent to which subsequent accreditation cycles impacts on the variation in

quality.

Page 4 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 5 of 30

22/08/18








slope for the quality measures. The second – Pre-survey Phase - occurs within 3 to 6 months









METHODS

Study population

The study was conducted in a publicly funded 650-bed, multi-specialty, acute tertiary care

hospital in Abu Dhabi, UAE. The annual inpatient census is approximately 18,000. The

hospital treats approximately 220,000 ambulatory care patients per year.

Page 5 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 6 of 30

22/08/18

Patient involvement


Data collection






quality rather than a single process. The measures represent both important indicators of

quality which are primarily reviewed during survey tracers- including patient assessment,

surgical procedures, infection control and patient safety –and 9 of the 14 chapters of the JCI

Hospital Standards manual.[14]

The outcome measures for the time series analysis incorporated clinical quality measures and

were expressed as percentages, proportions or rates which minimises ceiling effects (Table

1). These performance differences were compared across monthly intervals between four

time segments, one year pre- accreditation, three years post-accreditation cycle one, three

years post accreditation cycle two and one year post accreditation cycle 3 for the selected

quality measures. This study had more than the minimum number of eight data points before

and after the intervention and thus had sufficient power to estimate the regression

coefficients.[15] The larger number of data points (96) permit more stable estimates for

forecasting pre-intervention trends had the intervention not occurred. a random sample of 14,

500 patient records were drawn from 10% of the patient population enabling statements to be

made about the population at a 0.01 % level of significance. The total number of

observations was 388,800. The method of data collection was through closed medical record

reviews, the hospital adverse event reporting system and electronic medical record data

abstracts were used for discrete data.




Page 6 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 7 of 30

22/08/18


thus equating higher values to good quality.

Table 1: Quality measure descriptions for the SKMC (Hospital B) Time Series Analysis

Measures Value Rationale JCI chapter

Y1 Percentage of patients

with complete

Medical History and

Physical Examination

done within 24 hours

of admission

Percentage To monitor the completion of

History and Physical

Examination reports

Assessment of

patient

Y2 Percentage of

inpatients who have

allergies assessed and

documented on

admission

Percentage To provide appropriate

treatment to patients with

allergies

Assessment of

patient

Medication

management and

use

Y3 Hospital acquired

pressure ulcer

incidence

(transformed)

Percentage Care of patient

Y4 STAT laboratory

orders completed

within 1 hour

Percentage STAT orders are laboratory

requests requiring a TAT of

less than 60 minutes usually

due to medical emergency.

The indicator provides a

valuable tool for addressing

the medical and logistical

necessities underlying STAT

ordering practices

Assessment of

patient

Y5 STAT emergency

room Troponin orders

with a turnaround

time (TAT) within 1

hour

Percentage Monitors the efficiency of the

total testing cycle, from order

entry to availability of results,

for STAT Troponin orders

from all emergency locations

International

patient safety

goal 2

Y6 STAT Potassium

order with TAT

within 1 hour

Percentage Monitors the processing

efficiency (from specimen

receipt to result verification)

for STAT and Routine orders

from all locations. Potassium

is the Chemistry indicator.

Assessment of

patient

Y7 STAT Hemoglobin

with TAT within 1

hour

Percentage Monitors the processing

efficiency (from specimen

receipt to result verification)

for STAT and Routine orders

from all locations.

Hemoglobin is the

Hematology indicator.

Assessment of

patient

Y8 Percentage of patients Percentage Monitors surgical procedure Care of patient

Page 7 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 8 of 30

22/08/18

with myocardial

infarction within 72

hours after coronary

artery bypass graft

surgery (

transformed)

complications Quality and

patient safety

Y9 Percentage of

Completed Pre-

Anaesthesia

Assessments

Percentage Monitors anesthesia

compliance with the standards

Anesthesia and

surgical care


with completed pre-

induction

assessments.

Percentage Monitors whether patient is fit

for anesthesia

Anesthesia and

surgical care


with post-dural

headache post

anesthesia

(transformed)

Percentage Monitors this as a

complication within 72 hours

of surgery done under

epidural or spinal anesthesia,

or after delivery under

epidural labor analgesia

Anesthesia and

surgical care


with a prolonged post

anesthesia care unit

(PACU) stay (>2

hours) (transformed)

Percentage To measure delays in recovery Anesthesia and

surgical care

Quality and

patient safety

Y13 Red blood cell (RBC)

unit expiration rate (

transformed)

Percentage Monitors the RBC expiration

rate. It ensures that RBC

wastage is kept to a minimum

Assessment of

patients

Y14 Percentage of STAT

Cross matches done

within 1 hour

Percentage Monitors the efficiency (from

specimen receipt in the Blood

Bank to the completion of the

cross match to antihuman

globulin phase with the red

cell unit(s) appropriately

tagged and ready for release)

of STAT cross match orders

required for immediate

transfusion

Assessment of

patients

Y15 Percentage of correct

documents in the

medical record

Percentage Monitors the accuracy of the

documents filed in the

medical record.

Management of

information

Y16 Percentage of “Do

Not Use

Abbreviations”

documented in the

Medical Record (

transformed)

Percentage Monitors the usage of

unapproved abbreviations in

the medical records

Management of

information

Y17 Central Line-

associated

Bloodstream

Percentage Monitors Bloodstream

Infection rate related to

Central Lines in the ICU

Prevention and

control of

infection

Page 8 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 9 of 30

22/08/18

Infection (CL-BSI)

rate in ICU per 1000

device days

( transformed)

Y18 Indwelling catheter-

associated Urinary

Tract Infection (UTI)

Rate in ICU per 1000

device days

(transformed)

Percentage Monitors Indwelling catheter-

associated Urinary Tract

Infections (UTI) in the ICU

Prevention and

control of

infection

Y19 Ventilator Associated

Pneumonia (VAP)

rate in per 1000

device days(

transformed)

Percentage Monitors Ventilator

Associated Pneumonia (VAP)

in the ICU

Prevention and

control of

infection

Y20 Overall Health Care

Associated Infection

Rate / 1000 patients

bed days (

transformed)

Percentage Rate of the main health care

associated infections that are

being monitored in the

hospital per 1000 patients

days

Prevention and

control of

infection

Y21 Percentage of Supply

Wastage value in the

Consumable Store (

transformed)

Percentage Monitors capital due of

Expired items in Consumable

Store

Governance

leadership and

direction

Y22 Pulmonary TB cases

reported to the Health

Authority within 24

hours of diagnosis

Percentage Ensures that newly diagnosed

TB cases are reported as per

the law

Governance

leadership and

direction

Y23 Percentage of adverse

events reported per

1000 patient days

Percentage Monitors the culture of safety

in the organization

Quality and

patient safety

Y24 Readmissions within

48 hours per 1000

discharges (

transformed)

Percentage Rate of readmitted patients is

an important balancing

measure to indicate if changes

to patient flow through the

system are negatively

affecting care.

Quality and

patient safety

Y25 Unplanned

Readmission rate

within one month per

1000 discharges

(transformed)

Percentage Monitors unplanned

readmission rates to hospital

within one month following

discharge. Readmissions may

be indications of quality

issues related to shortened

length of stay and premature

discharge.

Quality and

patient safety

Y26 Hand Hygiene

Observation Rate

Percentage Compliant hand hygiene

patient care practices per 100

patient care practices

International

patient safety

goal 5

Y27 Inpatient fall rate per

1000 patients days (

Percentage Patient falls occurring during

hospitalization can result in

International

patient safety

Page 9 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 10 of 30

22/08/18

transformed) serious harm. goal 6

Source: Author

Study design





variables, while controlling for seasonality and secular trends. Shifts in level (intercept) or

slope, with P<0.05, were defined as statistically significant. Segmented regression models

fit a least squares regression line to each segment of the independent variable, time, and thus

assume a linear relationship between time and the outcome within each segment.[15-19] The

following linear regression equation is specified to estimate the levels and trends in the

dependent variable before each of three accreditations, and the changes in levels and trends

after each accreditation:









Page 10 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 11 of 30

22/08/18


error term.

Data analysis













Autoregressive Moving Average (ARIMA), ARMA and dynamic regression. A range of

model-checking techniques have been used including plotting residuals and partial

autocorrelation functions as well as sensitivity analyses. Fifth, there were no significant

hospital changes (i.e. change in ownership, construction, capacity or scope change greater

than 25% of the patient volume, addition of services or mergers) that occurred during the

study period based on the JCI accreditation participation requirements (APR 3). [14]

Furthermore, the leadership and composition of the quality and safety program remained the

same throughout. Therefore, it may be assumed that the accreditation interventions were the

Page 11 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 12 of 30

22/08/18

key events to impact the time series. The analysis was conducted using EViews 7.0. In order

to verify whether the accreditation process exhibits the life cycle effect, the statistical

predictions specified for the 27 measures were tested.




unweighted mean monthly value of the series. The composite measure assumed that all of

the 27 indicators have the same weight.

RESULTS

Table 2 outlines the interrupted time series equations for the 27 quality compliance measures.

Several equations display autocorrelation, in which cases the autoregressive (AR) or moving

average (MA) variable was included to correct for it. First, 78% of the β1 coefficients (the

slope prior to the first accreditation) are positive, as predicted and half are statistically

significant correlating with the pre-survey ramp up phase in the Life Cycle Model (Table 2).

Conversely, 26% of the coefficients are negative, but only three are significant. Second, the

β2 coefficients – the change in level following the first accreditation – are negative and

significant in five cases and positive and significant in six. Hence, in 60% of cases the first

intervention effect is not significant. The β3 slope coefficient results are more mixed

following the first accreditation: in five cases, coefficients are both negative and significant,

and also five are positive and significant. Conversely, for 63% of cases there is no

significant effect. Fourth, in the case of the second intervention, β4, seven coefficients are



Page 12 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 13 of 30

22/08/18


intervention – are not statistically significant; and, finally, 86% of the post-accreditation

slopes (β7) are not significant (Table 2). The mixed results at the level of individual

measures provide limited support for the Life Cycle Model with the exception of the Pre-

Survey ramp up phase (β1).





The slope prior to the first accreditation (β1) is positive and highly significant (pre-survey

ramp up phase), as predicted by the Life Cycle Model of Devkaran and O’Farrell.[8,12] The

change in level following the first accreditation survey (β2) is unexpectedly positive, but is

not significant. The post-accreditation slope (β3), however, is negative and statistically

significant (post-accreditation slump), as postulated by the model. The changes in level

following the second and third accreditation surveys are both negative, but are not significant

(Table 3). Similarly, the post-accreditation slopes following these two later surveys are both

negative, as hypothesised, but are not significant. The R2 value for the composite model with

the AR(1) function indicates that over 87% of the variation in quality compliance outcomes

is explained (Table 3). There is, however, a problem with multicollinearity. Inspection of

the three post-accreditation slopes in Figure 1 shows a long gently undulating plateau of

compliance which is consistent with the non-significance of the second and third

accreditations; and is substantiated by the evidence that the mean compliance level before the

first accreditation was 89.2% and, following the three accreditations, the mean levels were

Page 13 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 14 of 30

22/08/18

95.2%, 96.3% and 97.4%, respectively. The evidence for the Life Cycle Model is stronger in

the case of the first hospital accreditation survey than in the subsequent accreditations. Given

that our model has a high R2 value of 0.87; it is a useful predictive tool, although it results in










survey, it is highly likely to be maintained (Figure 1).

Finally, we compare the results of the composite model (YC) for the 27 measures (hospital B)

with that of the 23 quality measures for the 150 bed hospital A (Figure 2) [8,12]. A









Page 14 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 15 of 30

22/08/18







hospital B compliance average of 96% also exceeds that of hospital A (93%). It is important

to note that both hospitals adopted the same approach to accreditation and survey preparation

by following the JCI roadmap to accreditation.[13,14]









confirmatory test of the proposed Life Cycle Model, using a composite score (YC) of the 27

quality measures, provide proof of the Life Cycle Model.

Page 15 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 16 of 30

22/08/18


Response

Variable

Model


Intercept

Mean

(βo)

Pre-

accreditat

ion

Time(β1)

Accredita

tion 1

Interventi

on (β2)

After

accreditati

on 1

Tim_Aft_I

nt (β3)

Accreditati

on 2

Interventi

on (β4)

After

accreditat

ion 2

t_aft_int_

(β5)

Accreditati

on

3interventi

on(β6)

After

accreditati

on 3

t_aft_int_(

β7)

AR(1) MA(1)

F-

statisti

cs

R2

Autocorrelation

(AC) Check

Durbin Watson

Test for

Seasonality/

Stationarity:

(Dickey

Fuller Unit

Root Test)

Coefficien

t

p-value

Coefficien

t

p-value

Coefficien

t

p-value

Coefficien

t

p-value

Coefficient

p-value

Coefficien

t

p-value

Coefficient

p-value

Coefficient

p-value

Coefficien

t

p-value

Coeffici

ent

p-value

p-value

Result of Durbin

Watson

p-value

Result

(Y1) % patients

with complete

Medical

History and

Physical

Examination

within 24 hrs

Full Model

25.59

≤0.001

4.51

≤0.001

4.04

0.29

-4.47

≤0.001

0.52

0.86

0.00

0.99

-0.10

0.98

-0.03

0.96 ≤0.001 87.0%

2.74

1.26

No

Autocorrelation

≤0.001

Series is

stationary

(Y2) % of

inpatients who

have an

allergies

documented

on admission

Full Model

86.00

≤0.001

0.40

0.06

3.94

0.07

-0.29

0.19

-2.18

0.20

-0.03

0.68

-0.18

0.94

-0.13

0.71 ≤0.001 53.3%

1.61

2.39

No

Autocorrelation

≤0.001

Series is

stationary

(Y3) Hospital

acquired

pressure ulcer

incidence

(transformed)

Full Model

99.20

≤0.001

-0.01

0.58

-0.18

0.24

0.03

0.05

-0.04

0.75

-0.02

0.00

-0.01

0.95

0.00

0.91 ≤0.001 52.3%

1.64

2.36

No

Autocorrelation

≤0.001

Series is

stationary

Y4) STAT

(within 1 hour)

order rate (first

differencing)

75.20

≤0.001

0.32

0.03

0.92

0.50

-0.17

0.26

2.45

0.02

-0.26

≤0.001

2.51

0.13

-0.24

0.27

0.85

≤0.001

≤0.001 78.0%

1.85

2.15

No

autocorrelation

≤0.001

Series is

stationary

after first

differencing

Page 16 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 17 of 30

22/08/18

(Y5) STAT ER

Troponin with

TAT within 1

hour (with

MA1)

94.93

≤0.001

-0.16

0.27

4.05

≤0.001

0.18

0.25

-1.61

0.17

0.09

0.12

-0.41

0.82

-0.17

0.47

0.52

≤0.001

≤0.001 45.7%

1.91

2.09

No

autocorrelation

≤0.001

Series is

stationary

(Y6) STAT

Potassium with

TAT within 1

hour(with

MA1)

82.80

≤0.001

1.21

≤0.001

-4.18

≤0.001

-1.16

≤0.001

-2.32

0.01

0.01

0.91

0.91

-0.09

-0.03

0.86

0.74

≤0.001

≤0.001

79.7%

1.73

2.27

No

autocorrelation

≤0.001

Series is

stationary

(Y7) STAT

Hemoglobin

with TAT

within 1 hour

(with MA1)

96.28

≤0.001

0.04

0.45

-0.03

0.95

0.00

0.98

0.95

0.04

-0.07

≤0.001

-0.62

0.36

0.18

0.36

0.33

≤0.001

≤0.001

44.0%

1.88

2.12

No

autocorrelation

≤0.001

Series is

stationary

(Y8) Percentage

of patients

without

myocardial

infarction

within 72

hours after

coronary

artery bypass

graft surgery

99.94

≤0.001

-0.04

0.77

-1.15

0.44

0.09

0.54

-1.41

0.25

0.01

0.89

0.00

1.00

0.00

1.00

0.50

5.35%

0.00

2.19

No

autocorrelation

0.50

Series is

stationary

(Y9)

Completion of

pre-

anaesthesia

assessments

59.72

≤0.001

3.10

≤0.001

-6.68

0.05

-3.38

≤0.001

5.80

0.03

0.47

≤0.001

-1.41

0.73

-0.42

0.45

≤0.001 65.95%

0.00

1.57

No

autocorrelation

≤0.001

Series is

stationary

(Y10)

Completion of

Immediate

pre-induction

assessments

55.22

≤0.001

3.13

≤0.001

-8.60

≤0.001

-3.03

≤0.001

0.23

0.91

-0.03

0.81

-0.22

0.95

-0.08

0.85

≤0.001 80.11

%

0.00

1.83

No

autocorrelation

≤0.001

Series is

stationary

(Y11) Percent of

patients

without post

dural puncture

headaches

100.00

≤0.001

0.00

1.00

-1.61

0.04

0.06

0.44

-0.59

0.33

-0.06

0.04

0.00

1.00

0.00

1.00

0.02

12.37%

0.00

2.10

No

autocorrelation

≤0.001

Series is

stationary

(Y12) Prolonged

recovery more

than 2 hours

(transformed)

(First

96.85

≤0.001

0.04

≤0.001

-3.53

≤0.001

-0.05

0.72

-2.18

0.04

0.04

0.50

0.07

0.96

-0.07

0.74

0.37

≤0.001 ≤0.001 28.20%

0.00

1.83

No

autocorrelation

≤0.001

Series is

stationary

Page 17 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 18 of 30

22/08/18

differencing)

(Y13)

percentage of

red blood cell

units expired

(transformed)

99.74

≤0.001

0.02

0.82

-0.17

0.87

-0.03

0.79

-2.62

≤0.001

-0.09

0.03

-0.40

0.74

-0.12

0.47

0.01 18.96

%

0.00

2.07

No

autocorrelation

≤0.001

Series is

stationary

(Y14) Stat

crossmatch

within 1 hour

90.11

≤0.001

0.05

≤0.001

2.05

0.06

0.06

0.44

-0.82

≤0.001

-0.03

0.66

0.18

0.93

-0.05

0.85

0.02

47.69%

0.00

1.49

No

autocorrelation

≤0.001

Series is

stationary

(Y15)

percentage of

correct

documents

filed in the

record

96.81

≤0.001

0.07

≤0.001

2.20

0.13

0.70

≤0.001

0.72

0.70

-0.13

0.16

-0.14

0.96

0.01

0.97

≤0.001

40.90%

0.00

1.85

No

autocorrelation

≤0.001

Series is

stationary

(Y16)

Percentage of

unapproved

abbreviations

used

( transformed)

15.23

≤0.001

-0.26

≤0.001

2.72

0.62

2.33

0.30

-0.04

0.99

0.53

0.05

0.00

1.00

0.00

1.00

≤0.001

23.82%

0.00

1.69

No

autocorrelation

≤0.001

Series is

stationary

(Y17) central

line associated

bloodstream

infections

(transformed)

99.51

≤0.001

0.04

≤0.001

0.21

0.01

0.00

0.10

-2.18

0.04

0.04

0.50

0.07

0.96

-0.07

0.74

≤0.001

28.20%

0.00

1.83

No

autocorrelation

≤0.001

Series is

stationary

(Y18) Indwelling

catheter-

associated

Urinary Tract

Infection (UTI)

Rate

(transformed)

99.76

≤0.001

0.00

0.07

0.16

0.01

0.01

0.53

-0.03

0.74

-0.01

≤0.001

0.24

0.05

-0.03

0.05

≤0.001

23.21%

0.00

1.96

No

autocorrelation

≤0.001

Series is

stationary

(Y19) Ventilator

Associated

Pneumonia

(VAP) rate

(transformed)

First

differencing

99.51

≤0.001

0.00

0.03

0.21

0.05

0.09

≤0.001

0.06

0.58

-0.03

≤0.001

0.07

0.68

0.00

0.85

0.48

≤0.001

≤0.001

40.22%

0.00

2.05

No

autocorrelation

≤0.001

Series is

stationary

Page 18 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 19 of 30

22/08/18

(Y20) Overall

Health Care

Associated

Infection Rate

( transformed)

99.92

≤0.001

0.00

0.09

-0.00

0.91

0.01

0.03

0.02

0.11

-0.00

0.38

-0.02

0.35

0.01

0.26 0.03

15.38

%

0.00

1.80

No

autocorrelation

≤0.001

Series is

stationary

(Y21)

Percentage of

Supply

Wastage value

( transformed)

first

differencing

99.97

≤0.001

-0.01

0.01

0.01

0.78

0.02

≤0.001

-0.03

0.29

-0.01

≤0.001

0.00

1.00

0.00

1.00

0.60

≤0.001

≤0.001

55.61%

0.00

2.05

No

autocorrelation

≤0.001

Series is

stationary

(Y22)

Pulmonary TB

cases reported

to PMD within

24 hours of

diagnosis

first

differencing

29.05

0.01

0.25

0.21

25.26

0.08

-0.93

0.58

-24.29

0.06

-0.24

0.71

-26.04

0.19

4.44

0.09

0.26

≤0.001

≤0.001

27.39%

0.00

2.05

No

autocorrelation

0.02

Series is

stationary

(Y23) Adverse

event reports

per 1000

patient days

( transformed)

First

differencing

99.76

≤0.001

-0.02

≤0.001

0.32

0.24

-0.03

0.32

0.72

≤0.001

0.01

0.61

-0.06

0.85

-0.01

0.87

0.68

≤0.001

≤0.001

75.67%

0.00

2.07

No

autocorrelation

≤0.001Series

is stationary

(Y24)

Readmission

within 48

hours per 1000

discharges

( transformed)

First

differencing

98.43

≤0.001

0.05

0.22

-0.37

0.35

-0.05

0.22

-0.70

0.03

0.02

0.14

-2.25

≤0.001

0.37

≤0.001

0.70

≤0.001

≤0.001

48.36%

0.00

1.91

No

autocorrelation

≤0.001Series

is stationary

(Y25)

Unplanned

Readmission

rate within one

month per

1000

discharges

( transformed)

First

differencing

91.76

≤0.001

0.13

0.43

-0.51

0.73

-0.15

0.36

-3.36

≤0.001

0.22

≤0.001

-9.58

≤0.001

1.20

≤0.001

0.74

≤0.001

≤0.001

45.10%

0.00

1.79

No

autocorrelation

0.04

Series is

stationary

Page 19 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 20 of 30

22/08/18

(Y26) Hand

Hygiene

Observation

Rate

First

differencing

67.05

≤0.001

-0.02

0.96

8.19

0.09

0.36

0.50

-5.23

0.17

-0.33

0.08

4.36

0.45

-0.61

0.44

0.69

≤0.001

≤0.001

59.03%

0.00

1.93

No

autocorrelation

0.01

Series is

stationary

(Y33) Inpatient

fall rate

( transformed)

First

differencing

99.9

≤0.001

0.0

0.1

0.0

≤0.001

0.00

0.10

0.02

0.11

0.00

0.79

-0.01

0.51

0.00

0.73

0.26

≤0.001

≤0.001

20.01%

0.00

1.76

No

autocorrelation

0.02

Series is

stationary

AR: Autoregressive variable: MA: Moving average variable

Page 20 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 21 of 30

22/08/18


Interce

pt

(mean)

(βo)

Pre-

accreditati

on

Time

(β1)

Accreditation

1

Intervention

(β2)

After

accreditation

1

Tim_Aft_Int

(β3)

Accreditation

2

Intervention

(β4)

After

accreditation

2

t_aft_int_2

(β5)

Accreditation 3

intervention

(β6)

After

accreditation 3

t_aft_int_3

(β7)

AR

(1)

F-

statisti

cs

R2

Autocorrelation

(AC) Check

Durbin Watson

Statistic

Test for

Seasonality/

Stationarity

(Dickey Fuller

Unit Root

Test)

Coeffic

ient

p-

value

Coefficient

p-value

Coefficient

p-value

Coefficient

p-value

Coefficient

p-value

Coefficient

p-value

Coefficient

p-value

Coefficient

p-value

Coefficient

p-value

p-

value

Result of

Durbin Watson

p-value

Result

*YC

Comp

osite

Model

Full

Mode

l

87.20

≤0.001

0.49

≤0.001

0.79

0.17

-0.45

≤0.001

-0.03

0.94

-0.01

0.71

-0.44

0.53

-0.01

0.95 ≤0.001

86.6

0%

1.46

2.54

Positive

autocorrelation

≤0.001Series

is stationary

Full

Mode

l with

AR (1)

86.89

≤0.001

0.52

≤0.001

0.60

0.25

-0.48

≤0.001

-0.07

0.95

-0.01

0.87

-0.40

0.91

-0.01

0.99

0.25

≤0.001 ≤0.001

87.3

7%

1.91

2.09

No

autocorrelation

≤0.001Series

is stationary

*Composite quality measure (Yc) is the mean of the 27 quality measures, AR: Autoregressive variable

Page 21 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 22 of 30

22/08/18


SUMMARY

Groups Count Sum Average Variance

Pre-survey Phase 16 40.28 2.52 3.00




Page 22 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 23 of 30

22/08/18

DISCUSSION




about whether it results in measurable improvements in healthcare delivery and outcomes?[3-

7, 20] While accreditation enhances quality performance, its major benefit lies in





perform for the ‘test’; and, when the accreditation survey is completed, facilities may return





surveys compared with announced surveys.[21,22]









Page 23 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 24 of 30

22/08/18














reliability.







dependent on the quality of documentation in the patient record. For instance, if the

documentation was deficient then this was reflected in the measure. Second, the choice of


study is set in the UAE and may not be generisable to hospitals in other settings. Fourth, both

Page 24 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 25 of 30

22/08/18

study hospitals provide acute tertiary care and have limited generalisability to specialty

hospitals or primary/ secondary care healthcare facilities. Fifth, interrupted time series

analysis is limited by time-varying confounding improvement initiatives that may have

occurred at the department level however, since this methodology evaluates changes in

rates of an outcome at a system-level, confounding by individual level variables will not

introduce serious bias unless it occurs simultaneously with the intervention. Finally, more

studies are required to evaluate methodologies for achieving continuous survey readiness.

Page 25 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 26 of 30

22/08/18

Acknowledgements:

Many thanks to the SKMC Quality Team along with all SKMC caregivers, whose

commitment supported the outcomes achieved.

Contributors



for the paper. SD and PO’F revisited the article for important intellectual content. SE

provided access to the data as well as review of the article. RA provided technical assistance

with the statistical analysis. All authors reviewed and approved of the final manuscript.

Funding


profit sectors.

Competing interests

None declared.





Page 26 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 27 of 30

22/08/18

REFERENCES





3. Mumford V, Greenfield D, Hogden A, et al. Counting the costs of accreditation

in acute care: an activity-based costing approach. BMJ Open 2015;5:e008850



Health Care 2009; 18: 162–163.





6. Mumford V, Reeve R, Greenfield D, et al. Is accreditation linked to hospital

infection rates? A 4-year, data linkage study of Staphylococcus aureus rates and

accreditation scores in 77 Australian acute hospitals. Int J Qual Health Care

2015;27:478–84. Brubakk K, Vist GE, Bukholm G, et al. A systematic review of

hospital accreditation: the challenges of measuring complex intervention effects.

BMC Health Serv Res 2015;15:280.




Page 27 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 28 of 30

22/08/18



20:363-71.

9. Chandra A, Glickman SW, Ou FS, et al. An analysis of the association of









15:137 doi:10.1186/s12913-015-0784-5.


Getting Started. 2nd

edition. Oakbrook Terrace, IL: Joint Commission Resources,

2010.


Standards for Hospitals: 5nd

edition. Oakbrook Terrace, IL: Joint Commission

Resources, 2014.

14. Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health

care quality improvements. Acad Pediatr 2013;13(6 Suppl):S38–44.

doi:10.1016/j.acap.2013.08.002



2002; 27:299-309.

Page 28 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 29 of 30

22/08/18

16. Bogh SB, Falstie-Jensen AM, Hollnagel E et al. Predictors of the effectiveness of

accreditation on hospital performance: A nationwide stepped-wedge study,

International Journal for Quality in Health Care 2017; 29: 477–483,

doi.org/10.1093/intqhc/mzx052

17. Jandoc R, Burden AM, Mamdani M et al. Interrupted time series analysis in drug

utilization research is increasing: systematic review and recommendations

Journal of Clinical Epidemiology 2015; 68: 950-956

18. Dowding DW, Turley M, Garrido T. The impact of an electronic health record on

nurse sensitive patient outcomes: an interrupted time series analysis, Journal of

the American Medical Informatics Association 2012:19: 615–620,

https://doi.org/10.1136/amiajnl-2011-000504





21. Ehlers LH, Simonsen KB, Jensen MB, et al. Unannounced versus announced

hospital surveys: a nationwide cluster-randomized controlled trial. Int J Qual

Health Care. 2017;1:6.


Approach to Enhancing Organizational Performance. 2nd

edition. San Francisco:



Milbank Q 2013;91:459–90.

Page 29 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 30 of 30

22/08/18

Figures legend

Figure 1. Phases of the accreditation life cycle: empirical evidence over 8 years

Figure 2. Life Cycle Model comparison between Hospital A ( previous study) and

Hospital B ( current study)

Figure 3. Boxplot comparing variation in performance between the accreditation

cycles


Page 30 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from




297x209mm (300 x 300 DPI)

Page 31 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from




297x209mm (300 x 300 DPI)

Page 32 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from




297x209mm (300 x 300 DPI)

Page 33 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from




297x210mm (300 x 300 DPI)

Page 34 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



�

�� !"#�

$��%�� &��%� ��&��

��'�� (��)� �� !��)�

��

��

(��

�� !"#�� #�� $�%�

&�'�� $� (��#��'��)��#��#�� #�� $�%�

*��

(�� %� �� ##�� $)�%��*�

(�� +� ,�� )��)��)��#��)��"#��)��#)��

��

��+�-�

��#�� .� ��/0�� )�� #��#��1�,��

�� #�

�� /0�� )��

��1�0��

�� /0�� )�� #��#��

��+�-�

��/2�� )�� "#��"#��

�� /2�� )�� #��

��

3�� -� 4��)��"#��)�#��)�#��)��1�0��

��)��##��

��-�*�

,�� 56� �2�� )�� 1�,��

��#�� #�

��-�*�

�� *� ,��#�� *��7�

(��8�� 7� !"#�� 8�� )�*��7�

9�� !"#�� :�� 1��##��)�� #��

��

��*��7�

(�� ,�� )�� *��

��,�� "��#�� 7��

��!"#��

��/��##��)��"#�� #��

�� /��##��)��"#��

�� /��##��)�� #��

��

� ��,�� *��7�

Page 35 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




�

��

��#�� $6� ��;�#�� /��#��)��"��)�

��)�� )��#��#)��

��

� � ��0��#��#��

� � ��4��

,��#�� %6� ��0�� #��#��# �)��)��"#��

#��

��

� � ��#��#��

� � ��/(��#��)��

&�� +6� ��/;�#��

� � �� ;�#�� "#��)��"#��

� � �� ;�#�� %�

<�� .� ��0��'��)��##��)��'�� #��)�*+=�

��1�<�� '��

��

� � ��;�#�� 8��

� � ��)��#��

&� �� -� ;�#�� /��#��)��

+��

>�� 5� (�� '�� $��%�

?�� *� ,�� )��#��#��1�,��

��#��

��-��5�

��#�� 7� 0��#��'��)��)��#��)��

��)��

��-��5�

0�� ,�� "�� -��5�

��

2�� 0�� #��)��##��)��

� � �� #��

��-)�5�

�

60��#��)��##��)��"#��"#��#�� 1�

,��-�@��!"#��!�� #�� "�#��#��#��1�A ��(A;&�!�

��'�� B��?�(�<�� #C ��1#��1�� )�@��<��

��#C ��1��1�� )��!#�� #C ��1�#��1�� 1�� (A;&�!��1��1��1�

Page 36 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



For peer review onlyImpact of Repeated Hospital Accreditation Surveys on

Quality and Reliability? An Eight Year Interrupted Time Series Analysis.

Journal: BMJ Open

Manuscript ID bmjopen-2018-024514.R2


Date Submitted by the Author: 09-Oct-2018

Complete List of Authors: Devkaran, Subashnie; Cleveland Clinic AbuDhabi, Quality and Patient Safety; Edinburgh Business School, O'Farrell, Patrick ; Heriot-Watt University Edinburgh Business School, Emeritus Professor of EconomicsEllahham, Samer; Sheikh Khalifa Medical CityArcangel, Randy; Statistical Society-Central Luzon State University

<b>Primary Subject Heading</b>: Health policy

Secondary Subject Heading: Health services research

Keywords:Joint Commission International, healthcare quality measures, high reliability, life cycle model, interrupted time series analysis, hospital accreditation


BMJ Open on F


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 1 of 3309/10/18

TITLE PAGE:

Impact of Repeated Hospital Accreditation Surveys on Quality and Reliability? An Eight Year Interrupted Time Series Analysis











Emirates.




Word count: 3665

Page 1 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 2 of 3309/10/18

ABSTRACT



quality measures.







records.





27 quality measures to produce a composite score (YC) and to fit an ITS regression equation

to the unweighted monthly mean of the series.

Results. The results provide some evidence for the validity of the four phases of the life cycle

namely, the initiation phase, the pre-survey phase, the post-accreditation slump and the

stagnation phase. Furthermore, the life cycle model explains 87 percent of the variation in

quality compliance measures (R2 =0.87). The best fit ITS model contains two significant

variables ( 1 and 3) (p≤ 0.001).The Levene’s test (p ≤ 0.05) demonstrated a significant 𝛽 𝛽

reduction in variation of the quality measures (YC) with subsequent accreditation cycles.

Page 2 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 3 of 3309/10/18






The study uses segmented regression interrupted time series analysis, an alternative to the

randomised control trial which is a gold standard by which effectiveness is measured in

clinical disciplines.

This is the second interrupted time series analysis onhospital accreditation.

This is also the first study on hospital accreditation over 3 accreditation cycles and


The study is limited to one hospital in the UAE.

The quality measures were dependent on the accuracy of documentation in the patient

record.

INTRODUCTION











Page 3 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 4 of 3309/10/18













The primary objective of this paper is to evaluate whether hospital re-accreditation results in

an improvement in quality and safety standards over three accreditation cycles by testing the

effect of accreditation on 27 quality measures by comparing the results of this hospital (

Hospital B) accreditation time series with our previous study hospital, a 150 bed , multi-

speciality, acute care hospital (hospital A) in Abu Dhabi, UAE. The secondary objective is to

evaluate the extent to which subsequent accreditation cycles impacts on the variation in

quality.




Page 4 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 5 of 3309/10/18





slope for the quality measures. The second – Presurvey Phase - occurs within 3 to 6 months









METHODS

Study population

The study was conducted in a publicly funded 650-bed, multi-specialty, acute tertiary care

hospital in Abu Dhabi, UAE. The annual inpatient census is approximately 18,000. The

hospital treats approximately 220,000 ambulatory care patients per year.

Patient involvement


Data collection




Page 5 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 6 of 3309/10/18



quality rather than a single process. The measures represent both important indicators of

quality which are primarily reviewed during survey tracers- including patient assessment,

surgical procedures, infection control and patient safety –and 9 of the 14 chapters of the JCI

Hospital Standards manual.[14]

The outcome measures for the time series analysis incorporated clinical quality measures and

were expressed as percentages, proportions or rates which minimises ceiling effects (Table

1). These performance differences were compared across monthly intervals between four

time segments, one year pre- accreditation, three years post-accreditation cycle one, three

years post accreditation cycle two and one year post accreditation cycle 3 for the selected

quality measures. This study had more than the minimum number of eight data points before

and after the intervention and thus had sufficient power to estimate the regression

coefficients. [15] The larger number of data points (96) permit more stable estimates for

forecasting pre-intervention trends had the intervention not occurred. The principal data

source was the electronic medical record. Slovin’s formula was used to calculate the sample

size per month based on a 95% confidence interval from an average monthly inpatient census

of 1400 patients. Each month (during the entire investigation period), a simple random

sample of 10% of patient records was selected and audited from the monthly population

resulting in a total of 388,800 observations from 14,500 records. The internal data validation

process in place within the hospital included: re-collecting the data by second person not

involved in the original data collection; using a statistically valid sample of records, cases or

other data; comparing the original data with the re-collected data; calculating the accuracy by

dividing the number of data elements found to be same by the total number of data elements

and multiplying that total by 100. A 90% accuracy level was considered as an acceptable

benchmark. When the data elements differed, the reasons were noted (for example, unclear

data definitions) and corrective actions were taken. A new sample was collected after all

corrective actions have been implemented to ensure the actions resulted in the desired

accuracy level. The sources used for the data validation included, but were not limited to: the

electronic medical record and data abstracts; enterprise resource planning software; electronic

insurance claims and the adverse event reporting system.



Page 6 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 7 of 3309/10/18



thus equating higher values to good quality.

Table 1: Quality measure descriptions for the SKMC ( Hospital B) Time Series Analysis

Measures Value Rationale JCI chapterY1 Percentage of patients

with complete Medical History and Physical Examination done within 24 hours of admission

Percentage To monitor the completion of History and Physical Examination reports

Assessment of patient

Y2 Percentage of inpatients who have allergies assessed and documented on admission

Percentage To provide appropriate treatment to patients with allergies

Assessment of patient,Medication management and use

Y3 Hospital acquired pressure ulcer incidence (transformed)

Percentage Care of patient

Y4 STAT laboratory orders completed within 1 hour

Percentage STAT orders are laboratory requests requiring a TAT of less than 60 minutes usually due to medical emergency. The indicator provides a valuable tool for addressing the medical and logistical necessities underlying STAT ordering practices


Y5 STAT emergency room Troponin orders with a turnaround time (TAT) within 1 hour

Percentage Monitors the efficiency of the total testing cycle, from order entry to availability of results, for STAT Troponin orders from all emergency locations

International patient safety goal 2

Y6 STAT Potassium order with TAT within 1 hour

Percentage Monitors the processing efficiency (from specimen receipt to result verification) for STAT and Routine orders from all locations. Potassium is the Chemistry indicator.


Y7 STAT Hemoglobin with TAT within 1 hour

Percentage Monitors the processing efficiency (from specimen receipt to result verification) for STAT and Routine orders from all locations. Hemoglobin is the Hematology indicator.


Page 7 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 8 of 3309/10/18

Y8 Percentage of patients with myocardial infarction within 72 hours after coronary artery bypass graft surgery ( transformed)

Percentage Monitors surgical procedure complications

Care of patient,Quality and patient safety

Y9 Percentage of Completed Pre-Anaesthesia Assessments

Percentage Monitors anesthesia compliance with the standards

Anesthesia and surgical care

Y10 Percentage of patients with completed pre-induction assessments.

Percentage Monitors whether patient is fit for anesthesia


Y11 Percentage of patients with post-dural headache post anesthesia (transformed)

Percentage Monitors this as a complication within 72 hours of surgery done under epidural or spinal anesthesia, or after delivery under epidural labor analgesia


Y12 Percentage of patients with a prolonged post anesthesia care unit (PACU) stay (>2 hours) (transformed)

Percentage To measure delays in recovery Anesthesia and surgical care,Quality and patient safety

Y13 Red blood cell (RBC) unit expiration rate ( transformed)

Percentage Monitors the RBC expiration rate. It ensures that RBC wastage is kept to a minimum

Assessment of patients

Y14 Percentage of STAT Cross matches done within 1 hour

Percentage Monitors the efficiency (from specimen receipt in the Blood Bank to the completion of the cross match to antihuman globulin phase with the red cell unit(s) appropriately tagged and ready for release) of STAT cross match orders required for immediate transfusion

Assessment of patients

Y15 Percentage of correct documents in the medical record

Percentage Monitors the accuracy of the documents filed in the medical record.

Management of information

Y16 Percentage of “Do Not Use Abbreviations” documented in the Medical Record ( transformed)

Percentage Monitors the usage of unapproved abbreviations in the medical records

Management of information

Y17 Central Line-associated

Percentage Monitors Bloodstream Infection rate related to

Prevention and control of

Page 8 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 9 of 3309/10/18

Bloodstream Infection (CL-BSI) rate in ICU per 1000 device days ( transformed)

Central Lines in the ICU infection

Y18 Indwelling catheter-associated Urinary Tract Infection (UTI) Rate in ICU per 1000 device days (transformed)

Percentage Monitors Indwelling catheter-associated Urinary Tract Infections (UTI) in the ICU

Prevention and control of infection

Y19 Ventilator Associated Pneumonia (VAP) rate in per 1000 device days( transformed)

Percentage Monitors Ventilator Associated Pneumonia (VAP) in the ICU


Y20 Overall Health Care Associated Infection Rate / 1000 patients bed days ( transformed)

Percentage Rate of the main health care associated infections that are being monitored in the hospital per 1000 patients days


Y21 Percentage of Supply Wastage value in the Consumable Store ( transformed)

Percentage Monitors capital due of Expired items in Consumable Store

Governance leadership and direction

Y22 Pulmonary TB cases reported to the Health Authority within 24 hours of diagnosis

Percentage Ensures that newly diagnosed TB cases are reported as per the law

Governance leadership and direction

Y23 Percentage of adverse events reported per 1000 patient days

Percentage Monitors the culture of safety in the organization

Quality and patient safety

Y24 Readmissions within 48 hours per 1000 discharges ( transformed)

Percentage Rate of readmitted patients is an important balancing measure to indicate if changes to patient flow through the system are negatively affecting care.


Y25 Unplanned Readmission rate within one month per 1000 discharges (transformed)

Percentage Monitors unplanned readmission rates to hospital within one month following discharge. Readmissions may be indications of quality issues related to shortened length of stay and premature discharge.


Y26 Hand Hygiene Observation Rate

Percentage Compliant hand hygiene patient care practices per 100 patient care practices

International patient safety goal 5

Y27 Inpatient fall rate per Percentage Patient falls occurring during International

Page 9 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 10 of 3309/10/18

1000 patients days ( transformed)

hospitalization can result in serious harm.

patient safety goal 6

Source: author

Study design





variables, while controlling for seasonality and secular trends. Shifts in level (intercept) or

slope, with P<0.05, were defined as statistically significant. Segmented regression models

fit a least squares regression line to each segment of the independent variable, time, and thus

assume a linear relationship between time and the outcome within each segment.[15-19] The

following linear regression equation is specified to estimate the levels and trends in the

dependent variable before each of three accreditations, and the changes in levels and trends

after each accreditation:









Page 10 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 11 of 3309/10/18


error term.

Data analysis













Autoregressive Moving Average (ARIMA), ARMA and dynamic regression. A range of

model-checking techniques have been used including plotting residuals and partial

autocorrelation functions as well as sensitivity analyses. Fifth, there were no significant

hospital changes (i.e. change in ownership, construction, capacity or scope change greater

than 25% of the patient volume, addition of services or mergers) that occurred during the

study period based on the JCI accreditation participation requirements (APR 3) [14].

Furthermore, the leadership and composition of the quality and safety program remained the

same throughout. Therefore, it may be assumed that the accreditation interventions were the

key events to impact the time series. The analysis was conducted using EViews 7.0. In order

Page 11 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 12 of 3309/10/18

to verify whether the accreditation process exhibits the life cycle effect, the statistical

predictions specified for the 27 measures were tested.




unweighted mean monthly value of the series. The composite measure assumed that all of

the 27 indicators have the same weight.

RESULTS

The descriptive statistics of the dependent variables are depicted in Table 2 and demonstrate

that 88% of measures had a mean and median above 90%. The data was symmetrical as the

means and medians were similar for all measures. In terms of dispersion, 74% of measures

have a standard deviation of 3 or less. The measure Y22 has the lowest mean and the highest

standard deviation. Table 3 outlines the interrupted time series equations for the 27 quality

compliance measures. Several equations display autocorrelation, in which cases the

autoregressive (AR) or moving average (MA) variable was included to correct for it. First,

78% of the β1 coefficients (the slope prior to the first accreditation) are positive, as predicted

and half are statistically significant correlating with the pre-survey ramp up phase in the Life

Cycle Model (Table 3). Conversely, 26% of the coefficients are negative, but only three are

significant. Second, the β2 coefficients – the change in level following the first accreditation

– are negative and significant in five cases and positive and significant in six. Hence, in 60%

of cases the first intervention effect is not significant. The β3 slope coefficient results are

more mixed following the first accreditation: in five cases, coefficients are both negative and

significant, and also five are positive and significant. Conversely, for 63% of cases there is

Page 12 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 13 of 3309/10/18

no significant effect. Fourth, in the case of the second intervention, β4, seven coefficients are




intervention – are not statistically significant; and, finally, 86% of the post-accreditation

slopes (β7) are not significant (Table 3). The mixed results at the level of individual

measures provide limited support for the Life Cycle Model with the exception of the pre-

survey ramp up phase (β1).





The slope prior to the first accreditation (β1) is positive and highly significant (pre-survey

ramp up phase), as predicted by the Life Cycle Model of Devkaran and O’Farrell.[8,12] The

change in level following the first accreditation survey (β2) is unexpectedly positive, but is

not significant. The post-accreditation slope (β3), however, is negative and statistically

significant (post-accreditation slump), as postulated by the model. The changes in level

following the second and third accreditation surveys are both negative, but are not significant

(Table 4). Similarly, the post-accreditation slopes following these two later surveys are both

negative, as hypothesised, but are not significant. The R2 value for the composite model with

the AR(1) function indicates that over 87% of the variation in quality compliance outcomes

is explained (Table 4). There is, however, a problem with multicollinearity. Inspection of

the three post-accreditation slopes in Figure 1 shows a long gently undulating plateau of

Page 13 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 14 of 3309/10/18

compliance which is consistent with the non-significance of the second and third

accreditations; and is substantiated by the evidence that the mean compliance level before the

first accreditation was 89.2% and, following the three accreditations, the mean levels were

95.2%, 96.3% and 97.4%, respectively. The evidence for the Life Cycle Model is stronger in

the case of the first hospital accreditation survey than in the subsequent accreditations. Given

that our model has a high R2 value of 0.87 , it is a useful predictive tool, although it results in










survey, it is highly likely to be maintained ( Figure. 1).

Finally, we compare the results of the composite model (YC) for the 27 measures (hospital

B) with that of the 23 quality measures for the 150 bed hospital A ( Figure.2) [8,12]. A






Page 14 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 15 of 3309/10/18










hospital B compliance average of 96% also exceeds that of hospital A (93%). It is important

to note that both hospitals adopted the same approach to accreditation and survey preparation

by following the JCI roadmap to accreditation.[13,14]









confirmatory test of the proposed Life Cycle Model, using a composite score (YC) of the 27

quality measures, provide proof of the Life Cycle Model.

Page 15 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 16 of 3309/10/18

Table 2: Results of descriptive statistical analysis for the 27 quality measures

Variable Measure description Mean StandardDeviation

Median Quartile 1 Quartile 3 Inter-Quartile Range (IQR)

YC Composite variable 94.90 3.08 95.60 94.03 97.15 3.12

Y1 Percentage of patients with complete Medical History and Physical Examination done within 24 hours of admission

93.04 16.78 100.00 96.75 100.00 3.25

Y2 Percentage of inpatients who have allergies assessed and documented on admission 97.08 4.97 99.32 96.49 100.00 3.51

Y3 Hospital acquired pressure ulcer incidence (Transformed) 99.46 0.36 99.57 99.32 99.69 0.38

Y4 STAT laboratory orders completed within 1 hour 84.06 3.81 84.84 81.64 87.44 5.79

Y5 STAT emergency room Troponin orders with a turnaround time (TAT) within 1 hour

97.28 2.95 98.17 96.89 98.83 1.93

Y6 STAT Potassium order with TAT within 1 hour 96.44 3.30 97.22 96.24 97.90 1.65

Y7 STAT Hemoglobin with TAT within 1 hour 98.05 1.27 98.29 97.18 99.10 1.92

Y8 Percentage of patients with myocardial infarction within 72 hours after coronary artery bypass graft surgery (transformed)

99.41 2.49 100.00 100.00 100.00 0.00

Y9 Percentage of Completed Pre-Anesthesia Assessments 94.92 9.30 99.90 93.65 100.00 6.35

Y10 Percentage of patients with completed pre-induction assessments. 94.80 9.48 97.75 94.90 100.00 5.10Y11 Post Dural Puncture headache(Transformed) 99.82 1.30 100.00 100.00 100.00 0.00

Y12 Percentage of patients with a prolonged post anesthesia care unit (PACU) stay (>2 hours) (Transformed)

96.10 2.47 96.75 95.64 97.61 1.97

Y13 Red blood cell (RBC) unit expiration rate (Transformed) 99.30 1.80 99.76 99.12 100.00 0.88

Y14 Percentage of STAT Cross matches done within 1 hour 94.13 3.45 95.08 93.16 96.16 3.00

Y15 Percentage of correct documents in the medical record (Transformed) 97.06 4.71 99.28 96.79 99.61 2.82

Y16 Percentage of “Do Not Use Abbreviations” documented in the Medical Record (Transformed)

91.36 19.78 100.00 95.00 100.00 5.00

Y17 Central Line-associated Bloodstream Infection (CL-BSI) rate in ICU per 1000 device days (Transformed)

99.77 0.25 99.80 99.65 100.00 0.35

Page 16 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 17 of 3309/10/18

Y18 Indwelling catheter-associated Urinary Tract Infection (UTI) Rate in ICU per 100 device days(Transformed)

99.83 0.18 99.85 99.73 100.00 0.27

Y19 Ventilator Associated Pneumonia (VAP) rate per 100 device days (Transformed) 99.60 0.33 99.69 99.46 99.85 0.39

Y20 Overall Health Care Associated Infection Rate / 1000 patients bed days (Transformed)

99.93 0.03 99.93 99.91 99.95 0.04

Y21 Percentage of Supply Wastage value in the Consumable Store 99.96 0.09 100.00 99.95 100.00 0.05

Y22 Pulmonary TB cases reported to the Health Authority within 24 hours of diagnosis 62.67 30.75 66.67 42.56 85.71 43.15

Y23 Percentage of adverse events reported per 1000 patient days (Transformed) 98.85 0.70 99.12 98.43 99.38 0.95

Y24 Readmission within 48 hours per 100 discharges(Transformed) 98.61 0.79 98.58 98.13 99.08 0.95

Y25 Unplanned Readmission rate within one month per 1000 discharges(Transformed) 92.83 3.14 92.01 90.67 93.82 3.15

Y26 Hand Hygiene Observation Rate 79.36 9.53 82.05 75.38 85.09 9.71

Y27 Inpatient fall rate per 1000 patient days (Transformed) 99.94 0.03 99.94 99.93 99.95 0.03

Page 17 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 18 of 3309/10/18



InterceptMean(βo)

Pre-accreditat

ionTime(β1)

Accreditation 1

Intervention (β2)

After accreditat

ion 1(β3)

Accreditation 2

Intervention (β4)

After accreditat

ion 2 (β5)

Accreditation 3

intervention(β6)

After accreditation 3 (β7)

AR(1) MA(1)F-

statistics

R2Autocorrelation

(AC) Check Durbin Watson

Test for Seasonality/Stationarity:

(Dickey Fuller Unit Root Test)

Response VariableModel

Coefficient

p-value

Coefficient

p-value

Coefficient

p-value

Coefficient

p-value

Coefficientp-value

Coefficient

p-value

Coefficientp-value

Coefficientp-value

Coefficient

p-value

Coefficient

p-valuep-value Result of Durbin

Watson

p-valueResult

(Y1) % patients with complete Medical History and Physical Examination within 24 hrsFull Model

25.59≤0.001

4.51≤0.001

4.040.29

-4.47≤0.001

0.520.86

0.000.99

-0.100.98

-0.030.96 ≤0.001 87.0%

2.741.26No

Autocorrelation

≤0.001Series is stationary

(Y2) % of inpatients who have an allergies documented on admissionFull Model

86.00≤0.001

0.400.06

3.940.07

-0.290.19

-2.180.20

-0.030.68

-0.180.94

-0.130.71 ≤0.001 53.3%

1.612.39No

Autocorrelation


(Y3) Hospital acquired pressure ulcer incidence (transformed)Full Model

99.20≤0.001

-0.010.58

-0.180.24

0.030.05

-0.040.75

-0.020.00

-0.010.95

0.000.91 ≤0.001 52.3%

1.642.36No

Autocorrelation


Y4) STAT (within 1 hour) order rate (first differencing)

75.20

≤0.001

0.32

0.03

0.92

0.50

-0.17

0.26

2.45

0.02

-0.26

≤0.001

2.51

0.13

-0.24

0.27

0.85

≤0.001≤0.001 78.0%

1.852.15No

autocorrelation

≤0.001Series is stationary after first differencing

Page 18 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 19 of 3309/10/18

(Y5) STAT ER Troponin with TAT within 1 hour (with MA1)

94.93

≤0.001

-0.16

0.27

4.05

≤0.001

0.18

0.25

-1.61

0.17

0.09

0.12

-0.41

0.82

-0.17

0.47

0.52

≤0.001≤0.001 45.7%

1.912.09No

autocorrelation


(Y6) STAT Potassium with TAT within 1 hour(with MA1)

82.80

≤0.001

1.21

≤0.001

-4.18

≤0.001

-1.16

≤0.001

-2.32

0.01

0.01

0.91

0.91

-0.09

-0.03

0.86

0.74

≤0.001

≤0.001

79.7%

1.732.27No

autocorrelation


(Y7) STAT Hemoglobin with TAT within 1 hour (with MA1)

96.28

≤0.001

0.04

0.45

-0.03

0.95

0.00

0.98

0.95

0.04

-0.07

≤0.001

-0.62

0.36

0.18

0.36

0.33

≤0.001

≤0.001

44.0%

1.882.12No

autocorrelation


(Y8) Percentage of patients without myocardial infarction within 72 hours after coronary artery bypass graft surgery

99.94

≤0.001

-0.04

0.77

-1.15

0.44

0.09

0.54

-1.41

0.25

0.01

0.89

0.00

1.00

0.00

1.00

0.50 5.35%0.00 2.19No

autocorrelation

0.50Series is stationary

(Y9) Completion of pre-anaesthesia assessments

59.72

≤0.001

3.10

≤0.001

-6.68

0.05

-3.38

≤0.001

5.80

0.03

0.47

≤0.001

-1.41

0.73

-0.42

0.45≤0.001 65.95%

0.001.57No

autocorrelation


(Y10) Completion of Immediate pre-induction assessments

55.22

≤0.001

3.13

≤0.001

-8.60

≤0.001

-3.03

≤0.001

0.23 0.91

-0.03

0.81

-0.22

0.95

-0.08

0.85≤0.001 80.11

%

0.001.83No

autocorrelation

≤0.001 Series is stationary

(Y11) Percent of patients without post dural puncture headaches

100.00

≤0.001

0.00

1.00

-1.61

0.04

0.06

0.44

-0.59

0.33

-0.06

0.04

0.00

1.00

0.00

1.00

0.02 12.37%

0.00 2.10No

autocorrelation


(Y12) Prolonged recovery more than 2 hours (transformed) (First

96.85

≤0.001

0.04

≤0.001

-3.53

≤0.001

-0.05

0.72

-2.18

0.04

0.04

0.50

0.07

0.96

-0.07

0.74

0.37≤0.001 ≤0.001 28.20%

0.001.83No

autocorrelation


Page 19 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 20 of 3309/10/18

differencing)

(Y13) percentage of red blood cell units expired (transformed)

99.74

≤0.001

0.02

0.82

-0.17

0.87

-0.03

0.79

-2.62

≤0.001

-0.09

0.03

-0.40

0.74

-0.12

0.470.01 18.96

%

0.002.07No

autocorrelation


(Y14) Stat crossmatch within 1 hour

90.11

≤0.001

0.05

≤0.001

2.05

0.06

0.06

0.44

-0.82

≤0.001

-0.03

0.66

0.18

0.93

-0.05

0.85

0.02 47.69%

0.00 1.49No

autocorrelation


(Y15) percentage of correct documents filed in the record

96.81

≤0.001

0.07

≤0.001

2.20

0.13

0.70

≤0.001

0.72

0.70

-0.13

0.16

-0.14

0.96

0.01

0.97

≤0.001

40.90%

0.001.85No

autocorrelation


(Y16) Percentage of unapproved abbreviations used ( transformed)

15.23

≤0.001

-0.26

≤0.001

2.72

0.62

2.33

0.30

-0.04

0.99

0.53

0.05

0.001.00

0.001.00

≤0.001

23.82%

0.00 1.69No

autocorrelation


(Y17) central line associated bloodstream infections (transformed)

99.51

≤0.001

0.04

≤0.001

0.21

0.01

0.00

0.10

-2.18

0.04

0.04

0.50

0.07

0.96

-0.07

0.74

≤0.001

28.20%

0.001.83No

autocorrelation


(Y18) Indwelling catheter-associated Urinary Tract Infection (UTI) Rate (transformed)

99.76

≤0.001

0.00

0.07

0.16

0.01

0.01

0.53

-0.03

0.74

-0.01

≤0.001

0.24

0.05

-0.03

0.05

≤0.001

23.21%

0.00

1.96

No autocorrelation

≤0.001Series is

stationary

(Y19) Ventilator Associated Pneumonia (VAP) rate(transformed) First differencing

99.51

≤0.0010.00

0.03

0.21

0.05

0.09

≤0.001

0.06

0.58

-0.03

≤0.001

0.07

0.68

0.00

0.85

0.48

≤0.001

≤0.001

40.22%

0.002.05No

autocorrelation

≤0.001Series is

stationary

Page 20 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 21 of 3309/10/18

(Y20) Overall Health Care Associated Infection Rate ( transformed)

99.92≤0.001

0.000.09

-0.000.91 0.01

0.030.020.11

-0.000.38

-0.020.35

0.010.26 0.03

15.38%

0.001.80

No

autocorrelation

≤0.001Series is

stationary

(Y21) Percentage of Supply Wastage value( transformed) first differencing

99.97≤0.001

-0.010.01

0.010.78

0.02≤0.001

-0.030.29

-0.01≤0.001

0.001.00

0.001.00

0.60≤0.001

≤0.001

55.61%

0.002.05

No

autocorrelation

≤0.001Series is

stationary

(Y22) Pulmonary TB cases reported to PMD within 24 hours of diagnosis first differencing

29.050.01

0.250.21

25.260.08

-0.930.58

-24.290.06

-0.240.71

-26.040.19

4.440.09

0.26≤0.001

≤0.001

27.39%

0.002.05No

autocorrelation

0.02Series is

stationary

(Y23) Adverse event reports per 1000 patient days ( transformed) First differencing

99.76≤0.001

-0.02≤0.001

0.320.24

-0.030.32

0.72≤0.001

0.010.61

-0.060.85

-0.010.87

0.68≤0.001

≤0.001

75.67%

0.002.07 No

autocorrelation


(Y24) Readmission within 48 hours per 1000 discharges ( transformed) First differencing

98.43≤0.001

0.050.22

-0.370.35

-0.050.22

-0.700.03

0.020.14

-2.25≤0.001

0.37≤0.001

0.70≤0.001

≤0.001

48.36%

0.001.91 No

autocorrelation


(Y25) Unplanned Readmission rate within one month per 1000 discharges ( transformed) First differencing

91.76≤0.001

0.130.43

-0.510.73

-0.150.36

-3.36≤0.001

0.22≤0.001

-9.58≤0.001

1.20≤0.001

0.74≤0.001

≤0.001

45.10%

0.001.79No

autocorrelation

0.04Series is

stationary

Page 21 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 22 of 3309/10/18

(Y26) Hand Hygiene Observation Rate First differencing

67.05≤0.001

-0.020.96

8.190.09

0.360.50

-5.230.17

-0.330.08

4.360.45

-0.610.44

0.69≤0.001

≤0.001

59.03%

0.001.93 No

autocorrelation

0.01Series is

stationary

(Y27) Inpatient fall rate ( transformed)First differencing

99.9≤0.001

0.00.1

0.0≤0.001

0.000.10

0.020.11

0.000.79

-0.010.51

0.000.73

0.26≤0.001

≤0.001

20.01%

0.001.76 No

autocorrelation

0.02Series is

stationary

AR: Autoregressive variable: MA: Moving average variable

Page 22 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 23 of 3309/10/18


Diagnostic tests

Intercept

(mean)(βo)

Pre-accreditati

on Time(β1)

Accreditation1

Intervention (β2)

After accreditation

1 (β3)

Accreditation 2

Intervention (β4)

After accreditation

2 (β5)

Accreditation 3 intervention

(β6)

After accreditation 3

(β7)

AR(1)

F-statisti

csR2

Autocorrelation (AC) Check

Durbin Watson Statistic

Test for Seasonality/Stationarity

(Dickey Fuller Unit Root

Test)

Response

Variable

Model

Coefficientp-

value

Coefficientp-value

Coefficient p-value

Coefficient p-value

Coefficient p-value

Coefficient p-value

Coefficient p-value

Coefficient p-value

Coefficient p-value

p-value Result of

Durbin Watsonp-valueResult

Full Model

87.20≤0.001

0.49≤0.001

0.790.17

-0.45≤0.001

-0.030.94

-0.010.71

-0.440.53

-0.010.95 ≤0.001 86.6

0%

1.462.54

Positive autocorrelation

≤0.001Series is stationary*YC

Composite Model

Full Model with AR (1)

86.89≤0.001

0.52≤0.001

0.600.25

-0.48≤0.001

-0.070.95

-0.010.87

-0.400.91

-0.010.99

0.25≤0.001 ≤0.001 87.3

7%

1.912.09No

autocorrelation


*Composite quality measure (Yc) is the mean of the 27 quality measures, AR: Autoregressive variable

Page 23 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 24 of 3309/10/18


SUMMARYGroups Count Sum Average Variance

Presurvey Phase 16 40.28 2.52 3.00Post Survey Phase after accreditation 1 36 28.62 0.80 0.41Post Survey Phase after accreditation 2 34 12.37 0.36 0.08Post Survey Phase after accreditation 3 10 2.53 0.25 0.04

Page 24 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 25 of 3309/10/18

DISCUSSION




about whether it results in measurable improvements in healthcare delivery and outcomes?[3-

7, 20] While accreditation enhances quality performance, its major benefit lies in




Unannounced Surveys


perform for the ‘test’; and, when the accreditation survey is completed, facilities may return





surveys compared with announced surveys.[21, 22]

Continual Survey Readiness






Page 25 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 26 of 3309/10/18










Conclusion








reliability.






Page 26 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 27 of 3309/10/18


dependent on the quality of documentation in the patient record. For instance, if the

documentation was deficient then this was reflected in the measure. Second, the choice of


study is set in the UAE and may not be generisable to hospitals in other settings. Fourth, both

study hospitals provide acute tertiary care and have limited generalisability to specialty

hospitals or primary/ secondary care healthcare facilities. Fifth, interrupted time series

analysis is limited by time-varying confounding improvement initiatives that may have

occurred at the department level however, since this methodology evaluates changes in

rates of an outcome at a system-level, confounding by individual level variables will not

introduce serious bias unless it occurs simultaneously with the intervention. Sixth, although

ceiling effects were minimized, it is acknowledged that if a measure is close to 100% then

any subsequent improvement will only be small. However, our analysis has shown

conclusively that each successive accreditation lead to an increase in means and also to a

decrease in the variances of the composite measure. Finally, more studies are required to

evaluate methodologies for achieving continuous survey readiness.

Page 27 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 28 of 3309/10/18

Acknowledgements:

The authors thank the SKMC Quality Team and SKMC caregivers, whose commitment

supported the outcomes achieved.

Contributors



for the paper. SD and PO’F revisited and revised the article for important intellectual content.

SE provided access to the data as well as review of the article. RA provided technical

assistance with the statistical analysis. All authors reviewed and approved of the final

manuscript.

Funding


profit sectors.

Competing interests

None declared.

Page 28 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 29 of 3309/10/18





Figure Legends:


Figure 2. Life cycle model comparison between Hospital A (previous study) and Hospital B

(current study)



REFERENCES





3. Mumford V, Greenfield D, Hogden A et al. Counting the costs of accreditation

in acute care: an activity-based costing approach. BMJ Open 2015;5:e008850



Health Care 2009; 18: 162–163.

Page 29 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from

https://www.jointcommissioninternational.org/about-jci/jci-accredited-organizations/

https://www.jointcommissioninternational.org/about-jci/jci-accredited-organizations/



Page 30 of 3309/10/18





6. Mumford V, Reeve R, Greenfield D et al. Is accreditation linked to hospital

infection rates? A 4-year, data linkage study of Staphylococcus aureus rates and

accreditation scores in 77 Australian acute hospitals. Int J Qual Health Care

2015;27:478–84. Brubakk K, Vist GE, Bukholm G et al. A systematic review of

hospital accreditation: the challenges of measuring complex intervention effects.

BMC Health Serv Res 2015;15:280.






20:363-71.

9. Chandra A, Glickman SW, Ou FS, et al . An analysis of the association of







Page 30 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 31 of 3309/10/18



15:137 DOI 10.1186/s 12913-015-0784-5.


Getting Started. 2nd edition. Oakbrook Terrace, IL: Joint Commission Resources,

2010.


Standards for Hospitals: 5nd edition. Oakbrook Terrace, IL: Joint Commission

Resources, 2014.

14. Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health

care quality improvements. Acad Pediatr 2013;13(6 Suppl):S38–44.

doi:10.1016/j.acap.2013.08.002



2002; 27:299-309.

16. Bogh SB, Falstie-Jensen AM, Hollnagel E et al. Predictors of the effectiveness of

accreditation on hospital performance: A nationwide stepped-wedge study,

International Journal for Quality in Health Care 2017; 29: 477–483,

doi.org/10.1093/intqhc/mzx052

17. Jandoc R, Burden AM, Mamdani M et al. Interrupted time series analysis in drug

utilization research is increasing: systematic review and recommendations

Journal of Clinical Epidemiology 2015; 68: 950-956

18. Dowding DW, Turley M, Garrido T. The impact of an electronic health record on

nurse sensitive patient outcomes: an interrupted time series analysis, Journal of

Page 31 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from

https://doi.org/10.1093/intqhc/mzx052



Page 32 of 3309/10/18

the American Medical Informatics Association 2012:19: 615–620,

https://doi.org/10.1136/amiajnl-2011-000504





21. Ehlers LH, Simonsen KB, Jensen MB et al.. Unannounced versus announced

hospital surveys: a nationwide cluster-randomized controlled trial. Int J Qual

Health Care. 2017;1:6.


Approach to Enhancing Organizational Performance. 2nd edition. San Francisco:



Milbank Q 2013;91:459–90.

Page 32 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



Page 33 of 3309/10/18

Page 33 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from




297x209mm (300 x 300 DPI)

Page 34 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from




297x209mm (300 x 300 DPI)

Page 35 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from




297x209mm (300 x 300 DPI)

Page 36 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from




297x210mm (300 x 300 DPI)

Page 37 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ebruary 2019. Dow

nloaded from



�

�� !"#�

$��%�� &��%� ��&��

��'�� (��)� �� !��)�

��

��

(��

�� !"#�� #�� $�%�

&�'�� $� (��#��'��)��#��#�� #�� $�%�

*��

(�� %� �� ##�� $)�%��*�

(�� +� ,�� )��)��)��#��)��"#��)��#)��

��

��+�-�

��#�� .� ��/0�� )�� #��#��1�,��

�� #�

�� /0�� )��

��1�0��

�� /0�� )�� #��#��

��+�-�

��/2�� )�� "#��"#��

�� /2�� )�� #��

��

3�� -� 4��)��"#��)�#��)�#��)��1�0��

��)��##��

��-�*�

,�� 56� �2�� )�� 1�,��

��#�� #�

��-�*�

�� *� ,��#�� *��7�

(��8�� 7� !"#�� 8�� )�*��7�

9�� !"#�� :�� 1��##��)�� #��

��

��*��7�

(�� ,�� )�� *��

��,�� "��#�� 7��

��!"#��

��/��##��)��"#�� #��

�� /��##��)��"#��

�� /��##��)�� #��

��

� ��,�� *��7�

Page 38 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




�

��

��#�� $6� ��;�#�� /��#��)��"��)�

��)�� )��#��#)��

��

� � ��0��#��#��

� � ��4��

,��#�� %6� ��0�� #��#��# �)��)��"#��

#��

��

� � ��#��#��

� � ��/(��#��)��

&�� +6� ��/;�#��

� � �� ;�#�� "#��)��"#��

� � �� ;�#�� %�

<�� .� ��0��'��)��##��)��'�� #��)�*+=�

��1�<�� '��

��

� � ��;�#�� 8��

� � ��)��#��

&� �� -� ;�#�� /��#��)��

+��

>�� 5� (�� '�� $��%�

?�� *� ,�� )��#��#��1�,��

��#��

��-��5�

��#�� 7� 0��#��'��)��)��#��)��

��)��

��-��5�

0�� ,�� "�� -��5�

��

2�� 0�� #��)��##��)��

� � �� #��

��-)�5�

�

60��#��)��##��)��"#��"#��#�� 1�

,��-�@��!"#��!�� #�� "�#��#��#��1�A ��(A;&�!�

��'�� B��?�(�<�� #C ��1#��1�� )�@��<��

��#C ��1��1�� )��!#�� #C ��1�#��1�� 1�� (A;&�!��1��1��1�

Page 39 of 39


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



by copyright

Documents