hospital readmission rates: signal of failure or success? 201… · 5 introduction a large amount...
TRANSCRIPT
Hospital readmission rates: signal of failure or success?
Discussion paper 2012/02
February 2012
Mauro Laudicella, Paolo Li Donni, Peter C Smith
1
Hospital readmission rates: signal of failure or success?
Mauro Laudicella*, Paolo Li Donni**, Peter C Smith*
(*)Imperial College Business School and Centre for Health Policy (**) Department of Economics, University of Palermo
February 2012
Abstract
Hospital readmission rates are increasingly being used as signals of hospital performance and a basis
for hospital reimbursement. However for some interventions their interpretation may be
complicated by differential patient survival rates after the initial intervention. If patient
characteristics are not perfectly observable and hospitals differ in their mortality rates, then
hospitals with low mortality rates are likely to have a larger share of un-observably sicker patients at
risk of a readmission. Their performance on readmissions with respect to other hospitals will then be
underestimated. We therefore examine hospitals’ performance on readmission rates relaxing the
assumption of independence between the data generating process for mortality and readmissions
implicitly adopted in the vast majority of empirical applications. We use administrative data on
emergency admissions for fractured hip in 290,000 patients aged 65 and over from 2003-2008 in
England. We find strong evidence of sample selection bias in the identification of hospitals’
performance on 28 days emergency readmissions when the residual correlation between mortality
and readmissions is ignored. We use a bivariate sample selection model to allow for the selection
process and the dichotomous nature of the outcome variables. Our study suggests that when, as in
this example, the residual correlation is different from zero, inference from traditional models of
hospital performance on readmissions might be invalid, and we offer a more appropriate method of
inferring performance. The results have important implications for performance assessment and
financial penalties related to readmissions.
Acknowledgements
This work was funded by the Health Foundation. We are grateful to comments from participants at
seminars at Imperial College London and the University of York.
2
Executive Summary
1. Unplanned readmissions to hospital after an initial treatment are increasingly being used as a key performance indicator of hospital performance. The presumption is that readmission is an indication of a potential failure in the initial treatment, and therefore of hospital quality. There is nevertheless a recognition that, because of the stochastic nature of patients’ responses to healthcare, there may at times be unavoidable readmissions after treatment, even after optimal treatment. Furthermore, the probability of readmission will usually depend on patient characteristics, such as severity of condition on admission, comorbidities, age, and other personal characteristics.
2. The conventional approach towards measuring a hospital’s readmission performance is therefore to undertake a risk adjustment of the observed rate of readmissions, to adjust for the characteristics of the group of patients under scrutiny. This involves development of a statistical model of the ‘expected’ number of readmissions, given the observable characteristics of the patients, and to create an index based on the observed number relative to this expectation.
3. A fundamental difficulty with any such risk adjustment is that there may be unobservable patient characteristics that are not captured in the data available to the analyst. In particular, when mortality after treatment is a significant risk, an issue that is rarely considered is the extent to which, by succeeding in keeping a higher proportion of patients alive, a high quality provider might create a frailer population of surviving patients than a lower quality provider. That is, by being successful in an initial objective of minimizing mortality amongst its patients, a high quality provider might find it harder to score well on the second objective of keeping the rate of readmissions low.
4. There is an implication in the performance literature that performance indicators such as hospital mortality rate and readmission rate are independent, and that a good quality provider should score well on both. In contrast, the above argument suggests that there may be a trade-off, in the sense that there is a negative correlation between success on mortality and success on readmission. The implication is that – if this hypothesis is sustained – some adjustment may be required to take account of the sicker pool of surviving patients created by higher quality providers. Only after such an adjustment is made will it be valid to compare readmission rates of hospitals with different mortality rates.
5. If processes such as those described above exist, there may be important bias in conventional secondary outcome measures, such as readmission rates, whenever there is a significant risk of mortality after initial treatment. The pool of patients surviving initial treatment may be influenced by the quality of the hospital as well as patient characteristics, and statistical analysis should in principle take this into account.
3
6. This issue has become especially urgent given the increasing interest amongst policymakers in imposing financial penalties on providers that report high readmission rates. From 2011/12 the English National Health Service has withdrawn payment for hospital admissions within 30 days of hospital discharge from initial elective treatment, and the US Congress has passed legislation that allows the Centers for Medicare & Medicaid Services (CMS) to reduce payments for readmitted patients. Unless such incentive schemes are informed by good analysis, and designed with care, there is a risk that high quality providers might be unfairly penalized.
7. In this paper we implement advanced econometric methods designed to test whether the ‘survival’ effect exists, and then to indicate how readmission rates can be adjusted to take account of any such effect. The methods involve estimating separate equations for survival and readmissions, and examining the correlation between the residuals from these equations. A ‘bivariate sample selection model’ is then implemented to show how the readmissions can be properly modelled under such circumstances.
8. To examine the phenomenon, we study NHS patients aged 65 and over admitted with a fractured hip to English hospitals over the fiscal years 2003/04 to 2008/09. Mortality and readmission are measured within 28 days of admission.
9. This group of patients is chosen for several reasons. First, there are well-established medical guidelines on the standard of services and processes of care for this type of admission and clear links between the guidelines and both mortality and readmission outcomes. Second, age and sex standardised unplanned readmissions from this population of patients are one of the hospital outcome indicators currently used by the Health Care Quality Commission to monitor the performance of English hospitals. Third, a series of national audits on the status of service of falls and bone health in older people were conducted by the Royal College of Physicians over this period. Several aspects of the organisation and of the process of care were audited against the National Institute for Clinical Excellence (NICE) guidance for best practice and low rates of compliance were generally found across different providers of health services. There are therefore concerns about poor standards of care and large variations in health outcomes in these services.
10. Individual probit models of survival and readmissions are estimated, in which the outcome of interest is measured as a function of patient characteristics such as age, sex, comorbidities, small area of residence characteristics, and procedures undertaken. We then find substantial negative
correlation (ρ = -0.56) at the individual level between the two outcomes, as measured by the residuals from these equations. Also, we find some evidence that , on average, hospitals that perform well on survival rates(after adjusting for the stated characteristics) perform less well on readmissions.
11. This suggests that the sample of patients that die in hospital may have been at higher risk of a readmission had they survived their first admission compared to patients who survive. Therefore, the population of patients admitted to the hospital and the sample of patients that survive the first admission differ in their risk of being readmitted after controlling for all observable characteristics. The group of survivors should therefore not be used as a basis for making inferences on the probability of being readmitted until appropriate correction for the associated bias is undertaken.
4
12. The conventional univariate model of readmissions results in an underestimation of relative readmission rates amongst hospitals with lower survival rates, and therefore an upward bias in their relative ranking. Implementation of the bivariate sample selection model suggests that material changes to hospital rankings based on readmission rates occur if proper account is taken of such bias.
13. The results also account for some of the annual rise in observed readmissions observed over the study period, suggesting that much of this rise is due to improved performance, in the form of increasing probability of survival of patients.
14. The results are important because hospital readmission indicators are increasingly being used as measures of hospital performance, with financial penalties and other sanctions attached to poor performance. Conventional metrics take no account of the sample selection bias described above, and might therefore offer misleading signals of performance. Using inappropriate indicators of performance might put some hospitals under unwarranted pressure (and conversely may ignore weak performance in other hospitals) and even generate perverse incentives for hospital behaviour. Furthermore, policy or treatment evaluations may offer misleading inferences if hospital readmissions are used as a performance metric and are not specified correctly.
5
Introduction
A large amount of empirical research has sought to explain the variation in hospital readmission
rates observed in many high income countries (Boutwell, Johnson, Rutherford, Watson, Vecchioni,
Auerbach et al., 2011; Friedman & Basu, 2004; Vest, Gamm, Oxford, Gonzalez, & Slawson, 2010;
Westert, Lagoe, Keskimaki, Leyland, & Murphy, 2002; Yam, Wong, Chan, Wong, Leung, & Yeoh,
2010). Identifying the reasons for readmissions can be crucial to securing a reduction in readmissions
that are potentially avoidable, thereby reducing costs and improving health outcomes. Hospital
mortality and readmission rates are important indicators of hospital outcomes that are frequently
used to assess and publicise hospital and physician performance. They are also often used in health
services research to assess issues such as the impact of service organisation (Coyte, Young, &
Croxford, 2000; Evans & Kim, 2006; Ho & Hamilton, 2000; Lorch, Baiocchi, Silber, Even-Shoshan,
Escobar, & Small, 2010), the relationship between hospital inputs and outcomes (Heggestad, 2002;
Schreyogg & Stargardt, 2010), the effect of introducing new policies (Evans, Garthwaite, & Wei,
2008) and the impact of new technologies (Xian, Holloway, Chan, Noyes, Shah, Ting et al., 2011).
Hospital readmission rates are a frequently used outcome-based indicator of hospital quality
analogous to hospital mortality rates. The idea behind outcome-based quality indicators is that, if
appropriate adjustment is made for patient case-mix and external environmental factors, then
differences in reported hospital mortality or readmission rates are likely to be driven by differences
in the (unobservable) quality of hospital services, as reflected in the processes of hospital care and
service organisation. For example, the provision of appropriate rehabilitation services for fall and
fracture patients is known to have an impact on the risk of readmission (National Institute for Clinical
Excellence (NICE), 2004); similarly an efficient management of the surgical theatre and staff shifts
can reduce the delay before the patients are treated and thus their mortality risk (Bottle & Aylin,
2006). The intrinsic quality attributes are often unobservable by the researcher, because collection
of the necessary data is either impossible or highly costly. However, we would expect that hospitals
with better quality should have on average better outcomes (as defined above) than their lower
quality peers, after controlling for their differences in patient characteristics and environmental
factors. Many empirical applications therefore examine unplanned readmissions occurring within 30
days from previous discharge of patients admitted with a similar primary diagnosis, such as hearth
failures, AMI, strokes, pneumonia or hip fracture.
The advantage of outcome-based quality indicators is that they can be constructed by using routine
administrative data on patient discharges without the need for additional information on the
process of care, which are often prohibitively expensive to collect. Outcome-based quality indicators
can make it feasible for large populations of patients and hospitals to be included in a study and
followed for several years. However, these indicators are often inaccurate (large standard errors)
and criticised in the medical literature for their lack of clinical relevance (Lilford & Pronovost, 2010;
Shahian, Wolf, Iezzoni, Kirle, & Normand, 2010). Moreover, some of these indicators have low
correlation with more accurate measures of quality based on the process of care (Bradley, Herrin,
Elbel, McNamara, Magid, Nallamothu et al., 2006; Luthi, Burnand, McClellan, Pitts, & Flanders, 2004
).
6
Gowrisankaran and Town (1999) shed some light on the inconsistency between outcome-based and
process-based measures of quality. Using patient admitted with pneumonia in South California
hospitals from 1989 to 1994, they show that hospital risk adjusted mortality rates are affected by
selection bias that invalidates inferences on the quality of care provided. Specifically, if patients’
health conditions are not perfectly observable and patients are able to choose the hospital of
treatment, then (unmeasurably) sicker patients are more likely to select high quality hospitals.
Therefore, the differences in mortality rates across hospitals may be determined in part by
difference in the quality of care they provide and in part by differences in unobservable patient
health conditions. The latter effect systematically disadvantages high quality hospitals, and
measures of the processes and outcomes of care might show low correlation. Geweke et al. (2003)
provide an elegant econometric solution to correct for this bias by using a structural model that
takes into account the patient choice of hospital and the two determinants of the mortality variable.
In general, observational studies based on hospital administrative data have only limited information
on the heterogeneity in patient and treatment characteristics, which are therefore only partially
observable. In contrast, other study designs in the medical and epidemiology literature, such as
retrospective studies or prospective cohort studies, often have access to data describing such
heterogeneity and thus are able to provide a better direct control for the latter. Therefore,
observational studies need to pay more attention to the characteristics of the data generating
process before any meaningful inference can be made on variations in hospital quality of care, and
on the determinants of such variations.
In spite of the large number of empirical applications studying hospital readmissions, only a few
have devoted attention to the characteristics of the data generating process. Schreyoegg and
Stargardt (2010) model the hazard of hospital deaths and the hazard of readmissions using two
separate Cox regression models and allow for the event of death to be a competing risk for the
event of a readmission. Their model for readmissions includes patients dying in hospital as censored
observations assuming independence between mortality and readmissions. Papanicolas and McGuire
(2011) uses a vector of autoregressive (VAR) model to measure the quality of English hospitals over
1996-2008 following the method described in McClellan and Staigger (2000). In a first step they
estimate hospital risk adjusted mortality and readmission rates from patient level regressions
separately, i.e. assuming independence between these outcomes. In a second step, they estimate a
VAR model using the hospital level quality indicators obtained in the first step. Their VAR model
provides a synthetic indicator of hospital quality that takes into account information from a
hospital’s present and past performance on mortality and readmissions estimated in the first step. In
contrast, most empirical applications model hospital readmissions using multilevel single index
model (e.g. logit or probit) or hazard model (e.g. Cox regression model) without paying much
attention to the relationship between the event of a hospital death and a hospital readmission.
Outcome-based measures of quality, such as hospital risk adjusted mortality and 28 days
readmissions rates from specific type of admissions, are publicly released in the US by the Centres
for Medicare and Medicaid Services (CMS), in the UK by the National Centre for Health Outcomes
Development (NCHOD), and in Australia by the Australian Institute of Health and Welfare (AIHW) to
inform patient choice of hospital and to monitor hospital performance. Moreover, the English NHS
7
has introduced new rules for the reimbursement payments that seek to address rising trends in
emergency admissions. From the financial year 2011/12, hospital automatic payments will stop for
all emergency admissions occurring within 30 days of a previous discharge. Emergency readmissions
following elective admissions will receive no payment, while emergency readmissions following non-
elective admissions will receive no payment beyond a threshold based on at least a 25%
improvement in the historic rate of readmission (Department of Health, 2011). Similarly, the US
Congress has passed legislation that allows the CMS to hold hospitals accountable for their
readmissions rate (Foster & Harkness, 2010), with the objective of reducing the associated costs and
volume of treatment. The Patient Protection and Affordable Care act gives the CMS the authority to
penalise hospitals for excess readmissions by reducing reimbursement payments from fiscal year
2013. The initial scope will be limited to 30 days readmissions after heart failure, acute myocardial
infarctions (AMI) and pneumonia admissions. Under policies such as these, providing accurate
measures of hospital performance on readmission will be crucial if distorted incentives and
inefficiencies are to be avoided.
In this study we measure hospitals’ performance on readmissions relaxing the assumption of
independence between the data generating process of patient survival1 and readmission implicitly
adopted in most previous empirical applications. It finds strong evidence of sample selection bias in
the identification of the hospital effect that might affect the predictions from the usual models of
risk-adjusted readmissions. The bias originates from the residual correlation between the data
generating process of survival and readmissions.
The mechanism generating the bias can be described as follows. Suppose patients’ risk of negative
health outcomes (e.g. their underlying health status on admission) is not perfectly observable, and
that hospitals differ in their performance on survival rates (quality of care). Then (other things equal)
hospitals that are more successful in saving patients’ lives are likely to have a larger share of patients
at higher risk surviving the first admission as compared with other hospitals. In these circumstances,
hospitals’ relative performance on readmissions is determined in part by their difference in the
quality of care provided and in part by their difference in the share of patients with un-observably
higher risk of negative health outcomes that survive the first admission. High quality hospitals will
then have upward biased readmission rates due to the residual correlation between the data
generating process of survival and readmissions that systematically disadvantages such hospitals. In
the extreme case, one could observe a positive (negative) correlation between the hospitals’
performance in survival (mortality) and readmission rates, with hospitals with high survival rates
experiencing higher readmission rates, and vice versa.
This identification problem can give rise to two important problems. First it may lead to incorrect
inferences about the quality of care provided by individual hospitals. And secondly, it may lead to
incorrect inferences about the efficacy of a new policy or technology. If patients’ risk characteristics
are only partially observable, patients surviving their first admissions for the effect of the policy are
1Survival rates and mortality rates are complementary terms, i.e. the probability of a patient surviving her/his
first admission equals 1 minus the probability of dying in hospital on the first admission. We prefer to refer to survival rates rather than mortality rates for consistency with the specification of our empirical model that is defined over survival rates.
8
likely to be at higher risk of a negative outcome and thus the new policy might be associated with a
rise in patient readmissions. Ignoring sample selection might result in an incorrect ranking of
hospital performance, and in misrepresenting the effect on readmissions of policies with a genuine
positive effect on both survivals and readmissions.
In this study we first examine sample selection bias in the identification of hospitals’ performance on
unplanned readmissions occurring within 28 days of discharge of patients with a primary diagnosis
of fractured hip. We quantify the bias at the patient level in terms of the unexplained correlation
between the residuals of two separate probit models for survival and readmissions, similar to the
models used in many applied studies. Second, having identified a bias, we propose a solution to the
sample selection problem by using a bivariate sample selection model that allows for the correlation
between survival and readmissions and for the non linear nature of the data generating process
(Greene, 2000). This model, drawn from the literature on education and labour participation, is
simple to implement and provides accurate information on both the outcome of interest and the
underlying selection process.
We study patients aged 65 and over admitted with a fractured hip to English hospitals over the fiscal
years 2003/04 to 2008/09. This group is chosen for several reasons. First, there are well established
medical guidelines on the standard of services and processes of care for this type of admissions and
clear links between the guidelines and both mortality and readmission outcomes (National Institute
for Clinical Excellence (NICE), 2004). Second, rates of unplanned readmissions from this population
of patients standardised for age and sex are routinely published by the NCHOD and used by the
Health Care Quality Commission to monitor the performance of English hospitals. Finally, admissions
for hip fracture have substantial economic and health implications. It is estimated that fracture and
frailty related falls in older people accounted for more than 4 millions hospital bed days in 2006/7 in
England for. The combined cost of social and hospital care for this type of injury are reported to be
in excess of £1.8 billion per year in the UK (Treml, Husk, Lowe, & Vasilakis, 2011). Injuries from falls
are the leading cause of accident-related mortality in older people, and half of the people suffering a
hip fracture never return to their original level of independence (Treml et al., 2011).
The paper is organised as follows: section 2 explains our model, section 3 describes the data, section
4 presents results, and section 5 concludes.
The Model
Many empirical applications model the probability of a readmission using univariate single index
models, such as the logit or the probit. In this section, we start by describing the basic formulation of
a probit model of readmissions and then link it to the additional specifications that define our
model.
The probability of readmission can be modelled as follow:
9
(1)
(2)
(3)
Equation 1 defines the latent variable, , the propensity of being readmitted for patient i as a
function of the vector . The latter includes: a Cx1 vector of variables, , describing individual
characteristics, such as age, sex, health conditions; a Hx1 vector of dummy variables, ,capturing
the hospital of first admission; a Zx1 vector of area level variables, , capturing external
environmental factors, such as area level characteristics influencing the demand for and supply of
health services. Equation 2 describes the probability of a readmission, ,as function of the latent
process. Assuming that is normally distributed with zero mean and variance normalised to 1,
the probability of a readmission can be calculated using Equation 3 where is the standard
normal cumulative density function (CDF). The vector of marginal effects (also known as partial
effects) is:
(4)
Where is the standard normal density function.
Hospital performance on readmission can be measured by the hospital marginal probabilities, i.e.
the difference in the probability of being readmitted between patients discharged from hospital h
and patient discharged from the baseline hospital after controlling for confounding factors and
. In the presence of unobserved heterogeneity, such as unobservable patient health conditions,
the marginal probabilities described in equation 4 are calculated and averaged over the full
population of patients obtaining a vector of average partial effects (APE). The model described in
equation 3 is often estimated over a population of patients that can be assumed to be randomly
allocated to the hospitals after adjusting for observable confounders, such as emergency admissions
from hip fractures, strokes or AMI, in order to avoid the identification problems described in
Gowrisankaran and Town (1999). Now assume that the patients admitted to the hospital enter a
selection process before being discharged, for example a subsample of patients die during the
treatment. This process can be described by a probit model analogous to the one for readmission, in
which the latent variable indicates survival propensity:
(5)
(6)
(7)
10
Notice that the model in equation 7 is estimated over the total population of patients admitted to
the hospital; in contrast, the model in equation 3 is estimated only over the population of patients
that survive the first admission. This might result in a sample selection bias whenever we examine
only the subsample of patients that survive the first admission when we ideally want to make
inferences on the total population of patients admitted.
The sample selection problem can be formulated in terms of an omitted variable problem in
equation 1 (Heckman, 1979):
(8)
Equation 8 states that the parameters are not identified if the last term of the equation is
different from zero.
If we assume that:
1. ( , ) are bivariate standard normally distributed with correlation coefficient ρ
2. ( , ) are independent from ( )
Then we can define the last term of equation 8:
(9)
With
the inverse Mills ratio.
When survival and readmissions are uncorrelated, i.e. , inference on the population of patient
admitted to the hospital can be made by using equation 3. This might be the case if the two
processes are truly independent, or equivalently if we are able to make the two processes
independent after controlling for the residual heterogeneity conditioning on x, e.g. in a clinical trial
study design.
If survival and readmissions are correlated, i.e. , then using equation 3 to make inferences over
the population of patients admitted to the hospital results in omitted variable bias described by the
term (9).
Why should we expect the survival and readmission process to be correlated? The answer comes
from the combination of two factors: first, the characteristics of patients that influence their
underlying risk of a negative health outcome are only partially observable, e.g. patient health
conditions; and second, unobservable characteristics of patients influencing their mortality risk are
also likely to influence their risk of a readmission. Thus, if we are unable to provide appropriate
control for these risk factors in the readmission equation, then ex ante patients surviving their first
admission are expected to have a lower risk of being readmitted than patients dying in the hospital.
This condition can be summarized in the following expression:
(10)
11
When we are able to control for all the relevant risk factors, , the conditional probability of being
readmitted for patients surviving the first admission equals the conditional probability for the
patients that die in hospital, i.e. the two processes are uncorrelated and . Otherwise, we
expect that the conditional probability of being readmitted to be smaller in the subsample of
patients that survive the first admission because, for example, they are un-observably healthier than
other patients, i.e. . Of course, we only observe on the subsample of patients that survive
the first admission, but the counterfactual probability of a readmission for patients that die in
hospital can be estimated by comparing their characteristics with readmitted patients.
We now assume that hospitals differ in their performance on survival rates after conditioning on
observable confounders. In other words, the best performing hospitals are most successful in
reducing mortality of patients at higher risk of a negative health outcome. Then we would expect a
larger share of un-observably riskier patients to survive their first admission in these hospitals, i.e.
there is a larger share of patients with large values of in equation 8. Therefore, the relative
performance of such hospitals on readmissions is not identified by using equation 3 either with
respect to the population of patients that survive their first admission or with respect to the total
population of patients admitted. The problem of sample selection bias translates to a problem of
identification of hospital performance because patients surviving the first admission are no longer
randomly assigned to hospitals. We can also predict the sign of the bias. Since the inverse Mills ratio
is non-negative and we expect ρ < 0, then the performance of hospital with high survival rates is
underestimated due to the effect of the sample selection bias.
In general, sample selection bias might affect the identification of any variable having an effect on
patient survival rates, such as the introduction of a new health policy or technology that increases
the probability of survival. Since the patient characteristics that determine her/his risk of a negative
health outcome are only partially observable, then patients surviving due to the effect of the policy
are likely to have larger values of than other patients. Ignoring the sample selection might then
result in underestimating the effect of the policy on readmissions if the policy has a genuine positive
impact on readmissions, or in attributing an artificial negative effect to the policy on readmissions
even if the policy has no real impact. Inference based only on patients surviving the first admission is
therefore problematic.
We use a bivariate sample selection model to allow for the selection bias and estimate the model
over the total population of patient admitted to the hospital, which can be assumed to be randomly
allocated after controlling for observable confounders. This model is attractive because it takes into
account the non-linear nature of the process that defines mortality and readmissions. The model
consists of two equations as follows.
First a selection equation defines the probability of surviving the first admission, , as a function of
the latent propensity of surviving :
(11)
(12)
The parameterisation of equation 12 is described in equations 5-7.
12
Second, an outcome equation describes the probability of being readmitted, , as a function of the
latent propensity of being readmitted observed only when
(13)
(14)
With:
a. ( , ) are bivariate standard normally distributed with correlation coefficient ρ
b. ( , ) are independent from ( )
The maximum likelihood function is defined over the probabilities of three possible events:
1. surviving and being readmitted:
(15)
2. Surviving and not being readmitted:
(16)
3. Dying in hospital:
(17)
The maximum likelihood is (Green or van de ven and van praag):
(18)
Where the first patients survive and are readmitted, the following patients survive and
are not readmitted, and the last die in hospital.
The probability of interest is the probability of a readmission conditional on having survived the first
admission:
(19)
(20)
13
In the case of no sample selection this probability is given by equation 3. The performance of
hospital j in readmissions can be measured in terms of average partial effect (APE) of being treated
in hospital A as compare to a baseline hospital. Specifically, the APE is defined as the difference
between the conditional probability of a readmission in hospital A and the baseline hospital
averaged over all patients in the population:
(21)
The vectors and define the characteristics and the external environmental factors associated
with patient i; and is a dummy variable identifying the hospital j.
Alternatively, the performance of the hospital j can be defined in terms of average conditional
probability, i.e. the first term of equation 21:
(22)
Expression 22 describes the probability of a readmission in hospital j averaged over the total
population of patients and can be interpreted as the performance on readmission that hospital j
would have if it had treated the whole population of patients. This measure of hospital performance
has three appealing characteristics: 1) it is purged of differences across hospital case mix and
external environmental factors, 2) it is measured on a ratio scale, i.e. has no arbitrary zero value, 3) it
does not depend on a baseline hospital. In contrast, the APE benefits from only the first of these
desirable properties.
The bivariate sample selection model is identified under the two assumptions described in a and b.
The first states that the error terms in equation 13 and 11 are independent of all regressors. We
have shown in equation 8 that the hospital effects are potentially correlated with unobservable
patient characteristics in the error term. However, such a correlation is an effect of the sample
selection bias only, since after controlling for observable confounders patients are assumed to be
randomly allocated to hospital on admission.
The second assumption is a parametric assumption that is needed for the model identification
arising from the functional form of the probit models. In order to improve the identification of the
model we provide a set of exclusion restrictions, i.e. variables that explain the variation in the
probability of surviving (the selection equation 11-12) and are uncorrelated with the probability of a
readmission (the outcome equation 13-14) after controlling for other factors. We discuss our
approach to the exclusion restriction in the following section.
The Data
Population of interest and health outcome variables
Data on patient admissions are extracted from the Hospital Episode Statistics (HES), which comprise
records of all publicly funded patients admitted to hospitals in England. We include in our study all
14
hospital emergency admissions during the fiscal year 2003/4 to 2008/92 of patients aged 65 and over
with a primary diagnosis of a fractured hip (ICD-10 codes S70.0, S70.1 and S70.2) at the time of
admission. We track the full hospital history of these patients from their first admission to the final
discharge home taking into account transfers across different hospitals occurring within the period
of inpatient stay. Hospitals with less than 50 relevant admissions per year are excluded from the
analysis.
Unplanned readmissions are identified as emergency admissions occurring within 28 days of the
patient’s last discharge, and wherever they occur they are attributed to the hospital where the
patient was first admitted and treated for the fractured hip. We exclude patients admitted or
discharged under a mental health specialty and avoid double counting of patients having multiple 28
days readmissions for a fractured hip by including only the first one3. Our identification of patient
population and readmissions follows the methodology used by the NCHOD in producing hospital
standardised readmission rates to monitor hospitals’ performance.
We identify in-hospital patient mortality as reported by the hospital at the point of discharge. We do
not have data on patients dying at home within 28 days of discharge for the full period covered by
our study. However, we have data on mortality occurring within 28 days in any setting (home,
hospital or other institution) from 2003/4 to 2006/7 and are therefore able to test the robustness of
our model to the inclusion of such deaths.
Patient characteristics
We include dummy variables for patient age (7 groups) and gender. We measure patient health
characteristics on admission (observable risk of a negative heath outcome) by using the Charlson co-
morbidity index and a set of dummy variables controlling for specific conditions separately (Bottle &
Aylin, 2006): dementia or Alzheimer’s (ICD-10 codes F00-F03, G30), diabetes (E10-E14), chronic
ischemic heart disease (I20, I23-I25), chronic lower respiratory disease (J40-J47), heart failure (I50),
renal failure (N17-N19), and malignant melanoma (any C code). Also, we include a variable counting
the total number of secondary diagnosis in the first episode of care after the patient’s admission
(Wray, Hollingsworth, Peterson, & Ashton, 1997). We include dummies for the main type of
operations performed, i.e. fixation procedure including primary open or closed reduction and
internal or external fixation (OPCS-4 codes W19-25), prostatic replacement of head of femur (W46-
48), other procedures including non orthopaedic ones, and no procedure carried out (the baseline).
We follow the classification used in similar studies (Bottle & Aylin, 2006). The controls for the type of
operation acts as a proxy for patient health conditions rather than as hospital decision variables,
since the scope for varying the choice of procedure is limited for these type of patients. All the
variables described above are measured at the individual level and are included in both the patient
survival and readmission equations.
2 The fiscal year runs from 1 April to 31 March.
3 Also, we take into account readmissions occurring in the last month of the fiscal year 2002/3 and the first
month of 2009/10.
15
Environmental characteristics at small area level
We provide control for external environmental factors that influence hospital performance but are
outside the control of the hospital. We use a battery of indicators capturing the characteristics of the
patient small area of residence, known as the lower super output area (LSOA). These are
geographical units developed by the Office for National Statistics with an average population of
1,500 individuals and a standard deviation of 200. We control for the socioeconomic deprivation in
the patient area of residence by using an indicator of income deprivation among older people
(IDOPI). This indicator is one of the subdomains of the indices of multiple deprivation 2007 (Noble,
mcLennan, Wilkinson, Whitworth, & Barnes, 2008) and measures the proportion of area residents
aged 65 and over living in family relying on means-tested income benefits. We divided the IDOPI into
4 quartiles representing increasing level of deprivation and include these in both the survival and
readmission equations.
The distance of the hospital from the patient’s place of residence may influence the probability of a
readmission, as patients living closer to the hospital have lower costs in accessing hospital services.
This could influence the performance of hospitals located in urban areas relative to those located in
rural areas where the population is sparse. We include the distance variable both in the mortality
and readmission equation.
Characteristics of the supply of and demand for health care services in the patient area of residence
might influence her/his propensity of being admitted to the hospital and hence of being readmitted
(Epstein, Jha, & Orav, 2011). For example, a relatively high supply of primary services might reduce
hospital care utilisation, while a relatively high supply of hospital providers might increase it.
Similarly, the nature of local demand for health services may influence the propensity of hospital
readmissions. For example, the total population, the age and gender composition, and the
prevalence of disease are likely to put hospital services under different degree of pressure. In order
to control for all these factors, we construct an indicator of the expected volume of all cause
emergency admissions in the patient area of residence “a”:
Where is obtained by regressing all cause emergency admissions occurring in area “a” against the
area level characteristics of supply and demand captured by the vector . The latter includes
number of GPs per 10,000 population, number of hospitals within 30km, total area population, age
and gender composition and prevalence of disease (i.e. Atrial fibrillation, Cancer, Chronic kidney
disease, Chronic obstructive pulmonary disease, Coronary heart disease, Diabetes, Epilepsy, Heart
failure, Hypertension, Hypothyroidism, Obesity, Stroke and transient ischemic attack). We estimate
by running a separate OLS regression using the total population of English LSOAs. This indicator of
emergency admissions propensity is then used as a control in the readmission equation only.
Data on the prevalence of disease are submitted yearly by GP practices to the national Quality
Management and Analysis System (QMAS) and show the proportion of individuals registered with a
GP practice recorded with that condition. We attribute this to small area level using the Attribution
Dataset of patient registration addresses within GP practices. The attribution process assumes that
prevalence for a particular small area is a weighted sum of the prevalence in each GP practice
serving that small area, with weights proportional to the number of the area’s residents registered
16
with each GP practice. Both the QMAS data and practice to small area attribution data were
obtained from the NHS Information Centre. Number of GPs per 10,000 population is based on GP
practice level administrative data on whole time equivalent GPs per registered patient, from the
General Medical Services database. This GP practice level variable is then attributed to small level
using the same procedure described above, as a weighted average based on the share of GP practice
registered patients resident in the small area.
Exclusion restrictions
In order to improve the identification of our model, we use a set of variable explaining variation in
the mortality equation, but assumed to be uncorrelated with patient readmissions. For this purpose
we construct indicators for patients being admitted during Christmas or Easter holidays and for the
weekday of admission. Hospitals experience difficulties in maintaining appropriate levels of staff
during weekends and over long holidays due to higher costs, hence nurse and specialist staff is
generally reduced over these periods, and patient mortality risk increases (Dr Foster Inteligence,
2011). However, these indicators can be assumed to be uncorrelated with the risk of a readmission,
which depends on post operative care that can be provided more flexibly over a long period of time
once survival has been assured. Also, being admitted over a particular weekday, Christmas or Easter
should not be correlated with unobservable characteristics of the patient risk of a negative health
outcome. We have tested the association between our exclusion restrictions and the probability of a
readmission by including the latter in the probit for readmission (equation 3) and find no statistically
significant association. Also, appendix 1 reports differences in mean survivals, readmission, patient
age, Charlson index and number of diagnoses disaggregated by Christmas, Easter and week day of
admission. There are only small differences in the characteristics of patients by time of admission.
Finally, a similar set of variables are used as instruments in a study of the effect of a delay in
treatment on mortality in hip fracture admissions (Hamilton, 1999).
Results
Descriptive statistics
Table 1 contains descriptive statistics for all the main variables used in the analysis pooled from the
financial year 2003 to 2008. The average age in our population of patients is 83.3 years with the
largest share falling in the 80-85 (25.6%) and 85-90 (24.6%) age bands; 77.8% are women since bone
frailty and osteoporosis are conditions more prevalent in this gender group. Patients admitted have
on average 5 diagnoses and their more frequent co-morbidities in the Charlson index are chronic
ischemic heart disease (13.4% of admissions) and chronic lower tract disease (10.9%). The most
frequent procedure is a fixation (42.7%), followed by prostatic replacement (37.5%), management of
the patient without procedure carried out (15.2%) and other procedures (4.7%). The average patient
comes from a small area characterised by 15% of the over 65 population relying on income benefits,
with an average distance of 12.8Km from the hospital of first admission and a predicted volume of
129 emergency admissions per year given the characteristics of the local demand and supply of
health services.
17
Table 1 Patient level descriptive statisitcs 2003-2008 Variable Obs Mean Std. Dev.
health outcomes readmissions 250700 0.1192581 0.3240926
survivals 289910 0.8647631 0.3419771
demographics age 289910 83.26665 7.434723
65-70 289910 0.047325 0.2123335
70-75 289910 0.0858473 0.280139
75-80 289910 0.1590563 0.3657292
80-85 289910 0.2514608 0.4338536
85-90 289910 0.2458039 0.4305635
90-95 289910 0.1553172 0.3622074
95 and over 289910 0.0551895 0.2283502
female 289910 0.7774516 0.4159582
health conditions dementia (ICD-10 codes F00-F03, G30) 289910 0.0553206 0.2286054
diabetes (E10-E14) 289910 0.098465 0.2979429
chronic ischemic heart disease (I20, I23-I25) 289910 0.1341175 0.3407791
chronic low tract respiratory disease (J40-J47) 289910 0.1091339 0.3118076
heart failure (I50) 289910 0.0574903 0.2327774
renal failure (N17-N19) 289910 0.0369046 0.1885278
malignant melanoma (any C codes) 289910 0.0345142 0.182546
charlson index 289910 0.7533614 1.109684
total diagnoses 289910 5.114377 2.505715
fixation procedure (OPCS-4 codes W19-25) 289910 0.4266634 0.4945934
prostatic replacement of head of femur (W46-48) 289910 0.3745266 0.4840013
other procedure 289910 0.0470767 0.2118033
no procedure performed 289910 0.1517354 0.3587648
environmental factors expected emergency admissions(*) 289910 128.5546 35.50588
distance from hospital 289910 12.84647 25.39315 income deprivation among older people index (IDAOPI) 289910 0.1524638 0.1128906
year dummies 2003 289910 0.1627735 0.3691595
2004 289910 0.1622767 0.3687051
2005 289910 0.1661159 0.3721853
2006 289910 0.164643 0.3708587
2007 289910 0.1719868 0.3773696
2008 289910 0.1722041 0.3775584
notes:
(*) expected values are obtained by regressing observed total emergency admissions in the patient area of residence against the characteristics of demand and supply of health services.
Figure1 shows annual trends in hospital mortality as total deaths over the total patients admitted
and annual trends in hospital readmissions as total readmissions over total patient discharged alive
after the first admission. Hospital mortality follows a decreasing trend over the full study period,
with a steeper trend from the financial year 2006/7. In contrast, readmissions rise noticeably until
2005/6, stay constant in the following two years and then fall in 2008/9.
18
Figure 1 Raw trends in mortality and readmissions
Table 2 shows annual trends in unadjusted outcomes at hospital level. The number of hospitals
included in the analysis4 each year ranges from 151 to 148 and their average volume of relevant
admissions rises progressively from 375 to 404. The average hospital survival rate increases
progressively from 85.0% to 88.4%, while their average readmission rate increases from 10.9% in
2003/4 to 13.0% in 2005/6-2006/7 and drop back to 11.8% in 2008/9. The variation in hospital
survival rates is stable from 2003/4 to 2006/7 with coefficient of variation (i.e. standard deviation
over mean) ranging from 4.3% to 4.1%. This variation drops in 2007/8 and 2008/9 when the
coefficient of variation is 3.6% and 3.4% respectively. The correlation between hospital unadjusted
survival rates and readmission rates is positive over the period with larger variation in hospital
survival (2003/4-2006/7) and becomes negative over the period with smaller variation in hospital
survival (2007/8-2008/9). In other words, the descriptive statistics show that hospitals with better
performance on survival rates have worse performance on readmissions when the variation across
hospital performance in survival rates is large. The positive correlation between the two health
outcomes is superficially puzzling: if hospital quality of care (e.g. organisation and clinical quality)
influences survivals and readmissions, then hospital with higher survivals might be expected to have
lower readmissions, in which case the correlation should be negative. Indeed, the correlation turns
negative when the variation across hospital survival rates is reduced. Using regression analysis we
shall show that the observed correlation is the result of a sample selection process in which hospitals
4 Only hospitals with more than 50 admissions per year are included and 3 hospitals merges together over the
period considered.
.1.1
1.1
2.1
3.1
4.1
5
2003 2004 2005 2006 2007 2008year
mortality rates readmission rates
19
with higher survival rates end up having a larger share of patients at high risk of a negative outcome
compared to hospitals with lower survival rates.
Table 2 Hospital level descriptive statistics by year
2003/4 2004/5 2005/6 2006/7 2007/8 2008/9
total hospitals 150 151 151 148 148 148
mean admissions 374.8 373.7 382.2 389.0 404.2 404.0
mean survival 0.850 0.853 0.857 0.865 0.872 0.884
mean readmissions 0.109 0.120 0.129 0.130 0.128 0.118
survival stand. deviat. 0.037 0.036 0.036 0.035 0.031 0.030
survival stand.deviat. /mean 0.043 0.043 0.042 0.041 0.036 0.034
correlation survival readmissions 0.045 0.120 0.168 0.085 -0.040 -0.110
Regression analysis
Table 3 contains the estimated average partial effects (APEs) obtained from the probit model on
readmission described in equations 1-3 (column 1), a bivariate sample selection model described in
equations 11-20 (column 2), and a probit model for survival described in equation 5-7 (column 3). All
models are estimated over pooled observations from 2003/4 to 2008/9 and include dummy
indicators capturing the hospital effects. The bivariate sample selection model reports a significant
and negative residual correlation between the probit on survival and the probit on readmission (ρ = -
0.56). This suggests that the sample of patients that die in hospital would be at higher risk of a
readmission had they survived their first admission compared to patients who survive (expression
10). Therefore, the population of patients admitted to the hospital and the sample of patients that
survive the first admission differ in their risk of being readmitted after controlling for all observable
confounders. This implies that the group survivors cannot be used as a basis for making inferences
on the conditional probability of being readmitted before appropriate correction for the sample
selection bias is provided.
20
Table 3 Estimated Average Partial Effects (APE) from regression analysis
Probit on readmissions
Bivariate sample
selection model
Probit on survival
VARIABLES APE se APE se APE se
demographics (baseline: age 65-70) 70-75 0.00700** (0.00329) 0.0116*** (0.00379) -0.0140*** (0.00261)
75-80 0.0162*** (0.00322) 0.0285*** (0.00430) -0.0379*** (0.00302)
80-85 0.0287*** (0.00335) 0.0495*** (0.00536) -0.0632*** (0.00350)
85-90 0.0421*** (0.00362) 0.0746*** (0.00705) -0.0975*** (0.00420)
90-95 0.0501*** (0.00396) 0.0992*** (0.00963) -0.151*** (0.00522)
95 and over 0.0489*** (0.00477) 0.116*** (0.0129) -0.214*** (0.00664)
female -0.0258*** (0.00155) -0.0456*** (0.00306) 0.0562*** (0.00127)
health conditions dementia 0.0255*** (0.00323) 0.0270*** (0.00362) 0.00117 (0.00247)
diabetes 0.0133*** (0.00253) 0.00237 (0.00330) 0.0332*** (0.00175)
chronic ischemic heart disease 0.00999*** (0.00215) 0.0152*** (0.00255) -0.0129*** (0.00173)
chronic low tract respiratory disease 0.00336 (0.00237) 0.00850*** (0.00286) -0.0175*** (0.00203)
heart failure 0.0103*** (0.00357) 0.0491*** (0.00708) -0.0922*** (0.00312)
renal failure 0.000866 (0.00444) 0.0479*** (0.00900) -0.117*** (0.00412)
malignant melanoma -0.0153*** (0.00436) -0.0140*** (0.00515) -0.00660* (0.00372)
charlson index 0.00955*** (0.000996) 0.0168*** (0.00147) -0.0185*** (0.000758)
total diagnoses 0.00373*** (0.000347) 0.0106*** (0.00109) -0.0207*** (0.000279)
procedure (baseline: no procedure) fixation procedure -0.0272*** (0.00197) -0.0596*** (0.00463) 0.0978*** (0.00149)
prostatic replacement of head of femur -0.0294*** (0.00199) -0.0624*** (0.00469) 0.0992*** (0.00152)
other procedure -0.0289*** (0.00353) -0.0470*** (0.00462) 0.0465*** (0.00342)
environmental factors (baseline: least income deprived') 2nd quartile 0.00160 (0.00196) 0.00379* (0.00230) -0.00707*** (0.00173)
3rd quartile 0.00797*** (0.00208) 0.0127*** (0.00250) -0.0130*** (0.00178)
4th quartile (most income deprived) 0.0136*** (0.00227) 0.0206*** (0.00277) -0.0187*** (0.00192)
distance from hospital -8.37e-05*** (2.94e-05) -0.000154*** (3.66e-05) 0.000288*** (3.38e-05)
expected emergency admissions'' 0.000175*** (2.23e-05) 0.000196*** (2.52e-05) year dummies (baseline: 2003)
2004 0.00601** (0.00243) 0.00272 (0.00281) 0.0131*** (0.00184)
2005 0.0109*** (0.00246) 0.00410 (0.00302) 0.0259*** (0.00173)
2006 0.00866*** (0.00245) -0.00298 (0.00332) 0.0399*** (0.00163)
2007 0.00514** (0.00240) -0.0113*** (0.00369) 0.0533*** (0.00152)
2008 -0.00676*** (0.00233) -0.0299*** (0.00431) 0.0702*** (0.00139)
patient admitted on (baseline Saturday): Sunday
-0.00584** (0.00231)
Monday
-0.00333 (0.00222)
Tuesday
-0.00222 (0.00220)
Wednesday
-0.00255 (0.00221)
Thursday
0.000233 (0.00219)
Friday
-0.00391* (0.00222)
Christmas holidays
-0.0158*** (0.00384)
Easter holidays -0.0102** (0.00471)
rho
-0.56064 (-0.05782)
Observations 250700 289910 289910
Notes: Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 ' Income deprivation among older people index (IDAOPI)
'' Expected values are obtained by regressing observed total emergency admissions in the patient area of residence against the characteristics of demand and supply of health services.
21
The differences between the estimated coefficients of the probit and the sample selection model
are especially noticeable amongst the variables that describe patient characteristics. The conditional
probability of a readmission between each age group over the baseline (patients aged 65-70) almost
doubles after controlling for sample selection. In practical terms this means that hospitals
experiencing a rise in their share of admissions of older patients might underestimate their future
increment in readmissions if such projections are based solely on past readmissions of patients with
similar age. The difference in the conditional probability between women and men almost doubles (-
0.0258 probit; -0.0456 sample selection). The conditional probabilities for many health conditions
increase noticeably: chronic ischemic heart disease (0.010 probit; 0.015 sample selection), chronic
lower respiratory disease (0.003 probit; 0.009 sample selecton), heart failure (0.010 probit; 0.049
sample selection), renal failure (0.001 probit; 0.048 sample selection). Similar patterns are found in
the effect of variations in the Charlson index (0.010 probit; 0.017 sample selection) and total number
of diagnoses (0.004 probit; 0.011 sample selection). The conditional probabilities for each type of
operation doubles in the sample selection model reflecting the higher risk of a negative outcome
relative to patients managed with no operation (the baseline): fixation (-0.027 probit; -0.060 sample
selection), prostatic replacement (-0.029 probit; -0.062 sample selection) other operations (-0.029
probit -0.047 sample selection).
The estimated effects of the external environmental factors on readmissions show more modest
differences between the two models: patients in the second income deprived quartile (0.002 probit;
0.004 sample selection), third quartile (0.008 probit; 0.013 sample selection) and most deprived
quartile (0.014 probit; 0.021 sample selection) as compared with patients resident in the least
deprived quartile of area (the baseline); the effect of living 1km further away from the hospital (-
0.00008 probit; -0.00015 sample selection); the effect of the characteristics of the demand and
supply in the patient area of residence (0.00018 probit; 0.00020 sample selection).
The most remarkable difference is found in the annual trend in readmissions estimated by the two
models. The probit predictions mirrors the trend suggested by the descriptive statistics in figure 1. A
sharp rise in readmissions over the 2004/5 (0.006) and 2005/6 (0.011) as compared with 2003/4
(baseline), followed by a similar level in 2006/7 (0.009), a modest fall in 2007/8 (0.005) and then a
sharper fall in 2008/9 (-0.007). In contrast, the sample selection model identifies no significant
change in the year trend from 2003/4 to 2005/6 and a reduction in readmissions from 2007/8 (-
0.011) and in 2008/9 (-0.030). The differences in the two models’ predictions should be examined in
the light of the predictions from the probit on survivals, which describes the selection process. The
latter shows a significant and increasing trend in the probability of surviving the first admission
(0.013 in 2004/5, 0.026 in 2005/6, 0.040 in 2006/7, 0.053 in 2007/8 and 0.070 in 2008/9 as
compared with 2003/4). The rise in readmissions estimated by the probit model (table 3 column 1) is
generated by the following selection process. An increasing number of patients at risk of negative
health outcomes survive their first admission over time, the risk of a negative health outcome is only
partially controlled by the probit model, and hence risk adjusted readmissions are predicted to
increase over time. The 2003/4-2006/7 increment in readmissions disappears after allowing for the
sample selection process (table 3 column 2).
22
In contrast, the reduction in readmissions observed in 2007/8-2008/9 outweighs the selection effect
and therefore appears to reflect the extra effort made by the health care providers to improve the
standard of care. This effect is captured both by the sample selection model and by the probit model
(table 3 column 2 and 1 respectively), but is underestimated by the latter. It is also interesting to
note that this selection effect also explains the difference in magnitude between the risk adjusted
year trend estimated by the probit model (table 3 column 1) and the unadjusted trend shown in the
descriptive statistics in figure 1. Probit risk adjusted predictions show a more modest increase in
readmissions over time than descriptive statistics, since they capture the observable increase in
patient risk of a readmission over time. However, the probit model is unable to adjust for the
unobservable increase in patient risk generated by the selection process, and hence its trend
predictions are larger than the trend prediction of the bivariate sample selection model.
Column 3 of Table 3 reports the APEs of the probit model for survival. This model describes the
selection process that generates the sample of patients at risk of a readmission. The probability of
surviving the first admission decreases with the age of the patient, the total number of diagnoses,
the score of the Charlson index, and the income deprivation of the patient area of residence. Almost
all the comorbidity dummies are associated with a lower probability of surviving with the sole
exception of patients with diabetes. Patient having no operation performed are associated with a
lower probability of surviving than patients receiving a fixation or a prostatic replacement. The
distance from the hospital is positively associated with the probability of surviving although the
effect is virtually zero in magnitude (i.e. 100km increment in distance is associated with a 0.02
increment in the probability of surviving). This variable is likely to capture the effect of patients who
experience a fall away from their usual place of residence from which the distance is calculated.
Hence, these patients might be relatively more autonomous and healthier than other patients.
Moving to our exclusion restriction variables, we find that patients are less likely to survive if
admitted on Sunday and over Christmas and Easter holidays. Finally, the year dummies show a
progressive increase in the probability of surviving the first admission from 2003/4 (baseline) to
2008/9.
Figure 2 plots the hospital APE on readmissions from the probit model (left panel) and from the
sample selection model (right panel) against the hospital APE on survivals. The hospital APE are
estimated using the models in table 3 and are defined as the difference between the conditional
probability of a readmission (survival) in a given hospital and a baseline hospital averaged over of all
patients in the population (expression 21). The hospital APE provides a measure of hospital relative
performance in risk adjusted outcomes over the entire period 2003/4 to 2008/9. The slope of the
fitted line shows the correlation between the hospital performance on survival and readmissions.
The correlation is almost zero when hospital performance is estimated using a probit model for
readmission that does not correct for the sample selection, but becomes negative when the
performance is estimated using a bivariate sample selection model. Figure 2 provides evidence that
the sample selection at patient level biases the identification of the hospital performance on
23
readmissions, and that sample selection will lead to an underestimation of relative readmission rates
amongst hospitals with lower survival rates5.
Figure 2 Hospital performance in risk adjusted survival and readmission rates 2003-2008 - Hospital Average Partial Effects (APE)
In Table 4 we report the correlation between hospital performance in survival and readmissions
obtained from the probit and the bivariate sample selection models, disaggregated by two-year
period6. The correlation between risk adjusted survivals and readmissions is always underestimated
(in absolute value) by the probit with respect to the sample selection model. Also, the probit model
predicts a positive correlation in 2005/6-2006/7, i.e. hospitals with higher survival rates tend to
experience higher levels of readmissions.
5 As robustness check, we run the analysis excluding two hospital outliers reporting readmission rates 0.25 and
0.18 larger than the baseline hospital. Results at patient level are unchanged as well as the estimated residual
correlation coefficient ρ = -0.52. Hospital level correlation between risk adjusted survival and readmission rates increase by -0.20 under both the univariate and the bivariate probit models (from -0.01 to -0.20 and from -0.28 to -0.48 respectively). That is, hospital performance on readmissions is still underestimated under the univariate probit for hospital with high survival rates but now has the expected negative sign. However, these two outliers reports higher readmissions every year and hence are likely to be genuine observations.
6 This ensures a sufficient number of observations to identify the hospital effects. However results do not
change even when correlations are computed by each year. (Could we move this footnote after the full stop?
-.1
-.05
0
.05
.1.1
5.2
.25
.3
-.15 -.1 -.05 0 .05
Hospital APE Fitted values
Probit
-.1
-.05
0
.05
.1.1
5.2
.25
.3
-.15 -.1 -.05 0 .05me_ps
Hospital APE Fitted values
Bivariate Sample Selection
24
Table 4 Correlation between hospital risk adjusted survival and reamission rates
2003/4-2004/5
2005/6-2006/7
2007/8-2008/9
bivariate sample selection -0.306 -0.213 -0.390
probit -0.042 0.089 -0.073
Figure 3 ranks hospitals by increasing average conditional probability (AP) of a readmission as
defined in expression 22. This can be interpreted as the conditional probability of a readmission
expected for a given hospital had that hospital treated the whole population of patients. The
specification of the bivariate sample selection model is the same as in table 3 column 2. The bottom
and top quintile of hospitals have a significantly different performance in readmissions over the
period 2003/4-2008/9. Figure 4 shows the change in the hospital performance rank between the
bivariate sample selection model and the probit model. Hospitals on the 45 degree line experience
no change in their rank using both models, hospitals above (below) the diagonal show a worse
(better) performance under the probit model with respect to the sample selection model. The
largest changes in ranks affect middle rank hospitals, while hospitals at the extreme top and bottom
of the 45 degree line move less in their ranks.
25
Figure 3 Hospital risk adjusted readmission rates 2003-2008. Estimated average probabilities using a bivariate sample selection model
Figure 4 Hospitals' performance in readmission rates. Hospitals ranked using probit model
predictions versus bivariate sample selection predictions.
.1.2
.3.4
0 50 100 150
Hospital Average Performance 95% CI
National Average
26
Discussion and Conclusions
The main contribution of this study is to model hospital performance on readmissions relaxing the
assumption of independence between the data generating process of patient survival (or mortality)
and readmission that is implicitly adopted in the vast majority of studies on hospital readmissions.
We examine all emergency admissions for hip fractures of patients aged 65 and over occurring over
2003-2008 in English public hospitals. We find evidence that ignoring the correlation between
mortality and readmission results in material sample selection bias in the identification of the
hospital effect on readmissions. The bias originates from unobservable patient characteristics that
influence his/her risk of a negative health outcome, such as unmeasured patient health conditions,
and from differences in hospital mortality rates. Specifically, if patients’ health conditions are not
perfectly observable, then risk adjustment will be inadequate and hospitals with higher survival rates
are more likely to have a larger share of patients at higher risk of a readmission. Therefore,
hospitals’ performance in readmissions is determined in part by their difference in the quality of care
and in part by their difference in the share of unobservably sicker patients. If this hypothesis holds,
high quality hospitals with high survival rates will tend to have higher reported readmission rates,
and hence their true performance on readmissions will be underestimated.
Evidence of sample selection at patient level comes from the estimated correlation coefficient (ρ = -
0.56) between the residuals of a risk adjusted probit model on the patient probability of surviving
and a risk adjusted probit model on the patient probability of experiencing an emergency admission
within 28 days of previous discharge. Also, we find no correlation or positive correlation between
the hospital risk adjusted performance in survival and readmission estimated using the two separate
050
10
015
0
0 50 100 150
Probit Hospital Rank Biv Probit Hospital Rank
27
probits. The positive correlation suggests that hospitals with better performance in survival rates
have worse performance in readmission rates. This association is the opposite of what would be
expected if both survival and readmissions are driven by the underlying quality of hospital care, after
controlling for patient characteristics and external environmental factors that might influence
hospital performance. We argue that this estimated association is the result of ignoring the
correlation between the data generating process of survival and readmission.
We implement a solution to the sample selection bias problem by using a bivariate sample selection
model that allows for the residual correlation between the probability of survival and readmission.
This model is attractive because it also allows for the dichotomous nature of the two outcome
variables. Once the sample selection is taken into account, hospitals’ risk adjusted performance in
survival and readmission rates became negatively correlated with hospitals having high survival rates
also having low readmission rates.
The model also allows for sample selection in estimating the differences in the conditional
probabilities of a readmission by gender, age and co-morbidity groups. The estimates from the
sample selection model are noticeably different from those obtained from the probit model, which
assumes independence between survival and readmissions. Specifically, the conditional probabilities
by gender and by age groups are from 50% to 100% higher in the sample selection model as
compared with the probit model; similar results obtain for the conditional probabilities of patients
with chronic ischemic heart disease, heart failure, renal failure, and chronic low tract respiratory
disease. This is not surprising given that these patients are at higher risk of dying during their first
admission (relative to patients without the condition), and hence the sample that survives is subject
to an intense selection process.
Finally the annual trend estimates derived from the sample selection model differ from the annual
trend estimates from the probit model. The former predicts a flat trend in readmissions over 2003-
2005 followed by a fall in 2007-2008. In contrast, the latter predicts a rise in readmissions over the
2003-2005 years and a small drop in 2007-2008. The differences between the two models are
explained by the increasing trend in survival rates that characterised the 2003-2008 period. As the
share of patients surviving their first admission rises over time, so the risk of a negative outcome in
the survivors increases over time. The probit model fails to control for the increasing risk inherent in
the hospital case-mix, because patient health characteristics are only partially observable. In
contrast, the sample selection model provides a better risk adjustment by incorporating information
on the selection process over time.
Our study therefore offers strong evidence that ignoring the correlation between the data
generating process of survival and readmission may seriously corrupt any inference on readmission.
If the researcher were able to observe all relevant patient characteristics, then survival and
readmission probabilities can be estimated independently by conditioning on observables, and
hence a simple binary response model on readmission becomes an appropriate instrument of
analysis. Unfortunately, most studies, such those using hospital administrative data, have access
only to partial information on patient health conditions and treatment characteristics. In this case,
our study suggests that a simple test for the residual correlation between patient survival and
28
readmission can provide valuable information on the most appropriate model to use in any empirical
analysis of readmissions.
An increasing number of health systems have started to release public reports of hospital
performance on readmission rates to inform patient choice of provider and to monitor hospital
quality of care. In the US, 30 day emergency readmissions following hospitalisations for pneumonia,
acute myocardial infractions (AMI) or heart failure have been reported by the Centre for Medicare
and Medicaid Services (CMS) from 2009. In the UK, the NCHOD has from 1998 produced age and
gender standardised indicators of hospital 28 days emergency readmission rates following
admissions for hip fractures and strokes to inform quality regulators. Australia's National Agency for
Health and Information uses readmission to hospital within 28 days for selected types of surgery as
an indicator of the safety and quality of public hospital care. At the same time, hospitals in these and
other countries are under pressure to reduce mortality rates for the same type of admissions for
which they are required to reduce their readmission rates.
Conventional hospital readmission indicators currently take no account of the sample selection bias
described above, and may therefore offer misleading signals of performance. Using inappropriate
indicators of performance might put some hospitals under unwarranted pressure (and conversely
may ignore weak performance in other hospitals) and generate perverse incentives for hospital
behaviour. The recent efforts to link reimbursement to readmission performance indicators
increase the potential for perverse incentives associated with such measures. We find evidence that
hospital readmissions are likely to rise as a consequence of falling mortality rates over time, but that
reducing both mortality and readmissions is an achievable target. However, if adverse consequences
are to be avoided, it will be necessary to develop more analytically satisfactory measures of hospital
performance on readmissions along the lines described in this paper.
29
Appendix 1
Table 5 Patient mean outcomes and health characteristics by period of admission
survival readmissions age charslon index total diagnoses
Christmas
(yes) 0.849 0.125 83.419 0.755 5.220
(no) 0.865 0.119 83.263 0.753 5.112
Easter
(yes) 0.858 0.121 83.257 0.727 5.029
(no) 0.865 0.119 83.267 0.754 5.116
Week day
Sunday 0.861 0.121 83.348 0.753 5.125
Monday 0.864 0.119 83.237 0.760 5.117
Tuesday 0.864 0.119 83.243 0.752 5.113
Wednesday 0.864 0.121 83.324 0.762 5.109
Thursday 0.867 0.117 83.243 0.755 5.109
Friday 0.864 0.118 83.272 0.747 5.119
Saturday 0.869 0.119 83.207 0.745 5.109
30
References
Bottle, A., & Aylin, P. (2006). Mortality associated with delay in operation after hip fracture: observational study. BMJ, 332(7547), 947-951.
Boutwell, A. E., Johnson, M. B., Rutherford, P., Watson, S. R., Vecchioni, N., Auerbach, B. S., et al. (2011). An early look at a four-state initiative to reduce avoidable hospital readmissions. Health Aff (Millwood), 30(7), 1272-1280.
Bradley, E. H., Herrin, J., Elbel, B., McNamara, R. L., Magid, D. J., Nallamothu, B. K., et al. (2006). Hospital quality for acute myocardial infarction: correlation among process measures and relationship with short-term mortality. JAMA, 296(1), 72-78.
Coyte, P. C., Young, W., & Croxford, R. (2000). Costs and outcomes associated with alternative discharge strategies following joint replacement surgery: analysis of an observational study using a propensity score. Journal of Health Economics, 19(6), 907-929.
Department of Health. (2011). A simple guide to Payment by Results. http://www.dh.gov.uk/pbr. Dr Foster Inteligence. (2011). Dr Foster Hospital Guide 2001-2011. In Dr Foster Limited (Ed.). London. Epstein, A. M., Jha, A. K., & Orav, E. J. (2011). The Relationship between Hospital Admission Rates
and Rehospitalizations. New England Journal of Medicine, 365(24), 2287-2295. Evans, W. N., Garthwaite, C., & Wei, H. (2008). The impact of early discharge laws on the health of
newborns. Journal of Health Economics, 27(4), 843-870. Evans, W. N., & Kim, B. (2006). Patient outcomes when hospitals experience a surge in admissions.
Journal of Health Economics, 25(2), 365-388. Foster, D., & Harkness, G. (2010). Healthcare reform: pending changes to reimbursement for 30-day
readmissions. Thomson Reuters. Friedman, B., & Basu, J. (2004). The rate and cost of hospital readmissions for preventable
conditions. Med Care Res Rev, 61(2), 225-240. Geweke, J., Gowrisankaran, G., & Town, R. J. (2003). Bayesian Inference for Hospital Quality in a
Selection Model. Econometrica, 71(4), 1215-1238. Gowrisankaran, G., & Town, R. J. (1999). Estimating the quality of care in hospitals using
instrumental variables. Journal of Health Economics, 18(6), 747-767. Hamilton, B. H. (1999). The impact of the NHS reforms on queues and surgical outcomes in England:
Evidence from hip fracture patients. Economic journal, 109(457), 437. Heckman, J. J. (1979). Sample Selection Bias as a Specification Error. Econometrica, 47(1), 153-161. Heggestad, T. (2002). Do Hospital Length of Stay and Staffing Ratio Affect Elderly Patients' Risk of
Readmission? A Nation-wide Study of Norwegian Hospitals. Health Services Research, 37(3), 647-665.
Ho, V., & Hamilton, B. H. (2000). Hospital mergers and acquisitions: does market consolidation harm patients? Journal of Health Economics, 19(5), 767-791.
Lilford, R., & Pronovost, P. (2010). Using hospital mortality rates to judge hospital performance: a bad idea that just won't go away. BMJ, 340, c2016.
Lorch, S. A., Baiocchi, M., Silber, J. H., Even-Shoshan, O., Escobar, G. J., & Small, D. S. (2010). The role of outpatient facilities in explaining variations in risk-adjusted readmission rates between hospitals. Health Serv Res, 45(1), 24-41.
Luthi, J. C., Burnand, B., McClellan, W. M., Pitts, S. R., & Flanders, W. D. (2004). Is readmission to hospital an indicator of poor process of care for patients with heart failure? Qual Saf Health Care, 13(1), 46-51.
McClellan, M., & Staiger, D. (2000). Comparing the Quality of Health Care Providers. pp. 113-136): National Bureau of Economic Research, Inc.
National Institute for Clinical Excellence (NICE). (2004). Clinical practice guideline for the assessment and prevention of falls in older people. London: Royal College of Nursing.
Noble, M., mcLennan, D., Wilkinson, K., Whitworth, A., & Barnes, H. (2008). Indices of Deprivation 2007. Communities and Local Government Publications. London: Social Disadvantage Research Centre, University of Oxford.
31
Papanicolas, I. (2011). Using a vector autoregression framework to measure the quality of English NHS hospitals. Working Paper.
Schreyogg, J., & Stargardt, T. (2010). The trade-off between costs and outcomes: the case of acute myocardial infarction. Health Serv Res, 45(6 Pt 1), 1585-1601.
Shahian, D. M., Wolf, R. E., Iezzoni, L. I., Kirle, L., & Normand, S. L. (2010). Variability in the measurement of hospital-wide mortality rates. N Engl J Med, 363(26), 2530-2539.
Treml, J., Husk, J., Lowe, D., & Vasilakis, N. (2011). Report of the national audit of falls and bone health in older people 2010. The Royal College of Physician. London.
Vest, J. R., Gamm, L. D., Oxford, B. A., Gonzalez, M. I., & Slawson, K. M. (2010). Determinants of preventable readmissions in the United States: a systematic review. Implement Sci, 5, 88.
Westert, G. P., Lagoe, R. J., Keskimaki, I., Leyland, A., & Murphy, M. (2002). An international study of hospital readmissions and related utilization in Europe and the USA. Health Policy, 61(3), 269-278.
Wray, N. P., Hollingsworth, J. C., Peterson, N. J., & Ashton, C. M. (1997). Case-mix adjustment using administrative databases: a paradigm to guide future research. Med Care Res Rev, 54(3), 326-356.
Xian, Y., Holloway, R. G., Chan, P. S., Noyes, K., Shah, M. N., Ting, H. H., et al. (2011). Association between stroke center hospitalization for acute ischemic stroke and mortality. JAMA, 305(4), 373-380.
Yam, C. H., Wong, E. L., Chan, F. W., Wong, F. Y., Leung, M. C., & Yeoh, E. K. (2010). Measuring and preventing potentially avoidable hospital readmissions: a review of the literature. Hong Kong Med J, 16(5), 383-389.
www.imperial.ac.uk/business-school
Imperial College Business School
Tanaka Building
South Kensington Campus
London SW7 2AZ
United Kingdom
T: +44 (0)20 7589 5111
F: +44 (0)20 7594 9184
This paper has been produced by the Healthcare Management Group
at Imperial College Business School
Copyright © the authors 2012
All rights reserved
ISSN: 1744-6783