global infrastructure spotlight bench marking healthcare

Operating healthcare infrastructureAnalysing the evidence

GLOBAL INFRASTRUCTURE

KPMG INTERNATIONAL

Produced by KPMG in collaboration with:

Foreword 1

Introduction 2

Patient environment 3

Cleanliness 4

Cost of cleaning 5

Concluding remarks 7

A comment by an experienced infrastructure practitioner 7

Research methodology 8

Contents

© 2010 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG International provides no client services. All rights reserved.

ForewordWe are pleased to present this Infrastructure Spotlight Report, the first output of a research programme being undertaken by KPMG in the UK in collaboration with UCL (University College London), looking into the long-term performance of infrastructure. This report is intended to highlight the need to improve the quality of information on operational performance and cost of infrastructure. This report represents the first step on a long journey.

We decided to begin this journey by looking at patient environments in hospitals and specifically the impact of cleaning services on those environments.

These are the emerging headlines. Firstly, hospitals operated under the terms of PFI (Private Finance Initiative) are self-assessed as having better patient environments than conventionally procured hospitals. Secondly, such PFI hospitals are also self-assessed as being cleaner. On the cost side, the cost of cleaning appears similar in PFI and non-PFI hospitals.

New facilities are built partly because they are believed to provide benefits associated with better patient environments. The magnitude of these benefits may be influenced by how the buildings are operated. It is important to note that all PFI and non-PFI facilities analysed here were built after 1994, so making them all both recent and comparable in age.

Available National Health Service (NHS) in England source data is extensive but insufficiently analysed. We believe it is worthwhile to publish at this stage in order to raise awareness of what the data is capable of revealing and of areas where data availability could and should be improved. We hope this report will serve as the catalyst for both public and private sector organisations to collaborate with KPMG and UCL on this important initiative.

We hope that you will find this report interesting and that it will provoke thoughts on how best to make informed decisions when it comes to healthcare infrastructure.

Kai Rintala

Head of Infrastructure Intelligence for Global Infrastructure KPMG in the UK

OPERATING HEALTHCARE INFRASTRUCTURE 1


2 OPERATING HEALTHCARE INFRASTRUCTURE

1 In this publication Private Finance Initiative (PFI) is used to refer to a specific form of Public Private Partnership (PPP or P3) uuilding, financing and operating the facility.designing, b

2 All healthcare facilities covered by the datasets are used for patient care activity and were built after 1994. In this report we facilities in the dataset are hospitals, there are some treatment centres.

3 National Audit Office (2009) Private Finance Projects – A Paper for the Lords Economic Affairs Committee, HMSO, London.4 The analysis presented in this report has been produced as part of a UK government funded Knowledge Transfer Partnership

providing the academic input.5 KPMG and UCL are engaged in active and ongoing discussions with a number of public and private sector organisations.6 Patient environment ratings are collated by the National Patient Safety Agency and collected by its Patient Environment Acti

clinical aspects of patient surroundings and takes into account the organisation’s policy, cleanliness in various areas, infectioratings are produced using a self assessment proforma completed by inspectors most of whom are NHS employees (there

7 Cleanliness score is a percentage score against the National Specification for Cleanliness of the NHS. The score is producedaudit of 49 elements, such as the cleanliness of fixtures & fittings and equipment, in the functional areas of the facility.

8 The cost of cleaning measure was produced by dividing the cost for the hospital by the occupied floor area. The cost of cleaReturn Information Collection (ERIC). For outsourced cleaning this cost refers to contract price; for in-house cleaning provisi

9 The method used to arrive at our findings is detailed on the last pages of this report.10 The conventionally procured or non-PFI facilities are those which have been constructed using government finance, and pro

based methods.

sed in the UK. This involves the private sector taking responsibility for

have used the term hospital for simplicity. While the vast majority of the

between KPMG and UCL, with Andrew Edkins and Graham Ive of UCL

on Team (PEAT) inspectors. The patient environment rating assesses non-n control, general environment and conditions in access/external areas. The should be a patient representative). by self-assessment by NHS employees. The assessment is a pass or fail

ning is the cost to the NHS trust as reported by that trust in its Estates on, to material, equipment and staff direct costs plus certain on-costs.

cured using design-bid-build, design-and-build or other non-project-finance


IntroductionThe United Kingdom (UK) has often been regarded the test bed for the rest of the world when it comes to using output-based long-term service contracts, mainly under the PFI1 approach, to build and operate healthcare facilities.2 However, the UK government has not undertaken a systematic and robust evaluation of the effectiveness of its programme. The National Audit Office, an independent body responsible for assessing the effectiveness of public spending, has recently voiced criticisms of this shortfall, especially in respect of privately financed infrastructure.3

KPMG in the UK has collaborated with UCL4 for this research initiative with the aim of showing that, working together, the infrastructure community can rise to this challenge.5 There has been an absence of previous analysis of the operational performance of infrastructure over the long-term that groups facilities by method of provision. We believe this has been holding back the community’s ability to move to evidence-based decision making. We argue that such evidence can only be produced through rigorous analysis of consistent, comparable and quantifiable data on the performance of

the various operational arrangements on a like-for-like basis.

The NHS collects much data both at health trust and hospital level on operational performance and cost of facilities management. It would appear this data has been under analysed by method of provision.

This KPMG Infrastructure Spotlight Report presents the initial results of a preliminary analysis of this NHS data. Future publications will continue to expand the depth and breadth of the analysis.

We decided to begin with patient environment and cleaning in hospitals.To this end we use two measures of performance and one measure of costpatient environment ratings,6 cleanliness scores7 and cost of cleaning per m2 of occupied space.8 Patient environment ratings reflect thequality of cleaning as well as the availability and maintenance of facilities. Cleaning scores bear specifically upon the quality of cleaning.

:

Our preliminary findings9 are:• PFIhospitalshavebetterpatientenvironmentratingsthanconventionally

procured10 hospitals of comparable age, in which facilities management services are provided either in-house or by an external third party.

• ThePFIhospitalshavehighercleanlinessscoresthannon-PFIhospitalsofsimilar age.

• TheaveragecostofcleaningforPFIhospitalsappearssimilartothatofnon-PFI hospitals. However, the cost of cleaning in PFI hospitals appears to be less variable over the study period.

The remainder of this report describes the sampling methods, data sets and the forms of analysis undertaken to arrive at these findings.

The report also contains a clearly separate comment by Dr Tim Stone, the Chairman of KPMG’s Global Infrastructure and Projects Group, which sets out his views that seek to place our early findings into context.

Patient environmentDoes the method of operating healthcare infrastructure influence the non-clinical aspects of healthcare experience? We have attempted to inform the debate by using existing data on patient environments.

The patient environment scores shown in Graph 1 are for two different types of healthcare facilities. These are PFI buildings and non-PFI buildings with either outsourced or in-house facilities management. The data is for four calendar years from 2005 to 2008. The number of PFI sites in operation prior to 2005 was too low to permit statistical analysis.

The patient environment scores can vary from one (unacceptable) to five (excellent) for each hospital. The study samples, given in brackets in Graph 1, are different for each of the years. The smallest sample is 15 PFI facilities in 2005 whereas the largest sample is 98 non-PFI facilities in 2007. This varying of samples between years means that caution should be exercised when reading into the trends over the four years.

Graph 1: Patient environment scores by method of infrastructure procurement

2005 2006 2007 2008

1 un

acce

ptab

le –

5 e

xcel

lent

3.4

3.6

3.8

4.0

4.2

4.4

4.6

(n=84)

(n=15)

(n=95)

(n=20)

(n=98)

(n=29)

(n=88)

(n=37)

PFINon-PFI

Source: National Health Service Information Centre, England; Her Majesty’s Treasury

Analysis: KPMG in the UK, 2010




04/05 05/06 06/07 07/08

%

96

94

92

90

86

88

PFINon-PFI

(n=68)

(n=11)

(n=96)

(n=20)

(n=114)

(n=33)

(n=101)

(n=41)

The overall picture that is emerging from Graph 1 is that the patient environment in PFI hospitals is better than in non-PFI hospitals where facilities management is provided in-house or outsourced. There are statistically significant differences between the PFI and non-PFI results in three years (2005, 2006 and 2008). There is no statistically significant difference in 2007.

It is important to understand that the patient environment scores may be influenced by aspects such as the initial building design and construction quality, but facilities management will also influence the scores.

CleanlinessDoes the way that healthcare facilities are run have an impact on the standards of the accommodation services? The cleanliness score provides another indicator of the quality of the patient environment and the standards of facilities management services provided.

The cleanliness scores shown in Graph 2 compare the average results returned for the two groups of PFI and non-PFI hospitals. The data is for four financial years from the 2004/05 to 2007/08.

Graph 2: Cleanliness by method of infrastructure procurement



The graph shows consistently higher average cleanliness scores for PFI facilities compared to non-PFI. This result is statistically significant in three years. There is no statistically significant difference in 2004/05.



Cost of cleaningDo more favourable patient environments come at a cost? It is important to ask if the superior performance in PFI facilities in terms of cleanliness and patient environment is the result of extra cost. We are examining the costs of providing the facilities management services that sustain the patient environment, focussing initially on an important subset, the cleaning service.

Graph 3: Cost of cleaning (at constant prices) by method of infrastructure procurement

(n=103)

(n=16)

(n=113)

(n=21)

(n=114)(n=31)

(n=109)

(n=38)

04/05 05/06

£/m

2

PFINon-PFI

45

40

35

30

25

2006/07 07/08



A visual examination of Graph 3 reveals that in three of the four years the cost of cleaning of PFI facilities is higher than in non-PFI buildings. There is, however, no statistically significant difference in any of the years. The immediate conclusion should therefore be that the cost of cleaning PFI and non-PFI healthcare facilities is similar.

One proposition suggested by Graph 3 is that the cost of cleaning in more recent PFI hospitals, entering operation after 2005, is lower than that in earlier PFI hospitals. Another proposition is that the cost of cleaning in largely the same set of non-PFI facilities may be increasing between 2005/06 and 2007/08.




The boxplots in Graph 4 illustrate that the range of costs of cleaning in PFI hospitals is lower than in non-PFI hospitals. A visual inspection suggests that the costs are both more stable and predictable. The presence of outliers in PFI in previous years means the higher variance seen in the non-PFI sample is only statistically significant for 2007/08.

Graph 4: Cost of cleaning (at constant prices) by method of infrastructure procurement — boxplot



Are the facilities in the two samples fundamentally different from each other? Are PFI hospitals, for example, less likely to be on inner urban sites? The expectation would be that the PFI hospitals would on average be larger and more likely to be acute rather than mental health in comparison to the recently built non-PFI ones. What effect might difference of size, function or location have upon the quality and cost of services? The research required to answer such questions is ongoing.

04/05 05/06

£/m

2

PFINon-PFI

120

100

80

60

40

20

006/07 07/08

Boxplots

• Boxplotsareasimplewaytorepresentthedistributionofadataset

• Thehorizontallinerunningthroughtheboxisthemedian.Halfofthedatapointsinthedatasethaveavaluehigherthan the median and half a value lower than the median

• Theboxitselfcontainstherangewhichincludesthemiddle50percentofthedatapoints,i.e.the2ndand3rdquartiles lie within the box

• Outliersarerepresentedbycirclesandstars

• Circlesaremoderate outliers between 1.5 and 3 box lengths away from the box end

• Starsareextreme outliers more than 3 box lengths away

• Definingtheminimumandmaximumoftherangeasexcludingtheoutliers,thelinesextendingdownandupfromtheboxtothehorizontallinesrepresentthe1stand4thquartilesrespectively



Concluding remarksThe key findings of our analysis are:

• PFIhospitalsasjudgedbyNHSpatientenvironmentratingsandcleanlinessscores have better patient environments than those procured conventionally, in which facilities management is performed in-house or contracted out.

• ThecostofcleaningPFIhospitalsisonaveragesimilartothatofcleaningnon-PFI hospitals, but less variable.

The results give rise to a number of questions. In order to advance the debate and to provide more conclusive answers to these questions, further work is needed. KPMG and UCL are keen to work with both public and private sector organisations to achieve this.

A comment by an experienced infrastructure practitionerI was a member of KPMG in the UK’s team advising the Dartford and Gravesham NHS Trust on the Darent Valley PFI Hospital. In July 1997, it was the first hospital PFI contract to be let. A considerable amount of the thinking that went into that contract is evident today in the UK hospital PFI programme.

At the time, one of the issues we were trying to resolve was the “stop and go” nature of maintenance expenditure. The intention was very much to design a system of incentives which would encourage the private sector to build a quality hospital and to maintain that hospital over the long-term. We had to lock-in responsible long-term maintenance, and to take away the public sector’s ability to cut maintenance expenditure due to budgetary pressures. Today, almost 15 years since we started work on the approach, I hear it often referred to as the source of many headaches due to the safeguards in place. I do, however, take comfort that money has been set aside into a life cycle fund for responsible maintenance. In contrast, when I visit non-PFI hospitals, the conversations quickly turn to the generally astronomical backlog maintenance which is being held back.

I am not surprised by the results on patient environment and cleanliness in this publication. They begin to provide empirical evidence of what experience and intuition would lead me to expect. People generally tend to respond to financial and other incentives. I know from personal experience that in a PFI contract these incentives are very much in line with delivering an excellent patient environment and a clean hospital.

The results on cost of cleaning I approach with more scepticism. I know the way that cost information is analysed for a PFI contract (and reported to the trust). I also believe that this is not necessarily done comparably for non-PFI hospitals.

I was recently the first witness to the House of Lords Economic Affairs Committee inquiry into PFI. The main point I made was on public sector data. Each PFI contract and less extensively each outsourced contract is measured and monitored on performance, but projects procured conventionally are not necessarily exposed to the same rigours. On costs, I believe that the datasets on cost of cleaning for PFI and

non-PFI hospitals should include the costs of pensions, but do they? Maybe the reason for the greater range in the non-PFI set is that some do and some don’t? The net outcome of all of this is that there is not reliable and high quality information about the cost of operations within the public sector on which to make informed decisions.

This brings us to the current economic environment. Governments must get public expenditure under control, yet there is significant pressure to maintain investment in infrastructure in order to sustain economic growth and increase long-term productivity. The points that I take away from this are that PFI gives you better quality at similar, yet more predictable cost. However, this issue of reliable data needs to be resolved. In a time of fiscal constraints, its absence could hinder the provision of good quality healthcare.

Tim Stone Chairman, Global Infrastructure and Projects Group, KPMG in the UK



Research methodology New built healthcare facilities

• ThebasedatasetonUKhealthcarefacilities was obtained from Hospital Estates and Facilities Statistics (HEFS) website (www.hefs.ic.nhs.uk). The website is hosted by the NHS Information Centre.

• Allsitesthatareusedtoprovidenon-hospital care (such as Nursing Homes), all sites specially used for General Practitioner services and support facilities where patients are not treated or accommodated and any sites representing an aggregated facility were removed from the dataset.

• Allsiteswithanyfacilitiesbuiltbefore1995 were excluded from the dataset.

• LIFT(LocalImprovementFinanceTrust) facilities were excluded from the dataset.

• Theremainderofthesitesinthedataset consisted of Community Hospitals, General Acute Hospitals, Long Stay Hospitals, Multi-service Hospitals, Specialist Hospitals and Treatment Centres all built after 1994.

PFI facilities

• ThePFIhospitalswithintherecently-built facilities dataset were identified through a combination of using a list of PFI hospitals provided to us by the NHS Information Centre, the HM Treasury PFI signed projects list (dated September 2009) as well as other publicly available information such as NHS trusts’ websites.

Variables

• Performancedatafortheanalyseswas obtained from the NHS Information Centre and the National Patient Safety Agency.

• Patient environment rating is collated by the National Patient Safety Agency and collected by its Patient Environment Action Team (PEAT) inspectors. The patient environment rating assesses non-clinical aspects of patient surroundings and takes into account the organisation’s policy, cleanliness in various areas, infection control, general environment and conditions in access/external areas. The ratings are produced using a self assessment proforma completed by inspectors most of whom are NHS employees (there should be a patient representative). The data was obtained from the NHS Information Centre and the PEAT website (www.msnpsa.nhs.uk/Peat).

• Cleanliness score is a percentage score against the National Specification for Cleanliness of the NHS. The score is produced by self-assessment by NHS employees. The assessment is a pass or fail audit of 49 elements, such as the cleanliness of fixtures & fittings and equipment, in the functional areas of the facility.

• The cost of cleaning measure was produced by dividing the cost for the hospital by the occupied floor area. The cost of cleaning is the cost to the NHS trust as reported by that trust in its Estates Return Information Collection (ERIC). For outsourced cleaning this cost refers to contract price; for in-house cleaning provision, to material, equipment and staff


http://www.hefs.ic.nhs.uk

http://www.msnpsa.nhs.uk/Peat


direct costs plus certain on-costs. Each year’s cost of cleaning at current prices was converted to constant prices of April 2008 to make the years comparable with each other. For PFI hospitals, this is part of the unitary charge apportioned to cost of cleaning under the PFI contract.

Analysis

• Chi-squaredanalysiswasusedtoestablish whether the differences between the discrete patient environment scores in PFI and non-PFI hospitals were statistically significant. To reduce the number of cells within the frequency table for this analysis, the observations were grouped into three ranges of rating: 5, 4 and 3 or lower.

• Independentsamplet-testswereused to establish whether the differences between the cleanliness scores and cost of cleaning in PFI and non-PFI hospitals were statistically significant. If the result of Levene’s test for equality of variance was significant, the equal variances not assumed p-value was used.

• Ninetypercentconfidenceintervalswere used for all tests.

• SPSSwasusedtoundertakethestatistical analysis.

Sample size, time period, consistency and attrition

• Thesizeofthesamplesusedtoanalyzeeachvariableineachyearwasdriven by the availability of data on that variable.

• Samplesizesbefore2004weretoosmall for effective statistical analysis,

due to the relatively small number of operational PFI facilities.

Seventeenhospitalsinthe2004/05cleanliness score returns were found to have inadvertently reported their Patient Environment ratings. These were removed to form the study sample.

Sevenhospitalswithsimilarextremelow values for cost of cleaning were removed from the 2004/05 sample based on judgement as data error / aggregation of costs was suspected.

Thefacilitieswithineachsamplevaryslightly from year to year as data for some years is missing.

Samplingattritionisdemonstratedhere by the example of cost of cleaning in 2007/08:

− The untouched 2007/08 HEFS at site level reports on 1,965 sites

− R emoving all forms of aggregated site and sites where patients are neither treated nor accommodated and General Practitioners premises reduces this to 1,151

− Removing all sites with any part of the age asset profile dated before 1995 (or with missing data for age) reduces this to 155

− Of those 155, eight did not report the cost of cleaning services

− The remaining 147 became our base sample to be split between PFI and non-PFI hospitals.

•

•

•

•


KPMG’s Global Infrastructure professionals provide specialist advisory, tax, audit, accounting and compliance related assistance throughout the life of infrastructure projects and programs.

Our member firm teams have extensive local and global experience advising governmentorganizations,infrastructurecontractors,operatorsandinvestors.

We help our clients to ask the right questions and find strategies tailored to meet the specific objectives set for their businesses. Our teams can help set a solid foundation at the outset and combine the various aspects of infrastructure projects or programs – from strategy, to execution, to end-of-life or hand-back.

For further information regarding how KPMG’s Global Infrastructure practice can help, please visit us online or e-Mail: [email protected].

The information contained herein is of a general nature and is not intended to address the circumstances of any particular individual or entity. Although we endeavor to provide accurate and timely information, there can be no guarantee that such information is accurate as of the date it is received or that it will continue to be accurate in the future. No one should act on such information without appropriate professional advice after a thorough examination of the particular situation.

The views and opinions expressed herein do not necessarily represent the views and opinions of KPMG International and KPMG member firms.

© 2010 KPMG International Cooperative (“KPMG Interna-tional”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International have any such authority to obligate or bind any member firm. All rights reserved.

KPMG and the KPMG logo are registered trademarks of KPMG International Cooperative (“KPMG International”), a Swiss entity.

Publication name: Operating healthcare infrastructure

Publication number: 1004502

Publication date: April 2010

kpmg.com/infrastructure

www.kpmg.com/infrastructure

mailto:infrastructure%40kpmg.com?subject=

global infrastructure spotlight bench marking healthcare

Documents