pilot audit

8/14/2019 Pilot Audit

1/23

NATIONALN TLE

NATIONAL AUDIT OF CONTINENCE CARE:

LAYING THE FOUNDATION.

Sarah Mian BSc PhD, Project Manager,Adrian Wagg MB BS FRCP*, Associate DirectorPenny Irwin BA RGN SCM MSc,Derek Lowe MSc Cstat, Medical Statistician,Jonathan Potter MD FRCP, Programme DirectorMichael Pearson MD FRCP, Unit Director

Clinical Effectiveness and Evaluation Unit,

Royal College of Physicians,

London

Correspondence:

Dr Jonathan Potter, Clinical Effectiveness and Evaluation Unit,

Royal College of Physicians, 11 St Andrews Place, Regents Park,

London [email protected]

1


2/23

ABSTRACT

IntroductionNational audit provides a basis for establishing performance against national

standards, benchmarking against other service providers and improving standardsof care. For effective audit, clinical indicators are required that are valid, feasible toapply and reliable. This study describes the methods used to develop clinicalindicators of continence care in preparation for a national audit.

AimTo describe the methods used to develop and test clinical indicators of continencecare with regard to validity, feasibility and reliability.

MethodA multi-disciplinary working group developed clinical indicators that measured the

structure, process and outcome of care as well as case mix variables. Literaturesearching, consensus workshops and a Delphi process were used to develop theindicators. The indicators were tested in 15 secondary care sites, 15 primary caresites and 15 long-term care settings.

ResultsThe process of development produced indicators that received a high degree ofconsensus within the Delphi process. Testing of the indicators demonstrated aninternal reliability of 0.7 and an external reliability of 0.6. Data collection equiredsignificant investment in terms of staff time and training.

ConclusionThe method used produced indicators that achieved a high degree of acceptancefrom health care professionals. The reliability of data collection was high for thisaudit and was similar to the level seen in other successful national audits. Datacollection for the indicators was feasible to collect, however issues of time andstaffing were identified as limitations to such data collection. The study hasdescribed a systematic method for developing clinical indicators for national audit.The indicators proved robust and reliable in primary and secondary care as well aslong-term care settings.

2


3/23


4/23

1. Literature search to identify previous work on clinical standards, criteria andindicators,

2. Consensus Workshops,3. Delphi process,

4. Pre Pilot,5. Pilot.

1. Literature search to identify clinical criteria

A detailed analysis of the evidence-based literature on audit of urinary and/orfaecal incontinence was undertaken (Abrams et al 1999, Burgio et al 1999,Cheater et al 1998, Hopkins et al 1992, The Continence Foundation 2000,Georgiou et al 2001, Peet et al 1995, Royal College of Physicians 1995, 1998,1999, US Department of Health and Human Services 1996, Potteret al2002). Inaddition, three recent policy documents were used; Good practice in continence

services (Department of Health 2000), the National Service Framework for OlderPeople (Department of Health 2001a) and The Essence of Care (Department ofHealth 2001b). Indicators covering aspects of structure, process and outcomereflecting the quality of care provided to older people with urinary and/or faecalincontinence were identified. Structure indicators included resources required,staff training, the provision of equipment and the environment for continence care.Process indicators included communication, assessment, education,investigation, surgical and other therapeutics interventions, evaluation, anddocumentation. Outcome indicators included measures of the physical orbehavioural response to an intervention, reported health status, and level ofknowledge and satisfaction. Many of the outcomes were derived from work carriedout by the National Centre for Health Outcomes Development on UrinaryIncontinence (National Centre for Health Outcomes Development 1997). Clinicalindicators were drafted from these sources for discussion, development andagreement at the consensus workshops.

2 Consensus Workshops

2.1 Background

Two consensus development workshops were held (one for bladder problems, onefor bowel problems) during which a group of approximately 20 specialists in the

area of continence care met to discuss the proposed indicators. Participants wereasked to complete a questionnaire prior to the workshop in which they commentedon the validity, feasibility and acceptability of the proposed indicators and thesewere then discussed according to nominal group theory (Murphy et al1998)(Table2). The workshop was divided into 3 sessions during which case mix andoutcomes, process and structure was discussed. Participants were asked to

4


5/23

highlight any key areas that had been omitted and comment on the proposedindicators.

2.2 Results

Key issues that were highlighted included:

Ethnicity needed to be addressed in case-mix as well as service evaluationand person-centred care.

The terms continence specialist and continence training needed to beclearly defined, with specific examples to aid understanding for authors.

The inclusion of a patient or carer completed validated and condition-specific quality of life measure was considered important and this wasincluded as a quality standard

The supply and demand of all types of continence products needed to becarefully assessed in order to gauge the impact of rationing.

The inclusion of indicators relating to the environment and patient dignitywas imperative.

From the workshops the clinical indicators were refined for testing by a Dephiprocess. In addition a Help Booklet was developed which provided a rationale foreach indicator, definition of terms used and information as to how data should becollected. Workshop participants were invited to attend a second consensusworkshop following the completion of the Delphi Exercise.

3. The Delphi consensus process.

The Delphi technique has been defined as a method of systematic solicitation andcollation of ... informed judgment on a particular topic (Turoff M 1970). Theindicators developed by the consensus workshops were analysed and incorporatedinto a Delphi questionnaire.The questionnaire was responded to by 63 out of 95invited clinicians with special expertise in the management of incontinence in theelderly. The sample included continence advisers, nurses, hospital doctors,general practitioners and physiotherapists. The scale used to rate the level ofconsensus ranged from 1 (strongly disagree) to 5 (strongly agree) as used in morerecent studies in the UK (Gallagheret al1996).

The results of the Delphi exercise demonstrated a high level of consensusamongst the group (Figures 1 3). The percentage of Delphi participants agreeingor strongly agreeing with individual indicators ranged from 63% to 100% (Table 3).Comments returned by contributing participants were tabulated and fed back to theWorkshop participants. Each indicator was reviewed and revised in the light of thedegree of consensus from the Delphi exercise and feedback from the Delphiparticipants.

5


6/23

4 Pre-pilot

Prior to the full pilot, a pre-pilot was carried out to test the clinical indicators and thehelp booklet. Two sites were identified in secondary care, primary care and in long-term care settings. Each site was asked to complete each of the auditquestionnaires (organizational structure, process and case mix, outcomes) and toprovide feedback with regard to the practical usage of the paperwork.

5. Pilot5.1 BackgroundThe full pilot was designed to recruit 15 primary care, 15 long-term care and 15hospital settings. Each site audited patient records using a cross-sectional sample.The pilot was designed to test the robustness of the clinical indicators in terms ofvalidity, feasibility of data collection and reliability. In addition, as reportedelsewhere (Mian et al 2004a), the pilot tested the methods of selecting and

recruiting a representative sample of patients from each setting and provided aninitial indication as to the quality of continence care. With a minimum of 20 casesper institution the pilot sought to recruit at least 300 patients in each type of caresetting. This sample size was selected to allow first estimates of the distribution ofcontinence problems, accurate to within about +/- 5% (95% confidence interval).

Each site was required to return completed questionnaires relating to 1)organizational structure, 2) process and case mix data for 20 returns inconsecutive patients with urinary incontinence 3) process and case mix data for 10returns in consecutive patients with faecal or double incontinence, 4) outcome. Allsites were asked to collect data on older people aged 65 and over. To allow

comparison between centres of the same type an individual and confidential codenumber was given to each centre. Individual patient records were irreversiblyanonymised before any data left each centre. Auditors were mostly employees ofthe centre concerned and were from varying disciplines (Table 4). The project co-coordinator (SM) acted as an external auditor at some centres for the inter-raterreliability study. Completed forms were returned to the CEEU. Document DataCapture Ltd then scanned the data into SPSS databases. Analyses wereperformed within the CEEU using SPSS v11.5.

5.2 Results

A total of 30 hospitals were approached, with reminders as necessary in order torecruit 15 hospital sites (one site per hospital trust), of whom 13 participated. Aninitial total of 30 Primary Care Trusts (PCTs) were approached followed by asecond wave to another 18 so as to obtain 15 interested sites, of whom 11participated. A total of 90 Care Homes were approached of whom 8 participated.Three further homes were recruited via their PCTs. The total number completingthe audit was: 13 hospitals, 11 GP practices, and 11 care homes.

6


7/23

5.2.1 Validity of data collected, and of the audit tool for use in each sectorSites were asked for their procedures for checking the accuracy of data and forcorrecting invalid data. Various methods were used (Table 5) some of which weremore rigorous than others, though most sites were doing something. Primary caresites (7/7) thought the audit clinical questionnaires to be suitable for their sectorthough the views from secondary care (6/11) and the care homes (4/8) were moremixed. Various suggestions were made to improve the forms, in particular thecombining of the two separate bladder and bowel questionnaires into one. Mostsites felt that a national audit of this type would be useful to them in terms ofmeasuring change in performance, with less enthusiasm though coming from thecare home sector (Table 6).

5.2.2 FeasibilityThe whole audit took about 18 hours on average to complete, with considerablevariation between sites (Table 7). A minority of sites found the questionnairesdifficult to complete (Table 8) with the clinical audit being the most difficult and theorganizational audit least difficult.

5.2.3 Difficulties encountered in the auditSites were asked to rate various statements on a 1 to 5 scale of difficulty and tostate what they did to resolve any problems. The statements were groupedaccording to audit development (5 statements), audit implementation (6), auditmanagement (5), and audit collation (2). A summary of the difficulties encounteredfor each of these aspects is given below.

Audit Development:

Most general practices found it difficult to co-ordinate the various people requiredto collect data. Care homes had less people collecting data and consequentlyfound co-ordination easier. Establishing a working balance between the continencelead and clinical audit was generally easy. A few sites found it difficult to get peopleto do the reliability study, but most had little difficulty. Hospitals had difficultydefining a data collection start date.

Audit Implementation:

Responsibilities for completing the questions were clear. Questionnaire completion

was seen as time-consuming and there was a learning curve. There wereproblems in identifying cases and also problems in completing questionnaires dueto a lack of information in case-notes.

Audit Management:

Patient management was not that different than it would have been without theaudit. Some leads lacked the time or resources to manage the audit, though most

7


8/23

did manage it. Some implied that problems could go unresolved for some time. Fora few there was difficulty with the sense of ownership.

Audit Collation:

There were concerns about ambiguity in data recording from primary care, and to alesser extent from care homes. Two general practices found it very difficult to tracestaff completing the questionnaire to ask questions.

Sites were asked what additional resources, would be required if the audit wasrolled out nationally (Table 9) and virtually all of their answers were to do withwanting extra staff, time or resources.

5.2.4 Reliability

Sites were asked to internally audit the first 5 cases of urinary incontinence and the

first 5 cases of non-urinary incontinence twice using different auditors. It wassuggested that the second auditor was of a clinical background with an interest incontinence. If the local plan involved auditors working in pairs then it was alsoacceptable for the repeat audit to use (different) pairs. In a subset of sites anauditor external to the site (SM) visited the hospital to re-audit cases. The kappastatistic was used to measure agreement between first-time and duplicate auditors.Kappa values of 0.41 to 0.60 are said to indicate moderate agreement, values of0.61 to 0.80 indicate good agreement, whilst values of over 0.80 are very good(Altman 1991)

Table 10 indicates the amount of reliability data available for analysis, and given

these numbers the main assessment of the reliability of each audit item combinedall data for the three care sectors. The external reliability assessment was morelimited in scope and data was also combined for the three sectors.

Urinary incontinence

Internal reliability kappa statistics of agreement were computed for 94 items. Themedian kappa was 0.70 (range 0.01 to 1.00). For 73/94 items kappa was 0.60and above whilst for 10/94 kappa was below 0.50. Comparison with the externalauditor suggested the data were generally less reliable than the internalcomparisons, the Kappa median being 0.58 (range 0.11 to 1.00).

Faecal incontinence

Internal reliability kappa statistics of agreement were computed for 76 items. Themedian kappa was 0.73, (range 0.37 to 1.00). For 64/76 items kappa was 0.60and above whilst for 8/76 items kappa was below 0.50. Comparison with the

8


9/23

external auditor suggested the data were generally less reliable than the internalcomparisons, the kappa median being 0.61 (range 0.06 to 1.00).

DISCUSSION

The study has outlined the method for producing clinical indicators for use innational audit. The indicators that have been derived cover all aspects ofcontinence care in older people, in part derived from evidence-based research andin part from clinical consensus. The indicators have been successfully piloted inprimary and secondary care settings as well as care homes. The work donesuggests the indicators have face and content validity, and that the data arepractical to collect.Quality of care may be perceived differently by different groups i.e. clinicians,

managers, patients and carers. This study, while including representatives ofpatient groups on the working party, did not seek to include older patients in theconsensus process. It is recognized that it can be difficult for patients and theircarers to contribute within groups of specialist clinicians. A separate study is beingconducted to determine the views of older people with regard to what constitutesquality in continence care (Mian et al 2004b).

The workshops used to develop the indicators specifically required contributors toconsider validity in terms of suitability for implementation. The face validity of theindicators was confirmed by the Delphi consensus. In the development of previousaudit tools for other conditions at the Clinical Effectiveness and Evaluation Unit(CEEU) one Delphi round has been sufficient to obtain consensus with the backupof expert consensus workshops (from a mix of different disciplines) before andafter the survey. The disadvantages of repeating several rounds of a Delphi surveyinclude respondent fatigue, with the danger that, in trying to produce consensus inseveral rounds, dissent can be ignored, and opinion become centralised to thelowest common denominator. One round achieves external validation of the items,whilst highlighting the areas of disagreement. The second consensus workshopscan then make decisions accordingly. The process demonstrated a high degree ofagreement between clinicians from differing professional backgrounds. The findingin the pilot for most sites to regard the tool as suitable for rollout into a nationalaudit adds further support to the face and content validity of the tool. The validity ofthe data collection within the pilot was supported by the checks done by the sites,though these were of variable thoroughness. We do not know for sure howaccurate or how variable in quality the case-note extraction has been nor of theaccuracy of the information documented in local record systems in the first place.Data reliability is a necessary pre-requisite for data accuracy but does notguarantee accuracy but this is an issue that faces all retrospective audits. The pilotexperience has led to changes in the questionnaires and to the guidance notes and

9


10/23

these should further enhance the accuracy of data to be collected in the nationalaudit.

Considerable attention was paid to feasibility both of conducting the audit and of

data collection. Recruitment of sites in the first place was a problem in primary careand care home settings. Feedback from sites indicated that in secondary settingsthe mechanism for audit is well established and operational with both dedicatedaudit staff and clinicians involved. In primary care, structures are in place for auditbut there is not a well-established link between theclinicians and auditors. Hencein primary care no General Practitioners participated in data collection. In long-termcare settings the structures are generally not in place to permit audit as part ofroutine daily practice. The introduction of an audit is therefore a significant extraworkload for staff with little or no experience in the activity. Recruitment for thenational audit will require addressing these underlying issues. In all health caresettings audit must be seen as part of routine practice, staffing establishments

must be designed to allow time for such activity and expertise must be built intoestablishments both amongst clinicians and specialist audit staff.

The feasibility of data collection has also been explored in detail. The time taken tocomplete the audit (median time 18 hours) is similar to that of other national audits.The Help Booklet is a key component of the audit tool that will help simplify andspeed up the process for data collectors in the national audit. Different problemsarise in the various settings. In secondary care the required data is found in thecase notes on the wards. In primary care relevant data might be in differing placesincluding the records of the general practitioner, the district nurses and thecontinence advice team. Hence gathering the data becomes difficult. However, acontinence specialist who had knowledge of the patients and an understanding ofthe information required usually carried out the data gathering. In long-term caresettings, data again were stored in different places and were often not within theimmediate patient records being in the General Practitioner records or thecontinence advisors records. Furthermore, the staff was often not in a position fromtheir experience to provide the information required. For example, an assessmentof functional disability using the Barthel Index scores was one required measure.Many homes felt unqualified to determine individual Barthel Index scores. As aresult of the feasibility feedback, some indicators were discarded and others wererefined. The importance of the Help Booklet became apparent, and this wasdeveloped so the terms were more precise, the source of data more clearlyindicated and other issues presenting difficulty addressed.

Data collection is an additional task for staff and in some care settings local auditresources are often either limited or difficult to identify or both. Our experience isthat primary care and care home settings were not able to generate local supportand that participation depended on the personal commitment of individuals overand above their other duties. This is an issue that must be addressed to bothensure that complete and accurate data are collected and also to remove any

10


11/23

excuse for non-participation. Consideration should be given to the provision of anexternal researcher to visit sites that experience difficulty in data collection or to theprovision of incentive payments as happens in primary care - to ensure that auditoccurs.

Reliability of data collection is a great challenge. Interpretation of data within caserecords can vary greatly between individual data collectors. The results of this pilotdemonstrated inter rater reliability compatible with other national audits (stroke,COPD). The external reliability was less good which may relate to factors such asthe small numbers (sites & cases) for the external reliability assessment, thediffering sector & site mix for the internal and external reliability assessments andhaving only the one external auditor increased the possibility of there being asystematic bias.

In conclusion, the method used produced indicators , which achieved a high

degree of acceptance from health care professionals. The reliability of datacollection was high and was similar to the level seen in other successful nationalaudits. Data collection for the indicators was feasible to collect, however issues oftime and staffing were identified as limitations. The study has described asystematic method for clinical indicator development that proved robust andreliable in practice and lays the foundation for a national audit.

11


12/23

ACKNOWLEDGEMENT

The authors acknowledge the support and commitment of the multi-professionalworking group who provided invaluable supervision for the project.

The project was funded by a grant from the Health Foundation (previously the PPPFoundation) and was jointly commissioned by the Health Foundation and theNational Institute of Clinical Excellence.

12


13/23

References

Abrams P., Khoury S., Wein A. (eds). (1999) Incontinence: 1st International

Consultation on Incontinence. World Health Organisation and International Unionagainst cancer. Plymbridge Distributors, Plymouth,

Altman D.G. (1991) Practical statistics for medical research. Chapman & Hall.

Burgio K., Ouslander J.G. (1999) Effects of urge urinary incontinence on qualityof life in older people. J Am Geriat Soc 47:1032 1033

Cheater F., Lakhani M., Cawood C. (1998) Audit protocol. Assessment ofPatients with urinary incontinence. Eli Lily National Clinical Audit Centre, Leicester.

Continence Foundation (2000) Making the case for investment is an integratedcontinence service. The Continence Foundation. London.

Department of Health (1998) A first class service: quality in the new NHS.HMSO: London

Department of Health (2000) Good practice in continence services, Departmentof Health, London.

Department of Health (2001a) National Service Framework for Older People,Department of Health, Leeds.

Department of Health (2001b) The Essence of Care. Patient-FocusedBenchmarking for Health Care Practitioners, Department of Health, London.

Gallagher M., Bradshaw C., Nattress H. (1996) Policy priorities in diabetes care;a Delphi study. Qual Health Care.5: 3 8.

Georgiou A., Potter J.M., Brocklehurst J., Lowe D., Pearson M. (2001) Measuring

the quality of continence care in long-term care facilities: an analysisof outcome indicators.Age Ageing30(1):63-6

Grant R., Batty G., Aggarwal R., Lowe D., Potter J., Pearson M (2002) Nationalsentinel clinical audit of evidence based prescribing for older people:methodology and development. J Eval Clin Pract, 8, 189 98.

13


14/23

Hopkins A., Brocklehurst J., Dickinson E. (1992) The CARE (ContinuousAssessment Review and Evaluation) Scheme. Clinical audit of long-term care forelderly people. Royal College of Physicians, London.

Mian S. Lowe D., Potter J., Wagg A., Pearson M. The national audit of continencecare for older people: results of a pilot study. Neuro Urol Urodyn 2004;

Mian S., Wagg A., Billings J. Older peoples views of continence services.Proceedings of the ICS Annual meeting, Paris 2004

Murphy M.K., Black N.A., Lamping D.L., McKee C.M., Sanderson C.F.B., AskhamJ. et al. (1998) Consensus development methods and their use in clinical guidelinedevelopment. Health Technology Assessment, NHS R&D HTA Programme 2(3).Department of Health: London.

National Centre for Health Outcomes Development (1997) Working group onoutcome indicators for urinary incontinence. Report to the Department of Health.

NICE (National Institute for Clinical Excellence) (2002) Principles for best practicein clinical audit. Radcliffe Medical press: Oxford.

Peet S.M., Castleden S.M., Mc Grother C.W. (1995) Prevalence of urinary andfaecal incontinence in hospitals and residential and nursing homes for olderpeople. BMJ311:1063 1064

Potter J.M., Georgiou A., Pearson M. (2000) Measuring the quality of care forolder people. Royal College of Physicians: London

Potter J.M., Norton C., Cottenden A. eds. (2002) Bowel Care - Research andPractice. Royal College of Physicians, London.

Pruce D., Aggarwal R (2000) National Clinical Audits. A handbook for goodpractice. Royal College of Physicians: London

Roberts CM, Ryland I, Lowe D, Kelly Y, Bucknall CE, Pearson MG. (2002) Clinicalaudit indicators of outcome following admission to hospital with acute exacerbationof chronic obstructive pulmonary disease. Thorax57: 137 141.

Royal College of Physicians. (1995) Incontinence: causes, management andprovision of services. The Royal College of Physicians, London.

Royal College of Physicians. (1998) Promoting continence. Clinical auditscheme for the management of urinary and faecal incontinence. Royal College ofPhysicians, London.

14


15/23

Royal College of Physicians (1999) The Care Scheme. Clinical audit of long-termcare of elderly people. Royal College of Physicians, London.

Rudd A.G., Lowe D., Irwin P., Rutledge Z., Pearson M (2001) National stroke audit:

a tool for change? Quality in health Care, 6, 194 8.

Turoff M. (1970) The design of a policy Delphi. Technological Forecasting andSocial Change2: 149 171.

US Department of Health and Human Services (1996) Urinary incontinence inadults: acute and chronic management. US Department of Health and HumanServices. Clinical Practice Guideline No 2, Update. US Government PrintingOffice. Washington.

15


16/23

Table 1: Members of multi professional working party

Background Number

Continence Nurse Specialist Bladder 1Continence Nurse Specialist - Bowel 1

Patient Support Group Lay representative 1

- Professional representatives 2

Geriatric Medicine 1

General Practice 1

Physiotherapist 1

Occupational Therapist 1

Clinical Effectiveness staff (methodological advice) 2

16


17/23

Table 2: Example of questionnaire completed by experts at the consensusdevelopment workshops for urinary incontinence

IS IT MEASUREABLE? IS THIS AN ACCEPTABLEMEASURE OF SERVICE?

URINARY INCONTINENCE

Structure Indicator

IS THIS AGOOD THINGTO MEASURE?

Yes/No Yes/No Caveats Yes/No Caveats

WHAT IS BEINGMEASURED?

a. Serviceeffectivenessb. Patientsperspectivec.Cost-effectiveness

POLICIES &PROCEDURES

Facility should have a writtenpolicy concerning the promotionof urinary continence.

STAFF

There should be access tointegrated continence services.

There should be a structured andcomprehensive programme ofstaff training on promoting urinarycontinence.

ENVIRONMENT

All bladder assessment and careis given in an environment that isprivate and promotes patientdignity.

SERVICEEVALUATION

There should be a mechanism forassessing patient and carersatisfaction with the treatmentand continence services.

17


18/23

Table 3: The range of Delphi participants agreeing or strongly agreeingwith individual indicators.

Participants agree/strongly agreewith individual indicatorsRange

Structure indicators18 indicators 84% - 100%

Urinary IncontinenceProcess/ outcome/ case mix34 indicators

59% 96%

Faecal incontinenceProcess/ outcome/case mix34 indicators

63% - 96%

Table 4: Discipline of data collectors

DATA COLLECTORDISCIPLINE

PRIMARYCARE (10)

SECONDARYCARE (13)

CAREHOMES (11)

Doctor - 8 -

Nurse 8 8 7Therapist - - 1Manager 2 2 5

18


19/23

Table 5: Checks on data accuracy in different settings

Primary Care Secondary care Care Homes

Audit completed withstaff member who

knew that patientgroup. In additioncare pathways wereused to help completeBladderquestionnaires.

Clarification of datawith GP

First look throughdistrict nursing notesthen practice

computer (Microtest). Hand searched

patient records,continence servicerecords & computerrecords.

Systematic check withoriginal notes

Two people doublechecking notes.

Worked with a

colleague andpractice manager.

All data collected fromclinical records. Some

cross checks espwhere uncertaintyaround datarequirements.

Checked cases withward leads andchecked completedforms.

Checked nursingdocuments againstmedical notes. Used

fluid balance & drugcharts & checked withcolleagues.

Completed formsblind then discussedwith staff.

Correcting nursingand medical notes

Discussed issues withStroke Sp Nurse/SpRto ensure standard

approach to datacollection.

Phone up project co-ordinator & spoke withcolleague forclarification.

Studied notescarefully. Checkeddetails with nursesrecords & notes withhelp of senior nurse

on ward. Verified byconsultant.

Used RCP helpbooklet. First 5 caseswere double checked.

Checking hospitalinformation and care

plans. Clarification with a

colleague

Correlation forms withpatients notes,clarified withcolleague & liasedwith continenceadvisor from the PCT.

Correlation withclinical notes. And

clarification of datawith colleague.

Double checking ofnotes with colleague.

Each questionnairecompletedindependently andchecked againstnotes.

Forms doublechecked and

completed bycontinence advisor.

Looked throughpatients notes andcare plans andevaluation sheets.

19


20/23

Table 6: Suitability of audit tool for roll-out in a national audit

Suitability Primary care Secondary care Care homes

Yes 6 10 5

No - 1 3

Table 7: Time taken to complete the audit

Healthcare SettingTime taken in hours: Median (Range)

OrganisationalStructure

Case-mix &Process

Outcomes

Primary Care2.00

(1.00 10.00)11.00

(2.00 16.00)2.00

(1.00 8.00)

Secondary Care2.00

(0.33 10.00)

18.00

(2.00 40.00)

2.00

(0.25 10.00)Care Homes

2.25(0.50 10.00)

12.00(2.50-27.00)

2.00(0.50 6.00)

Table 8: Ease of data collection for the organisational, process & case-mix and outcomes audit tools

Health caresetting

Very easy

1

Quite Easy

2

Neither easyor difficult

3

Quitedifficult

4

Verydifficult

5

OrganisationPrimary Care - 5 - 3 -

Secondary care 1 4 4 2 -

Care homes - 4 3 - 1

Process&case-mix

Primary care - 4 1 2 1

Secondary care - 2 3 6 -

Care homes - 2 3 3 -

Outcome

Primary care 1 2 3 2 -

Secondary care - 4 3 2 -Care homes - 4 3 - 1

20


21/23

Table 9: Additional resources required for national roll-out

Primary Care Secondary Care Care homes

Additional clericalhours.

Dedicated timerequired for nurse tocompletequestionnaires.Reduce no. of casesrequired for audit.

Internal auditor.

Need training aroundincontinence &bladder training.Need a bladderscanner so problemscan be picked up insurgery. Need newalgorithms/pathwaysfor faecal bowelincontinence

Resource foreducation & trainingin primary care.Involvement fromGPs and practicenurses

Specified timelines fordata completion

The audit tool needsrefining. Perhaps itshould form par offallsmanagement/medicines management

Very little- a fewhours dedicated toaudit

Audit support andimproveddocumentation.

Continence nurse tojoin the team onrehabilitation ward.

Dedicated audit,admin & clinical time.

If management ofUI/FI were to improvemore nurse timewould be required.Regular voiding islabour intensive.

Increase incontinence advisortime.

More support fromaudit dept andtraining of audit.

Need planning inadvance, need morenotice period. Moreinput from nursing(esp continence) -

just appointedcontinence advisor.

Require considerablemanpower

Time

Time audit will haveto build into annualprogram of nationalaudits like Stroke

Additional staff .

Extra man hours.

More informationfrom hospital/socialworkers regardingpatient's continenceis required.

None (X2)

Time

21


22/23

Table 10: Reliability study number of sites and cases

AuditPRIMARY

CARESECONDARY

CARECARE

HOMES

Internalreliability 8 sites39 patients 12 sites57 patients 10 sites50 patientsUrinaryIncontinence External

reliability2 sites

20 patients3 sites

14 patients3 sites

10 patients

Internalreliability

6 sites17 patients

12 sites51 patients

10 sites47 patientsFaecal

Incontinence Externalreliability

1 sites2 patients

4 sites14 patients

3 sites13 patients

1

2

3

4

5

MeanAgreement

Structure Indicators

Figure 1: Mean level of Agreement for Structure Indicators for

Urinary and Faecal Incontinence

5 = strongly agree an indicator should be included and 1 = strongly disagree.Each bar represents the mean from 63 Delphi respondents.

22


23/23

1

2

3

4

5

Mean

Agreemen

t

Process, Outcome and

Casemix Indicators

Urinary Incontinence

Figure 2: Mean level of Agreement for Process, Outcome and Case mix Indicators

forUrinary Incontinence


1

2

3

4

5

Mean

Agreement

Process, Outcome and Case

mix Indicators

Faecal Incontinence

Figure 3: Mean level of Agreement for Process, Outcome and Case mix IndicatorsforFaecal Incontinence


pilot audit

Documents