patient satisfaction: indicator quality in disablement ...trigger improvements in the quality of...

Quality in Health Care 1995;4:31-36

Patient satisfaction: an indicator of quality indisablement services centres

C Smith, M McCreadie, J Unsworth, H I Wickings, A Harrison

AbstractObjectives-To develop a patient satisfac-tion system for disablement servicescentres and to report on how the initialfindings have been used in audit to im-prove their quality ofcare and services.Design-Interview survey of randomlyselected users attending in three centres:Birmingham (centre X), Oxford (centreY), and Cambridge (centre Z) to establishcore topics for developing a patient satis-faction questionnaire with incorporationinto a computer patient satisfactionsystem (PATSAT) to enable collation ofresponses to the questionnaire. A pilot ofthe questionnaire was undertaken in thecentres to assess the sensitivity of thequestionnaire, which was subsequentlyused as part of clinical audit processduring June 1991 and April 1992 in centreX and the patient satisfaction system usedto monitor changes in routine practice.Patients-123 amputees in the develop-ment phase, selected by cluster sampling,and 1103 amputees in the pilot study.Main measures-Satisfaction scores forcomponents ofthe service.Results-The questionnaire included 16core topics contributing to quality of careand services, including comfort of limbs,appointments, interpersonal aspects ofcare, a system of support and counselling,and organisation. The pilot survey demon-strated high satisfaction scores for aspectsof interpersonal care, organisation, andphysical surroundings of the centres andlower satisfaction for counselling services,comfort of the limb and the number ofalterations made before the limb wasconsidered acceptable. During the auditin centre X these results prompted changesto care and services which producedsignificant improvements in satisfaction.Conclusions-The early results suggestthat the questionnaire, coupled withPATSAT software system, enable users'views to be expressed, collated, and fedback to staff; the information provided hasalready prompted change, and the systemis sufficiently sensitive to measurechanges in satisfaction with the service.(Quality in Health Care 1995;4:31-36)

Keywords: patient satisfaction, quality indicator,disablement services, audit, amputees.

IntroductionMeasures of patient satisfaction are being usedincreasingly as part of the assessment of the

quality of care and they are being incorporatedinto the process of clinical audit. Satisfactionis an important component of the total assess-ment of the value of treatment and care.Satisfaction with care is an important influencedetermining whether a person seeks medicaladvice, complies with treatment, and maintainsa relationship with the practitioner.' For usersof artificial limbs the difficulties that resultfrom dissatisfaction with services may beprofound, and any alternative accessible help isunlikely.

Several studies have examined patientsatisfaction within the artificial limb service.None has attempted to develop a system toassess satisfaction routinely or to integrateinformation about satisfaction into the processof clinical audit. This paper describes thedevelopment of a patient satisfaction systemdeveloped for disablement services centres,reports how the initial findings were used in anaudit to improve the quality of care andservices provided to patients, and shows theeffect of incorporating users' views into theclinical audit of prosthetic services.

MethodsThe study was done at three disablementservices centres: Birmingham (centre X), oneof the largest centres in the country, with about5000 current prosthetic patients; Oxford(centre Y), with about 2500 current attendees;and Cambridge (centre Z), one of the smallercentres, with about 1700 patients. The servicesprovided by the three centres vary, as do theresources available at each.

DEVELOPMENT OF QUESTIONNAIREWe developed a patient satisfaction question-naire based on issues relevant to users of theservice. Users of artificial limbs and newamputees attending clinics in the three centreswere selected randomly by cluster sampling,and the selected 123 patients were interviewedat the centre after their clinic appointment.The purpose of the study was explained by anurse at the centre, who confirmed users'willingness to take part. Semistructuredinterviews were carried out by interviewersfrom CASPE Research, and the intervieweeswere asked to identify those components ofcare and services which influenced theirsatisfaction.

In addition, the views of managers, doctors,the prosthetist, and other clinicians wereobtained to identify the issues the staffconsidered important for inclusion in thequestionnaire. These interviewees were

CASPE Research,London W2 4HUC Smith, researcherH I Wickings, directorSheffield DisablementServices Centre,Northern GeneralHospital, SheffieldS5 7ATM McCreadie, clinicalaudit coordinatorRegionalRehabilitation Centre,SBHA, BirminghamB29 6JFJ Unsworth, directorA Harrison, generalmanager rehabilitationservicesCorrespondence to:Ms C Smith,PO Box 101 13 Gouger St,Adelaide, South Australia5000

Accepted for publication1 July 1994

31

on June 13, 2020 by guest. Protected by copyright.

http://qualitysafety.bmj.com

/Q

ual Health C

are: first published as 10.1136/qshc.4.1.31 on 1 March 1995. D

ownloaded from

http://qualitysafety.bmj.com/

Smith, McCreadie, Unsworth, Wickings, Harison

selected by purposive sampling, and interviewswere carried out using a different semi-structured interview schedule.Data from the interviews were ranked to

identify the topics that interviewees identifiedas being important to their satisfaction andthese were ranked in order of frequency. Alldata were entered on to a microcomputer,using general purpose database software. Thuscore topics for the questionnaire wereidentified and questions were designed toexplore each topic in the questionnaire. Inaddition to the core topics, each centre wasable to select up to eight topics which reflectedlocal staff concerns. The questions were

designed to be short and unambiguous.Once the questionnaire had been construc-

ted convenience sampling was carried out atthe three centres, and a further 50 service userswere selected and interviewed to confirm thecontent validity of the questions, by examiningtheir interpretation of each question. Forquestions scoring less than 85% agreement on

content, the question was reworded andretested. Where there were problems in inter-preting questions a detailed content analysiswas carried out, by examining the content ofwords or descriptions within a redefinedcoded category.Each question was posed on a 4 point scale:

very satisfied, satisfied, somewhat dissatisfied,and very dissatisfied, and scores were allocatedacross a Likert scale (1 very dissatisfied, 4 verysatisfied). Satisfaction indices were thencalculated for each topic and each respondentand the total score divided by the maximumpossible score. The range of scores across theindex of satisfaction was 0-25 (very dissatis-fied) to 1-0 (very satisfied). In this way indicesof satisfaction can be produced for any topic.The appropriateness of this scale has beenassessed previously.2 3 Additional research hasshown such indices are sufficiently sensitiveto detect changes in satisfaction at local levels,such as in wards or departments.6 Space on

the questionnaire was also available foradditional comments, which were analysed bya text and retrieval package which reports ontopics relating to, for example, limb comfort.

COMPUTER PROGRAMMING

The second phase of development entailedprogramming the computers. The patientsatisfaction system being developed here(PATSAT) and the disablement services centredatabase Limbsys (the management infor-mation system) were made compatible,enabling users' satisfaction scores to be linkedto information on aspects of service deliverysuch as the doctor, prosthetist, or site ofamputation. This facility allows assessment ofsatisfaction for each core topic according totype and level of amputation, age and sex ofrespondent, and the prosthetic contractor.This level of analysis is not reported in thispaper, although it is routinely produced forusers of the PATSAT system, allowingcomparisons to be made. In addition, therespondents' verbatim comments can be linkedto satisfaction scores through search andretrieval text software.

PILOT OF PATIENT SATISFACTION SYSTEM FOR

USERS OF DISABLEMENT SERVICES CENTRES

Once the questionnaire and software had beendeveloped, the system for monitoring patientsatisfaction was piloted in three centres for sixmonths from March 1991. Every patientvisiting the centre received a personalised letterand questionnaire from reception staff.Patients were identified from the appointmentsystem, which interacted with the PATSATsoftware and identified their eligibility to takepart in the study. Those who had received aquestionnaire in the previous six months wereexcluded from the study. Patients were

encouraged to complete their questionnaire on

site, and at the end of their appointment. Forthe pilot study the responders completedquestionnaires at various stages in the processof obtaining a limb. Non-responders were notfollowed up.

FEEDBACK OF RESULTS AND MONITORING

CHANGES IN ROUTINE PRACTICE

Data were processed routinely by CASPEResearch and reports were generated monthlyand disseminated to clinical and managerialstaff in the centres.

After introduction of the patient satisfactionsystem we examined whether the system couldtrigger improvements in the quality of care.

Accordingly, in centre X (Birmingham) satis-faction scores were analysed in June 1991 andwere compared nine months later, in April1992. In June 1991, 160 amputees completedthe questionnaire and in April 1992, 114 didso. Once again users could re-enter the studyonly every six months, therefore some of therespondents had an opportunity to commenttwice.

STATISTICAL ANALYSIS

Between the two periods the percentageof satisfied respondents was compared. Owing

Core topics identified from pilot1 Surroundings in the centre2. Facilities while waiting3. Friendliness and politeness of the doctors4. Friendliness and politeness of the

prosthetist (fitter)5. Waiting time in the prosthetist's (fitting)

room6. Kindness and help given to you by staff7. Staffs willingness to talk and listen to you8. Availability of information about your limb,

fitting, aftercare9. Availability of counselling and support

services10. Skill of the prosthetist (fitter) with fitting

your limb11. Comfort of your limb12. Usefulness of your limb for daily activities13. Number of alterations to your limb before it

was acceptable14. Arrangements for booking your

appointments15. Layout of the centre16. Overall way the centre is organised

32



/Q

ual Health C


ownloaded from


Patient satisfaction in disablement centres

Table 1 Response rate by sex and age group for three disablement services centres. Figures are perecentages (numbers)

Centre X Centre Y Centre Z(n = 1155) (n = 566) (n = 222)

Responders Non-responders Total* Responders Non-responders Total* Responders Non-responders Total*

SexMale 62 6(508) 37 4(303) 811 48-2(196) 51-8(211) 407 51.9(84) 48-1(78) 162Female 63-1(217) 36-9(127) 344 43-4(89) 56-6(90) 159 50 9(29) 49-1(28) 57

Age (years):10-19 72-4(21) 27.6(8) 29 ** ** 15 ** ** 320-29 57-9(44) 42-1(32) 76 42 6(20) 57-4(27) 47 41-7(10) 58-3(14) 2430-39 62-9(66) 37-1(39) 105 51-9(28) 48-1(26) 54 ** ** 1040-49 55 2(69) 44 8(56) 125 53-1(34) 46 9(30) 64 35-3(12) 64 7(22) 3450-59 64-6(102) 35-4(56) 158 52 4(44) 47-6(40) 84 61-5(16) 38-5(10) 2660-69 64-4(163) 35 6(90) 253 52 0(52) 48 0(48) 100 64-6(31) 35-4(17) 4870-79 66-7(174) 33-3(87) 261 48-0(61) 52 0(66) 127 61-1(33) 38-9(21) 5480-89 63-2(72) 36-8(42) 114 29-6(21) 70 4(50) 71 40 0(8) 60-0(12) 20

*Missing data in each case.**Data for <20 cases.Centre X (X2 = 7-16, df= 7; p > 0-05)Centre Y (X2 = I 1 -59, df = 7; p > 0-05)Centre Z (X2 = 15-67, df= 5; 0-01 < p < 0-05).

to the limited ability of the satisfaction scale todiscriminate between categories with smallnumbers, the scale was reduced to a 2 pointscale measuring, respectively, the percentagesof responders who were satisfied and of thosewho were dissatisfied.2 A test between sampleproportions was used to examine differencesbetween the proportions of users satisfiedbetween June 1991 and April 1992.

In all three centres the characteristics ofresponders and non-responders were exam-ined. A test between sample proportions wasused to examine differences in responsesbetween men and women and a x2 test toexamine differences in age between respondersand non-responders. Non-parametric testswere used to determine significance for datawith non-normal distributions.

ResultsDEVELOPMENT OF QUESTIONNAIREOver 40 topics were described as important tosatisfaction by limb amputees during thedevelopment phase of the questionnaire, whichincluded: comfort in wearing the limb,appointments, interpersonal aspects of care, asystem of support and counselling, and aspectsof organisation and efficiency of running of thecentres. The rank order of topics (box)determined the content of the questionnaire,the 16 topics forming the core questions. Noappreciable difference in ranking was evidentbetween the three centres. Content validity ofthe questionnaire was found to be high: onequestion, concerning the perceptions oforganisation of the centre, scored less than

Table 2 Response rate by level of amputation for two centres. Figures are percentages(numbers)

Centre X Centre Y

Site of amputation Responders Non-responders Total Responders Non-responders Total

Lowerlimb below knee 65 2(375) 34-8(200) 575 48-5(159) 51-5(169) 328Lower limb above knee 67-1(192) 32-9(94) 286 44 0(55) 56 0(70) 125Upper limb below elbow 42-9(21) 57-1(28) 49 ** ** 9Upper limb above elbow 42-9(12) 57-1(16) 28 ** ** 13Congenital amputation 40-0(12) 60-0(18) 30 ** ** 8Multiple amputation 59 0(46) 41-0(32) 78 74 3(29) 25-6(10) 39

**Data for <20 cases.Site X: x2= 20-15, df= 5; p <005.

85% agreement and was reworded to "theoverall way the centre is organised."

PILOT OF PATIENT SATISFACTION SYSTEM

In the six month pilot study a total of 1943amputees were sent questionnaires and askedto take part; 1103 responded, giving a responserate of 568%. For each centre the number ofresponders was 62-8% (725/1155, centre X)46-8% (265/566, centre Y), and 50 9%(113/222, centre Z).Tables 1 and 2 show characteristics of

responders and non-responders. No differencein the response rate between men and womenwas found. The age of responders and non-responders was similar in two centres (X andY). A x2 test of the data within each centreshowed a significant difference in age betweenresponders and non-responders only in centreZ, in which non-responders tended to beyounger than the responders.Over 70% (1380/1943) of users of the

disablement services were male (table 1), anduse of the services increased above the 50-59age group, with the greatest proportional useamong the 70-79 age group. Data on site ofamputation could be extracted only for centresX and Y (table 2). Amputations of the lowerlimb were the most common amputation incentres X and Y, and within this category therewas a greater proportion of below kneeamputations (66-7% and 72-4% respectively).In centre X a significant difference in responserate was found between upper and lower limbamputees. Too few data were available forcentre Y to enable analysis.

SATISFACTION INDICES

Figure 1 shows the satisfaction indices for alltopics for the three centres. A score of 1-0represents a response of "very satisfied" and0-25 "very dissatisfied." For all three centresuser satisfaction was high (0-80-0 90) orhigher for surroundings, layout, booking ofappointments, waiting facilities, and aspects ofinterpersonal care such as staff help and theattitudes of doctors and prosthetists, and limbfitting. Four main topics scored lower: waitingtime in the fitting room, availability of

33



/Q

ual Health C


ownloaded from


Smith, McCreadie, Unsworth, Wickings, Hamrson

1.00

0.75

0.50

0.25

1 00

a)" 075

0

utin

0.5

(1)

0-25

C0 %zo !;i?

Questionnaire topic

= Centre X (n = 545) m Centre Y (n = 265) Centre Z (n = 113)

Fig1 Patients' satisfaction with main aspects of care in disability services cenztres, March and August 1991

counselling services, comfort of the limb, andthe number of alterations before the limb was

considered to be acceptable.

AUDIT AT BIRMINGHAM DISABLEMENT SERVICES

CENTRE

Each centre had the opportunity to assess anduse information from the patients' views of itscare and services. The results from theBirmingham centre were chosen to illustratehow the results from the routine use of thequestionnaire linked to PATSAT softwarewere used in practice. Satisfaction scores forservices provided by the Birmingham centre (Xin fig 1) indicated satisfaction with aspects ofservice delivery and many of the interpersonalaspects of care. Lower levels of satisfactioncompared with the two other centres were

visually apparent (fig 1) for waiting time in thefitting room, information, counselling, fitting

Table 3 Comparison of users' satisfaction with selected aspects of services at Birminghamafter changes in practice

Topic No ofpatients Satisfied

JUiie 1991 April 1992 Significance*

Information 70(106/152) 87(90/104) p < 0 01Counselling 67(85/127) 81(84/104) p < 001Fitting of limb 85(126/148) 91(95/104) 0 01 <p<O005Comfort of limb 57(85/149) 74(81/110) p < 0 01Usefulness of limb 80(116/146) 84(92/109) p > 0 05Alterations to limb 67(95/141) 80(82/102) 001 <p< 005Booking appointments 84(127/152) 92(100/109) p < 0 01

*Test between sample proportions.

of the limb, comfort of the limb, and thenumber of alterations before the limb wasacceptable.These results were reviewed in the

Birmingham centre throughout the auditprocess during 1991, and in response severalchanges were introduced to improve thequality of care and services (box). Mostartificial limbs are prescribed in an environ-ment which does not offer the range ofdifferent surfaces on which the limb might beused. An outdoor track where users can testout their artificial limbs in more realisticconditions, such as on grass and slopes hasbeen introduced.Through the clinical audit process the

comfort and usefulness of limbs and thenumber of alterations to limbs were highlightedas topics of major concern to users. As a resultat the initial appointment the amputee,prosthetist, and healthcare professional now

agree on a realistic outcome as a result ofproviding a prosthesis. The amputee's progressis monitored against this goal throughout the

period of aftercare. In addition, the basis ofcontracts with the manufacturers of prostheseshas been changed to reflect the need to achieveimproved quality of care and user satisfactionthrough fewer visits to the centre and a more

comfortable limb fitting. This will continue tobe monitored and discussed every month withthe contractors, according to feedback receivedon user satisfaction.

MONITORING CHANGES IN PRACTICE

To examine whether the changes introduced inthe Birmingham centre led to improvedsatisfaction and quality of care, satisfactionrecorded in June 1991 was compared with thatin April 1992 for seven selected topics. Theresponse rate in June 1991 was 64% and in

April 1992, 53%. Table 3 shows that for eachselected core topic, except for usefulness of the

Changes made by Birmingham centreto improve service* Information folder on disability* Publicity on the information department* Nursing auxiliaries reassigned and given new

responsibilities* Fitting room made available for couples* Introduction of an outdoor test track* Funding to employ a social counsellor

34



/Q

ual Health C


ownloaded from


a)

(.1)

0-

35

100

90

80

70

60

50

40

30

20

10

0

Questionnaire topic

= June 1991 (n = 160) M April 1992 (n = 114)

Fig 2 Patients' satisfaction with main aspects of care in Birmingham (centre X) idn June 1991 and April 1992

limb, the percentage of satisfied respondents in

April 1992 had increased significantly and theimprovements were evident across the entiresatisfaction scale (fig 2)

DiscussionEarly results from the use of this questionnaireof users' satisfaction with the service providedby three disablement services centres show thatthis tool is successful in enabling users toexpress their opinion of the service standardsthey receive. The system was sufficientlysensitive to measure variations in satisfactionwith the service, it provided insights into areasof satisfaction and dissatisfaction with theprovision of artificial limbs, and it highlightedsome important differences between thecentres. Feeding back this information to thecentres has resulted in changes in servicedelivery whose effect can be measured bychanges in satisfaction indices.

The questionnaire was developed byranking topics raised by responders during theinitial interviews. The actual rankings andsatisfaction scores for each individual topicwere different, and this highlights a conceptualdifference between the two issues. To improveon future questionnaire design, incorporatinga weighting factor, such as the degree ofimportance, during the initial rankings mighthelp to identify the relative importance of thecore questions. This would facilitate a subse-quent review of satisfaction levels in audit andplanning of proposed action.The response rates were low in the pilot

study, for reasons that are not clear. Only one

set of questionnaires was sent out, and non-

responders were not followed up, owing to a

lack of resources. Among the respondentsupper limb amputees were underrepresented,which is likely to be related to their physicaldifficulty in completing the questionnaire.

Alternative methods of including the views ofthis group of users need to be considered.However, since the routine introduction ofthese questionnaires the response rates haverisen and now range from 63% to 89% in thethree centres.Once patient satisfaction data are regularly

generated, centres will be able to set localstandards including minimum standards(below which patient satisfaction should not beallowed to fall) and target standards (the levelof patient satisfaction which should beregularly achieved). Monitoring patient satis-faction consists of routinely comparing current

results against these standards. The intro-duction of a measure of one aspect of thequality of care which is available on a regularbasis is an important step forward.The facility to make comparisons lies at the

heart of any audit programme. The systemdescribed here allows such comparisons.Incorporation of PATSAT system into clinicalaudit (Committee on Clinical Audit Rehabili-tation Medicine, disablement services annualreport, April 1992) will provide information tohelp managers and health professionals to lookfor gaps in service provision and areas forimprovements. Not only does it provide a

framework for improving quality within theprovider unit it is also a mechanism whichpurchasers may require, and pay for, to ensure

the quality of services commissioned.This system has now been integrated into the

routine operation of 11 disablement servicescentres. Local and national user groups havebeen kept informed of the developments ofPATSAT. The patient satisfaction sysem has a

facility to explore topics in greater depth, andthis has been undertaken for comfort of thelimb. Future uses of these data includestandard setting for each of the topics includedin the questionnaire and monitoring the

100

90

80

70

60

50

40

Patient satisfaction in disablement centres



/Q

ual Health C


ownloaded from


Smith, McCreadie, Unsworth, Wickings, Hamrson

subsequent achievement of increased quality ofservice. The topics that are monitored may beincreased or changed and, of course, as

standards are achieved the target standard maybe raised.

1 Larsen DE, Rootman I. Physicians' role performance andpatient satisfaction. Soc Sci Med 1976;10:29-32.

2 Green J. On the receiving end. Health Services Journal 4 Aug1988:880- 1.

3 Smith C. Validatation of a patient satisfaction system in theUnited Kingdom. Qual Assur Health Care 1992;4:171-7.

4 Kerruish A, Wickings I, Tarrant P. Information frompatients as a management tool: empowering managers toimprove the quality of care. Hospital and Health ServicesReview April 1988.

5 Wickings I, Crown J, Harvey J, Kerruish A. Proof of thepudding. Health Services journal, 31 Aug 1987: 1070-1.

6 Brent Health District. Effect of presenting nmanageoientinfonnation to clinicallv accountable teanis. Brent: BrentHealth District, 1975. (Available from HealthInformation Unit and Management Research Team,Brent Health District.)

36



/Q

ual Health C


ownloaded from


patient satisfaction: indicator quality in disablement ...trigger improvements in the quality of...

Documents