total quality index: a benchmarking tool for total quality ... · total quality index: a...

21
Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle University, Philadelphia, Pennsylvania, USA Barbara Mohebbi Management Department, Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania, USA, and Dennis T. Kennedy Management Department, La Salle University, Philadelphia, Pennsylvania, USA Keywords Benchmarking, Total quality management, Decision making, Analytical hierarchy process, Delphi method Abstract The total quality index (TQI) proposed in this study is an information technology-supported benchmarking tool that helps managers assess a total quality management program by enabling the cost-effective measurement of key organizational processes. TQI utilizes the analytic hierarchy process and the Delphi technique to measure ideal and actual quality management along eight critical factors synthesized by Saraph et al. and supported by subsequent research. A study utilizing TQI was conducted to evaluate the progress of quality management in clinical and non-clinical settings. Eight clinical and six non-clinical departments were selected from four different hospitals to participate in this study. The results show that, contrary to the common beliefs, there is little difference in the actual and ideal scores on the eight critical factors between the clinical and non-clinical settings. Introduction Initially developed in manufacturing, total quality management (TQM) has become a widespread movement in health care (Burda, 1988). As the economy became increasingly service oriented, issues of quality and productivity began to spread to the service sector. However, the productivity of service workers has not kept pace with the increasing investment in technology in this sector (Roach, 1991). Also, global competition and deregulation potentially threaten the future of service organizations (Orr et al., 2001). As a result, organizations in this sector have turned to TQM for tools to help them to be competitive. Several issues including the need to contain rising costs, the growth of for-pro®t health care, and advances in medical information systems have contributed to the rapid adoption of TQM in health care (Berwick, 1989; Brashier et al., 1996). Health maintenance organizations (HMOs) were among the ®rst health care organizations to join the total quality management movement (Berwick, 1987). The Emerald Research Register for this journal is available at The current issue and full text archive of this journal is available at http://www.emeraldinsight.com/researchregister http://www.emeraldinsight.com/1463-5771.htm Total quality index 507 Benchmarking: An International Journal Vol. 10 No. 6, 2003 pp. 507-527 q MCB UP Limited 1463-5771 DOI 10.1108/14635770310505157

Upload: vanminh

Post on 01-Apr-2018

244 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

Total quality indexa benchmarking tool for total

quality managementMadjid Tavana

Management Department La Salle University PhiladelphiaPennsylvania USA

Barbara MohebbiManagement Department Wharton School University of Pennsylvania

Philadelphia Pennsylvania USA and

Dennis T KennedyManagement Department La Salle University Philadelphia

Pennsylvania USA

Keywords Benchmarking Total quality management Decision makingAnalytical hierarchy process Delphi method

Abstract The total quality index (TQI) proposed in this study is an informationtechnology-supported benchmarking tool that helps managers assess a total qualitymanagement program by enabling the cost-effective measurement of key organizationalprocesses TQI utilizes the analytic hierarchy process and the Delphi technique to measure ideal andactual quality management along eight critical factors synthesized by Saraph et al and supportedby subsequent research A study utilizing TQI was conducted to evaluate the progress of qualitymanagement in clinical and non-clinical settings Eight clinical and six non-clinical departmentswere selected from four different hospitals to participate in this study The results show thatcontrary to the common beliefs there is little difference in the actual and ideal scores on the eightcritical factors between the clinical and non-clinical settings

IntroductionInitially developed in manufacturing total quality management (TQM) hasbecome a widespread movement in health care (Burda 1988) As the economybecame increasingly service oriented issues of quality and productivity beganto spread to the service sector However the productivity of service workershas not kept pace with the increasing investment in technology in this sector(Roach 1991) Also global competition and deregulation potentially threatenthe future of service organizations (Orr et al 2001) As a result organizations inthis sector have turned to TQM for tools to help them to be competitive Severalissues including the need to contain rising costs the growth of for-proregt healthcare and advances in medical information systems have contributed to therapid adoption of TQM in health care (Berwick 1989 Brashier et al 1996)

Health maintenance organizations (HMOs) were among the regrst health careorganizations to join the total quality management movement (Berwick 1987)

The Emerald Research Register for this journal is available at The current issue and full text archive of this journal is available at

httpwwwemeraldinsightcomresearchregister httpwwwemeraldinsightcom1463-5771htm

Total qualityindex

507

Benchmarking An InternationalJournal

Vol 10 No 6 2003pp 507-527

q MCB UP Limited1463-5771

DOI 10110814635770310505157

The HMOsrsquo adoption of TQM was driven primarily by cost containmentbecause they operate on a regxed fee basis These programs focused onmeasuring quality but not improving it They experimented withnon-traditional forms of measurement and unrestricted access to informationand records Although these early forms of quality management did not looklike TQM they incorporated some of the essential principles such as beingdata driven and focusing on processes rather than on individuals Howeverthese early attempts at quality management programs did provide afoundation for the acceptance of other models including TQM andbenchmarking

There have been many successful applications of manufacturing techniquesin health care (Albert et al 1990 Berwick 1991 Buterbaugh 1992) and thehealth care industry has adopted the quality models for manufacturing (Burda1988) Furthermore the Joint Commission on Accreditation of HealthcareOrganizations has encouraged the implementation of quality improvementprograms for all health care organizations However the basic question is ordfCanthe tools of modern quality improvement with which other industries haveachieved breakthroughs in performance help in health care as wellordm (Berwicket al 1990)

Recently there has been a growing body of work critically examining theusefulness of TQM in health care The industry had difregculties when applyingTQM programs from other sectors Some problems are encountered in viewingthe patient as a client An objective of TQM is meeting customer needsTypically patients do not know what is best for them medically Should healthcare providers give patients what they want or what they need There is also aclash between the organizational models of TQM and health care Manyhospital workers are members of professional groups Reporting lines arecomplex and not always direct For example physicians often are notemployees of a hospital and they answer to a medical board that is separatefrom the administrative staff Furthermore physicians may be reluctant toparticipate in a TQM program because of their training Fried (1992) explainshow physicians are not trained to be team players as TQM encourages Theyare trained to make decisions independently and rely on their own judgmentAlso there is skepticism about TQM because of its perceived relationship toquality assurance Quality assurance is seen as regulation imposed fromoutside with its bad apple approach to quality assessment causing distrust(Fried 1992) Brashier et al (1996) observed ordfMany resent the fact that`outsidersrsquo are trying to tell them how to do their jobs They are not factoryworkers and cannot be managed as suchordm

There is also the perception that TQM is becoming a fad in health caremanagement The area has become fragmented with consultants emphasizingthose areas that represent their own strengths There are a number of ordfpoorlytrained consultants in the market selling programs with little chance of

BIJ106

508

workingordm (Burda 1991) In addition the cost-effectiveness of these programs isan issue because signiregcant amounts are invested in training and consultantswhile speciregc problems are not being addressed

The model and its objectivesThere is a large body of literature on multi-criteria modeling and problemsolving using gap analysis Pounds (1969) gave a general description of aproblem situation as a difference between some existing situation and somedesired situation Most descriptions or deregnitions begin with a difference orgap between the present situation and the desired situation (Bartee 1973MacCrimmon and Taylor 1976 Reitman 1964) called the ideal situation in thisstudy In terms of organizations Cyert and March (1963) describe a problem asa difference between actual performance and ideal performance Here theproblem gap is one involving differences between actual and ideal performancemeasures For Kiesler and Sproull (1982) a managerrsquos recognition of a probleminvolves a comparison of environmental stimuli with internal aspirationcriteria

Quality management involves the recognition of such a gap This gap is thediscrepancy between the signals that managers receive from the environmenton various aspects of the quality management process and their own idea ofwhat the process should be Benson et al (1991) have used the phrases actualquality management and ideal quality management to describe these twostates of quality management Ideal quality management is described as abusiness unit managerrsquos beliefs concerning what quality management shouldbe in the business unit while actual quality management is described as themanagerrsquos perceptions of the current practice of quality management in theunit (Benson et al 1991)

There are two generally accepted behavioral approaches to qualitymanagement First Demingrsquos (1982) Theory D offers 14 managerialprescriptions and proscriptions describing general ways that managersshould act and what they should do to improve individual and teamperformance job organization and managing external groups Second the lesspopular organizational behavior modiregcation (OBMod) approach (Luthansand Kreitner 1985) is a behavioral approach to performance improvementgrounded in the work of Skinner (1938 1966) While both approaches share thesame end the difference between them is their level of speciregcity (Redmon andDickinson 1987) Theory D is more of an organization-wide approach to culturalchanges while OBMod focuses on altering simple behaviors of individualswithin organizations Recently there have been efforts to merge the twoapproaches by extending the OBMod approach with social learning theory(Luthans and Thompson 1987) This adds the recognition of cognitiveprocesses to the operant processes of OBMod When this is done there are nomajor differences between the two approaches However a criticism has been

Total qualityindex

509

that neither approach is sufregciently speciregc for managers to operationalize(Luthans and Thompson 1987) Forker and Mendez (2001) note that the collapseof many quality management programs hinged on breakdowns in execution

Benchmarking is an effective vehicle for focusing continuous improvementon the basic processes that run an organization (Bhutta and Huq 1999 Prado2001 Simpson and Kondouli 2000) It consists of investigating processes andpractices that are employed and establishing metrics for assessing anorganizationrsquos performance (Camp 1989) Furthermore the benchmarkingapproach can be adapted to accommodate assessment input from health careprofessionals including doctors and minimize resistance and pushback Theutilization of information technology reduces the time and effort required toparticipate in the assessment process enhancing the likelihood of participationof other-directed health care professionals What are needed are theoretical andmethodological structures for the analysis (Bhutta and Huq 1999 Forker andMendez 2001 Kumar et al 1999)

Saraph et al (1989) synthesized critical areas of organizational qualitymanagement and sets of speciregc requirements for each of the eight criticalfactors They have also proposed a questionnaire presented in the Appendixthat can be used to evaluate actual and ideal quality management with the setsof requirements as operational measures of the critical factors Thequestionnaire was developed for evaluating quality management in eithermanufacturing or service organizations Empirical tests showed the measuresto be both reliable and valid Subsequent research has supported the contentvalidity of the instrument (Kumar et al 1999) An inherent advantage of theSaraph et al Benson and Schroeder questionnaire is that it minimizes the timeand cost for developing the concepts and operational measures essential for thebenchmarking approach to TQM

This research develops and applies an IT-supported benchmarking modelthat helps managers assess a quality management program by enabling thecost-effective measurement of critical organizational processes The modelutilizes AHP and Delphi to measure both actual and ideal quality managementalong the eight critical success factors synthesized by Saraph et al (1989)

To formulate the algebraic model assume

Fi The importance weight of a quality management critical factor (fori = 1 8)

fij The importance weight of an item associated with a qualitymanagement critical factor (for i = 1 8 and j = 1 ki)

Rij The ideal rating of an item associated with critical factor (for i =

1 8 and j = 1 ki)

Rtij The actual rating of an item associated with a critical factor for a given

time period (for i = 1 8 j = 1 ki and t = 1 n)

BIJ106

510

TQIThe ideal total quality index

TQIt The actual total quality index for a given time period

d t The gap between the ideal and actual total quality index for a giventime period

Ki The number of items within each quality management critical factor

n The number of time periods where the total quality index is measured

The objective of the model is to

Minimize d t = TQI 2 TQIt (1)

where

TQI =X8

i= 1

F i

XK i

j= 1

f ijRij

Aacute (2)

TQIt =X8

i= 1

F i

XK i

j= 1

f ijRtij

Aacute (3)

and

X8

i= 1

F i = 1XK i

j= 1

f ij = 1 1 Rij 5 1 Rt

ij 5

Both Fi and fij are developed with the analytic hierarchy process (AHP) AHPwas introduced by Saaty (1972) to assist DMs in the evaluation of complexjudgmental problems The department managers in this study used AHP toassign numerical values to the eight quality management critical factors (Fi)and the sub-factors ( fij) suggested by Saraph et al (1989) The process wasconregned to a series of pairwise comparisons Saaty (1972) argues that amanager naturally regnds it easier to compare two things than to compare all theitems in a list AHP also evaluates the consistency of the managers and allowsfor the revision of their responses Because of the intuitive nature of the processand its power in resolving the complexity in a judgmental problem AHP hasbeen applied to many diverse decisions In particular AHP has been a verypopular technique for determining weights in multi-criteria problems (Shim1989 Zahedi 1986) A comprehensive list of the major applications of AHPalong with a description of the method and its axioms can be found in theworks of Saaty (1972 1977a b 1980 1990 1994 1999) Weiss and Rao (1987)and Zahedi (1986)

Total qualityindex

511

There has been some criticism of AHP in the operations researchcommunity In response Harker and Vargas (1990) show that AHP has anaxiomatic foundation the cardinal measurement of preferences is fullyrepresented by the eigenvector method and the principles of hierarchicalcomposition and rank reversal are valid On the other hand Dyer (1990a) hasquestioned the theoretical basis underlying AHP and argues that it can lead topreference reversals based on the alternative set being analyzed Saaty (1990)explains how rank reversal is a positive feature when new reference points areintroduced In this study the geometric aggregation rule was used to avoidrank reversal which had varying degrees of importance to differentresearchers (Dyer 1990a b Harker and Vargas 1990 Saaty 1990)

Assuming manager i believes that c1 c2 cI are the I factors thatcontribute to the overall quality management program the managerrsquos next taskis to assess the relative importance of these factors with AHP by comparingeach possible pair of factors cj ck and indicating which factor is more importantand by how much

These judgments are represented by an I 3 I matrix

A = (ajk) (j k = 1 2 I )

If cj is judged to be of equal importance as ck then ajk = 1If cj is judged to be more important than ck then ajk 1If cj is judged to be less important than ck then ajk 1

ajk = 1=akj ajk sup1 0

Thus matrix A is a reciprocal matrix so that the entry ajk is the inverse of theentry akj ajk remacrects the relative importance of cj compared with factor ck Forexample a12 = 150 indicates that c1 is 150 times as important as c2

The vector w representing the relative weights of each of the I factors wasfound by computing the normalized eigenvector corresponding to themaximum eigenvalue of matrix A An eigenvalue of A is deregned as l whichsatisreges the following matrix equation

Aw = lw

where l is a constant called the eigenvalue associated with the giveneigenvector w Saaty has shown that the best estimate of w is the one associatedwith the maximum eigenvalue (lmax) of the matrix A Because the sum of theweights should be equal to 100 the normalized eigenvector is used Saatyrsquosalgorithm for obtaining this w is incorporated in the software Expert Choice(2000) utilized in this study

One of the advantages of AHP is that it assesses the consistency of themanagerrsquos pairwise comparisons When the judgments are perfectly consistentthe maximum eigenvalue (lmax) should equal the number of factors that are

BIJ106

512

compared (I) Typically the responses are not perfectly consistent and lmax isgreater than I The larger the lmax the greater is the degree of inconsistencySaaty deregnes a consistency index (CI) as (lmax 2 I )=(I 2 1) and provides arandom index (RI) table for matrices of order 3-10 This RI is based on asimulation of a large number of randomly generated weights (Table I)

Saaty recommends the calculation of a consistency ratio (CR) that is the ratioof CI to RI for the same order matrix A CR of 010 or less is consideredacceptable Each manager used Expert Choice (2000) an AHP-based softwareindividually to perform all necessary calculations When the CR wasunacceptable the manager was informed that the pairwise comparisons werelogically inconsistent and was asked to revise hisher Expert Choicejudgments

Numerical exampleTable II provides a numerical example illustrating the calculation of thecomponents of the measures used in this study The illustration assumes thatthere are three critical quality management factors (F1 F2 and F3) four items( f11 f12 f13 and f14) associated with F1 three items ( f21 f22 and f23) associatedwith F2 and four items (f31 f32 f33 and f34) associated with F3 Assume that thedecision maker (DM) has assigned the importance weights of 050 030 and020 to F1 F2 and F3 respectively using AHP and Expert Choice Furthermorethe DM has assigned the following importance weights to the items 040 030020 and 010 to f11 through f14 060 030 and 010 to f21 through f23 and 050020 020 and 010 to f31 through f34 using AHP as well Next assume for thisillustration that the DM used the scale proposed by Saraph et al (1989) to rateboth the ideal quality and the actual quality management of the eight criticalfactors Table II shows the calculations of the TQI for the current period(t = 1)

The TQI of 500 and the TQI1 of 264 in Table II are summary reggures for allthe critical factors and do not specify whether the gap is unacceptably large oracceptably small The total quality index for each critical factor (TQIi) givesmanagers more information about the quality management process in each ofthe critical areas For example health care organizations may experiencecoordination problems between the quality department typically anadministrative function and other departments such as the pathology lab orthe intensive care units These departments generally have their own hierarchywithin the medical staff and do not report directly to administration Thisproblem could be identireged in the model by a relatively signiregcant gap betweenthe actual TQIt

2 and ideal TQI2 for critical factor 2 (the role of the qualitydepartment) for a given time period If the gap is unacceptably large managers

n 3 4 5 6 7 8 9 10RI 058 090 112 132 141 145 149 151

Table IRandom index table

Total qualityindex

513

could be prompted either to form some intervention strategies to narrow it oradjust their belief about the ideal rating

If properly assessed the intervention would center around one or more of theitems associated with the critical factor In the previous example a managercould identify the source of the problem as a lack of coordination between thequality department and other departments Strategies could then be developedthat address this speciregc item A manager could also conclude that noproblems exist with any item or any that do exist are not as signiregcant asindicated by the measured gap of the factor This could lead to a change in howthe manager believes TQIi should be rated What causes managers either todevelop strategies for intervention or to adjust their ideal rating should beexamined over time Movement over time could remacrect interventions made bymanagers or changes in their ideal TQI Further investigation into themanagersrsquo responses to the feedback they receive from TQI tables as well asthe environment could then be undertaken to determine what actually hashappened

The studyA study was conducted to investigate the usefulness of the TQI in differenthealth care settings Four hospitals that are geographically dispersedthroughout the state of New Jersey participated in the study for a period ofone year Each hospital was already active in quality management Topmanagement had stated a commitment to quality management anddepartmental managers were familiar with the basic principles andterminology of quality management Hospitals that were either not involved

Critical factorweights (Fi)

Item weights( fij)

Ideal ratingR

ij

plusmn sup2 Actual ratingR1

ij

plusmn sup2F isbquof ijsbquoR

ij

plusmn sup2F isbquof ijsbquoR

1ij

plusmn sup2

F1= 050 f11= 040 5 3 100 060f12= 030 5 2 075 030f13= 020 5 2 050 020f14= 010 5 3 025 015

TQI1 = 250 TQI11 = 125

F2= 030 f21= 060 5 4 090 072f22= 030 5 2 045 018f23= 010 5 3 015 009

TQI2 = 150 TQI12 = 099

F3= 020 f31= 050 5 2 050 020f32= 020 5 3 020 012f33= 020 5 1 020 004f34= 010 5 2 010 004

TQI3 = 100 TQI13 = 040

TQI = 500 TQI1= 264

Table IIActual and ideal TQI foran example period

BIJ106

514

in quality management or in the beginning stages of involvement wouldrequire additional education and were not chosen for this study

In hospital A the radiology laboratory and pharmacy departmentsparticipated The radiology laboratory pharmacy and environmental servicesdepartments participated from hospital B Hospital C included the nuclearmedicine social services and food and nutrition departments in the studyFinally in hospital D the radiology applied clinical technology respiratorycare and pharmacy departments participated In health care departments aretypically categorized as either clinical or non-clinical Clinical departments aresubjected to broad regulations from local and state licensing authorities as wellas national accrediting organizations In addition clinical departments aretypically those with highly skilled medical professionals that provide directphysiological medical care Non-clinical departments provide care but in a lessinvasive manner The care provided by non-clinical departments is not a part ofthe core physiological medical care This research relies on this generallyaccepted distinction between clinical and non-clinical areas Table III shows theclassiregcation of the participating departments

The participation of all department members was voluntary While they wereinformed about the project and invited to participate by e-mail and inter-ofregcememos no personal persuasion was permitted At the regrst quarter assessment83 percent of non-clinical and 79 percent of clinical personnel participated acrossall four hospital systems There were no apparent differences among the hospitalparticipation rates At the end of the second quarterparticipation increased to 89percent of non-clinical and 92 percent of clinical personnel The participationpercentages remained at these levels for the third and fourth quarters Those notparticipating were primarily newly hired personnel and older departmentmembers who were approaching retirement

Initially all department members staff as well as managers attendedseveral hands-on training sessions to ensure that everyone thoroughlyunderstood the AHP pairwise comparison and Expert Choice Then themembers of each department used Expert Choice to assess the importanceweight (Fi) for each critical factor and the importance weight ( fij) for each item( j) associated with a critical factor (i) During the training sessions theparticipants agreed that they should have an opportunity to review theirjudgments after receiving anonymous feedback containing the judgments ofthe others in their own departments The revision of individual judgments inview of anonymous group feedback is a fundamental principle of the Delphitechnique Delphi was developed at the Rand Corporation to obtain the mostreliable consensus of opinion from a group of knowledgeable individuals aboutan issue not subject to objective solution (Dalkey and Helmer 1963) Thetechnique consists of iterative sequences of collecting judgments from a groupanonymous feedback and individuals reconsidering their judgments In healthcare Delphi has been used to identify issues affecting health care

Total qualityindex

515

Clinic

alN

on-c

linic

al

Hos

pit

als

Rad

iolo

gy

Applied

clin

ical

tech

nol

ogy

Lab

orat

ory

Nucl

ear

med

icin

eR

espir

ator

yca

reSoc

ial

serv

ices

Phar

mac

yE

nvir

onm

enta

lse

rvic

esF

ood

and

nutr

itio

n

A3

plusmn3

plusmnplusmn

plusmn3

plusmnplusmn

B3

plusmn3

plusmnplusmn

plusmn3

3plusmn

Cplusmn

plusmnplusmn

3plusmn

3plusmn

plusmn3

D3

3plusmn

plusmn3

plusmn3

plusmnplusmn

Table IIIMatrix of clinical andnon-clinical cases

BIJ106

516

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 2: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

The HMOsrsquo adoption of TQM was driven primarily by cost containmentbecause they operate on a regxed fee basis These programs focused onmeasuring quality but not improving it They experimented withnon-traditional forms of measurement and unrestricted access to informationand records Although these early forms of quality management did not looklike TQM they incorporated some of the essential principles such as beingdata driven and focusing on processes rather than on individuals Howeverthese early attempts at quality management programs did provide afoundation for the acceptance of other models including TQM andbenchmarking

There have been many successful applications of manufacturing techniquesin health care (Albert et al 1990 Berwick 1991 Buterbaugh 1992) and thehealth care industry has adopted the quality models for manufacturing (Burda1988) Furthermore the Joint Commission on Accreditation of HealthcareOrganizations has encouraged the implementation of quality improvementprograms for all health care organizations However the basic question is ordfCanthe tools of modern quality improvement with which other industries haveachieved breakthroughs in performance help in health care as wellordm (Berwicket al 1990)

Recently there has been a growing body of work critically examining theusefulness of TQM in health care The industry had difregculties when applyingTQM programs from other sectors Some problems are encountered in viewingthe patient as a client An objective of TQM is meeting customer needsTypically patients do not know what is best for them medically Should healthcare providers give patients what they want or what they need There is also aclash between the organizational models of TQM and health care Manyhospital workers are members of professional groups Reporting lines arecomplex and not always direct For example physicians often are notemployees of a hospital and they answer to a medical board that is separatefrom the administrative staff Furthermore physicians may be reluctant toparticipate in a TQM program because of their training Fried (1992) explainshow physicians are not trained to be team players as TQM encourages Theyare trained to make decisions independently and rely on their own judgmentAlso there is skepticism about TQM because of its perceived relationship toquality assurance Quality assurance is seen as regulation imposed fromoutside with its bad apple approach to quality assessment causing distrust(Fried 1992) Brashier et al (1996) observed ordfMany resent the fact that`outsidersrsquo are trying to tell them how to do their jobs They are not factoryworkers and cannot be managed as suchordm

There is also the perception that TQM is becoming a fad in health caremanagement The area has become fragmented with consultants emphasizingthose areas that represent their own strengths There are a number of ordfpoorlytrained consultants in the market selling programs with little chance of

BIJ106

508

workingordm (Burda 1991) In addition the cost-effectiveness of these programs isan issue because signiregcant amounts are invested in training and consultantswhile speciregc problems are not being addressed

The model and its objectivesThere is a large body of literature on multi-criteria modeling and problemsolving using gap analysis Pounds (1969) gave a general description of aproblem situation as a difference between some existing situation and somedesired situation Most descriptions or deregnitions begin with a difference orgap between the present situation and the desired situation (Bartee 1973MacCrimmon and Taylor 1976 Reitman 1964) called the ideal situation in thisstudy In terms of organizations Cyert and March (1963) describe a problem asa difference between actual performance and ideal performance Here theproblem gap is one involving differences between actual and ideal performancemeasures For Kiesler and Sproull (1982) a managerrsquos recognition of a probleminvolves a comparison of environmental stimuli with internal aspirationcriteria

Quality management involves the recognition of such a gap This gap is thediscrepancy between the signals that managers receive from the environmenton various aspects of the quality management process and their own idea ofwhat the process should be Benson et al (1991) have used the phrases actualquality management and ideal quality management to describe these twostates of quality management Ideal quality management is described as abusiness unit managerrsquos beliefs concerning what quality management shouldbe in the business unit while actual quality management is described as themanagerrsquos perceptions of the current practice of quality management in theunit (Benson et al 1991)

There are two generally accepted behavioral approaches to qualitymanagement First Demingrsquos (1982) Theory D offers 14 managerialprescriptions and proscriptions describing general ways that managersshould act and what they should do to improve individual and teamperformance job organization and managing external groups Second the lesspopular organizational behavior modiregcation (OBMod) approach (Luthansand Kreitner 1985) is a behavioral approach to performance improvementgrounded in the work of Skinner (1938 1966) While both approaches share thesame end the difference between them is their level of speciregcity (Redmon andDickinson 1987) Theory D is more of an organization-wide approach to culturalchanges while OBMod focuses on altering simple behaviors of individualswithin organizations Recently there have been efforts to merge the twoapproaches by extending the OBMod approach with social learning theory(Luthans and Thompson 1987) This adds the recognition of cognitiveprocesses to the operant processes of OBMod When this is done there are nomajor differences between the two approaches However a criticism has been

Total qualityindex

509

that neither approach is sufregciently speciregc for managers to operationalize(Luthans and Thompson 1987) Forker and Mendez (2001) note that the collapseof many quality management programs hinged on breakdowns in execution

Benchmarking is an effective vehicle for focusing continuous improvementon the basic processes that run an organization (Bhutta and Huq 1999 Prado2001 Simpson and Kondouli 2000) It consists of investigating processes andpractices that are employed and establishing metrics for assessing anorganizationrsquos performance (Camp 1989) Furthermore the benchmarkingapproach can be adapted to accommodate assessment input from health careprofessionals including doctors and minimize resistance and pushback Theutilization of information technology reduces the time and effort required toparticipate in the assessment process enhancing the likelihood of participationof other-directed health care professionals What are needed are theoretical andmethodological structures for the analysis (Bhutta and Huq 1999 Forker andMendez 2001 Kumar et al 1999)

Saraph et al (1989) synthesized critical areas of organizational qualitymanagement and sets of speciregc requirements for each of the eight criticalfactors They have also proposed a questionnaire presented in the Appendixthat can be used to evaluate actual and ideal quality management with the setsof requirements as operational measures of the critical factors Thequestionnaire was developed for evaluating quality management in eithermanufacturing or service organizations Empirical tests showed the measuresto be both reliable and valid Subsequent research has supported the contentvalidity of the instrument (Kumar et al 1999) An inherent advantage of theSaraph et al Benson and Schroeder questionnaire is that it minimizes the timeand cost for developing the concepts and operational measures essential for thebenchmarking approach to TQM

This research develops and applies an IT-supported benchmarking modelthat helps managers assess a quality management program by enabling thecost-effective measurement of critical organizational processes The modelutilizes AHP and Delphi to measure both actual and ideal quality managementalong the eight critical success factors synthesized by Saraph et al (1989)

To formulate the algebraic model assume

Fi The importance weight of a quality management critical factor (fori = 1 8)

fij The importance weight of an item associated with a qualitymanagement critical factor (for i = 1 8 and j = 1 ki)

Rij The ideal rating of an item associated with critical factor (for i =

1 8 and j = 1 ki)

Rtij The actual rating of an item associated with a critical factor for a given

time period (for i = 1 8 j = 1 ki and t = 1 n)

BIJ106

510

TQIThe ideal total quality index

TQIt The actual total quality index for a given time period

d t The gap between the ideal and actual total quality index for a giventime period

Ki The number of items within each quality management critical factor

n The number of time periods where the total quality index is measured

The objective of the model is to

Minimize d t = TQI 2 TQIt (1)

where

TQI =X8

i= 1

F i

XK i

j= 1

f ijRij

Aacute (2)

TQIt =X8

i= 1

F i

XK i

j= 1

f ijRtij

Aacute (3)

and

X8

i= 1

F i = 1XK i

j= 1

f ij = 1 1 Rij 5 1 Rt

ij 5

Both Fi and fij are developed with the analytic hierarchy process (AHP) AHPwas introduced by Saaty (1972) to assist DMs in the evaluation of complexjudgmental problems The department managers in this study used AHP toassign numerical values to the eight quality management critical factors (Fi)and the sub-factors ( fij) suggested by Saraph et al (1989) The process wasconregned to a series of pairwise comparisons Saaty (1972) argues that amanager naturally regnds it easier to compare two things than to compare all theitems in a list AHP also evaluates the consistency of the managers and allowsfor the revision of their responses Because of the intuitive nature of the processand its power in resolving the complexity in a judgmental problem AHP hasbeen applied to many diverse decisions In particular AHP has been a verypopular technique for determining weights in multi-criteria problems (Shim1989 Zahedi 1986) A comprehensive list of the major applications of AHPalong with a description of the method and its axioms can be found in theworks of Saaty (1972 1977a b 1980 1990 1994 1999) Weiss and Rao (1987)and Zahedi (1986)

Total qualityindex

511

There has been some criticism of AHP in the operations researchcommunity In response Harker and Vargas (1990) show that AHP has anaxiomatic foundation the cardinal measurement of preferences is fullyrepresented by the eigenvector method and the principles of hierarchicalcomposition and rank reversal are valid On the other hand Dyer (1990a) hasquestioned the theoretical basis underlying AHP and argues that it can lead topreference reversals based on the alternative set being analyzed Saaty (1990)explains how rank reversal is a positive feature when new reference points areintroduced In this study the geometric aggregation rule was used to avoidrank reversal which had varying degrees of importance to differentresearchers (Dyer 1990a b Harker and Vargas 1990 Saaty 1990)

Assuming manager i believes that c1 c2 cI are the I factors thatcontribute to the overall quality management program the managerrsquos next taskis to assess the relative importance of these factors with AHP by comparingeach possible pair of factors cj ck and indicating which factor is more importantand by how much

These judgments are represented by an I 3 I matrix

A = (ajk) (j k = 1 2 I )

If cj is judged to be of equal importance as ck then ajk = 1If cj is judged to be more important than ck then ajk 1If cj is judged to be less important than ck then ajk 1

ajk = 1=akj ajk sup1 0

Thus matrix A is a reciprocal matrix so that the entry ajk is the inverse of theentry akj ajk remacrects the relative importance of cj compared with factor ck Forexample a12 = 150 indicates that c1 is 150 times as important as c2

The vector w representing the relative weights of each of the I factors wasfound by computing the normalized eigenvector corresponding to themaximum eigenvalue of matrix A An eigenvalue of A is deregned as l whichsatisreges the following matrix equation

Aw = lw

where l is a constant called the eigenvalue associated with the giveneigenvector w Saaty has shown that the best estimate of w is the one associatedwith the maximum eigenvalue (lmax) of the matrix A Because the sum of theweights should be equal to 100 the normalized eigenvector is used Saatyrsquosalgorithm for obtaining this w is incorporated in the software Expert Choice(2000) utilized in this study

One of the advantages of AHP is that it assesses the consistency of themanagerrsquos pairwise comparisons When the judgments are perfectly consistentthe maximum eigenvalue (lmax) should equal the number of factors that are

BIJ106

512

compared (I) Typically the responses are not perfectly consistent and lmax isgreater than I The larger the lmax the greater is the degree of inconsistencySaaty deregnes a consistency index (CI) as (lmax 2 I )=(I 2 1) and provides arandom index (RI) table for matrices of order 3-10 This RI is based on asimulation of a large number of randomly generated weights (Table I)

Saaty recommends the calculation of a consistency ratio (CR) that is the ratioof CI to RI for the same order matrix A CR of 010 or less is consideredacceptable Each manager used Expert Choice (2000) an AHP-based softwareindividually to perform all necessary calculations When the CR wasunacceptable the manager was informed that the pairwise comparisons werelogically inconsistent and was asked to revise hisher Expert Choicejudgments

Numerical exampleTable II provides a numerical example illustrating the calculation of thecomponents of the measures used in this study The illustration assumes thatthere are three critical quality management factors (F1 F2 and F3) four items( f11 f12 f13 and f14) associated with F1 three items ( f21 f22 and f23) associatedwith F2 and four items (f31 f32 f33 and f34) associated with F3 Assume that thedecision maker (DM) has assigned the importance weights of 050 030 and020 to F1 F2 and F3 respectively using AHP and Expert Choice Furthermorethe DM has assigned the following importance weights to the items 040 030020 and 010 to f11 through f14 060 030 and 010 to f21 through f23 and 050020 020 and 010 to f31 through f34 using AHP as well Next assume for thisillustration that the DM used the scale proposed by Saraph et al (1989) to rateboth the ideal quality and the actual quality management of the eight criticalfactors Table II shows the calculations of the TQI for the current period(t = 1)

The TQI of 500 and the TQI1 of 264 in Table II are summary reggures for allthe critical factors and do not specify whether the gap is unacceptably large oracceptably small The total quality index for each critical factor (TQIi) givesmanagers more information about the quality management process in each ofthe critical areas For example health care organizations may experiencecoordination problems between the quality department typically anadministrative function and other departments such as the pathology lab orthe intensive care units These departments generally have their own hierarchywithin the medical staff and do not report directly to administration Thisproblem could be identireged in the model by a relatively signiregcant gap betweenthe actual TQIt

2 and ideal TQI2 for critical factor 2 (the role of the qualitydepartment) for a given time period If the gap is unacceptably large managers

n 3 4 5 6 7 8 9 10RI 058 090 112 132 141 145 149 151

Table IRandom index table

Total qualityindex

513

could be prompted either to form some intervention strategies to narrow it oradjust their belief about the ideal rating

If properly assessed the intervention would center around one or more of theitems associated with the critical factor In the previous example a managercould identify the source of the problem as a lack of coordination between thequality department and other departments Strategies could then be developedthat address this speciregc item A manager could also conclude that noproblems exist with any item or any that do exist are not as signiregcant asindicated by the measured gap of the factor This could lead to a change in howthe manager believes TQIi should be rated What causes managers either todevelop strategies for intervention or to adjust their ideal rating should beexamined over time Movement over time could remacrect interventions made bymanagers or changes in their ideal TQI Further investigation into themanagersrsquo responses to the feedback they receive from TQI tables as well asthe environment could then be undertaken to determine what actually hashappened

The studyA study was conducted to investigate the usefulness of the TQI in differenthealth care settings Four hospitals that are geographically dispersedthroughout the state of New Jersey participated in the study for a period ofone year Each hospital was already active in quality management Topmanagement had stated a commitment to quality management anddepartmental managers were familiar with the basic principles andterminology of quality management Hospitals that were either not involved

Critical factorweights (Fi)

Item weights( fij)

Ideal ratingR

ij

plusmn sup2 Actual ratingR1

ij

plusmn sup2F isbquof ijsbquoR

ij

plusmn sup2F isbquof ijsbquoR

1ij

plusmn sup2

F1= 050 f11= 040 5 3 100 060f12= 030 5 2 075 030f13= 020 5 2 050 020f14= 010 5 3 025 015

TQI1 = 250 TQI11 = 125

F2= 030 f21= 060 5 4 090 072f22= 030 5 2 045 018f23= 010 5 3 015 009

TQI2 = 150 TQI12 = 099

F3= 020 f31= 050 5 2 050 020f32= 020 5 3 020 012f33= 020 5 1 020 004f34= 010 5 2 010 004

TQI3 = 100 TQI13 = 040

TQI = 500 TQI1= 264

Table IIActual and ideal TQI foran example period

BIJ106

514

in quality management or in the beginning stages of involvement wouldrequire additional education and were not chosen for this study

In hospital A the radiology laboratory and pharmacy departmentsparticipated The radiology laboratory pharmacy and environmental servicesdepartments participated from hospital B Hospital C included the nuclearmedicine social services and food and nutrition departments in the studyFinally in hospital D the radiology applied clinical technology respiratorycare and pharmacy departments participated In health care departments aretypically categorized as either clinical or non-clinical Clinical departments aresubjected to broad regulations from local and state licensing authorities as wellas national accrediting organizations In addition clinical departments aretypically those with highly skilled medical professionals that provide directphysiological medical care Non-clinical departments provide care but in a lessinvasive manner The care provided by non-clinical departments is not a part ofthe core physiological medical care This research relies on this generallyaccepted distinction between clinical and non-clinical areas Table III shows theclassiregcation of the participating departments

The participation of all department members was voluntary While they wereinformed about the project and invited to participate by e-mail and inter-ofregcememos no personal persuasion was permitted At the regrst quarter assessment83 percent of non-clinical and 79 percent of clinical personnel participated acrossall four hospital systems There were no apparent differences among the hospitalparticipation rates At the end of the second quarterparticipation increased to 89percent of non-clinical and 92 percent of clinical personnel The participationpercentages remained at these levels for the third and fourth quarters Those notparticipating were primarily newly hired personnel and older departmentmembers who were approaching retirement

Initially all department members staff as well as managers attendedseveral hands-on training sessions to ensure that everyone thoroughlyunderstood the AHP pairwise comparison and Expert Choice Then themembers of each department used Expert Choice to assess the importanceweight (Fi) for each critical factor and the importance weight ( fij) for each item( j) associated with a critical factor (i) During the training sessions theparticipants agreed that they should have an opportunity to review theirjudgments after receiving anonymous feedback containing the judgments ofthe others in their own departments The revision of individual judgments inview of anonymous group feedback is a fundamental principle of the Delphitechnique Delphi was developed at the Rand Corporation to obtain the mostreliable consensus of opinion from a group of knowledgeable individuals aboutan issue not subject to objective solution (Dalkey and Helmer 1963) Thetechnique consists of iterative sequences of collecting judgments from a groupanonymous feedback and individuals reconsidering their judgments In healthcare Delphi has been used to identify issues affecting health care

Total qualityindex

515

Clinic

alN

on-c

linic

al

Hos

pit

als

Rad

iolo

gy

Applied

clin

ical

tech

nol

ogy

Lab

orat

ory

Nucl

ear

med

icin

eR

espir

ator

yca

reSoc

ial

serv

ices

Phar

mac

yE

nvir

onm

enta

lse

rvic

esF

ood

and

nutr

itio

n

A3

plusmn3

plusmnplusmn

plusmn3

plusmnplusmn

B3

plusmn3

plusmnplusmn

plusmn3

3plusmn

Cplusmn

plusmnplusmn

3plusmn

3plusmn

plusmn3

D3

3plusmn

plusmn3

plusmn3

plusmnplusmn

Table IIIMatrix of clinical andnon-clinical cases

BIJ106

516

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 3: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

workingordm (Burda 1991) In addition the cost-effectiveness of these programs isan issue because signiregcant amounts are invested in training and consultantswhile speciregc problems are not being addressed

The model and its objectivesThere is a large body of literature on multi-criteria modeling and problemsolving using gap analysis Pounds (1969) gave a general description of aproblem situation as a difference between some existing situation and somedesired situation Most descriptions or deregnitions begin with a difference orgap between the present situation and the desired situation (Bartee 1973MacCrimmon and Taylor 1976 Reitman 1964) called the ideal situation in thisstudy In terms of organizations Cyert and March (1963) describe a problem asa difference between actual performance and ideal performance Here theproblem gap is one involving differences between actual and ideal performancemeasures For Kiesler and Sproull (1982) a managerrsquos recognition of a probleminvolves a comparison of environmental stimuli with internal aspirationcriteria

Quality management involves the recognition of such a gap This gap is thediscrepancy between the signals that managers receive from the environmenton various aspects of the quality management process and their own idea ofwhat the process should be Benson et al (1991) have used the phrases actualquality management and ideal quality management to describe these twostates of quality management Ideal quality management is described as abusiness unit managerrsquos beliefs concerning what quality management shouldbe in the business unit while actual quality management is described as themanagerrsquos perceptions of the current practice of quality management in theunit (Benson et al 1991)

There are two generally accepted behavioral approaches to qualitymanagement First Demingrsquos (1982) Theory D offers 14 managerialprescriptions and proscriptions describing general ways that managersshould act and what they should do to improve individual and teamperformance job organization and managing external groups Second the lesspopular organizational behavior modiregcation (OBMod) approach (Luthansand Kreitner 1985) is a behavioral approach to performance improvementgrounded in the work of Skinner (1938 1966) While both approaches share thesame end the difference between them is their level of speciregcity (Redmon andDickinson 1987) Theory D is more of an organization-wide approach to culturalchanges while OBMod focuses on altering simple behaviors of individualswithin organizations Recently there have been efforts to merge the twoapproaches by extending the OBMod approach with social learning theory(Luthans and Thompson 1987) This adds the recognition of cognitiveprocesses to the operant processes of OBMod When this is done there are nomajor differences between the two approaches However a criticism has been

Total qualityindex

509

that neither approach is sufregciently speciregc for managers to operationalize(Luthans and Thompson 1987) Forker and Mendez (2001) note that the collapseof many quality management programs hinged on breakdowns in execution

Benchmarking is an effective vehicle for focusing continuous improvementon the basic processes that run an organization (Bhutta and Huq 1999 Prado2001 Simpson and Kondouli 2000) It consists of investigating processes andpractices that are employed and establishing metrics for assessing anorganizationrsquos performance (Camp 1989) Furthermore the benchmarkingapproach can be adapted to accommodate assessment input from health careprofessionals including doctors and minimize resistance and pushback Theutilization of information technology reduces the time and effort required toparticipate in the assessment process enhancing the likelihood of participationof other-directed health care professionals What are needed are theoretical andmethodological structures for the analysis (Bhutta and Huq 1999 Forker andMendez 2001 Kumar et al 1999)

Saraph et al (1989) synthesized critical areas of organizational qualitymanagement and sets of speciregc requirements for each of the eight criticalfactors They have also proposed a questionnaire presented in the Appendixthat can be used to evaluate actual and ideal quality management with the setsof requirements as operational measures of the critical factors Thequestionnaire was developed for evaluating quality management in eithermanufacturing or service organizations Empirical tests showed the measuresto be both reliable and valid Subsequent research has supported the contentvalidity of the instrument (Kumar et al 1999) An inherent advantage of theSaraph et al Benson and Schroeder questionnaire is that it minimizes the timeand cost for developing the concepts and operational measures essential for thebenchmarking approach to TQM

This research develops and applies an IT-supported benchmarking modelthat helps managers assess a quality management program by enabling thecost-effective measurement of critical organizational processes The modelutilizes AHP and Delphi to measure both actual and ideal quality managementalong the eight critical success factors synthesized by Saraph et al (1989)

To formulate the algebraic model assume

Fi The importance weight of a quality management critical factor (fori = 1 8)

fij The importance weight of an item associated with a qualitymanagement critical factor (for i = 1 8 and j = 1 ki)

Rij The ideal rating of an item associated with critical factor (for i =

1 8 and j = 1 ki)

Rtij The actual rating of an item associated with a critical factor for a given

time period (for i = 1 8 j = 1 ki and t = 1 n)

BIJ106

510

TQIThe ideal total quality index

TQIt The actual total quality index for a given time period

d t The gap between the ideal and actual total quality index for a giventime period

Ki The number of items within each quality management critical factor

n The number of time periods where the total quality index is measured

The objective of the model is to

Minimize d t = TQI 2 TQIt (1)

where

TQI =X8

i= 1

F i

XK i

j= 1

f ijRij

Aacute (2)

TQIt =X8

i= 1

F i

XK i

j= 1

f ijRtij

Aacute (3)

and

X8

i= 1

F i = 1XK i

j= 1

f ij = 1 1 Rij 5 1 Rt

ij 5

Both Fi and fij are developed with the analytic hierarchy process (AHP) AHPwas introduced by Saaty (1972) to assist DMs in the evaluation of complexjudgmental problems The department managers in this study used AHP toassign numerical values to the eight quality management critical factors (Fi)and the sub-factors ( fij) suggested by Saraph et al (1989) The process wasconregned to a series of pairwise comparisons Saaty (1972) argues that amanager naturally regnds it easier to compare two things than to compare all theitems in a list AHP also evaluates the consistency of the managers and allowsfor the revision of their responses Because of the intuitive nature of the processand its power in resolving the complexity in a judgmental problem AHP hasbeen applied to many diverse decisions In particular AHP has been a verypopular technique for determining weights in multi-criteria problems (Shim1989 Zahedi 1986) A comprehensive list of the major applications of AHPalong with a description of the method and its axioms can be found in theworks of Saaty (1972 1977a b 1980 1990 1994 1999) Weiss and Rao (1987)and Zahedi (1986)

Total qualityindex

511

There has been some criticism of AHP in the operations researchcommunity In response Harker and Vargas (1990) show that AHP has anaxiomatic foundation the cardinal measurement of preferences is fullyrepresented by the eigenvector method and the principles of hierarchicalcomposition and rank reversal are valid On the other hand Dyer (1990a) hasquestioned the theoretical basis underlying AHP and argues that it can lead topreference reversals based on the alternative set being analyzed Saaty (1990)explains how rank reversal is a positive feature when new reference points areintroduced In this study the geometric aggregation rule was used to avoidrank reversal which had varying degrees of importance to differentresearchers (Dyer 1990a b Harker and Vargas 1990 Saaty 1990)

Assuming manager i believes that c1 c2 cI are the I factors thatcontribute to the overall quality management program the managerrsquos next taskis to assess the relative importance of these factors with AHP by comparingeach possible pair of factors cj ck and indicating which factor is more importantand by how much

These judgments are represented by an I 3 I matrix

A = (ajk) (j k = 1 2 I )

If cj is judged to be of equal importance as ck then ajk = 1If cj is judged to be more important than ck then ajk 1If cj is judged to be less important than ck then ajk 1

ajk = 1=akj ajk sup1 0

Thus matrix A is a reciprocal matrix so that the entry ajk is the inverse of theentry akj ajk remacrects the relative importance of cj compared with factor ck Forexample a12 = 150 indicates that c1 is 150 times as important as c2

The vector w representing the relative weights of each of the I factors wasfound by computing the normalized eigenvector corresponding to themaximum eigenvalue of matrix A An eigenvalue of A is deregned as l whichsatisreges the following matrix equation

Aw = lw

where l is a constant called the eigenvalue associated with the giveneigenvector w Saaty has shown that the best estimate of w is the one associatedwith the maximum eigenvalue (lmax) of the matrix A Because the sum of theweights should be equal to 100 the normalized eigenvector is used Saatyrsquosalgorithm for obtaining this w is incorporated in the software Expert Choice(2000) utilized in this study

One of the advantages of AHP is that it assesses the consistency of themanagerrsquos pairwise comparisons When the judgments are perfectly consistentthe maximum eigenvalue (lmax) should equal the number of factors that are

BIJ106

512

compared (I) Typically the responses are not perfectly consistent and lmax isgreater than I The larger the lmax the greater is the degree of inconsistencySaaty deregnes a consistency index (CI) as (lmax 2 I )=(I 2 1) and provides arandom index (RI) table for matrices of order 3-10 This RI is based on asimulation of a large number of randomly generated weights (Table I)

Saaty recommends the calculation of a consistency ratio (CR) that is the ratioof CI to RI for the same order matrix A CR of 010 or less is consideredacceptable Each manager used Expert Choice (2000) an AHP-based softwareindividually to perform all necessary calculations When the CR wasunacceptable the manager was informed that the pairwise comparisons werelogically inconsistent and was asked to revise hisher Expert Choicejudgments

Numerical exampleTable II provides a numerical example illustrating the calculation of thecomponents of the measures used in this study The illustration assumes thatthere are three critical quality management factors (F1 F2 and F3) four items( f11 f12 f13 and f14) associated with F1 three items ( f21 f22 and f23) associatedwith F2 and four items (f31 f32 f33 and f34) associated with F3 Assume that thedecision maker (DM) has assigned the importance weights of 050 030 and020 to F1 F2 and F3 respectively using AHP and Expert Choice Furthermorethe DM has assigned the following importance weights to the items 040 030020 and 010 to f11 through f14 060 030 and 010 to f21 through f23 and 050020 020 and 010 to f31 through f34 using AHP as well Next assume for thisillustration that the DM used the scale proposed by Saraph et al (1989) to rateboth the ideal quality and the actual quality management of the eight criticalfactors Table II shows the calculations of the TQI for the current period(t = 1)

The TQI of 500 and the TQI1 of 264 in Table II are summary reggures for allthe critical factors and do not specify whether the gap is unacceptably large oracceptably small The total quality index for each critical factor (TQIi) givesmanagers more information about the quality management process in each ofthe critical areas For example health care organizations may experiencecoordination problems between the quality department typically anadministrative function and other departments such as the pathology lab orthe intensive care units These departments generally have their own hierarchywithin the medical staff and do not report directly to administration Thisproblem could be identireged in the model by a relatively signiregcant gap betweenthe actual TQIt

2 and ideal TQI2 for critical factor 2 (the role of the qualitydepartment) for a given time period If the gap is unacceptably large managers

n 3 4 5 6 7 8 9 10RI 058 090 112 132 141 145 149 151

Table IRandom index table

Total qualityindex

513

could be prompted either to form some intervention strategies to narrow it oradjust their belief about the ideal rating

If properly assessed the intervention would center around one or more of theitems associated with the critical factor In the previous example a managercould identify the source of the problem as a lack of coordination between thequality department and other departments Strategies could then be developedthat address this speciregc item A manager could also conclude that noproblems exist with any item or any that do exist are not as signiregcant asindicated by the measured gap of the factor This could lead to a change in howthe manager believes TQIi should be rated What causes managers either todevelop strategies for intervention or to adjust their ideal rating should beexamined over time Movement over time could remacrect interventions made bymanagers or changes in their ideal TQI Further investigation into themanagersrsquo responses to the feedback they receive from TQI tables as well asthe environment could then be undertaken to determine what actually hashappened

The studyA study was conducted to investigate the usefulness of the TQI in differenthealth care settings Four hospitals that are geographically dispersedthroughout the state of New Jersey participated in the study for a period ofone year Each hospital was already active in quality management Topmanagement had stated a commitment to quality management anddepartmental managers were familiar with the basic principles andterminology of quality management Hospitals that were either not involved

Critical factorweights (Fi)

Item weights( fij)

Ideal ratingR

ij

plusmn sup2 Actual ratingR1

ij

plusmn sup2F isbquof ijsbquoR

ij

plusmn sup2F isbquof ijsbquoR

1ij

plusmn sup2

F1= 050 f11= 040 5 3 100 060f12= 030 5 2 075 030f13= 020 5 2 050 020f14= 010 5 3 025 015

TQI1 = 250 TQI11 = 125

F2= 030 f21= 060 5 4 090 072f22= 030 5 2 045 018f23= 010 5 3 015 009

TQI2 = 150 TQI12 = 099

F3= 020 f31= 050 5 2 050 020f32= 020 5 3 020 012f33= 020 5 1 020 004f34= 010 5 2 010 004

TQI3 = 100 TQI13 = 040

TQI = 500 TQI1= 264

Table IIActual and ideal TQI foran example period

BIJ106

514

in quality management or in the beginning stages of involvement wouldrequire additional education and were not chosen for this study

In hospital A the radiology laboratory and pharmacy departmentsparticipated The radiology laboratory pharmacy and environmental servicesdepartments participated from hospital B Hospital C included the nuclearmedicine social services and food and nutrition departments in the studyFinally in hospital D the radiology applied clinical technology respiratorycare and pharmacy departments participated In health care departments aretypically categorized as either clinical or non-clinical Clinical departments aresubjected to broad regulations from local and state licensing authorities as wellas national accrediting organizations In addition clinical departments aretypically those with highly skilled medical professionals that provide directphysiological medical care Non-clinical departments provide care but in a lessinvasive manner The care provided by non-clinical departments is not a part ofthe core physiological medical care This research relies on this generallyaccepted distinction between clinical and non-clinical areas Table III shows theclassiregcation of the participating departments

The participation of all department members was voluntary While they wereinformed about the project and invited to participate by e-mail and inter-ofregcememos no personal persuasion was permitted At the regrst quarter assessment83 percent of non-clinical and 79 percent of clinical personnel participated acrossall four hospital systems There were no apparent differences among the hospitalparticipation rates At the end of the second quarterparticipation increased to 89percent of non-clinical and 92 percent of clinical personnel The participationpercentages remained at these levels for the third and fourth quarters Those notparticipating were primarily newly hired personnel and older departmentmembers who were approaching retirement

Initially all department members staff as well as managers attendedseveral hands-on training sessions to ensure that everyone thoroughlyunderstood the AHP pairwise comparison and Expert Choice Then themembers of each department used Expert Choice to assess the importanceweight (Fi) for each critical factor and the importance weight ( fij) for each item( j) associated with a critical factor (i) During the training sessions theparticipants agreed that they should have an opportunity to review theirjudgments after receiving anonymous feedback containing the judgments ofthe others in their own departments The revision of individual judgments inview of anonymous group feedback is a fundamental principle of the Delphitechnique Delphi was developed at the Rand Corporation to obtain the mostreliable consensus of opinion from a group of knowledgeable individuals aboutan issue not subject to objective solution (Dalkey and Helmer 1963) Thetechnique consists of iterative sequences of collecting judgments from a groupanonymous feedback and individuals reconsidering their judgments In healthcare Delphi has been used to identify issues affecting health care

Total qualityindex

515

Clinic

alN

on-c

linic

al

Hos

pit

als

Rad

iolo

gy

Applied

clin

ical

tech

nol

ogy

Lab

orat

ory

Nucl

ear

med

icin

eR

espir

ator

yca

reSoc

ial

serv

ices

Phar

mac

yE

nvir

onm

enta

lse

rvic

esF

ood

and

nutr

itio

n

A3

plusmn3

plusmnplusmn

plusmn3

plusmnplusmn

B3

plusmn3

plusmnplusmn

plusmn3

3plusmn

Cplusmn

plusmnplusmn

3plusmn

3plusmn

plusmn3

D3

3plusmn

plusmn3

plusmn3

plusmnplusmn

Table IIIMatrix of clinical andnon-clinical cases

BIJ106

516

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 4: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

that neither approach is sufregciently speciregc for managers to operationalize(Luthans and Thompson 1987) Forker and Mendez (2001) note that the collapseof many quality management programs hinged on breakdowns in execution

Benchmarking is an effective vehicle for focusing continuous improvementon the basic processes that run an organization (Bhutta and Huq 1999 Prado2001 Simpson and Kondouli 2000) It consists of investigating processes andpractices that are employed and establishing metrics for assessing anorganizationrsquos performance (Camp 1989) Furthermore the benchmarkingapproach can be adapted to accommodate assessment input from health careprofessionals including doctors and minimize resistance and pushback Theutilization of information technology reduces the time and effort required toparticipate in the assessment process enhancing the likelihood of participationof other-directed health care professionals What are needed are theoretical andmethodological structures for the analysis (Bhutta and Huq 1999 Forker andMendez 2001 Kumar et al 1999)

Saraph et al (1989) synthesized critical areas of organizational qualitymanagement and sets of speciregc requirements for each of the eight criticalfactors They have also proposed a questionnaire presented in the Appendixthat can be used to evaluate actual and ideal quality management with the setsof requirements as operational measures of the critical factors Thequestionnaire was developed for evaluating quality management in eithermanufacturing or service organizations Empirical tests showed the measuresto be both reliable and valid Subsequent research has supported the contentvalidity of the instrument (Kumar et al 1999) An inherent advantage of theSaraph et al Benson and Schroeder questionnaire is that it minimizes the timeand cost for developing the concepts and operational measures essential for thebenchmarking approach to TQM

This research develops and applies an IT-supported benchmarking modelthat helps managers assess a quality management program by enabling thecost-effective measurement of critical organizational processes The modelutilizes AHP and Delphi to measure both actual and ideal quality managementalong the eight critical success factors synthesized by Saraph et al (1989)

To formulate the algebraic model assume

Fi The importance weight of a quality management critical factor (fori = 1 8)

fij The importance weight of an item associated with a qualitymanagement critical factor (for i = 1 8 and j = 1 ki)

Rij The ideal rating of an item associated with critical factor (for i =

1 8 and j = 1 ki)

Rtij The actual rating of an item associated with a critical factor for a given

time period (for i = 1 8 j = 1 ki and t = 1 n)

BIJ106

510

TQIThe ideal total quality index

TQIt The actual total quality index for a given time period

d t The gap between the ideal and actual total quality index for a giventime period

Ki The number of items within each quality management critical factor

n The number of time periods where the total quality index is measured

The objective of the model is to

Minimize d t = TQI 2 TQIt (1)

where

TQI =X8

i= 1

F i

XK i

j= 1

f ijRij

Aacute (2)

TQIt =X8

i= 1

F i

XK i

j= 1

f ijRtij

Aacute (3)

and

X8

i= 1

F i = 1XK i

j= 1

f ij = 1 1 Rij 5 1 Rt

ij 5

Both Fi and fij are developed with the analytic hierarchy process (AHP) AHPwas introduced by Saaty (1972) to assist DMs in the evaluation of complexjudgmental problems The department managers in this study used AHP toassign numerical values to the eight quality management critical factors (Fi)and the sub-factors ( fij) suggested by Saraph et al (1989) The process wasconregned to a series of pairwise comparisons Saaty (1972) argues that amanager naturally regnds it easier to compare two things than to compare all theitems in a list AHP also evaluates the consistency of the managers and allowsfor the revision of their responses Because of the intuitive nature of the processand its power in resolving the complexity in a judgmental problem AHP hasbeen applied to many diverse decisions In particular AHP has been a verypopular technique for determining weights in multi-criteria problems (Shim1989 Zahedi 1986) A comprehensive list of the major applications of AHPalong with a description of the method and its axioms can be found in theworks of Saaty (1972 1977a b 1980 1990 1994 1999) Weiss and Rao (1987)and Zahedi (1986)

Total qualityindex

511

There has been some criticism of AHP in the operations researchcommunity In response Harker and Vargas (1990) show that AHP has anaxiomatic foundation the cardinal measurement of preferences is fullyrepresented by the eigenvector method and the principles of hierarchicalcomposition and rank reversal are valid On the other hand Dyer (1990a) hasquestioned the theoretical basis underlying AHP and argues that it can lead topreference reversals based on the alternative set being analyzed Saaty (1990)explains how rank reversal is a positive feature when new reference points areintroduced In this study the geometric aggregation rule was used to avoidrank reversal which had varying degrees of importance to differentresearchers (Dyer 1990a b Harker and Vargas 1990 Saaty 1990)

Assuming manager i believes that c1 c2 cI are the I factors thatcontribute to the overall quality management program the managerrsquos next taskis to assess the relative importance of these factors with AHP by comparingeach possible pair of factors cj ck and indicating which factor is more importantand by how much

These judgments are represented by an I 3 I matrix

A = (ajk) (j k = 1 2 I )

If cj is judged to be of equal importance as ck then ajk = 1If cj is judged to be more important than ck then ajk 1If cj is judged to be less important than ck then ajk 1

ajk = 1=akj ajk sup1 0

Thus matrix A is a reciprocal matrix so that the entry ajk is the inverse of theentry akj ajk remacrects the relative importance of cj compared with factor ck Forexample a12 = 150 indicates that c1 is 150 times as important as c2

The vector w representing the relative weights of each of the I factors wasfound by computing the normalized eigenvector corresponding to themaximum eigenvalue of matrix A An eigenvalue of A is deregned as l whichsatisreges the following matrix equation

Aw = lw

where l is a constant called the eigenvalue associated with the giveneigenvector w Saaty has shown that the best estimate of w is the one associatedwith the maximum eigenvalue (lmax) of the matrix A Because the sum of theweights should be equal to 100 the normalized eigenvector is used Saatyrsquosalgorithm for obtaining this w is incorporated in the software Expert Choice(2000) utilized in this study

One of the advantages of AHP is that it assesses the consistency of themanagerrsquos pairwise comparisons When the judgments are perfectly consistentthe maximum eigenvalue (lmax) should equal the number of factors that are

BIJ106

512

compared (I) Typically the responses are not perfectly consistent and lmax isgreater than I The larger the lmax the greater is the degree of inconsistencySaaty deregnes a consistency index (CI) as (lmax 2 I )=(I 2 1) and provides arandom index (RI) table for matrices of order 3-10 This RI is based on asimulation of a large number of randomly generated weights (Table I)

Saaty recommends the calculation of a consistency ratio (CR) that is the ratioof CI to RI for the same order matrix A CR of 010 or less is consideredacceptable Each manager used Expert Choice (2000) an AHP-based softwareindividually to perform all necessary calculations When the CR wasunacceptable the manager was informed that the pairwise comparisons werelogically inconsistent and was asked to revise hisher Expert Choicejudgments

Numerical exampleTable II provides a numerical example illustrating the calculation of thecomponents of the measures used in this study The illustration assumes thatthere are three critical quality management factors (F1 F2 and F3) four items( f11 f12 f13 and f14) associated with F1 three items ( f21 f22 and f23) associatedwith F2 and four items (f31 f32 f33 and f34) associated with F3 Assume that thedecision maker (DM) has assigned the importance weights of 050 030 and020 to F1 F2 and F3 respectively using AHP and Expert Choice Furthermorethe DM has assigned the following importance weights to the items 040 030020 and 010 to f11 through f14 060 030 and 010 to f21 through f23 and 050020 020 and 010 to f31 through f34 using AHP as well Next assume for thisillustration that the DM used the scale proposed by Saraph et al (1989) to rateboth the ideal quality and the actual quality management of the eight criticalfactors Table II shows the calculations of the TQI for the current period(t = 1)

The TQI of 500 and the TQI1 of 264 in Table II are summary reggures for allthe critical factors and do not specify whether the gap is unacceptably large oracceptably small The total quality index for each critical factor (TQIi) givesmanagers more information about the quality management process in each ofthe critical areas For example health care organizations may experiencecoordination problems between the quality department typically anadministrative function and other departments such as the pathology lab orthe intensive care units These departments generally have their own hierarchywithin the medical staff and do not report directly to administration Thisproblem could be identireged in the model by a relatively signiregcant gap betweenthe actual TQIt

2 and ideal TQI2 for critical factor 2 (the role of the qualitydepartment) for a given time period If the gap is unacceptably large managers

n 3 4 5 6 7 8 9 10RI 058 090 112 132 141 145 149 151

Table IRandom index table

Total qualityindex

513

could be prompted either to form some intervention strategies to narrow it oradjust their belief about the ideal rating

If properly assessed the intervention would center around one or more of theitems associated with the critical factor In the previous example a managercould identify the source of the problem as a lack of coordination between thequality department and other departments Strategies could then be developedthat address this speciregc item A manager could also conclude that noproblems exist with any item or any that do exist are not as signiregcant asindicated by the measured gap of the factor This could lead to a change in howthe manager believes TQIi should be rated What causes managers either todevelop strategies for intervention or to adjust their ideal rating should beexamined over time Movement over time could remacrect interventions made bymanagers or changes in their ideal TQI Further investigation into themanagersrsquo responses to the feedback they receive from TQI tables as well asthe environment could then be undertaken to determine what actually hashappened

The studyA study was conducted to investigate the usefulness of the TQI in differenthealth care settings Four hospitals that are geographically dispersedthroughout the state of New Jersey participated in the study for a period ofone year Each hospital was already active in quality management Topmanagement had stated a commitment to quality management anddepartmental managers were familiar with the basic principles andterminology of quality management Hospitals that were either not involved

Critical factorweights (Fi)

Item weights( fij)

Ideal ratingR

ij

plusmn sup2 Actual ratingR1

ij

plusmn sup2F isbquof ijsbquoR

ij

plusmn sup2F isbquof ijsbquoR

1ij

plusmn sup2

F1= 050 f11= 040 5 3 100 060f12= 030 5 2 075 030f13= 020 5 2 050 020f14= 010 5 3 025 015

TQI1 = 250 TQI11 = 125

F2= 030 f21= 060 5 4 090 072f22= 030 5 2 045 018f23= 010 5 3 015 009

TQI2 = 150 TQI12 = 099

F3= 020 f31= 050 5 2 050 020f32= 020 5 3 020 012f33= 020 5 1 020 004f34= 010 5 2 010 004

TQI3 = 100 TQI13 = 040

TQI = 500 TQI1= 264

Table IIActual and ideal TQI foran example period

BIJ106

514

in quality management or in the beginning stages of involvement wouldrequire additional education and were not chosen for this study

In hospital A the radiology laboratory and pharmacy departmentsparticipated The radiology laboratory pharmacy and environmental servicesdepartments participated from hospital B Hospital C included the nuclearmedicine social services and food and nutrition departments in the studyFinally in hospital D the radiology applied clinical technology respiratorycare and pharmacy departments participated In health care departments aretypically categorized as either clinical or non-clinical Clinical departments aresubjected to broad regulations from local and state licensing authorities as wellas national accrediting organizations In addition clinical departments aretypically those with highly skilled medical professionals that provide directphysiological medical care Non-clinical departments provide care but in a lessinvasive manner The care provided by non-clinical departments is not a part ofthe core physiological medical care This research relies on this generallyaccepted distinction between clinical and non-clinical areas Table III shows theclassiregcation of the participating departments

The participation of all department members was voluntary While they wereinformed about the project and invited to participate by e-mail and inter-ofregcememos no personal persuasion was permitted At the regrst quarter assessment83 percent of non-clinical and 79 percent of clinical personnel participated acrossall four hospital systems There were no apparent differences among the hospitalparticipation rates At the end of the second quarterparticipation increased to 89percent of non-clinical and 92 percent of clinical personnel The participationpercentages remained at these levels for the third and fourth quarters Those notparticipating were primarily newly hired personnel and older departmentmembers who were approaching retirement

Initially all department members staff as well as managers attendedseveral hands-on training sessions to ensure that everyone thoroughlyunderstood the AHP pairwise comparison and Expert Choice Then themembers of each department used Expert Choice to assess the importanceweight (Fi) for each critical factor and the importance weight ( fij) for each item( j) associated with a critical factor (i) During the training sessions theparticipants agreed that they should have an opportunity to review theirjudgments after receiving anonymous feedback containing the judgments ofthe others in their own departments The revision of individual judgments inview of anonymous group feedback is a fundamental principle of the Delphitechnique Delphi was developed at the Rand Corporation to obtain the mostreliable consensus of opinion from a group of knowledgeable individuals aboutan issue not subject to objective solution (Dalkey and Helmer 1963) Thetechnique consists of iterative sequences of collecting judgments from a groupanonymous feedback and individuals reconsidering their judgments In healthcare Delphi has been used to identify issues affecting health care

Total qualityindex

515

Clinic

alN

on-c

linic

al

Hos

pit

als

Rad

iolo

gy

Applied

clin

ical

tech

nol

ogy

Lab

orat

ory

Nucl

ear

med

icin

eR

espir

ator

yca

reSoc

ial

serv

ices

Phar

mac

yE

nvir

onm

enta

lse

rvic

esF

ood

and

nutr

itio

n

A3

plusmn3

plusmnplusmn

plusmn3

plusmnplusmn

B3

plusmn3

plusmnplusmn

plusmn3

3plusmn

Cplusmn

plusmnplusmn

3plusmn

3plusmn

plusmn3

D3

3plusmn

plusmn3

plusmn3

plusmnplusmn

Table IIIMatrix of clinical andnon-clinical cases

BIJ106

516

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 5: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

TQIThe ideal total quality index

TQIt The actual total quality index for a given time period

d t The gap between the ideal and actual total quality index for a giventime period

Ki The number of items within each quality management critical factor

n The number of time periods where the total quality index is measured

The objective of the model is to

Minimize d t = TQI 2 TQIt (1)

where

TQI =X8

i= 1

F i

XK i

j= 1

f ijRij

Aacute (2)

TQIt =X8

i= 1

F i

XK i

j= 1

f ijRtij

Aacute (3)

and

X8

i= 1

F i = 1XK i

j= 1

f ij = 1 1 Rij 5 1 Rt

ij 5

Both Fi and fij are developed with the analytic hierarchy process (AHP) AHPwas introduced by Saaty (1972) to assist DMs in the evaluation of complexjudgmental problems The department managers in this study used AHP toassign numerical values to the eight quality management critical factors (Fi)and the sub-factors ( fij) suggested by Saraph et al (1989) The process wasconregned to a series of pairwise comparisons Saaty (1972) argues that amanager naturally regnds it easier to compare two things than to compare all theitems in a list AHP also evaluates the consistency of the managers and allowsfor the revision of their responses Because of the intuitive nature of the processand its power in resolving the complexity in a judgmental problem AHP hasbeen applied to many diverse decisions In particular AHP has been a verypopular technique for determining weights in multi-criteria problems (Shim1989 Zahedi 1986) A comprehensive list of the major applications of AHPalong with a description of the method and its axioms can be found in theworks of Saaty (1972 1977a b 1980 1990 1994 1999) Weiss and Rao (1987)and Zahedi (1986)

Total qualityindex

511

There has been some criticism of AHP in the operations researchcommunity In response Harker and Vargas (1990) show that AHP has anaxiomatic foundation the cardinal measurement of preferences is fullyrepresented by the eigenvector method and the principles of hierarchicalcomposition and rank reversal are valid On the other hand Dyer (1990a) hasquestioned the theoretical basis underlying AHP and argues that it can lead topreference reversals based on the alternative set being analyzed Saaty (1990)explains how rank reversal is a positive feature when new reference points areintroduced In this study the geometric aggregation rule was used to avoidrank reversal which had varying degrees of importance to differentresearchers (Dyer 1990a b Harker and Vargas 1990 Saaty 1990)

Assuming manager i believes that c1 c2 cI are the I factors thatcontribute to the overall quality management program the managerrsquos next taskis to assess the relative importance of these factors with AHP by comparingeach possible pair of factors cj ck and indicating which factor is more importantand by how much

These judgments are represented by an I 3 I matrix

A = (ajk) (j k = 1 2 I )

If cj is judged to be of equal importance as ck then ajk = 1If cj is judged to be more important than ck then ajk 1If cj is judged to be less important than ck then ajk 1

ajk = 1=akj ajk sup1 0

Thus matrix A is a reciprocal matrix so that the entry ajk is the inverse of theentry akj ajk remacrects the relative importance of cj compared with factor ck Forexample a12 = 150 indicates that c1 is 150 times as important as c2

The vector w representing the relative weights of each of the I factors wasfound by computing the normalized eigenvector corresponding to themaximum eigenvalue of matrix A An eigenvalue of A is deregned as l whichsatisreges the following matrix equation

Aw = lw

where l is a constant called the eigenvalue associated with the giveneigenvector w Saaty has shown that the best estimate of w is the one associatedwith the maximum eigenvalue (lmax) of the matrix A Because the sum of theweights should be equal to 100 the normalized eigenvector is used Saatyrsquosalgorithm for obtaining this w is incorporated in the software Expert Choice(2000) utilized in this study

One of the advantages of AHP is that it assesses the consistency of themanagerrsquos pairwise comparisons When the judgments are perfectly consistentthe maximum eigenvalue (lmax) should equal the number of factors that are

BIJ106

512

compared (I) Typically the responses are not perfectly consistent and lmax isgreater than I The larger the lmax the greater is the degree of inconsistencySaaty deregnes a consistency index (CI) as (lmax 2 I )=(I 2 1) and provides arandom index (RI) table for matrices of order 3-10 This RI is based on asimulation of a large number of randomly generated weights (Table I)

Saaty recommends the calculation of a consistency ratio (CR) that is the ratioof CI to RI for the same order matrix A CR of 010 or less is consideredacceptable Each manager used Expert Choice (2000) an AHP-based softwareindividually to perform all necessary calculations When the CR wasunacceptable the manager was informed that the pairwise comparisons werelogically inconsistent and was asked to revise hisher Expert Choicejudgments

Numerical exampleTable II provides a numerical example illustrating the calculation of thecomponents of the measures used in this study The illustration assumes thatthere are three critical quality management factors (F1 F2 and F3) four items( f11 f12 f13 and f14) associated with F1 three items ( f21 f22 and f23) associatedwith F2 and four items (f31 f32 f33 and f34) associated with F3 Assume that thedecision maker (DM) has assigned the importance weights of 050 030 and020 to F1 F2 and F3 respectively using AHP and Expert Choice Furthermorethe DM has assigned the following importance weights to the items 040 030020 and 010 to f11 through f14 060 030 and 010 to f21 through f23 and 050020 020 and 010 to f31 through f34 using AHP as well Next assume for thisillustration that the DM used the scale proposed by Saraph et al (1989) to rateboth the ideal quality and the actual quality management of the eight criticalfactors Table II shows the calculations of the TQI for the current period(t = 1)

The TQI of 500 and the TQI1 of 264 in Table II are summary reggures for allthe critical factors and do not specify whether the gap is unacceptably large oracceptably small The total quality index for each critical factor (TQIi) givesmanagers more information about the quality management process in each ofthe critical areas For example health care organizations may experiencecoordination problems between the quality department typically anadministrative function and other departments such as the pathology lab orthe intensive care units These departments generally have their own hierarchywithin the medical staff and do not report directly to administration Thisproblem could be identireged in the model by a relatively signiregcant gap betweenthe actual TQIt

2 and ideal TQI2 for critical factor 2 (the role of the qualitydepartment) for a given time period If the gap is unacceptably large managers

n 3 4 5 6 7 8 9 10RI 058 090 112 132 141 145 149 151

Table IRandom index table

Total qualityindex

513

could be prompted either to form some intervention strategies to narrow it oradjust their belief about the ideal rating

If properly assessed the intervention would center around one or more of theitems associated with the critical factor In the previous example a managercould identify the source of the problem as a lack of coordination between thequality department and other departments Strategies could then be developedthat address this speciregc item A manager could also conclude that noproblems exist with any item or any that do exist are not as signiregcant asindicated by the measured gap of the factor This could lead to a change in howthe manager believes TQIi should be rated What causes managers either todevelop strategies for intervention or to adjust their ideal rating should beexamined over time Movement over time could remacrect interventions made bymanagers or changes in their ideal TQI Further investigation into themanagersrsquo responses to the feedback they receive from TQI tables as well asthe environment could then be undertaken to determine what actually hashappened

The studyA study was conducted to investigate the usefulness of the TQI in differenthealth care settings Four hospitals that are geographically dispersedthroughout the state of New Jersey participated in the study for a period ofone year Each hospital was already active in quality management Topmanagement had stated a commitment to quality management anddepartmental managers were familiar with the basic principles andterminology of quality management Hospitals that were either not involved

Critical factorweights (Fi)

Item weights( fij)

Ideal ratingR

ij

plusmn sup2 Actual ratingR1

ij

plusmn sup2F isbquof ijsbquoR

ij

plusmn sup2F isbquof ijsbquoR

1ij

plusmn sup2

F1= 050 f11= 040 5 3 100 060f12= 030 5 2 075 030f13= 020 5 2 050 020f14= 010 5 3 025 015

TQI1 = 250 TQI11 = 125

F2= 030 f21= 060 5 4 090 072f22= 030 5 2 045 018f23= 010 5 3 015 009

TQI2 = 150 TQI12 = 099

F3= 020 f31= 050 5 2 050 020f32= 020 5 3 020 012f33= 020 5 1 020 004f34= 010 5 2 010 004

TQI3 = 100 TQI13 = 040

TQI = 500 TQI1= 264

Table IIActual and ideal TQI foran example period

BIJ106

514

in quality management or in the beginning stages of involvement wouldrequire additional education and were not chosen for this study

In hospital A the radiology laboratory and pharmacy departmentsparticipated The radiology laboratory pharmacy and environmental servicesdepartments participated from hospital B Hospital C included the nuclearmedicine social services and food and nutrition departments in the studyFinally in hospital D the radiology applied clinical technology respiratorycare and pharmacy departments participated In health care departments aretypically categorized as either clinical or non-clinical Clinical departments aresubjected to broad regulations from local and state licensing authorities as wellas national accrediting organizations In addition clinical departments aretypically those with highly skilled medical professionals that provide directphysiological medical care Non-clinical departments provide care but in a lessinvasive manner The care provided by non-clinical departments is not a part ofthe core physiological medical care This research relies on this generallyaccepted distinction between clinical and non-clinical areas Table III shows theclassiregcation of the participating departments

The participation of all department members was voluntary While they wereinformed about the project and invited to participate by e-mail and inter-ofregcememos no personal persuasion was permitted At the regrst quarter assessment83 percent of non-clinical and 79 percent of clinical personnel participated acrossall four hospital systems There were no apparent differences among the hospitalparticipation rates At the end of the second quarterparticipation increased to 89percent of non-clinical and 92 percent of clinical personnel The participationpercentages remained at these levels for the third and fourth quarters Those notparticipating were primarily newly hired personnel and older departmentmembers who were approaching retirement

Initially all department members staff as well as managers attendedseveral hands-on training sessions to ensure that everyone thoroughlyunderstood the AHP pairwise comparison and Expert Choice Then themembers of each department used Expert Choice to assess the importanceweight (Fi) for each critical factor and the importance weight ( fij) for each item( j) associated with a critical factor (i) During the training sessions theparticipants agreed that they should have an opportunity to review theirjudgments after receiving anonymous feedback containing the judgments ofthe others in their own departments The revision of individual judgments inview of anonymous group feedback is a fundamental principle of the Delphitechnique Delphi was developed at the Rand Corporation to obtain the mostreliable consensus of opinion from a group of knowledgeable individuals aboutan issue not subject to objective solution (Dalkey and Helmer 1963) Thetechnique consists of iterative sequences of collecting judgments from a groupanonymous feedback and individuals reconsidering their judgments In healthcare Delphi has been used to identify issues affecting health care

Total qualityindex

515

Clinic

alN

on-c

linic

al

Hos

pit

als

Rad

iolo

gy

Applied

clin

ical

tech

nol

ogy

Lab

orat

ory

Nucl

ear

med

icin

eR

espir

ator

yca

reSoc

ial

serv

ices

Phar

mac

yE

nvir

onm

enta

lse

rvic

esF

ood

and

nutr

itio

n

A3

plusmn3

plusmnplusmn

plusmn3

plusmnplusmn

B3

plusmn3

plusmnplusmn

plusmn3

3plusmn

Cplusmn

plusmnplusmn

3plusmn

3plusmn

plusmn3

D3

3plusmn

plusmn3

plusmn3

plusmnplusmn

Table IIIMatrix of clinical andnon-clinical cases

BIJ106

516

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 6: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

There has been some criticism of AHP in the operations researchcommunity In response Harker and Vargas (1990) show that AHP has anaxiomatic foundation the cardinal measurement of preferences is fullyrepresented by the eigenvector method and the principles of hierarchicalcomposition and rank reversal are valid On the other hand Dyer (1990a) hasquestioned the theoretical basis underlying AHP and argues that it can lead topreference reversals based on the alternative set being analyzed Saaty (1990)explains how rank reversal is a positive feature when new reference points areintroduced In this study the geometric aggregation rule was used to avoidrank reversal which had varying degrees of importance to differentresearchers (Dyer 1990a b Harker and Vargas 1990 Saaty 1990)

Assuming manager i believes that c1 c2 cI are the I factors thatcontribute to the overall quality management program the managerrsquos next taskis to assess the relative importance of these factors with AHP by comparingeach possible pair of factors cj ck and indicating which factor is more importantand by how much

These judgments are represented by an I 3 I matrix

A = (ajk) (j k = 1 2 I )

If cj is judged to be of equal importance as ck then ajk = 1If cj is judged to be more important than ck then ajk 1If cj is judged to be less important than ck then ajk 1

ajk = 1=akj ajk sup1 0

Thus matrix A is a reciprocal matrix so that the entry ajk is the inverse of theentry akj ajk remacrects the relative importance of cj compared with factor ck Forexample a12 = 150 indicates that c1 is 150 times as important as c2

The vector w representing the relative weights of each of the I factors wasfound by computing the normalized eigenvector corresponding to themaximum eigenvalue of matrix A An eigenvalue of A is deregned as l whichsatisreges the following matrix equation

Aw = lw

where l is a constant called the eigenvalue associated with the giveneigenvector w Saaty has shown that the best estimate of w is the one associatedwith the maximum eigenvalue (lmax) of the matrix A Because the sum of theweights should be equal to 100 the normalized eigenvector is used Saatyrsquosalgorithm for obtaining this w is incorporated in the software Expert Choice(2000) utilized in this study

One of the advantages of AHP is that it assesses the consistency of themanagerrsquos pairwise comparisons When the judgments are perfectly consistentthe maximum eigenvalue (lmax) should equal the number of factors that are

BIJ106

512

compared (I) Typically the responses are not perfectly consistent and lmax isgreater than I The larger the lmax the greater is the degree of inconsistencySaaty deregnes a consistency index (CI) as (lmax 2 I )=(I 2 1) and provides arandom index (RI) table for matrices of order 3-10 This RI is based on asimulation of a large number of randomly generated weights (Table I)

Saaty recommends the calculation of a consistency ratio (CR) that is the ratioof CI to RI for the same order matrix A CR of 010 or less is consideredacceptable Each manager used Expert Choice (2000) an AHP-based softwareindividually to perform all necessary calculations When the CR wasunacceptable the manager was informed that the pairwise comparisons werelogically inconsistent and was asked to revise hisher Expert Choicejudgments

Numerical exampleTable II provides a numerical example illustrating the calculation of thecomponents of the measures used in this study The illustration assumes thatthere are three critical quality management factors (F1 F2 and F3) four items( f11 f12 f13 and f14) associated with F1 three items ( f21 f22 and f23) associatedwith F2 and four items (f31 f32 f33 and f34) associated with F3 Assume that thedecision maker (DM) has assigned the importance weights of 050 030 and020 to F1 F2 and F3 respectively using AHP and Expert Choice Furthermorethe DM has assigned the following importance weights to the items 040 030020 and 010 to f11 through f14 060 030 and 010 to f21 through f23 and 050020 020 and 010 to f31 through f34 using AHP as well Next assume for thisillustration that the DM used the scale proposed by Saraph et al (1989) to rateboth the ideal quality and the actual quality management of the eight criticalfactors Table II shows the calculations of the TQI for the current period(t = 1)

The TQI of 500 and the TQI1 of 264 in Table II are summary reggures for allthe critical factors and do not specify whether the gap is unacceptably large oracceptably small The total quality index for each critical factor (TQIi) givesmanagers more information about the quality management process in each ofthe critical areas For example health care organizations may experiencecoordination problems between the quality department typically anadministrative function and other departments such as the pathology lab orthe intensive care units These departments generally have their own hierarchywithin the medical staff and do not report directly to administration Thisproblem could be identireged in the model by a relatively signiregcant gap betweenthe actual TQIt

2 and ideal TQI2 for critical factor 2 (the role of the qualitydepartment) for a given time period If the gap is unacceptably large managers

n 3 4 5 6 7 8 9 10RI 058 090 112 132 141 145 149 151

Table IRandom index table

Total qualityindex

513

could be prompted either to form some intervention strategies to narrow it oradjust their belief about the ideal rating

If properly assessed the intervention would center around one or more of theitems associated with the critical factor In the previous example a managercould identify the source of the problem as a lack of coordination between thequality department and other departments Strategies could then be developedthat address this speciregc item A manager could also conclude that noproblems exist with any item or any that do exist are not as signiregcant asindicated by the measured gap of the factor This could lead to a change in howthe manager believes TQIi should be rated What causes managers either todevelop strategies for intervention or to adjust their ideal rating should beexamined over time Movement over time could remacrect interventions made bymanagers or changes in their ideal TQI Further investigation into themanagersrsquo responses to the feedback they receive from TQI tables as well asthe environment could then be undertaken to determine what actually hashappened

The studyA study was conducted to investigate the usefulness of the TQI in differenthealth care settings Four hospitals that are geographically dispersedthroughout the state of New Jersey participated in the study for a period ofone year Each hospital was already active in quality management Topmanagement had stated a commitment to quality management anddepartmental managers were familiar with the basic principles andterminology of quality management Hospitals that were either not involved

Critical factorweights (Fi)

Item weights( fij)

Ideal ratingR

ij

plusmn sup2 Actual ratingR1

ij

plusmn sup2F isbquof ijsbquoR

ij

plusmn sup2F isbquof ijsbquoR

1ij

plusmn sup2

F1= 050 f11= 040 5 3 100 060f12= 030 5 2 075 030f13= 020 5 2 050 020f14= 010 5 3 025 015

TQI1 = 250 TQI11 = 125

F2= 030 f21= 060 5 4 090 072f22= 030 5 2 045 018f23= 010 5 3 015 009

TQI2 = 150 TQI12 = 099

F3= 020 f31= 050 5 2 050 020f32= 020 5 3 020 012f33= 020 5 1 020 004f34= 010 5 2 010 004

TQI3 = 100 TQI13 = 040

TQI = 500 TQI1= 264

Table IIActual and ideal TQI foran example period

BIJ106

514

in quality management or in the beginning stages of involvement wouldrequire additional education and were not chosen for this study

In hospital A the radiology laboratory and pharmacy departmentsparticipated The radiology laboratory pharmacy and environmental servicesdepartments participated from hospital B Hospital C included the nuclearmedicine social services and food and nutrition departments in the studyFinally in hospital D the radiology applied clinical technology respiratorycare and pharmacy departments participated In health care departments aretypically categorized as either clinical or non-clinical Clinical departments aresubjected to broad regulations from local and state licensing authorities as wellas national accrediting organizations In addition clinical departments aretypically those with highly skilled medical professionals that provide directphysiological medical care Non-clinical departments provide care but in a lessinvasive manner The care provided by non-clinical departments is not a part ofthe core physiological medical care This research relies on this generallyaccepted distinction between clinical and non-clinical areas Table III shows theclassiregcation of the participating departments

The participation of all department members was voluntary While they wereinformed about the project and invited to participate by e-mail and inter-ofregcememos no personal persuasion was permitted At the regrst quarter assessment83 percent of non-clinical and 79 percent of clinical personnel participated acrossall four hospital systems There were no apparent differences among the hospitalparticipation rates At the end of the second quarterparticipation increased to 89percent of non-clinical and 92 percent of clinical personnel The participationpercentages remained at these levels for the third and fourth quarters Those notparticipating were primarily newly hired personnel and older departmentmembers who were approaching retirement

Initially all department members staff as well as managers attendedseveral hands-on training sessions to ensure that everyone thoroughlyunderstood the AHP pairwise comparison and Expert Choice Then themembers of each department used Expert Choice to assess the importanceweight (Fi) for each critical factor and the importance weight ( fij) for each item( j) associated with a critical factor (i) During the training sessions theparticipants agreed that they should have an opportunity to review theirjudgments after receiving anonymous feedback containing the judgments ofthe others in their own departments The revision of individual judgments inview of anonymous group feedback is a fundamental principle of the Delphitechnique Delphi was developed at the Rand Corporation to obtain the mostreliable consensus of opinion from a group of knowledgeable individuals aboutan issue not subject to objective solution (Dalkey and Helmer 1963) Thetechnique consists of iterative sequences of collecting judgments from a groupanonymous feedback and individuals reconsidering their judgments In healthcare Delphi has been used to identify issues affecting health care

Total qualityindex

515

Clinic

alN

on-c

linic

al

Hos

pit

als

Rad

iolo

gy

Applied

clin

ical

tech

nol

ogy

Lab

orat

ory

Nucl

ear

med

icin

eR

espir

ator

yca

reSoc

ial

serv

ices

Phar

mac

yE

nvir

onm

enta

lse

rvic

esF

ood

and

nutr

itio

n

A3

plusmn3

plusmnplusmn

plusmn3

plusmnplusmn

B3

plusmn3

plusmnplusmn

plusmn3

3plusmn

Cplusmn

plusmnplusmn

3plusmn

3plusmn

plusmn3

D3

3plusmn

plusmn3

plusmn3

plusmnplusmn

Table IIIMatrix of clinical andnon-clinical cases

BIJ106

516

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 7: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

compared (I) Typically the responses are not perfectly consistent and lmax isgreater than I The larger the lmax the greater is the degree of inconsistencySaaty deregnes a consistency index (CI) as (lmax 2 I )=(I 2 1) and provides arandom index (RI) table for matrices of order 3-10 This RI is based on asimulation of a large number of randomly generated weights (Table I)

Saaty recommends the calculation of a consistency ratio (CR) that is the ratioof CI to RI for the same order matrix A CR of 010 or less is consideredacceptable Each manager used Expert Choice (2000) an AHP-based softwareindividually to perform all necessary calculations When the CR wasunacceptable the manager was informed that the pairwise comparisons werelogically inconsistent and was asked to revise hisher Expert Choicejudgments

Numerical exampleTable II provides a numerical example illustrating the calculation of thecomponents of the measures used in this study The illustration assumes thatthere are three critical quality management factors (F1 F2 and F3) four items( f11 f12 f13 and f14) associated with F1 three items ( f21 f22 and f23) associatedwith F2 and four items (f31 f32 f33 and f34) associated with F3 Assume that thedecision maker (DM) has assigned the importance weights of 050 030 and020 to F1 F2 and F3 respectively using AHP and Expert Choice Furthermorethe DM has assigned the following importance weights to the items 040 030020 and 010 to f11 through f14 060 030 and 010 to f21 through f23 and 050020 020 and 010 to f31 through f34 using AHP as well Next assume for thisillustration that the DM used the scale proposed by Saraph et al (1989) to rateboth the ideal quality and the actual quality management of the eight criticalfactors Table II shows the calculations of the TQI for the current period(t = 1)

The TQI of 500 and the TQI1 of 264 in Table II are summary reggures for allthe critical factors and do not specify whether the gap is unacceptably large oracceptably small The total quality index for each critical factor (TQIi) givesmanagers more information about the quality management process in each ofthe critical areas For example health care organizations may experiencecoordination problems between the quality department typically anadministrative function and other departments such as the pathology lab orthe intensive care units These departments generally have their own hierarchywithin the medical staff and do not report directly to administration Thisproblem could be identireged in the model by a relatively signiregcant gap betweenthe actual TQIt

2 and ideal TQI2 for critical factor 2 (the role of the qualitydepartment) for a given time period If the gap is unacceptably large managers

n 3 4 5 6 7 8 9 10RI 058 090 112 132 141 145 149 151

Table IRandom index table

Total qualityindex

513

could be prompted either to form some intervention strategies to narrow it oradjust their belief about the ideal rating

If properly assessed the intervention would center around one or more of theitems associated with the critical factor In the previous example a managercould identify the source of the problem as a lack of coordination between thequality department and other departments Strategies could then be developedthat address this speciregc item A manager could also conclude that noproblems exist with any item or any that do exist are not as signiregcant asindicated by the measured gap of the factor This could lead to a change in howthe manager believes TQIi should be rated What causes managers either todevelop strategies for intervention or to adjust their ideal rating should beexamined over time Movement over time could remacrect interventions made bymanagers or changes in their ideal TQI Further investigation into themanagersrsquo responses to the feedback they receive from TQI tables as well asthe environment could then be undertaken to determine what actually hashappened

The studyA study was conducted to investigate the usefulness of the TQI in differenthealth care settings Four hospitals that are geographically dispersedthroughout the state of New Jersey participated in the study for a period ofone year Each hospital was already active in quality management Topmanagement had stated a commitment to quality management anddepartmental managers were familiar with the basic principles andterminology of quality management Hospitals that were either not involved

Critical factorweights (Fi)

Item weights( fij)

Ideal ratingR

ij

plusmn sup2 Actual ratingR1

ij

plusmn sup2F isbquof ijsbquoR

ij

plusmn sup2F isbquof ijsbquoR

1ij

plusmn sup2

F1= 050 f11= 040 5 3 100 060f12= 030 5 2 075 030f13= 020 5 2 050 020f14= 010 5 3 025 015

TQI1 = 250 TQI11 = 125

F2= 030 f21= 060 5 4 090 072f22= 030 5 2 045 018f23= 010 5 3 015 009

TQI2 = 150 TQI12 = 099

F3= 020 f31= 050 5 2 050 020f32= 020 5 3 020 012f33= 020 5 1 020 004f34= 010 5 2 010 004

TQI3 = 100 TQI13 = 040

TQI = 500 TQI1= 264

Table IIActual and ideal TQI foran example period

BIJ106

514

in quality management or in the beginning stages of involvement wouldrequire additional education and were not chosen for this study

In hospital A the radiology laboratory and pharmacy departmentsparticipated The radiology laboratory pharmacy and environmental servicesdepartments participated from hospital B Hospital C included the nuclearmedicine social services and food and nutrition departments in the studyFinally in hospital D the radiology applied clinical technology respiratorycare and pharmacy departments participated In health care departments aretypically categorized as either clinical or non-clinical Clinical departments aresubjected to broad regulations from local and state licensing authorities as wellas national accrediting organizations In addition clinical departments aretypically those with highly skilled medical professionals that provide directphysiological medical care Non-clinical departments provide care but in a lessinvasive manner The care provided by non-clinical departments is not a part ofthe core physiological medical care This research relies on this generallyaccepted distinction between clinical and non-clinical areas Table III shows theclassiregcation of the participating departments

The participation of all department members was voluntary While they wereinformed about the project and invited to participate by e-mail and inter-ofregcememos no personal persuasion was permitted At the regrst quarter assessment83 percent of non-clinical and 79 percent of clinical personnel participated acrossall four hospital systems There were no apparent differences among the hospitalparticipation rates At the end of the second quarterparticipation increased to 89percent of non-clinical and 92 percent of clinical personnel The participationpercentages remained at these levels for the third and fourth quarters Those notparticipating were primarily newly hired personnel and older departmentmembers who were approaching retirement

Initially all department members staff as well as managers attendedseveral hands-on training sessions to ensure that everyone thoroughlyunderstood the AHP pairwise comparison and Expert Choice Then themembers of each department used Expert Choice to assess the importanceweight (Fi) for each critical factor and the importance weight ( fij) for each item( j) associated with a critical factor (i) During the training sessions theparticipants agreed that they should have an opportunity to review theirjudgments after receiving anonymous feedback containing the judgments ofthe others in their own departments The revision of individual judgments inview of anonymous group feedback is a fundamental principle of the Delphitechnique Delphi was developed at the Rand Corporation to obtain the mostreliable consensus of opinion from a group of knowledgeable individuals aboutan issue not subject to objective solution (Dalkey and Helmer 1963) Thetechnique consists of iterative sequences of collecting judgments from a groupanonymous feedback and individuals reconsidering their judgments In healthcare Delphi has been used to identify issues affecting health care

Total qualityindex

515

Clinic

alN

on-c

linic

al

Hos

pit

als

Rad

iolo

gy

Applied

clin

ical

tech

nol

ogy

Lab

orat

ory

Nucl

ear

med

icin

eR

espir

ator

yca

reSoc

ial

serv

ices

Phar

mac

yE

nvir

onm

enta

lse

rvic

esF

ood

and

nutr

itio

n

A3

plusmn3

plusmnplusmn

plusmn3

plusmnplusmn

B3

plusmn3

plusmnplusmn

plusmn3

3plusmn

Cplusmn

plusmnplusmn

3plusmn

3plusmn

plusmn3

D3

3plusmn

plusmn3

plusmn3

plusmnplusmn

Table IIIMatrix of clinical andnon-clinical cases

BIJ106

516

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 8: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

could be prompted either to form some intervention strategies to narrow it oradjust their belief about the ideal rating

If properly assessed the intervention would center around one or more of theitems associated with the critical factor In the previous example a managercould identify the source of the problem as a lack of coordination between thequality department and other departments Strategies could then be developedthat address this speciregc item A manager could also conclude that noproblems exist with any item or any that do exist are not as signiregcant asindicated by the measured gap of the factor This could lead to a change in howthe manager believes TQIi should be rated What causes managers either todevelop strategies for intervention or to adjust their ideal rating should beexamined over time Movement over time could remacrect interventions made bymanagers or changes in their ideal TQI Further investigation into themanagersrsquo responses to the feedback they receive from TQI tables as well asthe environment could then be undertaken to determine what actually hashappened

The studyA study was conducted to investigate the usefulness of the TQI in differenthealth care settings Four hospitals that are geographically dispersedthroughout the state of New Jersey participated in the study for a period ofone year Each hospital was already active in quality management Topmanagement had stated a commitment to quality management anddepartmental managers were familiar with the basic principles andterminology of quality management Hospitals that were either not involved

Critical factorweights (Fi)

Item weights( fij)

Ideal ratingR

ij

plusmn sup2 Actual ratingR1

ij

plusmn sup2F isbquof ijsbquoR

ij

plusmn sup2F isbquof ijsbquoR

1ij

plusmn sup2

F1= 050 f11= 040 5 3 100 060f12= 030 5 2 075 030f13= 020 5 2 050 020f14= 010 5 3 025 015

TQI1 = 250 TQI11 = 125

F2= 030 f21= 060 5 4 090 072f22= 030 5 2 045 018f23= 010 5 3 015 009

TQI2 = 150 TQI12 = 099

F3= 020 f31= 050 5 2 050 020f32= 020 5 3 020 012f33= 020 5 1 020 004f34= 010 5 2 010 004

TQI3 = 100 TQI13 = 040

TQI = 500 TQI1= 264

Table IIActual and ideal TQI foran example period

BIJ106

514

in quality management or in the beginning stages of involvement wouldrequire additional education and were not chosen for this study

In hospital A the radiology laboratory and pharmacy departmentsparticipated The radiology laboratory pharmacy and environmental servicesdepartments participated from hospital B Hospital C included the nuclearmedicine social services and food and nutrition departments in the studyFinally in hospital D the radiology applied clinical technology respiratorycare and pharmacy departments participated In health care departments aretypically categorized as either clinical or non-clinical Clinical departments aresubjected to broad regulations from local and state licensing authorities as wellas national accrediting organizations In addition clinical departments aretypically those with highly skilled medical professionals that provide directphysiological medical care Non-clinical departments provide care but in a lessinvasive manner The care provided by non-clinical departments is not a part ofthe core physiological medical care This research relies on this generallyaccepted distinction between clinical and non-clinical areas Table III shows theclassiregcation of the participating departments

The participation of all department members was voluntary While they wereinformed about the project and invited to participate by e-mail and inter-ofregcememos no personal persuasion was permitted At the regrst quarter assessment83 percent of non-clinical and 79 percent of clinical personnel participated acrossall four hospital systems There were no apparent differences among the hospitalparticipation rates At the end of the second quarterparticipation increased to 89percent of non-clinical and 92 percent of clinical personnel The participationpercentages remained at these levels for the third and fourth quarters Those notparticipating were primarily newly hired personnel and older departmentmembers who were approaching retirement

Initially all department members staff as well as managers attendedseveral hands-on training sessions to ensure that everyone thoroughlyunderstood the AHP pairwise comparison and Expert Choice Then themembers of each department used Expert Choice to assess the importanceweight (Fi) for each critical factor and the importance weight ( fij) for each item( j) associated with a critical factor (i) During the training sessions theparticipants agreed that they should have an opportunity to review theirjudgments after receiving anonymous feedback containing the judgments ofthe others in their own departments The revision of individual judgments inview of anonymous group feedback is a fundamental principle of the Delphitechnique Delphi was developed at the Rand Corporation to obtain the mostreliable consensus of opinion from a group of knowledgeable individuals aboutan issue not subject to objective solution (Dalkey and Helmer 1963) Thetechnique consists of iterative sequences of collecting judgments from a groupanonymous feedback and individuals reconsidering their judgments In healthcare Delphi has been used to identify issues affecting health care

Total qualityindex

515

Clinic

alN

on-c

linic

al

Hos

pit

als

Rad

iolo

gy

Applied

clin

ical

tech

nol

ogy

Lab

orat

ory

Nucl

ear

med

icin

eR

espir

ator

yca

reSoc

ial

serv

ices

Phar

mac

yE

nvir

onm

enta

lse

rvic

esF

ood

and

nutr

itio

n

A3

plusmn3

plusmnplusmn

plusmn3

plusmnplusmn

B3

plusmn3

plusmnplusmn

plusmn3

3plusmn

Cplusmn

plusmnplusmn

3plusmn

3plusmn

plusmn3

D3

3plusmn

plusmn3

plusmn3

plusmnplusmn

Table IIIMatrix of clinical andnon-clinical cases

BIJ106

516

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 9: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

in quality management or in the beginning stages of involvement wouldrequire additional education and were not chosen for this study

In hospital A the radiology laboratory and pharmacy departmentsparticipated The radiology laboratory pharmacy and environmental servicesdepartments participated from hospital B Hospital C included the nuclearmedicine social services and food and nutrition departments in the studyFinally in hospital D the radiology applied clinical technology respiratorycare and pharmacy departments participated In health care departments aretypically categorized as either clinical or non-clinical Clinical departments aresubjected to broad regulations from local and state licensing authorities as wellas national accrediting organizations In addition clinical departments aretypically those with highly skilled medical professionals that provide directphysiological medical care Non-clinical departments provide care but in a lessinvasive manner The care provided by non-clinical departments is not a part ofthe core physiological medical care This research relies on this generallyaccepted distinction between clinical and non-clinical areas Table III shows theclassiregcation of the participating departments

The participation of all department members was voluntary While they wereinformed about the project and invited to participate by e-mail and inter-ofregcememos no personal persuasion was permitted At the regrst quarter assessment83 percent of non-clinical and 79 percent of clinical personnel participated acrossall four hospital systems There were no apparent differences among the hospitalparticipation rates At the end of the second quarterparticipation increased to 89percent of non-clinical and 92 percent of clinical personnel The participationpercentages remained at these levels for the third and fourth quarters Those notparticipating were primarily newly hired personnel and older departmentmembers who were approaching retirement

Initially all department members staff as well as managers attendedseveral hands-on training sessions to ensure that everyone thoroughlyunderstood the AHP pairwise comparison and Expert Choice Then themembers of each department used Expert Choice to assess the importanceweight (Fi) for each critical factor and the importance weight ( fij) for each item( j) associated with a critical factor (i) During the training sessions theparticipants agreed that they should have an opportunity to review theirjudgments after receiving anonymous feedback containing the judgments ofthe others in their own departments The revision of individual judgments inview of anonymous group feedback is a fundamental principle of the Delphitechnique Delphi was developed at the Rand Corporation to obtain the mostreliable consensus of opinion from a group of knowledgeable individuals aboutan issue not subject to objective solution (Dalkey and Helmer 1963) Thetechnique consists of iterative sequences of collecting judgments from a groupanonymous feedback and individuals reconsidering their judgments In healthcare Delphi has been used to identify issues affecting health care

Total qualityindex

515

Clinic

alN

on-c

linic

al

Hos

pit

als

Rad

iolo

gy

Applied

clin

ical

tech

nol

ogy

Lab

orat

ory

Nucl

ear

med

icin

eR

espir

ator

yca

reSoc

ial

serv

ices

Phar

mac

yE

nvir

onm

enta

lse

rvic

esF

ood

and

nutr

itio

n

A3

plusmn3

plusmnplusmn

plusmn3

plusmnplusmn

B3

plusmn3

plusmnplusmn

plusmn3

3plusmn

Cplusmn

plusmnplusmn

3plusmn

3plusmn

plusmn3

D3

3plusmn

plusmn3

plusmn3

plusmnplusmn

Table IIIMatrix of clinical andnon-clinical cases

BIJ106

516

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 10: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

Clinic

alN

on-c

linic

al

Hos

pit

als

Rad

iolo

gy

Applied

clin

ical

tech

nol

ogy

Lab

orat

ory

Nucl

ear

med

icin

eR

espir

ator

yca

reSoc

ial

serv

ices

Phar

mac

yE

nvir

onm

enta

lse

rvic

esF

ood

and

nutr

itio

n

A3

plusmn3

plusmnplusmn

plusmn3

plusmnplusmn

B3

plusmn3

plusmnplusmn

plusmn3

3plusmn

Cplusmn

plusmnplusmn

3plusmn

3plusmn

plusmn3

D3

3plusmn

plusmn3

plusmn3

plusmnplusmn

Table IIIMatrix of clinical andnon-clinical cases

BIJ106

516

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 11: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

administration (Hudak et al 1993) to assess interventions and policies in themental health industry (Bijl 1992) and to construct a model for project fundingdecisions at the National Cancer Institute (Hall et al 1992)

Next the managers and staff in each department completed the qualitymanagement questionnaire in the Appendix to capture their perceptions of theideal rating R

ij

plusmn sup2and the actual rating Rt

ij

plusmn sup2for each item Again the

department members had the opportunity to review their assessments afterreceiving anonymous feedback about the judgments of others in theirdepartment The assessment of the actual state was repeated for fourconsecutive quarters

The data were collected from the eight clinical and six non-clinicaldepartments and were used to calculate the difference (d t) between the actual(TQIt) and ideal scores (TQI) using the model and equations (1)-(3) At the endof the study the average of all d t s for each department was treated as anobservation Assuming that these observations are two independent randomsamples from two populations with mean and variance m c s2

c

iexcl centfor the

clinical population and m N s2N

iexcl centfor the non-clinical the hypothesis that there

is a difference between the perception of actual and ideal quality managementfor the clinical and non-clinical populations can be stated as

H 0 m c = mN

HA m c sup1 m N

Table IV shows the ideal and actual means on the eight critical factors forclinical and non-clinical departments Two different t-tests were performed onthe data to test for a difference between the means of the clinical andnon-clinical groups The regrst assumes equality of error variances while thesecond allows for the inequality of variances The results of both tests areshown in Table V

Levinersquos test for the equality of variances showed that there is a differencebetween the variances of the clinical and non-clinical groups for factors F1(p-level = 0014) and F2 (p-level = 0032) For these factors t-test 2 was usedbecause the equality of variances could not be assumed For the other sixfactors t-test 1 was used Except for factor F7 the p-level is above the criticallevels for a = 005 and 010 For F7 quality data and reporting the p-level is0011 For this factor the difference between the ideal and actual is higher forthe clinical departments (0254) than the non-clinical (0143) Thus in all thecases except quality data and reporting there is no signiregcant differencebetween the mean scores of the clinical and non-clinical groups[1]

ConclusionIn hospitals the perception of clinical superiority has a long history There is along history of clinical superiority within hospitals This is understandable

Total qualityindex

517

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 12: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

Cri

tica

lfa

ctor

sR

ole

ofdep

artm

enta

lm

anag

emen

tan

dqual

ity

pol

icy

Rol

eof

qual

ity

man

agem

ent

per

sonnel

Tra

inin

gSer

vic

edes

ign

Supplier

qual

ity

man

agem

ent

Pro

cess

man

agem

ent

oper

atin

gpro

cedure

s

Qual

ity

dat

aan

dre

por

ting

Em

plo

yee

rela

tion

s1

23

45

67

8

Clin

ical

Idea

l0

696

069

10

704

044

70

475

062

20

539

081

5A

llquar

ter

aver

age

(act

ual

)0

460

043

70

428

025

50

274

040

50

295

049

1Id

ealac

tual

mea

ndif

fere

nce

023

60

254

027

60

192

020

10

217

024

40

324

Non

-clin

ical

Idea

l1

017

065

50

872

043

00

436

056

30

354

069

8A

llquar

ter

aver

age

(act

ual

)0

699

040

20

497

025

30

268

031

20

210

040

2Id

ealac

tual

mea

ndif

fere

nce

031

80

253

037

50

177

016

80

251

014

40

296

Clinic

aln

on-c

linic

alm

ean

dif

fere

nce

(00

82)

000

1(0

099

)0

015

003

3(0

034

)0

100

002

8

Table IVIdeal vs actual criticalfactor means (clinical vsnon-clinical cases)

BIJ106

518

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 13: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

Lev

inersquo

ste

stfo

req

ual

ity

ofvar

iance

st-

test

for

equal

ity

ofm

eans

Cri

tica

lfa

ctor

t-te

sta

Fp-lev

elt

df

p-lev

elM

ean

bdif

fere

nce

F1

Rol

eof

Dep

M

gt

and

Qual

ity

Pol

icy

18

196

001

4(2

039

)12

00

006

4(0

082

)2

(17

86)

565

012

7F

2R

ole

ofQ

ual

ity

Mgt

Per

sonnel

15

892

003

20

527

120

00

608

000

12

047

66

610

650

F3

Tra

inin

g1

031

80

583

(12

68)

120

00

229

(00

99)

2(1

167

)7

290

280

F4

Ser

vic

edes

ign

11

009

033

50

518

120

00

614

001

52

048

17

590

644

F5

Supplier

qual

ity

man

agem

ent

11

976

018

50

662

120

00

520

003

32

060

36

840

566

F6

Pro

cess

mgt

oper

atin

gpro

cedure

s1

237

20

149

(08

76)

120

00

398

(00

34)

2(0

797

)6

880

452

F7

Qual

ity

dat

aan

dre

por

ting

11

566

023

53

009

120

00

011

010

02

314

111

99

000

9c

F8

Em

plo

yee

rela

tion

s1

165

50

223

074

612

00

047

00

028

20

691

752

051

1

Note

s

aIn

t-te

st1

equal

var

iance

sar

eas

sum

edw

hile

int-

test

2eq

ual

var

iance

sar

enot

assu

med

bcl

inic

alm

inus

non

-clinic

al

c signireg

cant

at( a

=0

05an

d0

10le

vel

ofco

nreg

den

ceplusmn

two

tail

test

)

Table VPairwise t-test forequality of means

(clinical vs non-clinical)

Total qualityindex

519

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 14: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

because clinical functions are the essence of a hospitalrsquos purpose All otherfunctions are considered ancillary This is true even though what drives theindustry is changing from a medical mission to provide care regardless of costto an economic perspective to provide care in the most proregtable waySimilarly the literature makes an overwhelming case for clinical departmentsperforming better in quality management (Anderson and Daigh 1991 Berwick1989 Berwick et al 1990 Dwyer and Amundson 1992 Godfrey et al 1992Ummel 1990)

However in this study the comparison of the ideal and actual qualityassessments reveals no signiregcant difference between the clinical andnon-clinical departments on seven of the eight critical factors in the Saraph et al(1989) instrument These results suggest the extent to which perceptions candiffer from reality The regndings of this study may be attributable to anormalization of the expectations of actual quality management that are thenremacrected in the weights assigned to ideal quality management Theseexpectations may not differ across clinical and non-clinical departments alongthe rating scale used In other words it is possible that managers in each areaadjust their belief of what ideal should be in accordance with what actual isperceived to be The result is a gap whose size is similar for both clinical andnon-clinical areas Another possibility is that there may be at least a portion ofthe gap that remains constant That is there would be some gap under anycircumstances even under the most favorable operating conditions This couldresult from either of two staff positions First a constant gap could imply thatthe staff are never wholly content with their quality management environmentThere may be some aspect of the environment such as employee relationswhere there may always be tension This tension can translate into a gapbetween where the staff would like to be and where they perceive they are at agiven point of time Second this gap may remacrect a situation that is not stableover time There may be shifts in both the belief of what ideal qualitymanagement is and staffsrsquo perceptions of how actual quality management ispracticed These possible explanations represent hypotheses for futureresearch

The statistical analysis shows a difference between the clinical andnon-clinical departments on critical factor F7 quality data and reporting Onthis factor clinical departments have a larger gap than the non-clinical Clinicalemployees have a tradition of data collection and analysis This includesphysicians and nurses who do research as well as other skilled clinicalemployees who must maintain records to satisfy licensing agents or regulatorybodies This ongoing process of data collection and analysis makes clinicaldepartments more acutely aware of weaknesses in actual practice This resultmay also remacrect the absence of a management philosophy that uses theinformation collected in a way that has a positive impact on the quality ofhealth care provided In other words the data cannot be interpreted without a

BIJ106

520

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 15: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

clearly deregned and understood management policy For all of the participatinghospitals there is an explicit commitment to quality management and itsencompassing philosophy Therefore it can be inferred that at least for F7clinical employees either fail to grasp the basic philosophy of qualitymanagement or they choose to neglect it for more traditional approaches

This research indicates that there may be a connection between perceptionsand actual quality management Future research can be done to investigate theinmacruence that perceptions have on actual quality management In thebehavioral sciences it has been asserted that perceptions can be a powerfuldetermining factor of reality It would be useful to examine the nature of thisinmacruence as well as the degree to which perceptions determine actual qualitymanagement Also it would be useful to examine the mindset of employeesregarding their status within hospitals on a clinical versus non-clinicalgrouping along with issues of authority and responsibility The degree towhich perceptions have a role in determining this status can be the subject offuture research

In addition to the speciregc results this study demonstrated that qualityprograms can be structured in a way that overcomes the frequently observedresistance to externally imposed TQM programs and motivate professionals toparticipate in the process The experiences in this study indicate that avoluntary process built on assessment input from health care professionals canachieve the desired outcome of acceptance remacrected in broad participation

Note

1 For factor F1 t-test 1 is signiregcant at a = 010 However the Levine test indicates that theassumption of equality of variances cannot be accepted and t-test 2 should be used Thist-test suggests a non-signiregcant difference at a = 010

References

Albert J et al (1990) ordfReady for quality How one hospital introduced the Deming methodordmHospital Topics Vol 68 No 2 pp 7-10

Anderson CA and Daigh RD (1991) ordfQuality mind-set overcomes barriers to successordmHealthcare Financial Management Vol 45 No 2 pp 20-32

Bartee EM (1973) ordfA holistic view of problem solvingordm Management Science Vol 20pp 439-48

Benson PG Saraph JV and Schroeder RG (1991) ordfThe effects of organizational context onquality management an empirical investigationordm Management Science Vol 37 No 9pp 1107-24

Berwick DM (1987) ordfMonitoring quality in HMOrsquosordm Business and Health Vol 5 No 1 p 10

Berwick DM (1989) ordfContinuous improvement as an ideal in health careordm New England Journalof Medicine Vol 320 No 1 pp 53-6

Berwick DM (1991) ordfBlazing the trail of quality the HFHS quality management processordmFrontiers of Health Services Management Vol 7 No 4 pp 47-50

Total qualityindex

521

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 16: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

Berwick DM Godfrey AB and Roessner J (1990) Curing Health Care New Strategies forQuality Improvement Jossey-Bass San Francisco CA

Bhutta KS and Huq F (1999) ordfBenchmarking plusmn best practices an integrated approachordmBenchmarking Vol 6 No 3 pp 254-68

Bijl R (1992) ordfDelphi in a future scenario study on mental health and mental health careordmFutures Vol 24 No 3 pp 232-50

Brashier LW Sower E Motwani J and Savoie M (1996) ordfImplementation of TQMCQI in thehealth-care industry a comprehensive modelordm Benchmarking for Quality Managementand Technology Vol 3 No 2 p 31

Burda D (1988) ordfProviders look to industry for quality modelsordm Modern Healthcare Vol 18No 29 p 24

Burda D (1991) ordfTotal quality management becomes big businessordm Modern Healthcare Vol 21No 4 p 25

Buterbaugh L (1992) ordfTQM the quality care revolutionordm Medical World News

Camp RC (1989) Benchmarking The Search for Industry Best Practices that Lead to SuperiorPerformance ASQC Press Milwaukee WI

Cyert RM and March JG (1963) A Behavioral Theory of the Firm Prentice-Hall EnglewoodCliffs NJ

Dalkey NC and Helmer O (1963) ordfAn experimental application of the Delphi method to the useof expertsordm Management Science Vol 9 pp 458-67

Deming EW (1982) Quality Productivity and Competitive Position MIT Press CambridgeMA

Dwyer WM and Amundson JA (1992) ordfTQM for suppliersordm in Melum MM and SiniorisMK (Eds) Total Quality Management The Health Care Pioneers American HospitalPublishing pp 367-73

Dyer JS (1990a) ordfRemarks on the analytic hierarchy processordm Management Science Vol 36No 3 pp 249-58

Dyer JS (1990b) ordfA clariregcation of remarks on the analytic hierarchy processordm ManagementScience Vol 36 No 3 pp 274-5

Expert Choice (2000) Decision Support Software McLean VA

Forker L and Mendez D (2001) ordfAn analytical method for benchmarking best peer suppliersordmInternational Journal of Operations and Production Vol 21 p 195

Fried RA (1992) ordfA crisis in health careordm Quality Progress Vol 25 No 4 p 68

Godfrey AB Berwick DM and Roessner J (1992) ordfCan quality management really work inhealth careordm Quality Progress Vol 22 No 4 pp 23-7

Hall NG Hershey JC Kessler LG and Stotts RC (1992) ordfA model for making projectfunding decisions at the National Cancer Instituteordm Operations Research Vol 40 No 6pp 1040-52

Harker PT and Vargas LG (1990) ordfReply to remarks on the analytic hierarchy process by JSDyerordm Management Science Vol 36 No 3 pp 269-73

Hudak RP Brooke PP Jr Finustuen K and Riley P (1993) ordfHealth care administration in theyear 2000 practitionersrsquo views of future health issues and job requirementsordm Hospital andHealth Service Administration Vol 38 No 2 pp 181-95

Kiesler S and Sproull L (1982) ordfManagerial response to changing environments perspectiveson problem sensing from social cognitionordm Administrative Science Quarterly Vol 27pp 548-70

BIJ106

522

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 17: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

Kumar A Motwani J Douglas C and Das N (1999) ordfA quality competitiveness index forbenchmarkingordm Benchmarking Vol 6 pp 112-21

Luthans F and Kreitner R (1985) Organizational Behavior Modiregcation and Beyond Foresmanand Company Glenview IL

Luthans F and Thompson KR (1987) ordfTheory D and OBMod synergistic or oppositeapproaches to performance improvementordm Journal of Organizational BehaviorManagement Vol 9 No 1 pp 105-24

MacCrimmon KR and Taylor RN (1976) ordfDecision making and problem solvingordm inDunnette MD (Ed) Handbook of Industrial and Organizational Psychology RandMcNally Chicago IL pp 1397-453

Orr S Sohal AS Gray K and Harbrow J (2001) ordfThe impact of information technology on asection of the Australian health care industryordm Benchmarking Vol 8 No 2 p 108

Pounds WF (1969) ordfThe process of problem regndingordm Industrial Management Review Vol 11pp 1-19

Prado JC (2001) ordfBenchmarking for the development of quality assurance systemsordmBenchmarking Vol 8 No 1 pp 62-9

Redmon WK and Dickinson MD (1987) ordfA comparative analysis of statistical process controlTheory D and behavior analytic approaches to quality controlordm Journal of OrganizationalBehavior Management and Statistical Process Control Vol 9 No 1 p 52

Reitman WR (1964) ordfHeuristic decision procedures open constraints and the structure ofill-deregned problemsordm in Shelly M II and Bryan GL (Eds) Human Judgements andOptimality Wiley New York NY

Roach SS (1991) ordfServices under siege plusmn the restructuring imperativeordm Harvard BusinessReview Vol 69 No 5 pp 82-91

Saaty TL (1972) An Eigenvalue Allocation Model for Prioritization and Planning EnergyManagement and Policy Center University of Pennsylvania Philadelphia PA

Saaty TL (1977a) ordfA scaling method for priorities in hierarchical structuresordm Journal ofMathematical Psychology Vol 15 pp 234-81

Saaty TL (1977b) ordfModeling unstructured decision problems a theory of analyticalhierarchiesordm Proceedings of the First International Conference on Mathematical Modelingpp 69-77

Saaty TL (1980) The Analytic Hierarchy Process McGraw-Hill New York NY

Saaty TL (1990) Multicriteria Decision Making The Analytic Hierarchy Process RWSPublications Pittsburgh PA

Saaty TL (1994) Fundamentals of Decision Making and Priority Theory with the AnalyticProcess RWS Publications Pittsburgh PA

Saaty TL (1999) Decision Making for Leaders The Analytic Hierarchy Process for Decisions ina Complex World RWS Publications Pittsburgh PA

Saraph JV Benson PG and Schroeder RG (1989) ordfAn instrument for measuring the criticalfactors of quality managementordm Decision Sciences Vol 20 No 4 pp 810-29

Shim JP (1989) ordfBibliographical research on the analytic hierarchy process (AHP)ordmSocio-Economic Planning Sciences Vol 23 pp 161-7

Simpson M and Kondouli D (2000) ordfA practical approach to benchmarking in three serviceindustriesordm Total Quality Management Vol 11 No 4-6 pp 623-30

Skinner BF (1938) The Behavior of Organisms An Experimental Analysis Appleton CenturyCrofts New York NY

Total qualityindex

523

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 18: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

Skinner BF (1966) ordfWhat is the experimental analysis of behaviorordm Journal of ExperimentalAnalysis of Behavior Vol 9 pp 213-8

Ummel SL (1990) ordfTotal quality management healthcarersquos complex and timely choiceordmComputers in Healthcare Vol 11 No 8 p 36

Weiss EN and Rao VR (1987) ordfAHP design issues for large-scale systemsordm Decision SciencesVol 18 pp 43-61

Zahedi F (1986) ordfThe analytical hierarchy process plusmn a survey of the method and itsapplicationsordm Interfaces Vol 16 pp 96-108

Appendix Quality management questionnaire for staff members (departmentmanagers) to assess the current state (ideal state) of quality managementThe purpose of this questionnaire is to assess your perceptions of the extent of effective qualitymanagement in your department The questionnaire captures the most important aspects ofeffective quality management as espoused by the leading practitioners and researchersThis is aconregdential survey Your name is not required to complete it

Please read each statement carefully and circle the number that best describes the currentpractice of quality management within your department (For the department managers thisstatement reworded to focus on the ideal state that best describes the ideal level of qualitymanagement in your department) Answer each statement as accurately as possible andremember that you are assessing your own perceptions of how quality management is practicedin your hospital (Table AI) (For the department managers should be practiced in yourdepartment)

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Role of divisional top management and quality policyExtent to which the top division executive(responsible for division proregt and loss) assumesresponsibility for quality performance 1 2 3 4 5Acceptance of responsibility for quality by majordepartment heads within the division 1 2 3 4 5Degree to which divisional top management (topdivisional executive and major department heads)is evaluated for quality performance 1 2 3 4 5Extent to which the division top managementsupports long-term quality improvement process 1 2 3 4 5Degree of participation by major departmentheads in the quality improvement process 1 2 3 4 5Extent to which the divisional top managementhas objectives for quality performance 1 2 3 4 5Speciregcity of quality goals within the division 1 2 3 4 5Comprehensivenessof the goal-setting process forquality within the division 1 2 3 4 5

(continued )Table AI

BIJ106

524

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 19: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Extent to which quality goals and policy areunderstood within the division 1 2 3 4 5Importance attached to quality by the divisionaltop management in relation to cost and scheduleobjectives 1 2 3 4 5Amount of review of quality issues in divisionaltop management meetings 1 2 3 4 5Degree to which the divisional top managementconsiders quality improvement as a way toincrease proregts 1 2 3 4 5Degree of comprehensiveness of the quality planwithin the division 1 2 3 4 5

Role of the quality departmentVisibility of the quality department 1 2 3 4 5Quality departmentrsquos access to divisional topmanagement 1 2 3 4 5Autonomy of the quality department 1 2 3 4 5Amount of coordination between the quality andother departments 1 2 3 4 5Effectiveness of the quality department inimproving quality 1 2 3 4 5

TrainingSpeciregc work-skills training (technical andvocational) given to hourly employees throughoutthe division 1 2 3 4 5Quality-related training given to hourlyemployees throughout the division 1 2 3 4 5Quality-related training given to managers andsupervisors throughout the division 1 2 3 4 5Training in the ordftotal quality conceptordm (iephilosophy of company-wide responsibility forquality) throughout the division 1 2 3 4 5Training in the basic statistical techniques (suchas histograms and control charts) in the divisionas a whole 1 2 3 4 5Training in advanced statistical techniques (suchas design of experiments and regression analysis)in the division as a whole 1 2 3 4 5Commitment of the divisional top management toemployee training 1 2 3 4 5Availability of resources for employee training inthe division 1 2 3 4 5

(continued ) Table AI

Total qualityindex

525

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 20: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Productservice designThoroughness of new productservice designreviews before the productservice is producedand marketed 1 2 3 4 5Coordination among affected departments in theproductservice development process 1 2 3 4 5Quality of new productsservices emphasized inrelation to cost or schedule objectives 1 2 3 4 5Clarity of productservice speciregcations andprocedures 1 2 3 4 5Extent to which implementationproducibility isconsidered in the productservice design process 1 2 3 4 5Quality emphasis by sales customer servicemarketing and PR personnel 1 2 3 4 5

Supplier quality management (supplier of goods andor services)Extent to which suppliers are selected based onquality rather than price or schedule 1 2 3 4 5Thoroughness of the supplier rating system 1 2 3 4 5Reliance on reasonably few dependable suppliers 1 2 3 4 5Amount of education of supplier by division 1 2 3 4 5Technical assistance provided to the suppliers 1 2 3 4 5Involvement of the supplier in the productdevelopment process 1 2 3 4 5Extent to which longer term relationships areoffered to suppliers 1 2 3 4 5Clarity of speciregcations provided to suppliers 1 2 3 4 5

Process managementoperating proceduresUse of acceptance sampling to acceptreject lots orbatches of work 1 2 3 4 5Amount of preventative equipment maintenance 1 2 3 4 5Extent to which inspection review or checking ofwork is automated 1 2 3 4 5Amount of incoming inspection review orchecking 1 2 3 4 5Amount of in-process inspection review orchecking 1 2 3 4 5Amount of regnal inspection review or checking 1 2 3 4 5Stability of production schedulework distribution 1 2 3 4 5Degree of automation of the process 1 2 3 4 5Extent to which process design is ordffool-proofordm andminimizes the chances of employee errors 1 2 3 4 5Clarity of work or process instructions given toemployees 1 2 3 4 5

(continued )Table AI

BIJ106

526

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527

Page 21: Total quality index: a benchmarking tool for total quality ... · Total quality index: a benchmarking tool for total quality management Madjid Tavana Management Department, La Salle

Rating of current practice in yourdepartment (rating of the ideal level of QM

in your department)Very low Low Medium High Very high

Quality data and reportingAvailability of cost of quality data in the division 1 2 3 4 5Availability of quality data (error rates defectrates scrap defects etc) 1 2 3 4 5Timeliness of the quality data 1 2 3 4 5Extent to which quality data (cost of qualitydefects errors scrap etc) 1 2 3 4 5Extent to which quality data are available tohourly employees 1 2 3 4 5Extent to which quality data are available tomanagers and supervisors 1 2 3 4 5Extent to which quality data are used to evaluatesupervisor and managerial performance 1 2 3 4 5Extent to which quality data control charts etcare displayed at employee work stations 1 2 3 4 5

Employee relationsExtent to which quality circle or employeeinvolvement type programs are implemented inthe division 1 2 3 4 5Effectiveness of quality circle or employeeinvolvement type programs in the division 1 2 3 4 5Extent to which employees are held responsiblefor error-free output 1 2 3 4 5Amount of feedback provided to employees ontheir quality performance 1 2 3 4 5Degree of participation in quality decisions byhourlynon-supervisory employees 1 2 3 4 5Extent to which quality awareness buildingamong employees is ongoing 1 2 3 4 5Extent to which employees are recognized forsuperior quality performance 1 2 3 4 5Effectiveness of supervisors in solvingproblemsissues 1 2 3 4 5 Table AI

Total qualityindex

527