8/2/2019 The Quality of Medical Care_1978

1/9

1 9 7 7 ) . T h e s u r v e y w a s d o n e b y p r i v a t e f i r m s f o rt h e S o c i a l S e c u r i t y A d m i n i s t r a t i o n .3 . T h e o d o s i a n C o d e , C . P h a r r , t r a n s l a t o r ( P r i n c e -t o n U n i v . P r e s s , P r i n c e t o n , N . J . , 1 9 5 2 ) 1 3 . 3 . 1 -1 9 , p p . 3 8 7 - 3 9 0 . I n l a t e Roman l a w , s o m e p h y s i -c i a n s a n d c e r t a i n o t h e r p r o f e s s i o n a l s w e r e e x -e m p t f r o m p u b l i c o b l i g a t i o n s , i n c l u d i n g p a y -i n g t a x e s a n d m i l i t a r y s e r v i c e . S e e a l s o J u s t i n -i a n ' s D i g e s t ( C e n t r a l T r u s t C o . , C i n c i n n a t i ,1 9 3 2 ) , 2 7 . 6 . 2 , v o l . 3 , p . 1 2 3 ; a n d 5 0 . 6 . 6 , v o l . 5 , p .2 3 5 .4 . N o t t i n g h a m E y r e ( 1 3 2 9 ) , f o l i o 2 1 8 ( E g er t o nm a n u s c r i p t N o . 2 8 1 1 , B r i t i s h M u s e u m ) . A l s o A .K . R . K i r a l f y , A S o u r c e B o o k o f E n g l i s h Law( S w e e t a n d M a x w e l l , L o n d o n , 1 9 5 7 ) , p . 1 8 4 .5 . S t r a t t o n v . S w a n l o n d , Y e a r B o o k H i l a r y , 4 8 E d -w a r d I I I , p l e a 1 1 , f o l i o 6 ( H i l a r y , 1 3 7 4 ) . Two r e -l a t e d c a s e s a r e Y e a r B o o k M i c h e l m a s , 4 3 E d -w a r d I I I , p l e a 3 8 , f o l i o 3 3 ; a n d Y e a r B o o k T r i n -i t y , 4 6 E d w a r d I I I , p l e a 1 9 , f o l i o 1 9 . A l l t h r e ea r e i n C . H . S . F i f o o t , H i s t o r y a n d S o u r c e s o ft h e Commpn L a w , T o r t a n d C o n t r a c t ( S t e v e n s ,L o n d o n , 1 9 4 9 ) , p p . 8 1 - 8 3 . T h e c o r r e s p o n d i n gp l e a r o l l i s i n A . K . R . K i r a l f y , A S o u r c e B o o k o fE n g l i s h Law ( S w e e t a n d M a x w e l l , L o n d o n ,1 9 5 7 ) , p p . 1 8 7 - 1 8 8 .6 . G . C l a r k , H i s t o r y o f t h e R o y a l C o l l e g e o f P h y s i -c i a n s o f L o n d o n ( C l a r en d o n, O x f o r d , 1 9 6 4 ) , v o l .1 , p p . 1 4 - 1 8 . S e e a l s o S . W. P . H o l l o w a y ,H i s t o r y 4 9 , 2 9 9 ( 1 9 6 4 ) .7 . P r i v i l e g e s a n d A u t h o r i t y o f P h y s i c i a n s i n L o n -d o n , 4 S t a t u t e s a t L a r g e , 1 4 H e n r y V I I I , c h a p .V , p . 1 5 5 ( 1 5 2 2 - 2 3 ) ; An A c t T o u c h i n g t h e C o r -p o r a t i o n o f t h e P h y s i c i a n s i n L o n d o n , 6 S t a t u t e sa t L a r g e , I M a r y , c h a p . V I , p p . 1 5 - 1 7 ( 1 5 5 3 ) .8 . D r . B o n h a m ' s C a s e , 8 C o k e R e p . 1 1 4 - 1 2 1( 1 6 1 0 ) ; C o l l e g e o f P h y s i c i a n s C a s e , 2 B r o w n l o wa n d G o l d e s . 2 5 6 - 2 6 6 ( 1 6 0 9 ) . S e e a l s o 2 B r o w n -l o w 2 5 5 - 2 6 7 ( 1 6 7 5 ) ; 1 2 8 E n g l . R e p . 9 2 8 - 9 3 4( 1 6 0 9 ) . Bonham w a s s e n t t o p r i s o n o n 7 N o v e m -b e r 1 6 0 6 , b u t w a s o u t w i t h i n a w ee k . T h er e w a sa s e c o n d a c t i o n a g a i n s t h i m i n l a t e 1 6 0 8 . F o r t h eR o y a l C o l l e g e ' s v i e w , s e e G . C l a r k , H i s t o r y o ft h e R o y a l C o l l e g e o f P h y s i c i a n s o f L o n d o n ( C l a -r e n d o n , O x f o r d , 1 9 6 4 ) , v o l . 1 , p p . 2 0 8 - 2 1 4 .9 . 8 C o k e R e p . 1 1 9 - 1 2 0 .1 0 . 8 C o k e R e p . 1 1 8 .1 1 . S e e g e n e r a l l y S . E . T h o r n e , La w Q . R e v . 1 4 , 5 4 3( 1 9 3 8 ) ; T . F . T . P l u c k n e t t , H a r v a r d La w R e v .4 0 , 3 0 ( 1 9 2 6 - 2 7 ) ; J . W. G o u g h , F u n d am e nt a lLa w i n E n g l i s h C o n s t i t u t i o n a l H i s t o r y ( C l a r e n -d on , O x fo r d, 1 9 5 3 ) , p p . 3 0 - 4 7 .1 2 . An A c t t o R e g u l a t e t h e Q u a l i t y o f P r a c t i t i o n e r s

i n M e d i c i n e a n d S u r g e r y , S t a t u t e s o f U n i t e dK i n g d o m , G r e a t B r i t a i n , a n d I r e l a n d , 2 2 V i c t o -r i a , c h a p . XC, p p . 2 9 9 - 3 1 2 ( 1 8 5 8 ) ; An A c t t oAmend t h e M ed i c a l A c t ( 1 8 5 8 ) , S t a t u t e s o f U n i t -e d K i n g d o m , G r e a t B r i t a i n , a n d I r e l a n d , 2 3V i c t o r i a , c h a p . L X V I , p p . 2 5 9 - 2 6 2 ( 1 8 6 0 ) .1 3 . S e e g e n e r a l l y R . H . S h y r o c k , M e d i c a l L i c e n s i n gi n t h e U n i t e d S t a t e s , 1 6 5 0 - 1 9 6 5 ( J o h n s H o p k i n sP r e s s , B a l t i m o r e , 1 9 6 9 ) ; R . C . D e r b y s h i r e ,M e d i c a l L i c e n s u r e a n d D i s c i p l i n e i n t h e U n i t e dS t a t e s ( J o h n s H o p k i n s P r e s s , B a l t i m o r e , 1 9 6 9 ) ,p p . 1 3 - 4 5 .1 4 . T . M . L o g a n , T r a n s . Am . M e d . A s s o c . 2 3 , 4 6 ( 9May 1 8 7 2 ) .1 5 . M i n u t e s o f H o u s e o f D e l e g a t e s , J . Am . M e d . A s -s o c . 7 4 , 1 3 1 9 ( 8 May 1 9 2 0 ) . S ee g e ne ml l y C . B .C h a p m a n a n d J . M . T a l m a d g e , P h a r o s 3 4 , 3 0( 1 9 7 1 ) .1 6 . P r a t t e t a l . v . B r i t i s h M e d i c a l A s s o c i a t i o n e t a l . ,B r i t i s h R u l i n g C a s e s 9 , 9 8 2 - 1 0 2 1 ( 1 9 1 5 ) .1 7 . U n i t e d S t a t e s v . A m e r i c a n M e d i c a l A s s o c i a t i o ne t a l . , 1 1 0 F e d . R e p . 2 n d s e r . 7 0 3 - 7 1 6 ( 1 9 4 0 ) .S e e a l s o M. F i s h b e i n , A H i s t o r y o f t h e A m e r i c a nM e d i c a l A s s o c i a t i o n ( S a u n d e r s , P h i l a d e l p h i a ,1 9 4 7 ) , p p . 5 3 4 - 5 5 0 .1 8 . T h e AMA i s t o d a y o f t e n s e v e r e l y c r i t i c i z e d o nt h e g r o u n d t h a t i t s i n t e r e s t w a s n o t i n i m p r o v i n gt h e t r a i n i n g o f d o c t o r s , b u t w a s r a t h e r i n p r e -s e r v i n g m o n o p o l y a n d d i s c o u r a g i n g c o m p e t i t i o nb y l i m i t i n g t h e s i z e a n d n u m b e r o f A m e r i c a nm e d i c a l s c h o o l s . T h e c r i t i c i s m i s e a s y t o l o d g e ,d i f f i c u l t t o r e f u t e , a n d i m p o s s i b l e t o s u b -s t a n t i a t e .1 9 . P l a t o , T h e S t a t e s m a n , H . N . F o w l e r , T r a n s l .( H ei n em an , L o n do n , 1 9 6 7 ) , p p . 1 4 7 - 1 5 7 .2 0 . E . F r e i d s o n , P r o f e s s i o n o f M e d i c i n e ( H a r p e r& R o w , New Y o r k , 1 9 7 0 ) , p p . 4 2 4 5 .2 1 . D i s s a t i s f a c t i o n w i t h p r o f e s s i o n a l p e r f o r m a n c e i si n h e r e n t i n a n u m b e r o f s t u d i e s , many d o n e b ym e m b e r s o f t h e p r o f e s s i o n : R . H . B r o o k , M . H .B e r g , P . A . S c h e c h t e r , A n n . I n t . M e d . 7 8 , 3 3 3( 1 9 7 3 ) ; C . W. E i s e l e , V . N . S l e e , R . G . H o f f -m a n n , i b i d . 4 4 , 1 4 4 ( 1 9 5 6 ) ; H . M. S o m e r s , M i l -b a n k Mem. F u n d Q . 5 5 , 1 9 3 ( 1 9 7 7 ) ; B . C . P ay n e,J . Am . M e d . A s s o c . 2 0 1 , 1 2 6 ( 1 9 6 7 ) ; J . P . B u n -k e r , B . A . B a r n e s , F . M o s t e l l e r , C o s t s , R i s k sa n d B e n e f i t s o f S u r g e r y ( O x f o r d U n i v . P r e s s ,New Y o r k , 1 9 7 7 ) ; R . S t ev e n s , A m e r i c a n M e d i -c i n e a n d t h e P u b l i c I n t e r e s t ( Y a l e U n i v . P r e s s ,New H a v e n , C o n n . , 1 9 7 1 ) ; 0 . L . P e t e r s o n , J .M e d . E d u c . 3 1 , 1 ( 1 9 5 6 ) ; D . G . W a r r e n , J . L e g .M e d . 2 1 , 2 3 ( 1 9 7 4 ) .2 2 . E . W. H . G r o v e s , B r . M e d . J . 2 , 1 0 0 8 ( 1 9 0 8 ) ; E .

A . C o d m a n , S u r g . G y n e c o l . O b s t e t . 1 8 , 4 9 1( 1 9 1 4 ) ; H . R . K u e h n , B u U l . Am . C o l l . S u r g . S 8 , 7( 1 9 7 3 ) .2 3 . C . C . H a v i g h u r s t , D u k e Law J . ( 1 9 7 5 ) , p p . 1 2 5 6 -1 2 6 3 ; J . O ' C o n n e l l , E n d i n g I n s u l t t o I n j u r y( U n i v . o f I l l i n o i s P r e s s , U r b a n a , 1 9 7 5 ) , p p . 9 7 -1 1 1 ; A m e r i c a n B a r A s s o c i a t i o n , 1 9 7 7 R e po r t o ft h e C o mm i s s i o n o n M e d i c a l P r o f e s s i o n a l L i a b i l -i t y ( A m e r i c a n B a r A s s o c i a t i o n , C h i c a g o , 1 9 7 7 ) ,p p . 9 9 - 1 0 0 .2 4 . 0 . W. H o l m e s , C o m m o n La w ( 1 8 8 1 ) ( H a r v a r dU n i v . P r e s s , C a m b r i d g e , M a s s . , 1 9 6 3 ) , p . 8 9 .2 5 . T h e A m er i c an S u r g i c a l A s s o c i a t i o n t o ok a n i m -p o r t a n t s t e p i n t h i s d i r e c t i o n i n i t s " S t a t e m e n to n p r o f e s s i o n a l l i a b i l i t y " [ N . E n g l . J . M e d . 2 9 5 ,1 2 9 2 ( 1 9 7 6 ) ] .2 6 . J . D . B l u m , P . M. G e r t m a n , J . R u b i n o w , PSROsa n d t h e Law ( A s p e n S y s t e m s , G e r m a n t o w n ,P a . , 1 9 7 7 ) , p p . 1 - 1 7 .2 7 . 4 2 U . S. C o d e A n n o t a t e d , N o . 5 0 1 , c h a p . 7 , s u b -c h a p . x i , 1 3 0 1 - 1 3 2 0 , p p . 3 0 9 - 3 3 3 ( p u b l i c l a w 9 2 -6 0 3 ) ( 1 9 7 2 ) ; s e e a l s o F e d . R e g . 3 8 ( p a r t 2 ) ,3 4 9 4 4 - 3 4 9 5 1 ( 2 0 D e c e m b e r 1 9 7 3 ) .2 8 . P . C a p e r , N . E n g l . J . M e d . 2 9 1 , 1 1 3 6 ( 1 9 7 4 ) .2 9 . H . E . S i m m o n s a n d J . R . B a l l , U n i v . T o l e d oLaw R e v . 6 , 7 3 9 ( 1 9 7 4 ) , p . 7 6 3 .3 0 . E . J . L e v i t , M. S a b s h i n , C . B . M u e l l e r , N . E n g l .J . M e d . 2 9 0 , 5 2 9 ( 1 9 7 4 ) , p . 5 4 5 .3 1 . T h e i n c i d e n c e o f c o m p e n s a b l e i n j u r y i s p r o b a b l yh i g h e r t h a n i t s h o u l d b e . P o c i n c k i e t a l . e s t i -m a t e d t h a t s u c h i n j u r y ( d u e t o n e g l i g e n c e ) o c -c u r s i n a b o u t 2 p e r c e n t o f a l l h o s p i t a l a d m i s -s i o n s , b u t t h a t o n l y I i n 1 7 o f t h o s e i n j u r e d f i l ec l a i m s { L . S . P o c i n k i , S . J . D o g g e r , B . P .S c h w a r t z , i n R e p o r t o f S e c r e t a r y ' s C o m m i s s i o no n M e d i c a l M a l p r a c t i c e [DHEW ( O S ) r e p o r tN o . 7 3 - 8 9 , D e p a r t m e n t o f H e a l t h , E d u c a t i o n ,a n d W e l f a r e , W a s h i n g t o n , D . C . , 1 9 7 3 ] , a p p e n -d i x , p . 5 0 } . I n a m o r e r e c e n t s t u d y i t w a s f o u n dt h a t " p o t e n t i a l l y c o m p e n s a b l e e v en t s " o c c u r r edi n 4 . 7 p e r c e n t o f a d m i s s i o n s t o t h e h o s p i t a l s t h e ys t u d i e d , b u t t h a t i n o n l y a b o u t a f i f t h o f s u c he v e n t s w o u l d c l a i m s h av e b ee n s u c c e s s f u l [ D .H . M i l l s , E d . , M e d i c a l I n s u r a n c e F e a s a b i l i t yS t u d y ( S u t t e r , S a n F r a n c i s c o , 1 9 7 7 ) , p p . 9 6 -1 0 5 ] .3 2 . B . B l e d s t e i n , T h e C u l t u r e o f P r o f e s s i o n a l i s m( N o r t o n , New Y o r k , 1 9 7 6 ) , p . 3 3 4 .3 3 . J . L . B e r l a n t , P r o f e s s i o n a l i s m a n d M o n o p o l y( U n i v . o f C a l i f o r n i a P r e s s , B e r k e l e y , 1 9 7 5 ) , p .3 0 7 .3 4 . E . F r e i d s o n , D o c t o r i n g T o g e t h e r ( E l s e v i e r ,New Y o r k , 1 9 7 5 ) , p . 2 4 9 .

D e f i n i t i o n o f C r i t e r i a a n d S t a n d a r d s

The Q u a l i t y o f M e d i c a l C a r eM e t h o d s f o r a s s e s s i n g a n d m o n i t o r i n g t h e q u a l i t y o fcare f o r r e s e a r c h a n d f o r q u a l i t y a ssura n ce programs.

A v e d i s Donabedian

W e h a v e g r a n t e d t h e h e a l t h p r o f e s -s i o n s a c c e s s t o t h e m o s t s e c r e t a n d s e n s i -t i v e p l a c e s i n o u r s e l v e s , a n d e n t r u s t e d t ot h e m m a t t e r s t h a t t o u c h o n o u r w e l l -b e i n g , h a p p i n e s s , a n d s u r v i v a l . I n r e -t u r n , we h a v e e x p e c t e d t h e p r o f e s s i o n st o g o v e r n t h e m s e l v e s s o s t r i c t l y t h a t we

n e e d h a v e n o f e a r o f e x p l o i t a t i o n o r i n -c o m p e t e n c e . T h e o b j e c t o f q u a l i t y a s -s e s s m e n t i s t o d e t e r m i n e how s u c c e s s f u lt h e y h a v e b e e n i n d o i n g s o ; a n d t h e p u r -p o s e o f q u a l i t y m o n i t o r i n g i s t o e x e r c i s ec o n s t a n t s u r v e i l l a n c e s o t h a t d e p a r t u r ef r o m s t a n d a r d s c an b e d e t e c t e d e a r l y a n dc o r r e c t e d . B u t , f i r s t , we m u s t s p e c i f yw h a t i t i s t h a t i s b e i n g a s s e s s e d a n d m o n -i t o r e d .

0 0 3 6 - 8 0 7 5 / 7 8 / 0 5 2 6 - 0 8 5 6 S 0 2 . 0 0 / o C o p y r i g h t 01 9 7 8 A A A S

F o r t h i s a r t i c l e I s h a l l a s s u m e t h a t t h eo b j e c t o f a s s e s s m e n t a n d m o n i t o r i n g i sm e d i c a l c a r e i t s e l f , w h i c h i s t h e i n t e r -a c t i o n b e t w e e n t h e p h y s i c i a n a n d h i s ( o rh e r ) c l i e n t . T h i s i n t e r a c t i o n i s i t s e l f d i v i s -i b l e i n t o t w o d o m a i n s . O n e i s t h a t o ft e c h n i c a l p e r f o r m a n c e . H e r e , t h e h e a r to f t h e m a t t e r i s t h e a p p l i c a t i o n o f m e d i -c a l k n o w l e d g e a n d t e c h n o l o g y i n a m a n -n e r t h a t m a x i m i z e s i t s b e n e f i t s a n d m i n i -m i z e s i t s r i s k s , t a k i n g a c c o u n t o f t h ep r e f e r e n c e s o f e a c h p a t i e n t . T h e o t h e rd o m a i n i s t h e m a n a g e m e n t o f t h e p e r s o n -a l r e l a t i o n s h i p w i t h t h e p a t i e n t i n a m a n -n e r t h a t c o n f o r m s t o e t h i c a l r e q u i r e -m e n t s , s o c i a l c o n v e n t i o n s , a n d t h e l e g i t i -m a t e e x p e c t a t i o n s a n d n ee d s o f t h e p a -t i e n t .F o r p u r p o s e s o f a s s e s s m e n t t h e d e f i n i -t i o n o f q u a l i t y m u s t b e m a d e p r e c i s e a n do p e r a t i v e i n t h e f o r m o f s p e c i f i c c r i t e r i aa n d s t a n d a r d s . H e r e o n e e n c o u n t e r s af u n d a m e n t a l p r o b l e m . I f q u a l i t y c o n s i s t si n a p r e c i s e a d j u s t m e n t o f c a r e t o t h e p a r -t i c u l a r r e q u i r e m e n t s o f e a c h c a s e , i s i tp o s s i b l e t o f o r m u l a t e d e t a i l e d s p e c i f i c a -

S C I E N C E , VOL. 2 0 0 , 2 6 M a y 1 9 7 8T h e a u t h o r i s p r o f e s s o r o f M e d i c a l C a r e O r g a n i z a -t i o n a t t h e S c h o o l o f P u b l i c H e a l t h , U n i v e r s i t y o fM i c h i g a n , An n A r b o r 4 8 1 0 9 .

8 5 6

8/2/2019 The Quality of Medical Care_1978

2/9

t i o n s o f w h a t c o n s t i t u t e s q u a l i t y t h a t a p -p l y t o g r o u p s o f c a s e s ? M o s t p h y s i c i a n sw o u l d a n s w e r i n t h e n e g a t i v e . T h e yw o u l d i n s i s t t h a t a d e f i n i t i v e a s s e s s m e n to f q u a l i t y m u s t b e b a s e d o n a k n o w l e d g eo f a l l t h e p a r t i c u l a r s i n a c a s e , s o t h a t a na s s e s s o r r e c o g n i z e d t o h a v e s u p e r i o rs k i l l c a n r e c o n s t r u c t i n h i s own m i n d t h ec o n d u c t o f c a r e t h a t h e w o u l d h a v e r e c -ommended u n d e r t h e c i r c u m s t a n c e s .S u c h a s s e s s m e n t s , w h i c h u s e w h a t a r ec a l l e d " i m p l i c i t " c r i t e r i a , a r e , o f c o u r s e ,t i m e - c o n s u m i n g a n d c o s t l y . T h e y a l s ot e n d t o b e u n r e l i a b l e u n l e s s p e r f o r m e db y e x t r e m e l y c o m p e t e n t a n d m o t i v a t e dp h y s i c i a n s who a r e a l s o s k i l l e d i n d o i n ga s s e s s m e n t s . B e s i d e s , t h e q u a l i f i c a t i o n so f a n y a s s e s s o r may b e c h a l l e n g e d . F o rt h e s e r e a s o n s , t h o s e wh o p r o p o s e t ok e e p m e d i c a l c a r e u n d e r c o n s t a n t s u p e r -v i s i o n h a v e r e s o r t e d t o t h e f o r m u l a t i o no f " e x p l i c i t c r i t e r i a " t h a t a r e s u p p o s e dt o r e p r e s e n t a t l e a s t a c c e p t a b l e p r a c t i c e( 1 ) . A t o n e e x t r e m e , t h e s e c r i t e r i a a n ds t a n d a r d s r e p r e s e n t w h a t l e a d i n g e x -p e r t s , u s i n g t h e b e s t s c i e n t i f i c e v i d e n c e ,c o n s i d e r t o b e t h e b e s t p r a c t i c e . A t t h eo t he r e x t r e m e t h e y may b e d e r i v e d f r o mt h e a v e r a g e p r a c t i c e o f p h y s i c i a n s i n ac o m m u n i t y . O b v i o u s l y , t h e s t r i n g e n c ya n d p r e s u m e d v a l i d i t y o f t h e s e t w o f o r -m u l a t i o n s w o u l d b e e x p e c t e d t o b e v e r yd i f f e r e n t a n d , i n p r a c t i c e , a n a t t e m p tmay b e m a d e t o a c c e p t s o m e t h i n g i n t e r -m e d i a t e .T h e i s s u e o f v a l i d i t y i s p a r t i c u l a r l yv e x i n g , n o m a t t e r w h a t k i n d o f c r i t e r i a i su s e d , b e c a u s e n o t e v e r y t h i n g i n m e d i c a lp r a c t i c e i s u n i v e r s a l l y a c c e p t e d o r f u l l ys u b s t a n t i a t e d b y " s c i e n t i f i c " e v i d e n c e .T h i s m e a n s t h a t t h e r e i s a w i d e m a r g i n o fd ou b t a b ou t s o m e o f t h e c r i t e r i a a n ds t a n d a r d s i n a l m o s t a n y f o r m u l a t i o n , a n dp r o v i d e s a n o t h e r r e a s o n why p h y s i c i a n sr e s i s t b e i n g j u d g e d b y c r i t e r i a a n d s t a n -d a r d s o t h e r t h a n t h e i r o w n . W i t h p r e -f o r m u l a t e d e x p l i c i t c r i t e r i a t h e r e i s t h ea d d i t i o n a l d i f f i c u l t y t h a t t h e c r i t e r i a c a n -n o t e a s i l y t ak e a c c o u n t o f t h e v a r i a b i l i t ya m o n g d i f f e r e n t c a s e s . T h i s i s h a n d l e d b ys u b c l a s s i f y i n g c a s e s i n t o r e a s o n a b l y h o -m o g e n e o u s c l a s s e s a n d b y d i v i d i n g t h ec r i t e r i a i n t o t w o t y p e s t h a t o n e m i g h t c a l l" c a t e g o r i c a l " a n d " c o n t i n g e n t . " T h ec a t e g o r i c a l c r i t e r i a a r e l i s t s o f p r o c e -d u r e s t h a t m u s t b e p e r f o r m e d i n e v e r yc a s e b e l o n g i n g t o a c l a s s , o r n e v e r p e r -f o r m e d i n s u c h c a s e s ( 2 ) . T h e c o n t i n g e n tc r i t e r i a a r e l i s t s o f p r o c e d u r e s t h a ts h o u l d b e p e r f o r m e d , o r may b e p e r -f o r m e d , i n some c a s e s b u t n o t i n o t h e r s ,d e p e n d i n g o n t h e n a t u r e a n d c i r c u m -s t a n c e s o f t h e c a s e s . A f u r t h e r r e f i n e -m e n t i s t o s p e c i f y f o r e a c h p r o c e d u r e t h ef r e q u e n c y w i t h w h i c h i t i s e x p e c t e d t ob e p e r f o r m e d i n a " r e p r e s e n t a t i v e "2 6 MAY 1 9 7 8

s a m p l e o f t h e c a s e s i n a n y g i v e n c l a s s ( 3 ) .M o s t s t u d e n t s o f t h e s u b j e c t w o u l da g r e e t h a t e x p l i c i t c r i t e r i a f o r m u l a t e d i nt h i s m a n n e r a r e u s e f u l f o r i d e n t i f y i n gc a s e s t h a t a r e s u s p e c t b e c a u s e o f n o n -c o m p l i a n c e , a n d t h a t t h e d e g r e e o f c o m -

c o m p l e t e , i n f e r e n c e s may b e d r a w n c o n -c e r n i n g " p r o c e s s " b y e x a m i n i n g e i t h e r" s t r u c t u r e " o r " o u t c o m e " ( 4 ) . B y" s t r u c t u r e " I mean t h e m a t e r i a l a n d s o -c i a l i n s t r u m e n t a l i t i e s t h a t a r e u s e d t op r o v i d e c a r e . T h e s e i n c l u d e t h e n u m b e r ,

S u m m a r y . T h i s a r t i c l e c l a s s i f i e s t h e m a j o r a p p r o a c h e s t o t h e a s s e s s m e n t o f t h ep r o c e s s a n d o u t c o m e s o f m e d i c a l c a r e . T h e a p p a r e n t n e e d t o s a f e g u a r d a n d e n h a n c et h e q u a l i t y o f c a r e h a s l e d t o t h e i n s t i t u t i o n o f m e c h a n i s m s t h a t s u b j e c t c a r e t o c o n -s t a n t r e v i e w s o t h a t d e f i c i e n c i e s may b e f o u n d a n d c o r r e c t e d . T h e a r t i c l e r e v i e w s t h ed e v e l o p m e n t s t h a t l e d t o t h e i n v o l v e m e n t o f t h e f e d e r a l g o v e r n m e n t i n t h i s a c t i v i t yt h r o u g h i t s s p o n s o r s h i p o f p r o f e s s i o n a l s t a n d a r d s r e v i e w o r g a n i z a t i o n s ( P S R O ' s ) .T h e m a j o r f e a t u r e s o f t h e P S R O ' s a r e d e s c r i b e d a n d t h e i r p o s s i b l e e f f e c t s d i s c u s s e d .I t i s t o o e a r l y t o s a y how t h e P S R O ' s w i l l f a r e , b u t s h o u l d t h ey f a i l t o a c c o m p l i s h t h e i ro b j e c t i v e s t h e p r e s s u r e f o r m o r e r a d i c a l s o l u t i o n s w i l l b e d i f f i c u l t t o r e s i s t .p l i a n c e i s a r o u g h m e a s u r e o f q u a l i t y .H o w e v e r , m o s t p h y s i c i a n s w i l l i n s i s tt h a t a d e f i n i t i v e j u d g m e n t i n a n y g i v e nc a s e c a n n o t r e s t o n c o m p l i a n c e w i t h c r i -t e r i a t h a t a r e m e a n t t o a p p l y t o t h e " a v -e r a g e c a s e . " I t i s s t i l l n e c e s s a r y t o s u b -j e c t e a c h c a s e o f q u e s t i o n a b l e c a r e t o aj u d g m e n t b y e x p e r t p h y s i c i a n s who a r eg i v e n a l l t h e r e l e v a n t f a c t s a nd e x pe c t e dt o u s e n o t o n l y t h e e x p l i c i t c r i t e r i a b u ta l s o t h e much l a r g e r s e t o f i n t e r n a l i z e di m p l i c i t c r i t e r i a w h i c h g o v e r n s t h e c a r eo f i n d i v i d u a l s i n a l l t h e i r c o m p l e x i t y .I t f o l l o w s t h a t m o s t s y s t e m s f o r m o n -i t o r i n g t h e q u a l i t y o f c a r e e m p l o y a t w o -s t a g e a p p r o a c h : o n e t h a t i d e n t i f i e s c a s e st h a t d o n o t c o n f o r m t o e x p l i c i t c r i t e r i aa n d a n o t h e r t h a t s u b m i t s t h e s e c a s e s t od e t a i l e d r e v i e w b y c o l l e a g u e s , t h a t i s ," p e e r r e v i e w . " R e v i e w e r s f r o m o u t s i d emay b e u s e d i n a d d i t i o n t o o r i n s t e a d o fc o l l e a g u e s w h e n t h e i n i t i a l j u d g m e n t i sc o n t e s t e d , w h e n a n o u t s i d e a g e n c y h a si n i t i a l o r s u p e r v i s o r y r e s p o n s i b i l i t y , o rw h e n r e s e a r c h i s b e i n g d o n e. T hi s c o m -b i n a t i o n o f i n i t i a l s c r e e n i n g f o l l o w e d b yd e t a i l e d r e v i e w , e i t h e r i n t e r n a l o r e x -t e r n a l t o t h e o r g a n i z a t i o n t h a t p r o v i d e sc a r e , m e e t s t h e o b j e c t i v e s o f m o n i t o r i n gw h e n e v e r t h e r e i s t h e w i l l a n d t h e a b i l i t yt o u s e i t p r o p e r l y . I t d o e s n o t , h o w e v e r ,f u l l y m e e t t h e m o r e r i g o r o u s r e q u i r e -m e n t s o f a v a l i d a n d r e l i a b l e j u d g m e n t o nt h e q u a l i t y o f c a r e . F o r t h a t i t i s n e c e s -s a r y t o s p e c i f y i n d e t a i l t h e a p p r o p r i a t es t r a t e g i e s o f c a r e a s j u d g e d b y t h e i r b e n -e f i t s , r i s k s , a n d c o s t s .A p p r o a c h e s t o A s s e s s m e n t

I t may b e i n f e r r e d f r o m t h e a b o v e t h a tq u a l i t y a s s e s s m e n t i s a j u d g m e n t o n t h ep r o c e s s o f c a r e p r o v i d e d b y p r a c t i t i o n e r se i t h e r i n d i v i d u a l l y o r a s a g r o u p . Whend i r e c t i n f o r m a t i o n c o n c e r n i n g t h e p r o -c e s s o f c a r e i s n o t a v a i l a b l e , o r i s i n -

m i x , a n d q u a l i f i c a t i o n s o f t h e s t a f f ; t h em a n n e r i n w h i c h t h e s t a f f i s o r g a n i z e da n d g o v e r n e d ; s p a c e , e q u i p m e n t , a n do t h e r p h y s i c a l f a c i l i t i e s ; a n d s o o n . T h ea s s es s m e n t o f s t r u c t u r e i s a j u d g m e n t o nw h e t h e r c a r e i s b e i n g p r o v i d e d u n d e rc o n d i t i o n s t h a t a r e e i t h e r c o n d u c i v e o ri n i m i c a l t o t h e p r o v i s i o n o f g o o d c a r e .S i n c e t h e r e l a t i o n b e t w e e n s t r u c t u r e a n dp r o c e s s i s p o o r l y u n d e r s t o o d , i n f e r e n c e sd r a w n f r o m t h e f o r m e r c a n b e s e r i o u s l yc h a l l e n g e d . T h e r e a r e s t r o n g e r g r o u n d sf o r u s i n g " o u t c o m e " t o i n d i c a t e t h eq u a l i t y o f a n t e c e d e n t c a r e .T h e o u t c o m e s o f c a r e a r e p r i m a r i l yc h a n g e s i n h e a l t h s t a t u s t h a t c a n b e a t -t r i b u t e d t o t h a t c a r e . A b r o a d e r v i e wi n c l u d e s c h a n g e s i n t h e h e a l t h - r e l a t e dk n o w l e d g e , a t t i t u d e s , a n d b eh a v i o r o ft h e c l i e n t ( 5 ) . H e a l t h s t a t u s c a n i t s e l f b ev i e w e d r a t h e r n a r r o w l y a s p h y s i c a l o rp h y s i o l o g i c a l f u n c t i o n o r , m o r e b r o a d l y ,t o i n c l u d e p s y c h o l o g i c a l f u n c t i o n a n d s o -c i a l p e r f o r m a n c e ( 6 ) . I n f a c t , t h e r e i sm u c h c u r r e n t r e s e a r c h i n t o w a y s o f c o m -b i n i n g a l l t h e s e e l e m e n t s i n t o a s i n g l em e a s u r e t h a t n o t o n l y r e f l e c t s s u r v i v a lb u t a l s o g i v e s a n i n d i c a t i o n o f t h e q u a l i t yo f l i f e ( 7 ) . I f s u c c e s s f u l , s u c h m e a s u r e sw o u l d e x p r e s s t h e q u a l i t y . o f c a r e i nt e r m s o f i t s c o n t r i b u t i o n t o t h e d u r a t i o na n d q u a l i t y o f l i f e . M o r e p r e c i s e l y , t h eq u a l i t y o f c a r e i s p r o p o r t i o n a l t o t h e e x -t e n t t o w h i c h p o s s i b l e i m p r o v e m e n t s i nt h e q u a l i t y o f l i f e a r e a t t a i n e d a s a r e s u l to f t h a t c a r e , w i t h t h e a s s u m p t i o n t h a tc o s t i s n o o b j e c t .I n r e c e n t y e a r s t h i s f o r m u l a t i o n h a sg a i n e d a l a r g e f o l l o w i n g , a n d i t h a s i n -t e n s i f i e d t h e c o n t r o v e r s y b e t w e e n t h o s ew h o e m p h a s i z e t h e a s s e s s m e n t o f p r o -c e s s a n d t h o s e who s w e a r b y o u t c o m e .I n m y o p i n i o n t h i s c o n t r o v e r s y a r i s e sf r o m a m i s c o n c e p t i o n . Q u a l i t y a s s e s s m e n ti s n o t c l i n i c a l r e s e a r c h w h i c h i s d e s i g n e dt o e s t a b l i s h t h e r e l a t i o n s b e t w e e n p r o c e s sa n d o u t c o m e . I t i s a j u d g m e n t o n t h e

8 5 7

http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/

8/2/2019 The Quality of Medical Care_1978

3/9

p r o c e s s o f c a r e t h a t u s e s w h a t i s a l r e a d yk n o w n a b o u t t h o s e r e l a t i o n s , g i v e n t h el i m i t s o f c u r r e n t m e d i c a l s c i e n c e . I t i st r u e t h a t p r o c e s s e l e m e n t s c a n b e u s e d a si n d i c a t o r s o f q u a l i t y o n l y i f t h e r e i s a v a l -i d r e l a t i o n b e t w e e n t h e s e e l e m e n t s a n dd e s i r e d o u t c o m e s . I t i s e q u a l l y t r u e t h a ts p e c i f i c o u t c o m e s c a n b e u s e d a s i n -d i c a t o r s o f t h e q u a l i t y o f c a r e o n l y t o t h ee x t e n t t h a t t h e r e i s a v a l i d r e l a t i o n b e -t w e e n t h e t w o . T h u s , v a l i d i t y r e s i d e s n o ti n t h e c h o i c e o f e l e m e n t s o f p r o c e s s o ro u t c o m e b u t i n w h a t i s known a b o u tt h e i r r e l a t i o n s h i p . I f a v a l i d r e l a t i o n e x -i s t s , e i t h e r may b e u s e d , d e p e n d i n g o nw h i c h c a n b e m o r e e a s i l y a n d a c c u r a t e l ym e a s u r e d ; i t n o t , n e i t h e r c a n b e u s e d .S t u d i e s o f t h e Q u a l i t y o f C a r e

E a c h s t u d y o f q u a l i t y c a n b e c a t e -g o r i z e d i n s o many w a y s , a n d t h e c l u s -t e r i n g s o f a t t r i b u t e s a r e s o i n d i s t i n c t t h a ti t h a s b e e n i m p o s s i b l e t o d e v i s e a s a t i s -f a c t o r y s i m p l e c l a s s i f i c a t i o n . I n t h i s a r -t i c l e I i g n o r e s t u d i e s t h a t r e l y m a i n l y o ne v a l u a t i o n s o f s t r u c t u r e a n d u s e t h ec l a s s i f i c a t i o n g i v e n i n T a b l e 1 f o r t h e r e -m a i n d e r . A b r i e f r e v ie w o f s e l e c t e d s t u d -i e s d r a w n f r o m t h i s c l a s s i f i c a t i o n c a n i l -l u s t r a t e a n d r a i s e q u e s t i o n s a b o u t s p e c i f -i c m e t h o d s o f a s s e s s m e n t , a s w e l l a s p r o -v i d e i n f o r m a t i o n a b o u t s o m e f a c t o r s t h a ti n f l u e n c e p e r f o r m a n c e . B u t , b e c a u s es o m e o f t h e s e s t u d i e s a r e o l d , a n d a l m o s ta l l h a v e e x a m i n e d h i g h l y c i r c u m s c r i b e ds i t u a t i o n s , t h e o n l y c o n c l u s i o n t h a t c a nb e d r a w n a b o u t l e v e l s o f q u a l i t y i n g e n e r -a l i s t h a t w h e n e v e r t h e q u a l i t y o f c a r eh a s b e e n e x a m i n e d , s e r i o u s a n d w i d e -s p r e a d d e f i c i e n c i e s h a v e b ee n f o u n d .T h i s may b e a c h a r a c t e r i s t i c o f a l l h u m a ne n d e a v o r - t h a t i s , i f s u f f i c i e n t l y s t r i c ts t a n d a r d s a r e u s e d we s h a l l a l l b e f o u n dt o h a v e f a i l e d i n some d e g r e e . I t i s c e r -t a i n l y s o f o r t h e p e r f o r m a n c e o f p h y s i -c i a n s .As t o t h e p r e v a i l i n g l e v e l s o f q u a l i t y i nt h e U n i t e d S t a t e s o r e l s e w h e r e , we h a v et o r e l y o n g r o s s m e a s u r e s o f l o n g e v i t y ,m o r t a l i t y , m o r b i d i t y , t h e u s e a n d d i s t r i -b u t i o n o f s e r v i c e s , t h e f r e q u e n c y o f s u r -g i c a l o p e r a t i o n s , a n d t h e l i k e . B u t t h e s ep h e n o m e n a , t h o u g h i m p o r t a n t , a r e i n f l u -e n c e d b y s o many u n e x a m i n e d v a r i a b l e st h a t i t w o u l d b e f o o l h a r d y t o u s e t h e mf o r c o n f i d e n t a s s e r t i o n s .S t u d i e s o f t h e P r o c e s s o f C a r e

T h e r e p u t a t i o n s o f p h y s i c i a n s amongt h e i r c o l l e a g u e s a r i s e t o a l a r g e e x t e n tf r o m t h e o p p o r t u n i t i e s t h a t t h e y h a v e t oo b s e r v e e a c h o t h e r a t w o r k . T h e o p e n -8 5 8

n e s s o f p r a c t i c e t o s u c h o b s e r v a t i o n i s , i nf a c t , a m a j o r s a f e g u a r d , a n d a c o g e n t a r -g u m e n t i n f a v o r o f o r g a ni z e d f o r m s o fp r a c t i c e . I t i s i n t e r e s t i n g , t h e r e f o r e , t of i n d t h e f i r s t i m p o r t a n t u s e o f d i r e c t , f o r -m a l o b s er v a t i o n w i t h a v i e w t o a s s e s s i n gt h e q u a l i t y o f c a r e i n a s t u d y o f r u r a l g e n -e r a l p r a c t i c e , t h a t m o s t i s o l a t e d a n d s e -c r e t c o r n e r o f m e d i c i n e- l an d ( 8 ) . T h em e t h o d u s e d w a s t o h a v e a q u a l i f i e d p h y -s i c i a n , w i t h t h e p e r m i s s i o n o f h i s h o s t ,o b s e r v e t h e l a t t e r a s h e c a r e d f o r p a t i e n t swho w e r e m a k i n g t h e f i r s t v i s i t f o r a newi l l n e s s . I n t h i s way i t w a s p o s s i b l e f o r t h eo b s e r v e r t o m a k e a j u d g m e n t a b o u t t h ec o m p l e t e n e s s o f t h e e x a m i n a t i o n , t h e a p -p r o p r i a t e n e s s o f f u r t h e r i n v e s t i g a t i o n ,a n d t h e s u i t a b i l i t y o f t r e a t m e n t . As a r e -s u l t , 2 5 p e r c e n t o f p r a c t i t i o n e r s w e r er a t e d s u p e r i o r o r g o o d , w h e r e a s 4 4 p e r -c e n t w e r e j u d g e d t o b e b e l o w a n " a v e r -a g e " o r a c c e p t a b l e l e v e l . T h e b e t t e rp r a c t i t i o n e r s w e r e m o r e l i k e l y t o b ey o u n g e r , t o s e e p a t i e n t s b y a p p o i n t m e n t ,a n d t o h a v e a c c e s s t o l a b o r a t o r y s e r -v i c e s ; b u t , a b o v e a l l , t h e y w e r e m o r el i k e l y t o h a v e h ad a p e r i o d o f t r a i n i n g i ni n t e r n a l m e d i c i n e s u b s e q u e n t t o g r a d u -a t i o n f r o m m e d i c a l s c h o o l . A l l t h e s e a r es t r u c t u r a l c h a r a c t e r i s t i c s c o n d u c i v e t ob e t t e r c a r e , t h o u g h t h e y d o n ot a s s u r e i t .O t h e r s t u d i e s u s i n g t h e s a m e a p p r o a c hs u g g e s t t h a t g e n e r a l p r a c t i c e i n o t h e rc o u n t r i e s may s u f f e r f r o m s i m i l a r c h a r a c -t e r i s t i c s a n d h a n d i c a p s ( 9 ) .

T h a t t h e o b s e r v at i o n o f p r a c t i c e i s am e t h o d w i t h w i d e r a p p l i c a b i l i t y i s s h o w nb y a s t u d y o f t h e i n t e r a c t i o n b e t w e e nn u r s i n g p e r s o n n el a n d r a n d o m l y s a m p l e dp a t i e n t s i n s e l e c t e d h o s p i t a l s i n t h e D e -t r o i t a r e a ( 1 0 ) . M o r e i n t e r e s t i n g t h a n t h el e v e l s o f p e r f o r m a n c e r e v e a l e d w e r e t h ef i n d i n g s s u g g e s t i v e o f d i f f e r e n c e s r e l a t e dt o t h e c h a r a c t e r i s t i c s o f p a t i e n t s . A s -p e c t s o f n u r s i n g c a r e t e n d e d t o b e l e s ss a t i s f a c t o r y f o r n o n w h i t e s , f o r p a t i e n t si n w a r d s w i t h many b e d s , f o r t h o s e whoh a d c a n c e r w i t h a p o o r p r o g n o s i s , f o ry o u n g e r f e m a l e s a n d f o r o l d e r m a l e s . B e -c a u s e o f t h e n a t u r e o f t h i s s t u d y t h e s ef i n d i n g s c a n n o t b e a c c e p t e d a s c o n c l u -s i v e , b u t t h e y d o i l l u s t r a t e a p r o b l e m o fg r e a t s o c i a l s i g n i f i c a n c e : T h e e x t e n t t ow h i c h t h e q u a l i t y o f c a r e may d i f f e r a c -c o r d i n g t o t h e s o c i a l o r e c o n o m i c c h a r a c -t e r i s t i c s o f c l i e n t s e i t h e r b e c a u s e t h es o u r c e s o f c a r e a r e d i f f e r e n t o r b e c a u s et h e s a m e s o u r c e s a r e g u i l t y o f d i s c r i m i -n a t o r y b e h a v i o r .T h e d i r e c t o b s e r v a t i o n o f p r a c t i c e i s ,o f c o u r s e , c o s t l y a n d t i m e - c o n s u m i n g . I tmay a l s o a l t e r t h e b e h av i o r b ei n g o b -s e r v e d , e x c e p t t h a t t h o s e w h o h a v e u s e di t s a y t h a t v e r y s o o n t h e p r e s e n c e o f t h eo b s e r v e r i s f o r g o t t e n a n d t h e s u b j e c tl a p s e s i n t o h i s u s u a l r o u t i n e . T h e a n a l y -

s i s o f m e d i c a l r ec o r ds i s l e s s o b t r u s i v ea n d m o r e e a s i l y s u b j e c t t o c h e c k i n g b ys e v e r a l j u d g e s , b u t i t s u f f e r s f r o m t h e l i m -i t a t i o n s i n t h e c o m p l e t e n e s s a n d v e r a c i t yo f t h e r e c o r d , e s p e c i a l l y i n o f f i c e p r a c -t i c e . T h i s h a s l e d t o c r i t i c i s m o f t h i sm e t h o d f o r b e i n g a n a s s e s s m e n t o f r e -c o r d i n g r a t h e r t h a n o f c a r e . T h i s h a sb e e n c o u n t e r e d b y t h e a r g u m e n t t h a t r e c -o r d i n g i s a n i m p o r t a n t e l e m e n t i n c a r ea n d t h a t t h e r e i s a n a s s o c i a t i o n b e t w e e nt h e q u a l i t y o f r e c o r d i n g a n d t h e q u a l i t yo f c a r e ( 1 1 ) .T h e a n a l y s i s o f t h e r e c o r d o f c a r ev a r i e s g r e a t l y i n b r e a d t h a n d d e t a i l . A to n e e x t r e m e a l l t h a t i s s o u g h t i s i n f o r m a -t i o n a b o u t a s m a l l n u m b e r o f c r i t i c a l e l e -m e n t s t h a t a r e i m p o r t a n t i n t h e m s e l v e sa n d w h i c h may a l s o b e t ak e n a s r e p r e -s e n t a t i v e o f a s p e c t s o f c a r e t h a t a r e n o td i r e c t l y o b s e r v e d . T h e s e c r i t i c a l e l e -m e n t s o r i n d e x e s c a n b e f o r m u l at e d s ot h a t t h e y a r e a p p l i c a b l e t o a l l p a t i e n t s o rt o s u b g r o u p s o f p a t i e n t s c h a r a c t e r i z e db y a g e , s e x , d i a g n o s i s , a n d t h e l i k e . F o re x a m p l e , i n t h e r e c o r d s o f o f f i c e c a r e o n ec a n l o o k f o r t h e f r e q u e n c y w i t h w h i c hb l o o d p r e s s u r e s a r e m e a s u r e d ; r e c t a l a n dv a g i n a l e x a m i n a t i o n s a r e d o n e ; t h e e y e -g r o u n d s a n d e a r s e x a m i n e d w i t h t h e a p -p r o p r i a t e i n s t r u m e n t s ; i n f a n t s a r e i m m u -n i z e d ; c h i l d r e n w i t h s o r e t h r o a t s h a v e at h r o a t c u l t u r e f o r s t r e p t o c o c c i ; p r e g n a n twomen h a v e t h e i r u r i n e t e s t e d ; s e d a -t i v e s , t r a n q u i l i z e r s , a n d a n t i b i o t i c s a r ep r e s c r i b e d ; a n d i n j e c t i o n s a r e g i v e nw h e n t h e d r u g s c o u l d h av e b e e n t a k e n b ym o u t h ( 1 2 ) . H o s p i t a l r e c o r d s o f f e r o p -p o r t u n i t i e s f o r t h e c o n s t r u c t i o n o f muchl a r g e r l i s t s o f s u c h i n d i c a t o r s w i t h g r e a t -e r a s s u r a n c e t h a t t h e n e c e s s a r y i n f o r m a -t i o n i s i n t h e r e c o r d ( 1 3 ) . A f a v o r i t e t y p eo f s l e u t h i n g i s t o l o c a t e r e p o r t s o f a b n o r -m a l l a b o r a t o r y f i n d i n g s w h i c h p h y s i c i a n sa g r e e r e q u i r e a t t e n t i o n , a n d t o d e t e r m i n eh ow o f t e n t h e s e g o u n n o t i c e d , a r e i g -n o r e d , o r a r e d e a l t w i t h i n a d e q u a t e l y .F o r e x a m p l e , i n t h e g e n e r a l c l i n i c o f o n eu n i v e r s i t y h o s p i t a l a b o u t a f i f t h o f s u c ha b n o r m a l i t i e s w e r e n o t f o l l o w e d u p ( 1 4 ) ;a n d i n o n e c o m m u n i t y h o s p i t a l m o r et h a n h a l f o f t h e a b n o r m a l f i n d i n g s w e r ee i t h e r i g n o r e d o r i n a d e q u a t e l y h a n d l e d( 1 5 ) . I n g e n e r a l , w h e n t h e r e s u l t s o f i n -v e s t i g a t i o n s t h a t a t t e m p t t o c h a r a c t e r i z ec r i t i c a l e l e m e n t s o f p r a c t i c e a r e a s s e m -b l e d , i t i s a s t o u n d i n g how v a r i a b l e p r a c -t i c e i s f o u n d t o b e , a n d how o f t e n i ts e e m s t o d e p a r t f r o m s t a n d a r d s o f s u p -p o s e d l y g o o d c a r e .D e v e l o p m e n t s i n d a t a a c q u i s i t i o n a n dp r o c e s s i n g h a v e s t i m u l a t e d t h e u s e o ft h i s a p p r o a c h t o a s s e s s m e n t a n d m o n -i t o r i n g , a n d g r e a t l y a m p l i f i e d i t s u s e f u l -n e s s . D at a f r o m r e c o r d s o f a m b u l a t o r yc a r e , a b s t r a c t s o f h o s p i t a l c h a r t s , a n d t h e

S C I E N C E , VOL. 2 0 0

http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sciencemag.org/http://www.sci