operations research results - shops plus project · the operations research results series presents...

Q U A L I T Y

A S S U R A N C E

P R O J E C T

Center for Human Serv ices • 7200 Wiscons in Avenue, Suite 600 • Bethesda , MD 20814-4811 • USA

O P E R A T I O N S R E S E A R C H R E S U L T S

Improving Provider-ClientCommunication: Reinforcing IPC/C

Training in Indonesia with Self-Assessment and Peer Review

The Quality Assurance Project (QAP) is funded by the U.S. Agency for International Development(USAID), under Contract Number HRN-C-00-96-90013. QAP serves countries eligible for USAIDassistance, USAID Missions and Bureaus, and other agencies and nongovernmental organizationsthat cooperate with USAID. The QAP team consists of prime contractor Center for Human Services(CHS), Joint Commission Resources, Inc. (JCI), Johns Hopkins University School of Hygiene andPublic Health (JHSPH), Johns Hopkins Center for Communication Programs (JHU/CCP), and theJohns Hopkins Program for International Education in Reproductive Health (JHPIEGO). It providescomprehensive, leading-edge technical expertise in the design, management, and implementationof quality assurance programs in developing countries. CHS, the nonprofit affiliate of UniversityResearch Co., LLC (URC), provides technical assistance and research for the design, management,improvement, and monitoring of healthcare systems and service delivery in over 30 countries.

Improving Provider-Client Communication:Reinforcing IPC/C Training in Indonesiawith Self-Assessment and Peer Review

O P E R A T I O N S R E S E A R C H R E S U L T S

Continued on page ii

Table of Contents

I. BACKGROUND: QUALITY IMPROVEMENT IN INDONESIA .............................. 1

II. METHODS .......................................................................................................... 2

A. Research Design ..................................................................................... 2

B. Data collection ......................................................................................... 3

C. Site profile ................................................................................................ 4

D. Service provider and client profiles ........................................................ 5

III. RESULTS ........................................................................................................... 5

A. Impact of training on provider performance and client communication .. 6

B. Impact of reinforcement activities .............................................................. 7

C. Comparing self-assessment and peer review ........................................... 9

D. Cost analysis ............................................................................................ 11

IV. DISCUSSION AND IMPLICATIONS ................................................................. 13

A. The value of IPC/C training and reinforcement activities ........................ 13

B. Successfully implementing self-assessment and peer review ............... 14

REFERENCES ....................................................................................................... 17

Abstract

To improve the quality of reproduc-tive healthcare in Indonesia,refresher training in interpersonalcommunication and counseling(IPC/C) has been offered toclinic-based service providerswho attend family planning clients.This study tested the effectivenessand feasibility of two low-costalternatives to supervision—self-assessment and peerreview—that may reinforceproviders’ skills after training.

The performance of three groupsof providers in East Java andLampung Provinces was com-pared. All 203 providers attendedan IPC/C training workshop in1997–98, after which they weredivided into three follow-up groupsby district. The control groupreceived no reinforcement aftertraining. Providers in the self-as-sessment only group conductedself-assessment exercises for 16weeks after training. Providers inthe peer review with self-assess-ment group attended peer reviewmeetings as well as conductingself-assessment exercises overthe same 16-week period.

Reinforcement activities boostedprovider-client interaction over thefour-month follow-up period evenas the length of consultations grewshorter. Provider facil itativecommunication increased from28 percent to 35 percent in thetwo reinforcement groups (the self-assessment only group and the

Abstract ContinuedContinuedContinuedContinuedContinued

peer review with self-assessmentgroup), while it declined from 29percent to 27 percent in the controlgroup. Self-assessment and peer re-view did not have a similar positiveimpact on information giving. Bothtypes of reinforcement proved feasiblefor a low-resource setting. Cost analy-sis showed that money spent on train-ing alone, without reinforcement,had minimal impact on providerperformance. Adding peer review toself-assessment proved cost-effectivedespite its relatively higher cost.

When providers returned to their homeclinics after training, self-assessmentand peer review helped them consoli-date their newly learned skills byfocusing their attention on importantissues, clarifying performance stan-dards, helping them identify weak-nesses, and motivating them to dobetter. The reinforcement strategiesalso taught providers how to workmore efficiently so that they were ableto maintain the quality of the interac-tion while shortening the duration ofthe session. In addition, the reinforce-ment strategies served as a mecha-nism of ongoing quality improvement,encouraging providers to continuestrengthening their skills beyond post-training levels. Results also confirmthat interventions directed to provid-ers can, indirectly, influence clientbehavior. Changes in provider com-munication behavior elicited moreactive communication from clients,probably as a result of increasedrapport between providers andclients. Lessons learned from thisstudy point to simple, affordable strat-egies to maximize the impact of costlytraining courses.

Acknowledgements

This paper was written by Young Mi Kim and Fitri Putjuk of JHU/CCPand consultants Adrienne Kols and Endang Basuki. The authors aregrateful to the service providers and the clients for their participationin the study interventions and their willingness to be audiotaped. ManyBKKBN staff members contributed to the success of the project,including Pudjo Rahardjo, H. Slamet Tjiptoraharjo, Djoko Rusmoro, M.Yatim Hamzah, Mastoni Sani, Sugiri Syarief, and Maman Sundjana.

The authors also wish to thank the field research team, Erlaini Uhanda,Emiria Sirman, Nurfina R. Bachtiar, H.R. Soedibyo, H.J. Mukono, H.M.Sulaksmono, Muchsin Badar, K. Wiryoseputro, and A. KantanAbdullah, and the administrative support and assistance of JHU/CCP/Indonesia staff, including Endang Iradati, Eugenita Garot (Nita), andChristiana Tri Desintawati (Sinta).

Technical comments were provided by Paul Richardson, BartonBurkhalter, Paula Tavrow, David Nicholas, Jim Heiby, and HanyAbadallah of QAP; Izhar M. Fihir, AVSC International; Yudo Priharto,YKB; and Russ Vogel, JHPIEGO. The IPC/C training benefited from theinput of Lucy Mize, Ninuk Widiyantoro, Ina Saraswati, Agustine S.Basri, Retno Asmanu, Zarni Amri, and Setiawati.

Project coordination was provided by Anne Palmer. Gary Lewis, J.Douglas Storey, Kirstin Bose, Bruce Morén, and Sung Hee Yun of JHU/CCP, Sharon Rudy (now at INTRAH), Debra Roter, and Susan Larsonof JHSPH provided technical guidance and facilitation for the project.The authors also thank Leslie B. Curtin, Lana Dakan, and BambangSamekto of USAID/Indonesia for their support of the project.

Recommended citation

Kim, Y.M., F. Putjuk, A. Kols and E. Basuki. 2000. Improving provider-client communication: Reinforcing IPC/C training in Indonesia withself-assessment and peer review. Operations Research Results 1(6).Published for the United States Agency for International Development(USAID) by the Quality Assurance Project (QAP): Bethesda, Maryland.

About this series

The Operations Research Results Series presents the results of QAPcountry or area research to encourage discussion and comment withinthe international development community. If you would like to obtainthe full research report of this study containing all relevant datacollection instruments, please contact [email protected].

Improving Provider-Client Communication in Indonesia ■ 1

Abbreviations

BKKBN State Ministry of Population/National Family PlanningCoordinating Board

COPE Client-Oriented,Provider-Efficient

IPC/C Interpersonal Communicationand Counseling

JHU/CCP Johns Hopkins University/Center for CommunicationPrograms

RIAS Roter Interaction AnalysisSystem

SDES Service Delivery ExpansionSupport Project

UNFPA United Nations PopulationFund

USAID U.S. Agency for InternationalDevelopment

Young Mi Kim, Fitri Putjuk, Adrienne Kols, and Endang Basuki

Improving Provider-Client Communication:Reinforcing IPC/C Training in Indonesiawith Self-Assessment and Peer Review

I. Background:Quality Improvement inIndonesia

Although contraceptive use inIndonesia is high at 55 percent, soare discontinuation rates: Fullyone-fourth of couples who adopt acontraceptive method discontinueits use within a year. At the sametime, pregnancy and childbirthcontinue to be major causes ofdeath among women of reproductiveage (Central Bureau of Statistics etal. l995). In response, the StateMinistry of Population/NationalFamily Planning Coordinating Board(BKKBN) is leading a nationalinitiative to improve the quality ofreproductive healthcare at thepuskesmas, or community healthclinic, level.

A chief component of this qualityimprovement initiative is refreshertraining in interpersonal communica-tion and counseling (IPC/C) forhealth workers who attend familyplanning clients. Good quality IPC/C

has proven to increase clientcompliance, promote contraceptivecontinuation, and improve healthoutcomes in a variety of healthcaresettings (Pariani et al. 1991; Kim etal. 1992; Abdel-Tawab 1995; Ong etal. 1995; Stewart 1996; Clark et al.1998; Roter and Hall 1998). Clientparticipation is especially essentialduring family planning consultations,because clients must discuss theirpersonal needs and priorities withthe provider and understand theiroptions in order to make an informedchoice of contraceptive methods.Yet formative research in Indonesiadocumented multiple weaknesses inthe interaction between clients andproviders in Central and West Java(JHU/CCP 1998). Clinic-basedproviders did not always explaintechnical matters accurately, clearly,and completely, and clients played apassive role, volunteering littleinformation (Kim et al. 1997).

Therefore, BKKBN in collaborationwith the Johns Hopkins University(JHU/CCP) developed new nationalIPC/C curricula for field workers andclinic-based workers to promotegreater dialogue between providerand client. The curricula wereemployed in 1997–98 when BKKBNconducted extensive refreshertraining for service providers in 13provinces with support from theUnited Nations Population Fund(UNFPA) and the U.S. Agency for

2 ■ Improving Provider-Client Communication in Indonesia

International Development (USAID)through the Service DeliveryExpansion Support Project (SDES).

Research has found that providersneed reinforcement after training ifthey are to apply and consolidatetheir newly acquired skills as part oftheir daily work (Best 1998; Kols andSherman 1998). Individual serviceproviders find it difficult to carry newskills learned in the supportiveenvironment of a workshop back totheir home clinics because cowork-ers are skeptical, needed materialsor equipment are lacking, providersdo not understand which behaviorsto change without regular feedbackon their performance, and/orproviders do not feel motivated tochange old habits (Garavaglia 1995;Sullivan and Smith 1996).

BKKBN searched for low-cost, low-maintenance forms of reinforcementto ensure that providers whoattended the IPC/C workshops usedtheir new skills after they returned totheir home clinics. Self-assessmentand peer review are affordable,sustainable, and potentially self-empowering alternatives to supervi-sion that can help providers applynewly learned skills on the job.Self-assessment calls on individualproviders to judge their own jobperformance against a set of outsidestandards. In peer review, smallgroups of providers give oneanother feedback and shareexperiences and ideas. Bothapproaches rely on self-adminis-tered tools (individual self-assess-ment questionnaires and groupdiscussion guides) that are espe-cially developed for these purposes.

A self-learning process is built intoboth self-assessment and peerreview to help providers changetheir behavior. After identifying their

weaknesses, providers set personalgoals for behavior change, try outnew behaviors, and assess theoutcomes of their efforts. This is acontinuous process in whichproviders establish new goals andrepeat the learning cycle until theyare satisfied with their performance.The success of self-assessment andpeer review depends on providers’own motivation, ability, and diligencein completing the tasks required,since there is little or no outsidesupervision.

While research on whether self-assessment and peer reviewimproves health workers’ perfor-mance is limited and inconclusive,studies suggest these interventionsdo have the potential to improvehealthcare providers’ communica-tion skills (Calhoun et al. 1990;Gordon 1992; Fincher and Lewis1994; Sobral 1994; Kaiser and Bauer1995; Roberts et al. 1997). An earlierIndonesian program tested theeffectiveness of peer assessmentby sending trained midwives toobserve, assess, and give directfeedback to their colleagues. Anevaluation found that this interven-tion enhanced midwives’ interper-sonal and clinical skills (MacDonald1995). Some of the key elements ofthe self-assessment and peer reviewapproaches also have been testedas part of the Client-Oriented,Provider-Efficient (COPE) interven-tion, in which clinic staff work as agroup to assess and solve prob-lems. Results from sub-SaharanAfrica and Asia suggest thatproviders are intrinsically motivatedto offer better services, considerthemselves responsible for self-improvement, and can continue tolearn through group self-assessmentand support (Lynam et al.1993;Beattie et al. 1994).

This study tests whether self-assessment and peer review aresustainable, cost-effective alterna-tives to supervision and refreshertraining for maintaining and improv-ing providers’ IPC/C skills. Unlikeprior studies, self-assessment andpeer review were the sole reinforce-ment mechanisms: Providers did notbenefit from continuing supervisionor receive refresher training.

II. Methods

A. Research designThis study employed a prospective,quasi-experimental research design.The investigators purposefullyselected three districts in each oftwo provinces, East Java andLampung, that were comparable insize, proximity to the provincialcapital, and clients’ socioeconomiccharacteristics. Within each district,clinics were randomly chosen for thestudy. In most clinics, the singleprovider responsible for familyplanning services was asked toparticipate. In large clinics, twoproviders were invited to participate.

All of the participating serviceproviders had attended a five-dayIPC/C workshop as part of a nationalintervention conducted by BKKBNwith funding from UNFPA andUSAID. After training, the serviceproviders were assigned to one ofthree groups (control, self-assess-ment only, or peer review with self-assessment) by district, that is, onedistrict from each province wasassigned to each study condition.Table 1 outlines the interventions ineach group. The control groupreceived training, but no follow-up.In contrast, the other two groupsreceived some kind of reinforcement


for a 16-week period followingtraining. Members of the self-assessment only group completedweekly self-assessment exercises.Members of the peer review withself-assessment group completedthe self-assessment exercises and,in addition, attended weekly peerreview sessions. Providers in bothreinforcement groups received anadditional half-day of training at theIPC/C workshop during which theywere taught about self-assessmentand/or peer review. This reportanalyzes data on 203 providers,from 172 clinics, who completed thetraining and all three rounds of datacollection, along with 1,209 of theirclients.

Self-assessment exercises consistedof a series of eight forms, eachcovering a different IPC/C skill area.During the training workshopproviders had been introduced tothe basic IPC/C concepts involved:listening, giving feedback, sharing,and receiving feedback. Providerswere asked to fill out the formimmediately after a randomlyselected family planning consulta-tion, but some busy providerscompleted the form at the end of theday instead. It took providers 15 to20 minutes to complete the form,which asked them to rate their ownand the client’s behaviors during theselected session. The form alsoprompted providers to reflect on theimpact of their behavior on the clientand to list specific behaviors thatthey wanted to change. Later, theyrecorded the outcome of their effortsto change. The forms were true self-assessments: They were not turnedin to or reviewed by supervisors.

Peer review took place during thesame 16-week period as self-assessment. Peer review consisted

Table 1

Study Conditions

Group Interventions

1. Control Training

2. Reinforcement: self-assessment only Training + self-assessment

3. Reinforcement: peer review with self-assessment Training + self-assessment + peer review

of a weekly 30- to 60-minute peerreview session with three or fourproviders from participating clinics.Most had to travel to a different,sometimes distant, clinic to attendthese meetings. No moderator orfacilitator was assigned, but provid-ers were given a brief discussionguide echoing that week’s self-assessment activity. Providers wereexpected to discuss issues thatemerged from the self-assessmentexercises but not to identify aspecific case or to share their self-assessment forms.

To ensure that providers understoodhow to conduct self-assessment andpeer review, project staff met oncewith participants, either individuallyor in groups, after they had begunthe reinforcement interventions.Project staff used these meetings tocheck how well the providers wereimplementing the interventions andto clarify problem areas.

B. Data collectionData were collected at three pointsin time over a six-month period:

1. Baseline round––conducted inDecember 1997 prior to training

2. Post-training round––conductedin February 1998 immediatelyafter training

3. Follow-up round––conducted inJune 1998 after the self-assess-ment and peer review interven-tions were completed

The primary source of data is codedclient-provider interactions. Duringeach round of data collection,research assistants selected twofamily planning clients per providerto participate in the study andaudiotaped their consultations.These tapes were then analyzedusing an adaptation of the RoterInteraction Analysis System (RIAS),which has been used extensively inboth developed and developingcountries (Roter l997). RIAS assignsa code to each utterance of theclient and the service provider inorder to generate quantitative dataabout client-provider communica-tion. An utterance is defined as acomplete thought, usually a phraseor sentence. The coders wereIndonesians who understood thelocal language and received specialtraining in the RIAS system.

RIAS codes were used to constructthe following three variables tomeasure the impact of training andreinforcement. (See Table 2 for acomplete list of communicationcategories.)

■ Provider facilitative communica-tion—utterances that promote aninteractive relationship between


client and provider by fosteringdialogue, rapport, and clientparticipation

■ Provider information giving—utterances that give clients thetechnical information and adviceabout family planning andmedical matters they need tomake informed decisions

■ Client active communication—utterances that allow the client totake an active part in the consul-tation and help shape its direc-tion. Includes social and personal

Table 2

Provider and Client Communication Categoriesfor Coded Transcripts

Clients

Active Communication

Asks questions of all kinds

Seeks clarification

Shows concern or worry; seeks reassurance

Expresses opinion, approval, disapproval; requestsservice

Makes personal or social remarks

Providers

Facilitative Communication

Asks lifestyle and psychosocial questions

Gives information and counsels on lifestyle andpsychosocial issues

Builds partnership with clients (self-disclosure,checks for understanding, asks for opinion, statesopinion, etc.)

Expresses positive emotion (approval, empathy,concern, reassurance)

Shows agreement or understanding

Makes personal or social remarks

Information Giving

Gives information on medical and family planningissues

Counsels on medical and family planning issues

Other Communication

Asks medical, family planning, and routine questions

Gives instructions

Expresses negative emotion (disapproval, criticism)

Miscellaneous (transition words, mechanicalrepetition, unintelligible)

Other Communication

Gives medical, family planning, and routineinformation

Gives lifestyle and psychosocial information

Shows agreement or understanding

Laughs (nervous or happy)

Miscellaneous (transition words, unintelligible, givesinstructions)

conversation that indicates theclient feels comfortable talkingwith the provider

Client exit interviews provided dataon clients’ subjective assessment oftheir own behavior during theconsultation and of the quality ofcare offered by the provider.Research assistants read a series ofstatements to clients, who had achoice of four responses: stronglyagree, agree, disagree, stronglydisagree. The statements weregrouped into indicators as shown in

Table 3. The first indicator, clientself-efficacy, is a precondition tobehavior change: It is the extent towhich a person believes that she orhe is able to act. Here, self-efficacyrefers to clients’ belief that they cansay what they want to the provider.The second indicator, self-expres-sion, is clients’ assessment of howmuch they spoke and what they saidduring the consultation; it is asubjective measure of client partici-pation. Two aspects of a thirdindicator, clients’ satisfaction withthe quality of care, were assessed:the personal attention they wereshown and whether they receivedthe help they came for.

Other data came from providerinterviews and clinic observations. Inaddition, providers’ performanceduring the final round of datacollection was rated by three outsideexperts and by service providers ona special checklist.

Steps were taken to ensure dataintegrity. Before data collection,providers and clients signed aconfidentiality and voluntary partici-pation consent form. Names ofclients and providers were notknown to those performing primaryor secondary analyses.

C. Site profileAbout three-quarters of the clinicswere located in rural areas, andmost of the remainder were in peri-urban areas. While 82 percent hadelectricity, only 66 percent hadrunning water. The size of theclinics, the number and type ofproviders, and client flow varieddramatically. While a single midwifewas responsible for family planningservices at most clinics, multiplemidwives, nurses, and doctors


Table 3

Client Assessments: Key Concepts and Exit Interview Items

Concept Interview Items

Self-evaluation1. When I come to the clinic, I feel confident that I can talk about whatever is on my

mind.

2. When I come to the clinic, I feel confident that I can ask for clarification when I donot understand something.

3. When I am asked a question by the provider, I feel confident that I can give morethan brief answers.

1. I feel that I spoke as much as I wanted today.

2. I feel that I had the chance to say, in my own words, what I wanted to say today.

3. I feel that I asked all the questions I wanted to ask today.

Self-efficacy

Self-expression

SatisfactionAttention and care 1. The provider took time to find out what I was concerned about today.

2. The provider answered my questions.

3. The provider listened carefully to everything I had to say.

4. The provider made me feel that she cared about me. (I felt attended by provider.)

5. The provider treated me well today.

1. I feel that I received the information and services I wanted today.

2. I feel that I got appropriate assistance for my particular needs.Needs met

offered family planning services atlarge clinics. The number of familyplanning visits in June 1998 rangedfrom less than 18 at the slowest 20percent of clinics to more than 90 atthe busiest 20 percent; a few clinicsrecorded more than 200 familyplanning visits during that month.

D. Service provider andclient profilesThe majority (91 percent) of theproviders who participated in thestudy were midwives, while theremainder were nurses. All werewomen. Four-fifths had more thanfive years of experience offering

family planning services. About half(49 percent) served ten or fewerfamily planning clients per week;only 21 percent saw more than 20family planning clients each week.The providers had a wide variety ofjob responsibilities; 58 percent wereresponsible for at least eightdifferent tasks, such as maternal andchild healthcare, control of commu-nicable diseases, nutrition, outpa-tient clinics, health education, andschool health services. Forty-twopercent split their time betweenfamily planning and other activities,13 percent spent more than half theirtime on family planning, and 45percent spent less than half theirtime on family planning.

During each round of data collec-tion, two family planning clients perprovider were asked to participate inthe study. The clients were chosenrandomly on the day of data collec-tion so they mirrored the generalfamily planning population inIndonesia, which consists mostly ofwomen and is limited to persons ofreproductive age. All but two of the1,209 clients were women, 99percent were married, 70 percenthad at least two living children, and55 percent were between the agesof 25 and 34. Most (76 percent) werecontinuing clients already usingcontraception. Of these continuingclients, about half had problems withtheir method, while the rest came forroutine check-ups, resupply or, in afew cases, the removal of an IUD orimplant. Injectables and IUDs werethe most popular contraceptivemethods, followed by implants andthe pill.

III. Results

Baseline data confirmed thatproviders dominate family planningconsultations in Indonesia, as theydo around the world. Providersaccounted for 64 percent of allutterances. Most provider communi-cation consisted of information (39percent) and questions (27 percent)regarding family planning andrelated medical issues, but 25percent of all provider communica-tion was facilitative. Providers didnot speak for long before givingclients an opportunity to reply, butclients rarely took advantage ofthese opportunities to volunteerinformation, ask questions, orotherwise play an active role in theconsultation. Most client commentswere brief responses to provider


Figure 1

Impact of Training on Provider FacilitativeCommunication and Information Gathering

Perc

enta

ge o

f Pro

vider

Utte

ranc

es

50

40

30

20

10

0Facilitative communication Medical/FP information and counseling

Source: JHU/CCP, Self-assessment study - 1998

Notes: Baseline n=397 clients; Post-training n=406 clients

After training, providers encouraged client participation with more facilitative communication andalso gave clients more technical information about contraception.

questions (56 percent) or anacknowledgment of what theproviders said (24 percent). Only 10percent of all client utterances wereactive, and clients asked 1.6questions per session, on average.

Client assessments of the consulta-tions were relatively high, rangingfrom 3.9 to 4.1 on a five-point scale.This is not surprising given thestrong social pressures againstexpressing disagreement in Indone-sia. Some clients, however, did havereservations about the extent of theirparticipation. For example, 7percent of clients did not feelconfident about asking for clarifica-tion, while an equal percentage saidthey did not speak as much as theywanted. While low, these levels ofdisagreement are meaningful in theIndonesian context.

A. Impact of training onprovider performance andclient communicationThe most dramatic impact of trainingwas to almost double the length offamily planning consultations froman average of 6 minutes to 11minutes. Longer sessions gave bothproviders and clients additional timeto talk, although the two-to-one ratioof provider to client communicationremained intact. The fluctuatinglength of the sessions makes it moredifficult to interpret changes inprovider and client communication.Percentage data control for thelength of the session but mayoverlook important changes inthe frequency of a behavior. Forexample, longer sessions allowedproviders to express positiveemotion more often (the frequencyrose from 2.6 to 4.5 utterances) eventhough the percentage of positiveemotion remained the same, at just

over 4 percent. Therefore, bothpercentage and frequency data arereported here.

Providers used most of the extratime in longer sessions to giveclients additional information andcounseling on medical and familyplanning issues: Both the number(27 to 58, p<.001) (Table 4) andproportion (39 percent to 48 per-cent, p<.001) of utterances in this

category rose sharply (Figure 1).Since prior research in Indonesiahas found that providers generallygive family planning clients sketchyinformation (Kim et al. 1997), theincrease in the amount of informa-tion marks an improvement in thequality of care. More important thanthe quantity of information is itsquality, but RIAS coding does notprovide data on the clarity, accuracy,and relevance of information given.

Table 4

Frequency of Selected Provider Communication Categories:Baseline and Post-Training Rounds

Type of Communication Average No. of Utterances p Value

Baseline Post-Training(n=397 clients) (n=406 clients)

All facilitative communication 14.9 30.5 .0001

Medical/FP information and counseling 26.5 57.9 .0001

Baselineround

Post-traininground

Baselineround

Post-traininground


Longer sessions also meant that thefrequency of most types of facilita-tive communication increasedmarkedly even when their percent-age remained flat. Overall, thenumber of facilitative utterancesdoubled after training from 15 to 31(p<.001) and, in percentage terms,increased from 25 percent to 28percent (p<.001) (Figure 1).

Training made the greatest impacton providers’ weakest skills so that,as a group, providers’ IPC/Cbehavior became less variable.Facilitative communication andinformation giving were inverselyrelated at the baseline: The group ofproviders that was least facilitativegave clients the most family plan-ning and medical information andvice versa. As Figures 2 and 3 show,providers who had the lowest levelsof facilitative communication duringthe baseline round made thegreatest gains in that behavior, whileinformation-giving increased most inthe group that started with the lowestbaseline levels. As a result, the gapbetween the most and least facilita-tive providers shrank from 19 to 5percentage points, while the gapbetween the providers giving themost and least information shrankfrom 14 to 4 percentage points.

Training also had an indirect impacton client participation. While thepercentage of active client commu-nication did not change significantlyafter the training workshop, longersessions gave clients more opportu-nities to communicate actively. Thefrequency of client active communi-cation rose from 3.3 utterances persession at the baseline to 7.0 aftertraining (p<.001). Most of theincrease was in acknowledging whatthe provider had said; this was aconsequence of the sharp rise inprovider information giving. How-

Figure 2

Impact of Training on Provider Facilitative Communication,by Level of Facilitative Behavior at Baseline

Perc

enta

ge o

f Pro

vider

Utte

ranc

es 40

30

20

10

0Baseline round Post-training round



Providers who used the least facilitative communication during the baseline round made thegreatest gains after training. As a result, after training there was less variation in the amount offacilitative communication offered by individual providers.

High facilitative

Medium facilitative

Low facilitative

Figure 3

Impact of Training on Provider Information Giving,by Level of Facilitative Behavior at Baseline

Perc

enta

ge o

f Pro

vider

Utte

ranc

es 60

50

40

30

20Baseline round Post-training round



During the baseline round, providers who used the most facilitative communication gave clientsthe least family planning information. After training, these providers made the greatest gains ininformation-giving, so that there was less variation between providers.

High facilitativeMedium facilitative

Low facilitative

ever, the average number of ques-tions also doubled from 1.6 to 3.3(p<.001) in the post-training round,and slipped only slightly to 3.2 atfollow-up even though the consulta-tions were shorter.

B. Impact of reinforcementactivitiesDuring the four-month follow-upperiod, consultations grew shorter inboth the self-assessment only and


peer review with self-assessmentgroups (Table 5). In contrast,consultations continued to growlonger in the control group.

Providers in the reinforcementgroups used their limited timeefficiently as self-assessment andpeer review activities helped themmaintain or continue to improvemany IPC/C skills. From the post-training to the follow-up round,provider facilitative communicationincreased from 28 percent to 35percent in the two reinforcement

Table 5

Duration of Consultation in Minutes, by Study Group

StudyGroup Baseline Post-Training Follow-Up

(n=397) (n=406) (n=403)

Control 6.1 9.7 10.4

Self-assessment only 5.1 9.9 8.8

Peer review with self-assessment 5.8 13.1 10.6

Figure 4

Impact of Reinforcement on Provider FacilitativeCommunication: Control versus Reinforcement Groups

Perc

enta

ge o

f Pro

vider

Utte

ranc

es 40

35

30

25

20Post-training round Follow-up round


Notes: Control group: post-training n=121 clients; follow-up n=119 clients

Reinforcement group: post-training n=285 clients; follow-up n=284 clients

Levels of facilitative communication in the control group changed little in the four months aftertraining. In contrast, facilitative communication increased significantly in the self-assessment onlyand peer review with self-assessment groups.

Control group

Reinforcement group

groups even as the sessions grewshorter; in contrast, facilitativecommunication declined from 29percent to 27 percent in the controlgroup (Figure 4). Given the chang-ing length of sessions from the post-training to the follow-up rounds, thistranslates into a marked decrease inthe number of facilitative utterancesamong the control group (from 33 to23, p<.001), while the number offacilitative utterances in the rein-forcement groups held steady at 29(Table 6).

Reinforcement had a greater impacton facilitative communication amongexperienced providers with morethan 10 years of experience offeringfamily planning services. Thisimplies that experienced providers,far from resisting change, werebetter able to understand, carry out,and apply lessons learned fromreinforcement activities than theirless experienced peers.

By every measure, the amount ofmedical and family planninginformation offered by providersdecreased in both the reinforcementand control groups (Table 6). Theproportion of medical and familyplanning information and counselingfell from 47 percent to 43 percent inthe reinforcement groups and from48 to 45 percent in the controlgroup. However, both remainedsignificantly higher than the baselinelevel of 39 percent.

Expert ratings and client assess-ments confirm the positive impact ofreinforcement. According to expertraters, providers in the reinforcementgroups more often encouragedclients to ask questions (in 36percent of sessions compared with20 percent, p<.01), complimentedclients when they asked questions(6 percent versus 0 percent, p<.01),asked clients about their feelings (15percent versus 1 percent, p<.001),and asked clients to return if theyhad a problem (55 percent versus41 percent, p<.01) compared withproviders in the control group.Similarly, client satisfaction and self-expression increased significantly inthe reinforcement groups, but not inthe control group, from the post-training to the follow-up rounds.

Client communication also changedin response to reinforcementstrategies. Client active communica-tion rose from 12 percent to 16


percent in the reinforcement groupsduring the four-month follow-upperiod, compared with a smallerincrease from 10 percent to 12percent in the control group (Figure5). The changes primarily reflect anincrease in social and personalconversation and were concentratedamong new clients and continuingclients with problems. Expert ratersalso noted that clients were morelikely to answer questions at lengthwhen their providers were participat-ing in reinforcement activities (40percent versus 23 percent, p<.01).

C. Comparing self-assessment and peer reviewPeer review heightened the impactof the self-assessment interventionon some, but not all, aspects ofprovider communication. Total levelsof facilitative communication rosefrom 28 percent to 37 percent in thepeer review with self-assessmentgroup, compared with 28 percent to33 percent in the self-assessmentonly group (Figure 6). This is due tosharper increases in social conver-sation and lifestyle/psychosocialinformation in the peer review withself-assessment group, as well asmore limited declines in positiveemotion and acknowledgment. Thepeer review with self-assessmentgroup did no better than the self-assessment only group in lifestyle/psychosocial questions and didworse in partnership-building.Frequency data show that thenumber of facilitative utterancesdeclined among providers in theself-assessment only group (from 29to 25, p<.05) while it did not changesignificantly in the peer review group(30 to 32, ns) (Table 7). While thepercentage and frequency ofprovider information giving declinedin both groups, the drop was more

Figure 5

Impact of Reinforcement on Client ActiveCommunication: Control versus Reinforcement Groups

Perc

enta

ge o

f Pro

vider

Utte

ranc

es 20

15

10

5



Notes: Control group: post-training n=121 clients; follow-up n=119 clients


Client active communication increased in every group during the four months after training.However, the increase was significantly greater in the self-assessment only and peer review withself-assessment groups than in the control group.

Control group

Reinforcement group

Table 6

Frequency of Selected Categories of Provider Communication, byControl and Reinforcement Groups

Category/Study Group p ValuePost-Training Follow-Up

All facilitative communication

Control 32.6 23.2 .0010

Reinforcement 29.6 28.4 .4201

Medical/FP information and counseling

Control 60.4 43.9 .0031

Reinforcement 56.8 38.6 .0001

Control group: post-training n=121 clients; follow-up n=119 clients


Average No. of Utterances

marked in the self-assessment onlygroup (Table 7).

Further analysis shows that peerreview benefited providers at all skilllevels, but that more highly skilledproviders gained the most from theintervention. During the four-month

follow-up period, levels of facilitativecommunication increased byone-fifth among providers in thepeer review with self-assessmentgroup who had the lowest baselinelevels of facilitative communication(Table 8). In contrast, they increased


by one-third among providers withmedium to high levels of facilitativecommunication at the baseline. Asimilar pattern prevailed in the self-

Figure 6

Impact of Self-Assessment and Peer Review onProvider Facilitative Communication

Perc

enta

ge o

f Pro

vider

Utte

ranc

es 40

35

30

25



Notes: Self-assessment only group: post-training n=142 clients; follow-up n=142 clients

Peer review group: post-training n=143 clients; follow-up n=142 clients

Provider facilitative communication increased in both of the reinforcement groups during the fourmonths after training. However, the increase was significantly greater in the peer review withself-assessment group than in the self-assessment only group.

Peer review group

Self-assessment only group

Table 7

Frequency of Selected Categories of Provider Communication, bySelf-Assessment Only and Peer Review

with Self-Assessment Groups

Type of Communication/Study Group p ValuePost-Training Follow-Up

All facilitative communication

Self-assessment only 28.7 24.9 .0417

Peer review with self-assessment 30.4 32.0 .4876

Medical/FP information and counseling

Self-assessment only 55.8 34.6 .0001

Peer review with self-assessment 57.9 42.7 .0037

Self-assessment only group: post-training n=142 clients; follow-up n=142 clients.

Peer review with self-assessment group: post-training n=143 clients; follow-up n=142 clients.

Average No. of Utterances

assessment only group, but thegains were far smaller than those inthe peer review with self-assessmentgroup.

As for clients, total active communi-cation increased more sharply in thepeer review with self-assessmentgroup (from 12 percent to 17percent) than in the self-assessmentonly group (from 12 percent to 15percent), mainly because of differ-ences in social and personalconversation (Figure 7). During thefinal round of data collection, clientsin the peer review with self-assess-ment group made 8.5 active utter-ances, including 4.9 questions,compared with 5.8 active utter-ances, including 3.3 questions, inthe self-assessment only group.

Self-assessment and peer reviewhad more impact on both providerfacilitative and client active commu-nication among clients with at least asecondary education. Bettereducated clients more often en-gaged in active communication thantheir less educated peers at thebaseline, and this gap widened overthe intervention period. This mayindicate that education gives clientsthe confidence to take advantage ofany opportunities to speak thatproviders offer, while providers maybe more open and responsive tobetter-educated clients with whomthey identify.

Despite documented improvementsin the quality of interpersonalcommunication, the number of familyplanning visits to clinics in all threestudy groups dropped during thefour-month reinforcement periodbecause of economic and socialdisruptions in Indonesia. Riots,contraceptive shortages, and newlyintroduced charges for contracep-tives all discouraged clients fromattending public clinics. Manyclients may have discontinued familyplanning or switched to privateproviders.


D. Cost analysis1

Data were obtained on the one-timetraining costs and 16-week operat-ing costs of the two reinforcementinterventions (self-assessment andpeer review). While the reinforce-ment interventions may havecontinued on in some locations, thisanalysis is restricted to the 16-weekperiod following training.

Each intervention had both directand opportunity costs. Direct costsinclude additional program expendi-tures on materials and supplies,trainers’ honoraria and transporta-tion, and providers’ per diems andtransportation. In contrast, opportu-nity costs consist of employee timethat is diverted away from regularduties to program activities, forexample, the time midwives andnurses spent at the training work-shop, filling out self-assessmentforms, and attending peer reviewmeetings. Opportunity costs werecalculated by prorating providers’and trainers’ salaries. The costanalysis did not consider develop-ment costs for the training curricu-lum, self-assessment forms, andpeer review discussion guides,since they are one-time costs thathad already been paid, nor didit consider opportunity costsassociated with longer counselingsessions.

Two scenarios were considered incalculating costs. The “minimal” costscenario includes only the directcosts of training and reinforcement,without supervision. The “full” costscenario includes both direct andopportunity costs, including the costof a single supervisory visit to thereinforcement groups.

1 This section is based on an unpublished paper analyzing the cost-effectiveness of reinforcement in Indonesia by Hany Abdallah andfurther analysis by Barton Burkhalter, both of QAP.

Table 8

Percentage of Facilitative Communication inReinforcement Groups by Provider’s Baseline Level

of Facilitative Communication

Baseline Level of Peer Review withProviders’ Facilitative Self-AssessmentCommunication Post-Training Follow-Up Post-Training Follow-Up

Low 26.3 32.6 27.4 33.1

Medium 28.6 33.6 27.8 36.8

High 31.0 33.0 29.6 39.9

Control group: post-training n=121 clients; follow-up n=119 clients.

Self-assessment only group: post-training n=142 clients; follow-up n=142 clients.

Peer review group: post-training n=143 clients; follow-up n=142 clients.

Self-Assessment Only

Figure 7

Impact of Self-Assessment and Peer Review onClient Active Communication

Perc

enta

ge o

f Pro

vider

Utte

ranc

es 20

15

10

5



Notes: Self-assessment only group: post-training n=142 clients; follow-up n=142 clients

Peer review group: post-training n=143 clients; follow-up n=142 clients

Client active communication increased in both reinforcement groups during the four months aftertraining. However, the increase was greater in the peer review with self-assessment group than inthe self-assessment only group.

Peer review group

Self-assessment only group


Training was the most expensiveintervention, costing a total of US$90 per provider in both direct andopportunity costs.2 Once providersand trainers were already gatheredfor IPC/C training, however, it costrelatively little to add an extra half-day of training on self-assessmentand/or peer review (Table 9). Self-assessment was the least expensiveintervention, costing less than US$ 2per provider to photocopy the formsplus another US$ 8 in opportunity(time and supervision) costs. Addingpeer review to self-assessmentraised reinforcement costs anotherUS$ 16, largely due to the cost oftransporting providers to peer reviewmeetings at other clinics. Overall, fullcosts for the three study groupsranged from US$ 90 for the controlgroup, to US$ 106 for the self-assessment only group, to US$ 122for the peer review group. Incomparison, the providers’ averagemonthly salary is approximatelyUS$ 76.

To calculate cost-effectiveness, theoutcome measures were defined asthe percentage gains in two types ofprovider communication: facilitativecommunication and informationgiving. Table 10 presents theincrease in the number of utterancesfrom the baseline to the follow-uprounds for each study group and themarginal cost to achieve thatincrease.

The higher costs in reinforcementgroups were more than matched bygreater gains in facilitative communi-cation. The average number offacilitative utterances per sessionincreased by 53 percent in the

Table 10

Cost-Effectiveness of Training, Self-Assessment, andPeer Review in Improving Provider Communication (in US$)

Average No. ofUtterances per Session % Gain over % GainBaseline Follow-Up Baseline1 per Dollar

Facilitative communication:Training only group 15.2 23.2 52.6 0.60Training + self-assessment (SA) 13.6 24.9 83.1 0.78Training + SA + peer review 15.8 32.0 102.5 0.84

Medical and family planning information:Training only group 20.1 44.3 120.4 1.34Training + SA 30.9 34.6 12.0 0.11Training + SA + peer review 27.3 42.7 56.4 0.46

1 % gain over baseline = (# of utterances at follow-up – # utterances at baseline) / (# of utterances at baseline).

Table 9

Direct and Opportunity Costs per Provider of Training,Self-Assessment, and Peer Review Interventions (in US$)

Intervention Minimal (Direct) Full (Direct + Opportunity)

Training:IPC/C 68.56 90.10Additional cost for self-assessment 4.61 6.39Additional cost for peer review 1.08 1.64

Reinforcement for 16 weeks:Self-assessment 1 1.56 9.58Additional cost for peer review 2 9.42 14.71

Total cost by group:IPC/C training only (control group) 68.56 90.10Training + self-assessment group 74.73 106.07Training + self-assessment + peer review group 85.23 122.42

1 Direct costs of self-assessment are photocopying forms; opportunity costs are provider time and supervision.2 Direct costs of peer review are photocopying forms and transportation to meetings; opportunity costs are provider time

and supervision.

Cost per provider

2 The cost-effectiveness analysis employed the exchange rate that prevailed when each expense was incurred. This generally rangedfrom 5,000 to 7,500 rupees to the dollar, but at one point rose as high as 13,500 rupees.


training only group, 83 percent in theself-assessment group, and 103percent in the self-assessment andpeer review group. When costs areconsidered, one dollar spent oninterventions in each of the threestudy groups led to 0.60, 0.78, and0.84 percentage point gains infacilitative communication, respec-tively. An analysis of the marginalcosts and gains associated withself-assessment and peer reviewfurther demonstrates the cost-effectiveness of reinforcement. Theself-assessment only group incurredUS$ 16 more in costs than thetraining group, but had an additional29 percent gain in facilitativeutterances. This means that onedollar spent on self-assessmentyielded a 1.83 percentage point gainin facilitative communication. Incontrast, one dollar spent on IPC/Ctraining yielded just a 0.6 percent-age point gain in facilitative commu-nication. Compared with training andself-assessment, peer review costan additional $16 but led to anadditional 19 percent gain infacilitative communication. Thus,one dollar spent on peer reviewyielded a 1.17 percentage point gainin facilitative communication.

Results proved far different forinformation giving, which was afeatured topic during training butwas largely ignored by the self-assessment and peer reviewmaterials. The control group had byfar the largest gain in informationgiving, 120 percent, compared to 12percent for the self-assessmentgroup and 56 percent for the self-assessment and peer review group.Thus, spending additional money onreinforcement actually reduced theimpact on information giving, anissue that is explored further in the

discussion section below. One dollarspent on the training only group ledto a 1.34 percentage point gain ininformation-giving, while a dollarspent on the reinforcement groupsled to 0.11 and 0.46 percentagepoint gains, respectively.

IV. Discussion andImplications

A. The value of IPC/Ctraining and reinforcementactivitiesThis study confirms the value of briefIPC/C training for primary healthcareproviders in developing countries.Providers credited the course withincreasing their patience with clientsand helping them feel closer toclients, as well as teaching them theimportance of giving clients exten-sive information and opportunities totalk. Objective assessments confirmthat the training workshop increasedthe degree of interaction betweenproviders and clients, the amount ofinformation provided, and theconsistency of care. To some extent,these improvements depended uponincreasing the length of consulta-tions, which could potentially strainthe resources of busy facilities andadd to clients’ waiting time. How-ever, providers in the reinforcementgroups were able to maintain goodcommunication with clients even astheir consultations grew shorter overthe course of the four-month follow-up period. Evidently, self-assessmentand peer review activities helpedproviders use their time moreefficiently by focusing on criticalelements in the consultation.Trainers could contribute to thisprocess by teaching providers to

eliminate unnecessary and counter-productive behaviors even as theytry valuable new behaviors.

Although the vast majority ofproviders believed they could applytheir newly learned IPC/C skills attheir workplace, they found littlesupport when they returned to theirhome clinics after training. Routinesupervision visits were irregular, andsupervisors rarely addressed client-provider communication. Trainedproviders also encountered skepti-cism from colleagues since only oneprovider was trained at most clinics.Even clients could pose an obstacle:Continuing clients were bewilderedwhen providers, contrary to pastexperience, acted unusually friendlyor offered lengthy explanations, andtheir reactions made providers feeluncomfortable.

Given the obstacles to behaviorchange, it is not surprising that thejob performance of newly trainedproviders tended to erode over timewithout reinforcement. But exactlyhow do self-assessment and peerreview work to counter this trend?Observations from this study andfindings from other self-assessmentstudies point to several possiblemechanisms (Abrams and Kelleyl974; Stuart et al. l980; Stackhouseand Furnham l983; Henbest andFehrsen l985): Regular self-assess-ment and peer review remindproviders to apply newly learnedskills and provide the support andmotivation they need to succeed.They teach providers how todiscriminate between good andpoor skills, so they can evaluatetheir abilities more realistically. Theyreduce providers’ anxiety andconfusion by clarifying standardsand focus providers’ attention on


important issues. They guideproviders through a systematicbehavior change process so theycan identify and correct weak-nesses. Last, but not least, theymotivate providers to do better bypointing out their improvement overtime as well as their deficiencies.

Reinforcement had a greater impacton provider facilitative and clientactive communication than oninformation giving, probablybecause the self-assessment andpeer review materials emphasizedthe need for more interactivesessions and barely touched ongiving complete and relevantinformation. This may have causedproviders to focus on boostingfacilitative communication and clientparticipation at the expense ofinformation giving. If this is the case,the lack of impact on informationgiving further demonstrates thepower of the self-assessment andpeer review interventions: Behaviorspromoted by reinforcement materials(i.e., provider facilitative and clientactive communication) increased,while behaviors not promoted by thereinforcement materials (i.e.,provider information giving)decreased. This suggests thatchanging the content of the rein-forcement materials to includeinformation giving along withfacilitative and active communica-tion might change the results.

The fact that reinforcement mayhave weakened information givingraises other important issues. Whichis more important to the quality ofthe client-provider interaction andhealthcare outcomes: giving clientsmore technical information oreliciting client participation? Andmust information giving and clientparticipation necessarily be trade-

offs? After all, a key goal of clientparticipation is to improve the qualityof the information exchangedbetween clients and providers.Actively participating clientsdisclose more information aboutthemselves, so providers cancounsel them better. Activelyparticipating clients also may elicitmore information from providers,allowing them to make a betterchoice of family planning methodsand to more fully understand how touse their chosen method safely andeffectively. Further research isneeded to explore the relationshipbetween information giving, clientparticipation, and family planningand healthcare outcomes.

A key assumption of this study isthat clients will feel more comfort-able and talk more openly withproviders who have good IPC/Cskills. This, in turn, means thatinterventions directed to providerscan influence client behavior.Findings support this assumption.By the end of the study, clients wereengaging in more active communi-cation and, according to providers,were speaking more freely. Most ofthe gains in client active communi-cation were in personal and socialconversation, suggesting that theintervention was more successful inpromoting rapport between provid-ers and clients than in actuallyboosting client input into theconsultation. However, clients didask twice as many questions by theend of the study.

B. Successfullyimplementing self-assessment and peer reviewThe self-assessment and peerreview interventions were designedto be low-cost and low-maintenance.This is critical for the sustainability ofany program but is especiallyimportant in Indonesia, where therecent economic crisis has intensi-fied cost concerns for policy makersand program managers. Whether ornot providers attempted the self-assessment exercises or partici-pated in the peer review meetings,and how much energy they put intothem, depended entirely on theirown diligence and motivation.Despite the fact that providersconducted the self-assessmentexercises in isolation from oneanother, nearly all of them com-pleted the full 16-week series.Similarly, absenteeism from the peerreview meetings was extremely low,and providers participated enthusi-astically in the discussions.

The smooth implementation of theself-assessment process stands incontrast to previous studies in whichunfamiliarity with the assessmentforms and provider resistance,disorientation, or distrust havecaused problems (Jelly and Fried-man l980; Abrams and Kelley l974;Geissler l973; Katz l970). Threefactors eased the implementation.First was the simple design of theself-assessment and peer reviewmaterials. Most providers did notfind it difficult to complete the forms,although it took somewhat longerthan anticipated (15–20 minutesinstead of 5–10 minutes). With just alittle outside help, all the providerswere able to apply the most criticalpart of self-assessment, that is, theself-learning cycle, which consists of


setting behavioral goals, trying outnew behaviors, and assessing theoutcomes.

A second factor was the consistencyin the contents of the trainingcurriculum and reinforcementactivities. The training workshopclearly defined the behaviors to beevaluated and their performancecriteria, so that providers understoodwhat each behavior meant and howto discriminate its quality. A compari-son of expert ratings and providerself-ratings found that, while self-ratings were inflated, they followedthe same pattern as the expertratings. That is, the weaknessescommonly identified by providersduring their self-assessmentexercises matched outside assess-ments of their deficiencies. Thisindicates that providers can ratetheir own performance well enoughfor learning purposes, if there isconsistency between the curriculumand reinforcement activities.

The third factor was providers’ highlevel of motivation. Training instilledproviders with the desire to givebetter service, as evidenced by theincreased length of sessionsimmediately afterwards. Furthermotivation may have come fromIndonesian service providers’identification with the servicedelivery system and their tendencyto comply with their institutionalobligations. In addition, providers atpublic clinics in Indonesia faceincreasing competition for clientsfrom private and village midwiveswho are more conveniently located.They believe that improving servicesis a powerful way to attract moreclients.

Despite the relatively smoothimplementation of the interventions,

it is clear that some refinementscould heighten their impact:

■ Training all the providers in aclinic and orienting the rest of thestaff to the curriculum contentwould eliminate the skepticismthat many providers faced, createa system of mutual support, andlower the cost of peer review byallowing providers to hold groupmeetings within the clinic.

■ Self-assessment would be morepowerful and more objective ifproviders audiotaped the consul-tation and listened to themselvesbefore they completed the form,rather than relying on memory. Ajob aid outlining key skills (suchas a composite of their lengthytraining materials) also wouldhelp providers complete theforms, especially the sectionasking them to choose specificgoals for behavior change.

■ Peer review sessions would bemore focused and more construc-tive if there were a skilled facilita-tor, such as a supervisor ortrained peer mentor, to lead themand if there were specific ex-amples of counseling to discuss,perhaps in the form of a role playor an excerpt from one of theprovider’s audiotapes.

■ Clients could be transformed intoa positive force for change by amass media campaign raisingtheir expectations about providerbehavior.

This study demonstrates that self-assessment and peer reviewinterventions can reinforce newlylearned skills, encourage healthcareproviders to change their behavioron the job, and contribute tocontinuing quality improvement. In

contrast, training without reinforce-ment has little long-term impact onproviders’ behavior and may bemoney wasted. Self-assessment andpeer review are effective reinforce-ment strategies because they focusproviders’ attention on importantissues, clarify performance stan-dards, teach providers to evaluatetheir abilities more realistically, helpthem identify weaknesses in their jobperformance, and motivate them todo better. They are sustainablebecause they require few materialsand virtually no supervision. Asproviders work to strengthen theirskills each week during self-assessment and peer reviewactivities, the quality of care contin-ues to improve long after trainingends.


References

Abdel-Tawab, N. 1995. Provider-Client Communication inFamily Planning Clinics in Egypt: Styles, Predictors, andAssociations with Client Outcomes. Doctoral Dissertation,School of Hygiene and Public Health, Johns HopkinsUniversity, Baltimore, Maryland.

Abrams, R. and M. Kelley. 1974. Student self-evaluation ina pediatric-operative technique course. Journal of DentalEducation 28(7):385–91.

Beattie, K., A.J. Faisel, M. Ahmed, and B.A. Pati. 1994.Introducing COPE in Asia: A quality management tool forFP services in Bangladesh. In Satia, J. and M.L. Liow, eds.Innovative Approaches to Population Programme Manage-ment: Quality of Care, vol. 1. Kuala Lumpur, Malaysia:International Council on Management of PopulationProgrammes. 15–30.

Best, K. 1998. Training involves many factors. Network19(1):19–23, 26.

Calhoun, J., J. Ten Haken, and J. Wooliscroft. 1990.Medical students’ development of self and peer assess-ment skills. Teaching and Learning in Medicine 2:25–29.

Central Bureau of Statistics [Indonesia], State Ministry ofPopulation/National Family Planning Coordinating Board,Ministry of Health, and Macro International Inc. 1995.Indonesia Demographic and Health Survey 1994.Calverton, Maryland: Central Bureau of Statistics andMacro International.

Clark, N., M. Gong, M. Schork, D. Evand, D. Roloff, M.Hurwitz, L. Maiman, and R. Mellins. l998. Impact of educa-tion for physicians on patient outcomes. Pediatrics101(5):831–36.

Fincher, R. and L. Lewis. l994. Learning, experience, andself-assessment of competence of third-year medicalstudents in performing bedside procedures. AcademicMedicine 69(4):291–95.

Garavaglia, P. 1995. Transfer of training: Making trainingstick. Info-Line No. 9512. Alexandria, VA: American Societyfor Training and Development.

Geissler, P. 1973. Student self-assessment in dentaltechnology. Journal of Dental Education 37(9):19–21.

Gordon, M. l992. Self-assessment programs and theirimplications for health professions training. AcademicMedicine 67(10):672–79.

Henbest, R. and G. Fehrsen. 1985. Preliminary study at theMedical University of Southern Africa on student self-assessment as a means of evaluation. Journal of MedicalEducation 60(1):66–68.

Jelly, E. and C. Friedman. 1980. An evaluation system forresidency training. Journal of Family Practice 10(1):73–80.

Johns Hopkins University Population CommunicationServices (JHU/CCP). 1998. Institutionalization of Interper-sonal Communication/Counseling (IPC/C) in Indonesia: AProcess Report. Baltimore: JHU/CCP and BKKBN.

Kaiser, S. and J. Bauer. 1995. Checklist self-evaluation in astandardized patient exercise. American Journal ofSurgery 169(4):418–20.

Katz, R. 1970. Self-Assessment Workshop (SAW) FinalResearch Report. Peace Corps Contract #80-1531, TaskOrder 3. Cambridge, Massachusetts: Human DevelopmentFoundation.

Kim, Y., J. Rimon, K. Winnard, C. Corson, I. Mako, S.Lawal, S. Babalola, and D. Huntington. 1992. Improvingthe quality of service delivery in Nigeria. Studies in FamilyPlanning 23(2):118–27.


Kim, Y.M., D. Storey, E. Basuki, F. Putjuk, and L. Mize.1997. Client-Provider Interaction Study: Program SupportResearch for the Quality Improvement Initiative in Indone-sia. Baltimore: BKKBN and JHU/CCP.

Kols, A.J. and J.E. Sherman. l998. Family PlanningPrograms: Improving Quality. Population Reports. Series J,No 47. Baltimore: Johns Hopkins University School ofPublic Health, Population Information Program.

Lynam, P., L. McNeil Rabinovitz, and M. Shobowale. 1993.Using self-assessment to improve the quality of familyplanning clinic services. Studies in Family Planning24(4):252–60.

MacDonald, P. 1995. The Peer Review Program of theIndonesian Midwives Association. Jakarta: UniversityResearch Co., LLC and Indonesian Midwives Association.

Ong, L., J. De Haes, A. Hoos, and F. Lammes. l995.Doctor-patient communication: A review of the literature.Social Science and Medicine 40(7):903–18.

Pariani, S., D.M. Heer, J. Van Arsdol, and D. Maurice. l991.Does choice make a difference to contraceptive use?Evidence from East Java. Studies in Family Planning22(6):384–90.

Roberts, B., E. Cleary, and J. Roberts. l997. Graded checklists to assist undergraduate students in self-directedlearning and assessment in general and systematicanatomical pathology. Pathology 29(4):370–73.

Roter, D. l997. The Roter Interaction Analysis System(RIAS) Coding Manual. Baltimore: Johns Hopkins Univer-sity School of Hygiene and Public Health.

Roter, D. and J. Hall. l998. Effectiveness of interventions toimprove patient compliance: A meta-analysis. MedicalCare 36(8):1138–61.

Sobral, D. 1994. Peer tutoring and student outcomes in aproblem-based course. Medical Education 28(4):284–89.

Stackhouse, J. and A. Furnham. 1983. A student-centeredapproach to the evaluation of clinical skills. British Journalof Disorders Communication 18(3):171–79.

Stewart, M. 1996. Effective physician-patient communica-tion and health outcomes: A review. Canadian MedicalAssociation Journal 152(9):1423–33.

Stuart, M., H. Goldstein, and F. Snope. 1980. Self-evalua-tion by residents in family medicine. Journal of FamilyPractice 10(4):639–42.

Sullivan, R.S. and T. Smith. l996. On-the-Job Training forFamily Planning Service Providers. Strategy Paper No. 3.Baltimore: Johns Hopkins Program for InternationalEducation and Training in Reproductive Health (JHPIEGO).

operations research results - shops plus project · the operations research results series presents...

Documents