This article was downloaded by: [220.255.1.173]On: 27 June 2013, At: 17:50Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

The Clinical SupervisorPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/wcsu20

Does Psychotherapy SupervisionContribute to Patient Outcomes?Considering Thirty Years of ResearchC. Edward Watkins Jr. a

a University of North Texas, Denton, Texas, United StatesPublished online: 29 Nov 2011.

To cite this article: C. Edward Watkins Jr. (2011): Does Psychotherapy Supervision Contribute toPatient Outcomes? Considering Thirty Years of Research, The Clinical Supervisor, 30:2, 235-256

To link to this article: http://dx.doi.org/10.1080/07325223.2011.619417

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representationthat the contents will be complete or accurate or up to date. The accuracy of anyinstructions, formulae, and drug doses should be independently verified with primarysources. The publisher shall not be liable for any loss, actions, claims, proceedings,demand, or costs or damages whatsoever or howsoever caused arising directly orindirectly in connection with or arising out of the use of this material.

Does Psychotherapy Supervision Contributeto Patient Outcomes? Considering Thirty

Years of Research

C. EDWARD WATKINS, JR.University of North Texas, Denton, Texas, United States

After a century of psychotherapy supervision and over half acentury of supervision research, what do we know empiricallyabout the contribution of psychotherapy supervision to patient out-comes? In this article, I address that question by (1) assembling allidentified supervision-patient outcome studies, from 1981 to 2006,referenced in four reviews (Ellis & Ladany, 1997; Freitas, 2002;Inman & Ladany, 2008; Wheeler & Richards, 2007) and (2) iden-tifying additional post-review studies by means of computersearches, spanning January 2006 through May 2011. A total of18 supervision outcome studies emerged, spanning the past gener-ation of supervision scholarship. Unfortunately, after closely scruti-nizing each investigation, eliminating misidentified studies(constituting over one-third of the 18 studies), and weighing thegravity of various methodological deficiencies across investiga-tions, the collective data appeared to shed little new light on thematter: We do not seem to be any more able to say now (as opposedto 30 years ago) that psychotherapy supervision contributes topatient outcome. What did emerge of considerable promise, how-ever, were three recent studies that were developed, organized,and prosecuted with the primary objective of evaluating the effectsof supervision on patient outcome. Those investigations were high-lighted because, in my view, they point the way for future studies tofollow (in ways not done before) and are prototypal in their designand execution. Although the difficulty in researching thesupervision-patient outcome matter has long been lamented inthe supervision literature, those few studies (especially Bambling,King, Raue, Schweitzer, & Lambert, 2006) indeed show us for

Address correspondence to C. Edward Watkins, Jr., 1155 Union Circle # 311280,Psychology, UNT, Denton, TX 76203-5017. E-mail: watkinsc@unt.edu

The Clinical Supervisor, 30:235–256, 2011Copyright # Taylor & Francis Group, LLCISSN: 0732-5223 print=1545-231X onlineDOI: 10.1080/07325223.2011.619417

235

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

the first time that research on supervision-patient outcome can bedone and be done well.

KEYWORDS clinical supervision efficacy, clinical supervisionoutcome, patient outcome, psychotherapy supervision efficacy,psychotherapy supervision outcome

INTRODUCTION

In psychiatric and psychological practice, accountability has become anincreasingly pressing, preeminent issue (Thomason, 2010). The concepts ofcompetent practice, evidence-based practice, and empirically supportedtherapies have now become an ever more vital part of our treatment lexicon;they have substantially impacted how we think about treatment and treat-ment education, and are now seemingly forever inextricably intertwined withour conceptualizations of responsible, informed, and ethical therapeuticimplementation (American Psychological Association, 2006; AmericanPsychiatric Association, 2001; Fisher & O’Donohue, 2006; Spring & Walker,2007; Weisz & Kazdin, 2010). We now also see the concepts of competentpractice and evidence-based practice increasingly becoming more centralconsiderations in how we think about, conduct, teach, and even supervisepsychotherapy supervision (Falender & Shafranske, 2007; Milne, 2009). Muchas psychotherapy has been called upon to account, psychotherapy super-vision now finds itself called to do so as well.

While concerns about evidence for and impact of psychotherapy haveactually been with us for more than a century (Coriat, 1917; Eysenck, 1952;Freud, 1909=1959; Smith, Glass, & Miller, 1980; Wampold, 2008), concernsabout evidence for and impact of psychotherapy supervision have compara-tively been a much more recent phenomenon. The first empirical efforts toinvestigate supervision did not occur until the 1950s (Harkness & Poertner,1989), and in many respects, it has only been within the past 30 years thatsupervision scholarship and study have truly exploded (cf. Bernard, 2005).As psychotherapy supervision has matured and become ever more substan-tial, questions about its efficacy have increasingly emerged (Inman & Ladany,2008; Lambert & Ogles, 1997). Those questions have tended to take one oftwo forms: Does supervision have a beneficial effect on supervisees (thepositive impact of supervisor on supervisee)?, or Does supervision actuallyhave a beneficial effect on supervisees’ patients (the positive impact ofsupervisor on supervisee, which in turn positively impacts patients)?

Research thus far suggests that psychotherapy supervision indeed has amost beneficial effect on supervisees. Some of those positive effects includesupervisee enhanced self-awareness, enhanced treatment knowledge, skillacquisition and utilization, enhanced self-efficacy, and strengthening of the

236 C. E. Watkins, Jr.

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

supervisee–patient relationship (Beutler & Kendall, 1995; Goodyear &Guzzardo, 2000; Holloway & Neufeldt, 1995; Inman & Ladany, 2008; Lambert& Ogles, 1997; Wheeler & Richards, 2007). But research that examines theimpact of psychotherapy supervision on patient outcomes has proven muchmore of a challenge. Admittedly, it is a most difficult endeavor to trace triadicimpact—the effect of the supervisor=supervision experience as processedthrough the supervisee upon the patient (Wampold & Holloway, 1997).Yet the importance of this type of outcome research cannot be overempha-sized; it was identified as a significant press or need for twenty-first-centurypsychotherapy supervision well over a decade ago (Watkins, 1998), con-tinues to be referred to as the real ‘‘acid test’’ or ‘‘gold standard’’ of super-vision efficacy (Bernard & Goodyear, 2009; Ellis & Ladany, 1997), and callsfor that test or standard to be substantively addressed continue to be issued(Lichtenberg, 2007; Watkins, 2011; Westefeld, 2009).

Attention to the supervision-patient outcome issue began to take moresolid form and gather momentum in the mid to late 1990s. Holloway andNeufeldt (1995), in their review of supervision effects, indicated that researchon the supervision-patient outcome matter was virtually nonexistent (cf.Neufeldt, Beutler, & Banchero, 1997). But shortly thereafter, Ellis and Ladany(1997) identified what to my knowledge was the first list of supervision-patient outcome studies, which included nine such investigations; their briefcritique highlighted a host of methodological problems with that researchand they concluded that there were ‘‘few justifiable conclusions [that couldbe drawn] from this set of studies’’ (p. 488). Building on that list, however,Freitas (2002) provided a more detailed summary and analysis of most ofthe nine studies identified by Ellis and Ladany (e.g., number of participantsinvolved, specific measures used) and included and detailed four otherstudies that had not earlier been referenced; he reported no supervisionoutcome studies appearing between 1997 and 2001. While not skirting themethodological problems evident across studies, Freitas presented a moreoptimistic view about the available research and focused his attention onwhat the data had to offer for future research considerations. Five years later,Wheeler and Richards (2007) devoted but a paragraph to supervision-patientoutcome, referenced only two other supervision outcome studies appearingsince the Freitas review, and noted the limited attention given to thisall-important matter. Inman and Ladany (2008) also devoted just 1 paragraphto supervision-patient outcome research, indicated that about 18 such studieshad been done so far, identified 1 new study, and much like previousreviewers, accentuated the difficulty in researching this subject.

In my view, the essence of this outcome concern—despite its problem-atic researchability—was perfectly captured by Lichtenberg (2007) in his briefcommentary: ‘‘ . . . the reason for providing supervision and the ethical justi-fication for requiring it are that it makes a difference with respect to clientoutcomes’’ . . . . ‘‘supervisors’ impact on psychotherapy outcomes is critical

Psychotherapy Supervision Outcomes 237

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

in terms of evaluating the effectiveness of supervision and the ethics ofsupervision practices’’ (p. 275). If we cannot show that supervision affectspatient outcome, then how can we continue to justify supervision? The ben-efits of supervision on supervisees alone are not necessarily sufficient; whilevaluable, they at best only provide us with an indirect link to patient out-come. At this point in time, evidence of a direct link seems to be increasinglyimperative. As Watkins (2011) has stated, ‘‘the effectiveness question must becompellingly addressed if supervision is to ever move beyond ‘the reason-able but unproven practice stage’ and convincingly justify itself . . . . After acentury of existence, proof of supervision effectiveness for patient outcomeseems long, long overdue’’ (p. 63).

As calls for supervision-patient outcome research continue to be made(Bernard & Goodyear, 2009; Ellis & Ladany, 1997; Lichtenberg, 2007;Watkins, 1998, 2011; Westefeld, 2009), as supervision has been and is beingincreasingly called upon to meet the competence and evidence-based chal-lenge (see Milne, 2009, 2010; Stoltenberg, 2009; Stoltenberg & McNeill, 2009),as some study of the supervision-patient outcome issue continues to beconducted (Bambling, King, Raue, Schweitzer, & Lambert, 2006; Bradshaw,Butterworth, & Mairs, 2007; White & Winstanley, 2010), and as we nowalready have about 18 such studies (Inman & Ladany, 2008) spanning a gen-eration of supervision research (from 1981 through mid-2011), I thought itmight be interesting to take a fresh look at this matter. Since Freitas’ (2002)review, which largely built upon the work of Ellis and Ladany (1997), noeffort has been made to integrate the first two decades of supervision-patientoutcome research with research produced in the past decade. Where do wenow stand with regard to those ‘‘acid test’’ data? I would like to consider thatquestion by assembling all previously identified supervision-patient outcomestudies, complement those by including any recent studies that have beenconducted, and then examine the group of investigations as a whole forany new insights or directions that they might have to offer. What doesour first generation of supervision-patient outcome research tell us?

DEFINITIONS AND METHOD

Psychotherapy will be defined as psychological treatment for issues that areprimarily psychological in nature, offered by mental health professionalsfrom various disciplines (e.g., psychology, psychiatry, social work, psychi-atric nursing, and counseling). Psychotherapy supervision (or clinical super-vision) will be defined as follows:

. . . an intervention provided by a more senior member of a profession to amore junior member or members of that same profession. This relation-ship is evaluative and hierarchical, extends over time, and has the simul-taneous purposes of enhancing the professional functioning of the more

238 C. E. Watkins, Jr.

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

junior person(s); monitoring the quality of professional services offered toclients that she, he, or they see; and serving as a gatekeeper for those whoare to enter the particular profession. (Bernard & Goodyear, 2009, p. 7)

That definition will be used here because (1) it is a widely accepted super-vision definition in the United States and abroad (Milne, 2007), (2) its ele-ments seem to nicely capture the essence of supervisory practice acrossmental health specialties (cf. Gold, 2006; Hess, Hess, & Hess, 2008; Munson,2001; Watkins, 1997), and (3) it distinguishes supervision as an actualintervention separate from graduate course work and psychotherapy skillstraining (cf. Hill & Lent, 2006; Robertson, 1995; Stein & Lambert, 1995). Asupervision-patient outcome study will be defined as follows: A study inwhich (1) one of its stated objectives is the investigation of a supervision-patient outcome link and=or (2) a measure (or measures) of patient outcomeis taken over time and that is then related back to supervision in some way. Ichose not to adopt stringent exclusionary review criteria that would onlyfurther restrict an already restricted field of supervision-patient outcome stu-dies. Instead, I chose to include any quantitative or qualitative investigationsthat emerged, but then subject them to critical examination.

Articles to be included in this review were drawn from two sources: (1)studies that had been previously identified by Ellis and Ladany (1997), Freitas(2002), Wheeler and Richards (2007), and Inman and Ladany (2008); and (2)a computer search for any new studies that would have appeared in the pastfew years. Drawing on the reviews of Ellis and Ladany, Freitas, Wheeler andRichards, and Inman and Ladany, 16 studies—which spanned from 1981through 2006—were identified for inclusion (see Table 1). Using PsycINFOand Google Scholar databases, computer searches were conducted for theJanuary 2006 through May 2011 period, with such keywords as ‘‘psycho-therapy supervision outcomes,’’ ‘‘clinical supervision outcomes,’’ and ‘‘super-vision outcomes’’ being inputted for article identification purposes. For thatapproximate five-year period, two additional articles were identified forinclusion (Bradshaw et al., 2007; White & Winstanley, 2010), bringing thetotal number of supervision-patient outcome articles from 1981 throughmid-2011 to 18. A synopsis of each of the 18 articles is provided in Table 1.

RESULTS AND DISCUSSION

Those 18 studies have involved the full spectrum of mental health disciplines,with the most recent investigations emerging from psychiatric nursing(Bradshaw et al., 2007; White & Winstanley, 2010). The studies have rangedfrom experimental research (Bambling et al., 2006), correlational research(Harkness, 1995), case studies (Alpher, 1991), to surveys of perceptionsand opinions (Vallance, 2004). Participating supervisors and patients have

Psychotherapy Supervision Outcomes 239

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

TABLE 1 Eighteen Actual or Purported Psychotherapy Supervision-Patient Outcome Studies, 1981–May 2011

Author Study description Findings Limitations

Alpher (1991) NSOS: Intensive case study analysis involving 1 female patient, 1male therapist, and 1 male supervisor. Purpose of study toexamine parallel process between therapy and supervision.Patient improvement ratings across treatment were made bypatient, therapist, and supervisor. Structural Analysis of SocialBehavior ratings made by patient (for therapist) and therapist(for patient and supervisor). Supervisor provided ratings of hisrelationship with therapist. The effects of supervision onpatient outcome not considered at any point in article.

Bambling,King, Raue,Schweitzer,& Lambert(2006)

127 patients (87 f, 40m), 127 therapists (96 f, 31m), and 40supervisors (31 f, 9m) participated. Patients with majordepression were randomly assigned to receive 8 sessions ofproblem-solving therapy (PST) from either a supervised orunsupervised therapist; the 3 supervision conditions, to whichtherapists were randomly assigned, were alliance skill focus,alliance process focus, and no supervision: all therapistsreceived manual-driven training on PST; all supervisorsreceived manual-driven training in either alliance skill oralliance process supervision. Effects of supervision onclient-rated working alliance and symptom reduction wereevaluated.

Patients in supervised asopposed to unsupervisedtreatment rated the workingalliance higher, theirsymptoms lower, theirsatisfaction with treatmenthigher, and were more apt tostay in treatment.

Supervision pretreatmenttraining session and therapistallegiance effects wereidentified as potentialconfounds; total power wasinsufficient to eliminatepossibility of Type II errors.

Bradshaw,Butterworth,& Mairs(2007)

89 schizophrenic patients (sex not specified), 23 mental healthnurses (14 f, 9m), and several nurse supervisors (number notspecified) participated. Supervisors received 2-day courseabout clinical supervision from study’s first author. All nursesreceived 36 days of formal training in PsychosocialIntervention (PSI) and small-group clinical supervision. Inaddition, those nurses assigned to experimental group alsoreceived workplace clinical supervision (whereas controlgroup nurses did not). Nurses’ knowledge about seriousmental illness and patients’ symptom changes were assessed.

Both experimental and controlgroups showed significantincreases in casemanagement knowledge andtheir patients demonstratedsignificant reductions inaffective and positivesymptoms and significantimprovements in socialfunctioning; nurses in

Nurses in experimental groupsignificantly older and moreexperienced than controlgroup nurses; onlysupervision training forsupervisors was a 2-daycourse, which was notdescribed; no non-PSIeducation control group;retrospective comparison

240

Downloaded by [220.255.1.173] at 17:50 27 June 2013

Data were gathered twice—at very beginning of PSI trainingand at its end.

experimental group,however, also showedgreater knowledge aboutpsychological interventionand schizophrenia and theirpatients demonstratedsignificantly greaterreductions in both positiveand total symptoms.

group used; studyquasi-experimental indesign; ‘‘[s]upervisors’ fidelityto the model of clinicalsupervision was assessed viamonthly meetings with thefirst author’’ (p. 6) butnothing beyond that singlestatement was offered tohelp us understand what thatentailed.

Couchon &Bernard(1984)

32 clients (19 f, 13m), 21 counselors (17 f, 4m), and 7 supervisors(3 f, 4m) participated. Clients were being seen at a universityclinic for a variety of personal issues. Effects of timing ofsupervision were examined across three conditions:supervision occurring (1) within 4 hours of next counselingsession, (2) 1 day before next counseling session, or (3) 2 daysbefore next counseling session. All supervision and counselingsessions were audio-taped and rated for strategies generatedand time orientation employed. Client satisfaction andcounselor satisfaction ratings were taken.

Supervision approximately4 hours before nextcounseling session emergedas more of a ‘‘planningsession’’ comparatively.Supervisor tended tofunction as more of aconsultant and be morefocused than in other 2treatment conditions.Supervision session timing,however, had no significanteffect on either client orcounselor satisfaction.

Inadequate sample size (Ellis &Ladany, 1997); while eachcounselor was to see adifferent client in each of the3 timing conditions, thatobjective was not achieved;instead, ‘‘some counselorssaw the same client twice(under different treatmentconditions)’’ (Couchon &Bernard, 1984, p. 6).Homemade measurescreated to assess counselorsatisfaction with supervisionand counseling.

Dodenhoff(1981)

59 master’s-level counseling student therapists (34 f, 25m) and 12supervisors (5 f, 7m; 8 PhDs, 2 master’s degrees, 2 doctoralstudents) participated; number of patients involved notspecified; focus of study was on supervision as a socialinfluence process; student therapists completed aninterpersonal attraction (to supervisor) measure at week 3 ofsemester, supervisors completed (1) an effectiveness measurefor their supervisees (week 3 and at semester’s end) and (2) anoutcome measure for their supervisees’ patients (around week

Student therapists who scoredhigher on interpersonalattraction toward supervisorwere rated to be moreeffective by their supervisors;higher ratings of patientoutcome were associatedwith a direct supervisorystyle.

Patient outcome only rated atone point in time (aroundfifth session); no pre-postpatient outcome datacollected; no randomassignment; modification ofnon-equivalent groupsdesign used.

(Continued )

241

Downloaded by [220.255.1.173] at 17:50 27 June 2013

TABLE 1 Continued

Author Study description Findings Limitations

5), and patients also completed the same outcome measurearound week 5.

Friedlander,Siegel, &Brenock(1989)

NSOS: Intensive case study analysis involving 1 patient, 1 femaletherapist, and 1 female supervisor. Purpose of study toexamine parallel process between therapy and supervision.Across triad, 7 pre-treatment measures taken, 12 measurestaken during treatment, and 4 measures taken posttreatment.Therapist rated patient satisfaction at end of treatment; patientreported being ‘‘too busy’’ to complete any posttreatmentmeasures. If a prime objective was to investigate effects ofsupervision on patient outcome, that objective was neverstated, and absent any patient posttreatment data, that seemsvirtually impossible to achieve.

Harkness(1995)

Same data set=measures=participants from Harkness andHensley (1991; see below) used. Test of Shulman’sinteractional helping theory. 36 correlations performed toexamine possible relations between supervisory skills,relationship, and helpfulness and practice and clientoutcomes.

Significant relationships foundbetween therapist ratings ofsupervisory empathy andclient ratings of contentment,supervisory helpfulness andclient contentment, andsupervisory relationship andclient contentment and goalattainment.

‘‘Weak experimental controlover volunteer subjectsincreased error variance’’(Harkness, 1995, p. 70);‘‘Causal inferences cannot bemade from the findings ofthis investigation, and thenarrow scope of its samplemitigates againstgeneralization about skills,relationships and outcomesof practice in other settings’’(Harkness, 1995, p. 70);‘‘ . . .one is left wonderingwhat can be inferred . . .’’(Freitas, 2002, p. 361).

Harkness(1997)

Same data set=measures=participants from Harkness andHensley (1991; Harkness, 1995; see subsequent entry) againused; test of Interactional Social Work theory; multiple

Findings interpreted assupporting and alteringinteractional view of social

Large number of causal testsconducted; limitations underHarkness (1995) and

242

Downloaded by [220.255.1.173] at 17:50 27 June 2013

regression in cross-lagged panel design employed; examinedcausal connections among the skills, relationships, andoutcomes of supervised practice; 81 causal tests of associationconducted.

work practice; supervisionproblem-solving andempathic skills found to havea causal influence on ratingsof supervisory relationshipand supervisory helpfulnessbut not in ways expected;empathy actually found tohave a direct negative effecton the supervisionexperience (a truly uniquefinding in empathy=supervision research).

Harkness and Hensley(1991) entries also applyhere (see above).

Harkness &Hensley(1991)

161 patients (87 f, 74m), 2 male therapists (master’s-levelpsychologists), 2 female therapists (master’s-level socialworkers), and 1 female supervisor (certified social worker)participated; 4 therapists first exposed to 8 weeks ofmixed-focus supervision (case management=consultation)followed by mix of 8 weeks of client-focus (individual=groupsupervision)=mixed-focus supervision. Effects of mixed-focusversus client-focus=mixed-focus supervision were evaluatedfor: depressive symptoms, patient satisfaction with therapisthelpfulness, goal attainment, and patient-therapist partnership.

Under client-focus=mixedfocus-supervision asopposed to mixed-focus(only) supervision, patientdepression decreased;ratings of therapisthelpfulness, goal attainment,and patient-therapistpartnership all increased.

Multiple-baseline researchdesign; ‘‘any combination oforder effects, sampling error,and the interaction of testingand treatment may haveconfounded the findings’’(Harkness & Hensley, 1991,p. 511); throughout article,client-focused supervision iscontrasted with mixed-focussupervision but in realityclient-focused group was asupervision amalgam, not apure type.

Iberg (1991) NSOS: Examined what were designated as 3 supervision themes:Give more empathic responses, don’t give advice orsuggestions, and don’t ask questions. 6 doctoral studenttherapists in clinical psychology (5 f, 1m) each recruited 8volunteer pseudo-clients (27 f, 21m) to participate. Therapistswere assigned to each level of the 3 experimental conditions(empathy: few versus many statements; suggestions:

(Continued )

243

Downloaded by [220.255.1.173] at 17:50 27 June 2013

TABLE 1 Continued

Author Study description Findings Limitations

volunteered versus withheld; questions: asked versus notasked) across their 8 volunteer clients. Results analyzed viaStatistical Control Theory. No supervisors or supervisioninvolved. This was a study of therapy skill developmentspecifically where pseudo-clients (e.g., friends) were used forrole-play purposes. Results could be useful in informingsupervision but had nothing to do with the actual effects ofsupervision on patient outcome.

Kivlighan,Angelone, &Swafford(1991)

48 undergraduate volunteers (39 f, 9m; awarded extra coursecredit), 48 master’s-level counseling student therapists (29 f,19m) and 17 supervisors (sex unspecified); 1 PhD and 16doctoral students) participated; recruited volunteer patientsseen for 4 50-minute therapy sessions; effects of videotapedsupervision (tape review) versus live supervision werecompared on patients’ ratings of working alliance and sessionsmoothness-ease and depth-value.

Patients in live as opposed tovideotaped supervision ratedthe working alliance higherand their sessions asrougher; authors concludedthat live supervision mayhave an accelerative learningeffect on supervisees.

Due to clerical error, halfof the working alliancequestionnaire data couldnot be used; no randomassignment involved;pre-experimental,non-equivalent groupsdesign used.

Mallinckrodt &Nelson(1991)

NSOS: 50 counselor-client dyads participated; after third session,counselor and client completed Working Alliance Inventory;effects of counselor training level (novice, advanced,experienced) on working alliance formation investigated;though having potential supervision implications, this studyinvolved no supervision at all.

Milne,Pilkington,Gracie, &James (2003)

NSOS: Intensive qualitative and quantitative case study analysisinvolving 1 male patient, 1 female therapist, and 1 malesupervisor. Purpose of study to examine thematic contentsimilarities between cognitive-behavioral therapy andsupervision. 10 supervision and 10 therapy sessions codedusing qualitative and quantitative content analysismethodology. Supervision themes nicely reflected in therapyprocess; a ‘‘parallelism’’ occurred. No measures of patientoutcome taken, effects of supervision on patient outcome notaddressed. Focus of study on the process of supervision tospecifically impact therapist behavior.

244

Downloaded by [220.255.1.173] at 17:50 27 June 2013

Sandell (1985) NSOS: 20 psychiatric outpatients (sex unspecified) and 5psychotherapists (sex unspecified) participated. Purpose ofstudy to test influence of patient’s ego level, therapistcompetence, and supervision on effects of Mann’s time-limitedpsychodynamic psychotherapy. Interview summary ratingsprovided the bulk of the data. Supervision was a binaryvariable, either received or not and did not involve aprofessional supervisor but was peer in nature. Supervisionwas conducted ‘‘in groups of 3-4 persons who met once aweek for mutual supervision of 1 case each’’ (Sandell, 1985,p. 105). Data were analyzed by means of path analysis, andsupervision was judged to have ‘‘had, if anything, a negativeinfluence’’ (p. 103).

Steinhelber,Patterson,Cliffe, &LeGoullon(1984)

Data gathered from 237 psychiatric patients (154 f, 83m) ofmixed diagnoses and 51 therapist trainees (sex not specified;primarily psychiatric residents and clinical psychology interns).Therapist trainees (1) completed questionnaire about patients,their treatment, and supervision matters and (2) completed aGlobal Assessment Scale (GAS) for each patient at treatment’sbeginning and at time of study. Effects of amount ofsupervision and therapist-supervisor theoretical congruenceon patient outcome were evaluated.

When therapist trainees andsupervisors shared similartheoretical orientation,therapist trainees’ GASratings reflected greaterpatient improvement;amount of supervision wasunrelated to trainee ratingsof patient GAS.

Survey study; ex post factodesign; patient outcomemeasured by but asingle-item global ratingmade by trainee at 2 differenttimes; cause-effectconclusions not possible.

Triantafillou(1997)

14 supervisory=management staff and 10 direct care workers (allwith at least a 3-year child=youth worker college diploma)received 4, 3-hour weekly training sessions in solution-focusedsupervision (SFS). To test effects of training, 5 problemresidents from one residential facility (where SFS was taught tostaff=workers) were compared against 7 problem residentsfrom a different facility (where SFS not taught) on seriousincidence behaviors and medication usage. Pilot study oftraining program package. No actual supervision observed ormeasured.

Both treatment and controlgroups showed reduction inserious incidence behaviorsduring study but treatmentgroup showed substantiallymore (75% less than controlgroup).

Pilot study with very smallsample size; no randomassignment; extremelylimited study details (entirestudy given but a single pagein 23-page article).

(Continued )

245

Downloaded by [220.255.1.173] at 17:50 27 June 2013

TABLE 1 Continued

Author Study description Findings Limitations

Vallance (2004) NSOS: Qualitative study of 19 practicing counselors receivingindividual supervision; objective to examine counselors’perceptions of supervision’s impact on their clients;perceptions assessed via open-ended questionnaires orsemi-structured interviews; no actual supervision observed,measured, or studied.

White &Winstanley(2010)

170 patients (sex not specified), 186 mental health nurses (sexnot specified), 54 unit staff (sex not specified), and 24 nursesupervisors (17 f, 7m) participated. 2 nurses received 4-dayresidential course in clinical supervision, which involvedtheory, practice, and direct feedback, and were then assignedto conduct yearlong group supervision of neophytesupervisees at their worksites. The control group was ano-supervision condition. Qualitative diary data were collectedmonthly from supervisors; quantitative data were collectedfrom mental health nurses (baseline=12 months), patients(baseline=6 and 12 months), and unit staff (6 months).

Nurse supervisors scoredsignificantly higher onManchester ClinicalSupervision Scale (MCSS)after 4-day training courseand maintained differencescores 12 months later;supervisee MCSS scores didnot change significantly over12 months of supervision;statistically significantdifferences not demonstratedin either quality of care orpatient satisfaction.

Supervision in this studyseemingly defined asoccurring in ‘‘small groups ofindividuals (n-6) attending apre-arranged meeting withan appropriately trainedclinical supervisor, for 45–60minutes per session, on amonthly frequency, forfacilitated reflectivediscussion, in confidence,around matters ofprofessional relevance andimportance’’ (p. 152);influence of middlemanagers facilitated orfrustrated supervision effortsacross work settings.

Notes. NSOS¼Not Supervision Outcome Study; if a study was judged NSOS, then no Findings or Limitations were provided. f¼ females, m¼males.

246

Downloaded by [220.255.1.173] at 17:50 27 June 2013

run the full gamut as well—from recruited undergraduate or community vol-unteer pseudo-clients (Iberg, 1991) to actual patients with schizophrenia(Bradshaw et al., 2007), from experienced supervisors (Bambling et al.,2006) to those given only two days or less of supervisory training (Bradshawet al., 2007; Triantafillou, 1997) before being designated ‘‘supervisor.’’ Withbut two exceptions, where qualitative analysis was employed exclusivelyor in part (Milne, Pilkington, Gracie, & James, 2003; Vallance, 2004), all ofthe studies were quantitative in design and analysis.

In surveying the first 20 years (1981–2000) and last 10 years (2001–mid-2011) of this outcome research, one of the most surprising results frommy perspective is this: Approximately 40% of those investigations actuallyidentified as being supervision-patient outcome in nature have been misi-dentified and are not really supervision-patient outcome studies at all. Sevenof the 18 studies fall into that category by my review: Alpher (1991);Friedlander, Siegel, and Brenock (1989); Iberg (1991); Kivlighan, Angelone,and Swafford (1991); Mallinckrodt and Nelson (1991); Milne and colleagues(2003); and Sandell (1985). Alpher (1991) and Friedlander and colleagues(1989) were both case studies of the parallel process phenomenon. Whilestudy of parallel process would seem to readily lend itself to supervision-patient outcome inferences, that is not necessarily a given. In Alpher(1991), patient improvement ratings were taken, but at no point in his articlewere those in any way considered in relation to supervision and patientoutcome. Friedlander and colleagues (1989) were also appropriately cautiousin considering the implications of their data; while the therapist providedratings about her patient’s outcome, her client failed to provide any post-treatment outcome data, thus limiting any conclusions that could be drawn.Friedlander and colleagues (1989) mentioned no supervision-outcome link atany point in their article. Milne and colleagues (2003) studied thematic simi-larities between cognitive-behavioral therapy and supervision (parallelism)in an intensive case design; their investigation’s focus was on how super-vision actually impacted therapist behavior. The effects of supervision onpatient, however, were not considered.

Mallinckrodt and Nelson (1991), while conducting a study that may havesupervision implications, investigated the impact of therapist training level(novice, advanced, experienced) on formation of the therapeutic workingalliance. This was strictly a study of psychotherapy; no supervision wasinvolved. Iberg (1991), while orienting his study around ‘‘supervisionthemes,’’ actually conducted an investigation of psychotherapy skills training(e.g., learning empathic skills), not supervision. Six doctoral-level clinical psy-chology students recruited pseudo-clients for role-play purposes, with theskills focus being on empathy, questions, and suggestions. No supervisionor supervisors were involved at any time. Sandell (1985) provided data about‘‘voluntary small-group peer supervision’’ on patient outcome in short-termpsychodynamic psychotherapy; again, trained professional supervision was

Psychotherapy Supervision Outcomes 247

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

not part of this study. Kivlighan and colleagues (1991) found that recruited,compensated (extra course credit) undergraduate ‘‘clients,’’ when exposedto supervisees receiving live versus videotaped supervision, rated their fourcounseling sessions as rougher and the working alliance as stronger duringthe live supervision condition. No actual change ratings were taken or made;this appeared to primarily be an investigation of how live versus videotapedsupervision modalities affected clients’ session and working alliance percep-tions. How that impacted overall outcome, however, was not assessed. It ismy contention that those seven studies have been identified as supervision-patient outcome studies (see Ellis & Ladany, 1997; Freitas, 2002; Wheeler &Richards, 2007) when in fact they are not. While some supervision implica-tions may be drawn from some of those studies, I believe we should bequite cautious about specifically drawing any supervision-patient outcomeimplications from them.

Unfortunately, I do not believe those are the only studies in this groupof 18 that give us reason for question or concern. We also have studies herethat are purely survey or opinion based (Steinhelber, Patterson, Cliffe, &LeGoullon, 1984; Vallance, 2004), pilot in nature (Triantafillou, 1997), involvedata duplication (see Harkness, 1995, 1997), or failed to adequately assesspatient outcome (Dodenhoff, 1981). Steinhelber and colleagues (1984) eval-uated therapist trainees’ questionnaire and single-item rating scale responsesabout patient diagnosis and progress, treatment variables, and supervision;this was really a study about therapists’ perceptions of self, their supervisors,and their patients. Though the trainees were all in supervision, no actualsupervision was observed or studied. Vallance (2004) conducted a qualitativestudy in which she investigated 19 counselors’ opinions about the impactof supervision on their treatment efforts and its ultimate impact on patientoutcome. This, too, was a study of counselors’ perceptions about self, theirclients, and their supervisors, tapped by means of either an open-endedquestionnaire or semi-structured interview. Again, though all participantswere being supervised, no actual supervision was observed or studied in thisresearch either. (While I think that therapist perceptual data alone can beimportant and have a place in supervision research, I also think that at thisparticular point in time we need much more than that for the supervision-patient outcome area of inquiry to move forward.)

With Triantafillou (1997), we have a supervision training package (four3-hour meetings of supervision instruction) pilot tested on 12 behaviorallydisturbed residents (5 in treatment group versus 7 in control group); super-visors trained in solution-focused supervision served the treatment group,whereas the control group served as a no-supervision condition. While muchattention and care appears to have been given to the training package pres-entation, no monitoring, observation, or study of actual supervision occurredonce that was complete. Harkness and Hensley (1991) conducted an interest-ing study that involved 1 supervisor, 4 supervisees, and 161 mental health

248 C. E. Watkins, Jr.

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

center patients, but much like cutting the same pie up time and time again,those very same data were reanalyzed on two other occasions (Harkness,1995, 1997). While Harkness’ two later studies have been recognized as partof our body of supervision outcome studies (see Freitas, 2002, who did men-tion the matter of piecemeal publication), I believe that we can certainlyquestion that wisdom. If nothing else, those are not new data sets, and whilewe may cautiously draw implications from them, any such implicationswould probably best be placed within the confines of Harkness andHensley’s first study (1991). In Dodenhoff’s (1981) investigation, both clientand supervisor completed a client outcome rating scale around the fifthcounseling session; no pretreatment or posttreatment measure was taken,so any type of pre=post comparison was not possible. This was foremost astudy about supervisor social influence, so the supervisor and client outcomeratings were used exclusively as measures of counselor trainee effectiveness(seen as being influenced by supervisor social power). Consideration ofsupervision-patient outcome was not a prime objective, if an objective at all.

What does this misidentification, data duplication, pilot, survey=opinion, and inadequate measurement attention and mention mean for ourexamination of psychotherapy supervision-patient outcome? We may havethought that we had a body of 18 studies on this subject but I do not believethat that is really the case at all. From my perspective, if we are to truly con-sider the more solidly constructed and conducted investigations, we actuallyhave only three such studies upon which we can now draw, all publishedwithin the past five years: Bambling and colleagues (2006); Bradshaw andcolleagues (2007); and White and Winstanley (2010). While investigationsby Couchon and Bernard (1984) and Harkness and Hensley (1991) alsoexamined supervision-patient outcome, those each suffered from some ser-ious methodological deficiencies that sorely limit the strength of their find-ings (e.g., condition confounds, homemade measures, uncontrolled Type Iand Type II error; Ellis & Ladany, 1997; Freitas, 2002). And while the researchof Bambling and colleagues (2006), Bradshaw and colleagues (2007), andWhite and Winstanley (2010) is certainly not deficiency free (see Limitationsin Table 1), their studies in my view provide us with new light—giving ussubstantive and prototypal direction for thinking about and tackling theever-challenging conundrum of supervision-patient outcome. Let us lookmore closely at each of those.

The truly stellar, model study is Bambling and colleagues (2006; seeTable 1 for more detail). This was foremost an investigation undertaken toplainly and simply research the supervision-patient outcome issue, and fourof its hypotheses directly compared supervised versus unsupervised treat-ment on patient variables, with depression symptom reduction and treatmentcompletion being two of them. Experienced therapists and supervisors(primarily from psychology and social work) were used, all patients had adiagnosis of major depression, and eight sessions of supervision were

Psychotherapy Supervision Outcomes 249

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

provided. There was one other major study difference: A supervision manualwas used to guide the work of supervision. (Therapists also were providedwith their own treatment manual.) To my knowledge, this is only the secondsupervision study (other than Patton & Kivlighan, 1997) to ever use a super-vision manual. While manuals have a definite history of being referenced andrecommended in the supervision literature (Bernard & Goodyear, 2009;Goodyear & Bernard, 1998; Goodyear & Guzzardo, 2000; Holloway, 1992;Lambert & Arnold, 1987; Lambert & Ogles, 1997; Watkins, 1998), their usein supervision research has been virtually absent. It may indeed be that the‘‘most promising possibility for outcome research on the effects of supervi-sion . . . appears to be in the area of large-scale outcome studies of manua-lized treatments . . .’’ (Neufeldt et al., 1997, pp. 519–520). Perhaps whatBambling and colleagues have done is to also show us that ‘‘manualized’’has a place in supervision too: Just as psychotherapy research has benefitedfrom a manualized approach, supervision research could do so as well.While manuals are by no means a panacea (Scott & Binder, 2002), they doprovide us with one viable means of gathering useful research data (seeLambert, 2004) and might be useful in exposing a supervision data mine thathas yet to be unearthed, explored, and excavated.

Two recent research investigations from psychiatric mental health nurs-ing (Bradshaw et al., 2007; White & Winstanley, 2010; see Table 1 for moredetail), though not as refined as Bambling and colleagues’ study, also presenta highly focused approach to the specific study of supervision and patientoutcome that could prove quite informative for future study (NURSINGtimes.net, 2010; Proctor, 2010). Bradshaw and colleagues (2007) examined theeffects of supervision (provided by nurse ‘‘supervisors’’ receiving a two-daycourse in clinical supervision) on nurses receiving Psychosocial InterventionTraining (PIT; family and cognitive-behavioral intervention) plus workplacesupervision (treatment group) versus a group of nurses receiving PTI only(no workplace supervision control group). Supervision was provided everyother week, with there being one supervisor and two supervisees per group.Eighty-nine patients with schizophrenia served as the patient group. Patientswhose caregivers received PIT plus workplace supervision (compared withthe control group) experienced greater reductions in positive and total psy-chotic symptoms measured. While the Bradshaw and colleagues investi-gation was not a randomized control trial (RCT) and was certainly theweakest methodologically of the three studies highlighted here, I include itbecause (1) it had as a preeminent intent the examination of supervision-patient outcome; (2) it provided a corresponding structure that directlyallowed that to be done (contrary to most of the other 18 studies); and (3)it clearly has the potential to stimulate useful heuristic thought about pushingthe matter under study forward. White and Winstanley (2010) examined theeffects of supervision (provided by nurse ‘‘supervisors’’ receiving a four-dayresidential course in clinical supervision) on mental health nurses receiving

250 C. E. Watkins, Jr.

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

yearlong supervision (treatment group) versus nurses who received nosupervision (control group). One hundred seventy patients from variousmental health facilities served as the patient group, but after a year of studyand multiple measures being taken, no significant differences emergedbetween patients in the treatment and control groups on treatment satisfac-tion and quality of care. While this was a much-involved RCT, it appears thatmanagement issues across some of the participating mental health facilitiesmay have greatly affected the prosecution of the entire investigation, thesupervision provided, and the resulting outcomes (Proctor, 2010; White &Winstanley, 2010).

The Bradshaw and colleagues (2007) and White and Winstanley (2010)investigations raise two questions: (1) Does a two-day or four-day course inclinical supervision ‘‘make’’ a supervisor? In psychology, psychiatry, andsocial work, the project of becoming and being a supervisor has tended tobe viewed as a developmental process (Rodenhauser, 1994; Watkins, 2010)that involves growth over time. Would such a brief training period be suffi-ciently powerful to render one a good-enough supervisor? (2) How fre-quently must supervision occur for it to have a beneficial effect? InBambling and colleagues (2006), individual supervision was provided oncea week; in Bradshaw and colleagues (2007), supervision was provided totwo supervisees at a time every other week; in White and Winstanley(2010), supervision was provided in groups consisting of anywhere fromsix to nine individuals once a month. If the supervisory alliance is a criticalcomponent of supervision, and accumulating research suggests that it is(Inman & Ladany, 2008; Ladany, 2004), can such an alliance actually beestablished in groups of 6 to 9 supervisees meeting only once a month forabout 45 minutes? It may well be that another reason White and Winstanley’snicely conducted RCT failed to find supervision-patient effects was that theamount of time devoted to supervision was not sufficiently powerful to havean effect. As Proctor (2010) stated, ‘‘. . .makes me shake my head at the ideaof nine supervisees in a 1 hour (or less) monthly group’’ (p. 171). It may alsobe possible that some of what led to positive results in Bradshaw and collea-gues’ (2007) study was the more frequent supervision conducted with buttwo supervisees. Those possibilities would at least be reasonable hypothesesto consider by means of future research.

Of all the studies considered here, those three—Bambling and collea-gues (2006), Bradshaw and colleagues (2007), and White and Winstanley(2010)—provide the best and clearest directions for further thought aboutconducting future successful research in the supervision-patient outcomearea. Those investigations show us that, despite jeremiads about the difficult-ies of researching this subject, it can be done and be done well. As concernmounts about more substantively demonstrating the effectiveness of super-vision on patient outcome, we have our first vestiges, tentative though theymay be, of evidence to that effect. But beyond that, I really do not believe

Psychotherapy Supervision Outcomes 251

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

that we can say any more right now. When we take into account study mis-identifications along with other issues mentioned earlier (e.g., data dupli-cation), I am not sure that we can safely draw any further conclusionsabout supervision outcome from this very limited group of research investi-gations. What we do have, in my opinion, is a beginning, and I believe someof the work done in the past few years opens a door that had not beenopened before. For the first time in supervision’s 100-year history, thepsychotherapy supervision-patient outcome problem does not have to bethe will-o’-the-wisp it has always been.

CONCLUSION

Does psychotherapy supervision positively affect patient outcomes? After acentury of psychotherapy supervision and over half a century of supervisionresearch, we still cannot empirically answer that question. But in this age ofaccountability, we can be assured that that question will continue to be anincreasingly preeminent press for psychotherapy supervision in thetwenty-first century. In surveying the last 30 years of supervision outcomeresearch (actual and purported), the drawing of any conclusions aboutsupervision’s effects on patient outcome seems premature. In my view, wehave not arrived at the point where we can safely do that. But some recent,nicely done studies produced in the past few years do provide us with sub-stantive examples, exciting possibilities, and charted directions upon whichwe can build in our quest to further unravel the psychotherapysupervision-patient outcome riddle.

REFERENCES

Alpher, V. S. (1991). Interdependence and parallel processes: A case study of struc-tural analysis of social behavior in supervision and short-term dynamic psycho-therapy. Psychotherapy, 28, 218–231.

American Psychiatric Association. (2001). Practice guidelines for the treatment ofpsychiatric disorders. Washington, DC: American Psychiatric Press.

APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-basedpractice in psychology. American Psychologist, 61, 271–285.

Bambling, M., King, R., Raue, P., Schweitzer, R., & Lambert, W. (2006). Clinicalsupervision: Its influence on client-rated working alliance and client symptomreduction in the brief treatment of major depression. Psychotherapy Research,16, 317–331.

Bernard, J. M. (2005). Tracing the development of clinical supervision. The ClinicalSupervisor, 24, 3–21.

Bernard, J. M., & Goodyear, R. K. (2009). Fundamentals of clinical supervision (4thed.). Upper Saddle River, NJ: Merrill.

252 C. E. Watkins, Jr.

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

Beutler, L. E., & Kendall, P. C. (Eds.) (1995). Introduction to the special section: Thecase for training in the provision of psychological therapy. Journal of Consult-ing and Clinical Psychology, 63, 179–183.

Bradshaw, T., Butterworth, A., & Mairs, H. (2007). Does structured clinical super-vision during psychosocial intervention education enhance outcome for mentalhealth nurses and the service users they work with? Journal of Psychiatric andMental Health Nursing, 14, 4–12.

Coriat, I. H. (1917). Some statistical results of the psychoanalytic treatment of thepsychoneuroses. The Psychoanalytic Review, 4, 209–216.

Couchon, W. D., & Bernard, J. M. (1984). Effects of timing of supervision on super-visor and counselor performance. The Clinical Supervisor, 2, 3–20.

Dodenhoff, J. T. (1981). Interpersonal attraction and direct-indirect supervisor influ-ence as predictors of counselor trainee effectiveness. Journal of CounselingPsychology, 28, 47–52.

Ellis, M. V., & Ladany, N. (1997). Inferences concerning supervisees and clients inclinical supervision: An integrative review. In C. E. Watkins, Jr. (Ed.), Handbookof psychotherapy supervision (pp. 447–507). New York: Wiley.

Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal ofConsulting Psychology, 16, 319–324.

Falender, C. A., & Shafranske, E. P. (2007). Competence in competency-based super-vision practice: Construct and application. Professional Psychology: Researchand Practice, 38, 232–240.

Fisher, J. E., & O’Donohue, W. T. (2006). Practitioner’s guide to evidence-basedpsychotherapy. New York: Springer.

Freitas, G. J. (2002). The impact of psychotherapy supervision on client outcome: Acritical examination of 2 decades of research. Psychotherapy, 39, 354–367.

Freud, S. (1909=1959). Analysis of a phobia in a five-year-old boy. Collected papers(vol. 3, pp. 149–289). London: Hogarth Press.

Friedlander, M. L., Siegel, S., & Brenock, K. (1989). Parallel process in counsel-ing and supervision: A case study. Journal of Counseling Psychology, 36,149–157.

Gold, J. H. (Ed.). (2006). Psychotherapy supervision and consultation in clinicalpractice. New York: Aronson.

Goodyear, R. K., & Bernard, J. M. (1998). Clinical supervision: Lessons from theliterature. Counselor Education and Supervision, 38, 6–22.

Goodyear, R. K., & Guzzardo, C. R. (2000). Psychotherapy supervision and training.In S. D. Brown & R. W. Lent (Eds.), Handbook of counseling psychology (3rd ed.,pp. 83–108). New York: Wiley.

Harkness, D. (1995). The art of helping in supervised practice: Skills, relationships,and outcomes. The Clinical Supervisor, 13, 63–76.

Harkness, D. (1997). Testing interactional social work theory: A panel analysis ofsupervised practice and outcomes. The Clinical Supervisor, 15, 33–50.

Harkness, D., & Hensley, H. (1991). Changing the focus of social work supervision:Effects on client satisfaction and generalized contentment. Social Work, 36,506–512.

Harkness, D., & Poertner, A. (1989). Research and social work supervision: Aconceptual review. Social Work, 34, 115–119.

Psychotherapy Supervision Outcomes 253

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

Hess, A. K., Hess, K. D., & Hess, T. H. (Eds.). (2008). Psychotherapy supervision:Theory, research, and practice (2nd ed.). Hoboken, NJ: Wiley.

Hill, C. E., & Lent, R. W. (2006). A narrative and meta-analytic review of helpingskills training: Time to revive a dormant area of inquiry. Psychotherapy, 43,154–172.

Holloway, E. L. (1992). Supervision: A way of teaching and learning. In S. D. Brown& R. W. Lent (Eds.), Handbook of counseling psychology (pp. 177–214). NewYork: Wiley.

Holloway, E. L., & Neufeldt, S. A. (1995). Supervision: Its contributions to treatmentefficacy. Journal of Consulting and Clinical Psychology, 63, 207–213.

Iberg, J. R. (1991). Applying Statistical Control Theory to bring together clinicalsupervision and psychotherapy research. Journal of Consulting and ClinicalPsychology, 59, 575–586.

Inman, A. G., & Ladany, N. (2008). Research: The state of the field. In A. K. Hess,K. D. Hess, & T. H. Hess (Eds.), Psychotherapy supervision: Theory, research,and practice (2nd ed., pp. 500–517). Hoboken, NJ: Wiley.

Kivlighan, D. M., Angelone, E. O., & Swafford, K. G. (1991). Live supervision in indi-vidual psychotherapy: Effects on therapist’s intention use and client’s evaluationof session effect and working alliance. Professional Psychology: Research andPractice, 22, 489–495.

Ladany, N. (2004). Psychotherapy supervision: What lies beneath. PsychotherapyResearch, 14, 1–19.

Lambert, M. J. (Ed.). (2004). Bergin and Garfield’s handbook of psychotherapy andbehavior change. New York: Wiley.

Lambert, M. J., & Arnold, R. C. (1987). Research and the supervision process.Professional Psychology: Research and Practice, 18, 217–224.

Lambert, M. J., & Ogles, B. M. (1997). The effectiveness of psychotherapysupervision. In C. E. Watkins, Jr. (Ed.), Handbook of psychotherapy supervision(pp. 421–446). New York: Wiley.

Lichtenberg, J. W. (2007). What makes for effective supervision? In search of clinicaloutcomes. Professional Psychology: Research and Practice, 38, 27–28.

Mallinckrodt, B., & Nelson, M. L. (1991). Counselor training level and the formationof the psychotherapeutic working alliance. Journal of Counseling Psychology,38, 133–138.

Milne, D. (2007). An empirical definition of clinical supervision. British Journal ofClinical Psychology, 46, 437–447.

Milne, D. (2009). Evidence-based clinical supervision: Principles and practice.Hoboken, NJ: Wiley.

Milne, D. (2010). Can we enhance the training of clinical supervisors? A national pilotstudy of an evidence-based approach. Clinical Psychology and Psychotherapy,17, 321–328.

Milne, D., Pilkington, J., Gracie, J., & James, I. (2003). Transferring skills from super-vision to therapy: A qualitative and quantitative N¼ 1 analysis. Behavioural andCognitive Psychotherapy, 31, 193–202.

Munson, C. (2001). Handbook of clinical social work supervision (3rd ed.).Binghamton, NY: Haworth Press.

254 C. E. Watkins, Jr.

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

Neufeldt, S. A., Beutler, L. E., & Banchero, R. (1997). Research on supervisor vari-ables in psychotherapy supervision. In C. E. Watkins, Jr. (Ed.), Handbook ofpsychotherapy supervision (pp. 508–524). New York: Wiley.

NURSINGtimes.net. (2010). Does clinical supervision lead to better patient outcomesin mental health nursing? (Research report). (Originally appeared April 26,2010). Retrieved August 20, 2010, from http://www.nursingtimes.net./nursing-practice-clinical-research/mental-health/does-clinical-supervision-lead-to-better-patient-outcomes-in-mental-health-nursing/5013977.article

Patton, M. J., & Kivlighan, D. M., Jr. (1997). Relevance of the supervisory allianceto the counseling alliance and to treatment adherence in counselor training.Journal of Counseling Psychology, 44, 108–115.

Proctor, B. (2010). Review: A randomised controlled trial of clinical supervision:Selected findings from a novel Australian attempt to establish the evidence basefor causal relationships with quality of care and patient outcomes, as aninformed contribution to mental health nursing practice development. Journalof Research in Nursing, 15, 169–172.

Robertson, M. H. (1995). Psychotherapy education and training. Madison, CT:International Universities Press.

Rodenhauser, P. (1994). Toward a multidimensional model for psychotherapy super-vision based on developmental stages. Journal of Psychotherapy Practice andResearch, 3, 1–15.

Sandell, R. (1985). Influence of supervision, therapist’s competence, and patient’sego level on the effects of time-limited psychotherapy. Psychotherapy andPsychosomatics, 44, 103–109.

Scott, S. J., & Binder, J. L. (2002). The effects of manual-based training on treatmentfidelity and outcome: A review of the literature on adult individual psycho-therapy. Psychotherapy, 39, 184–198.

Smith, M. L., Glass, G. V., & Miller, T. I. (1980). The benefits of psychotherapy.Baltimore, MD: Johns Hopkins University Press.

Spring, B., & Walker, B. B. (Eds.). (2007). Special issue: Evidence-based practice inclinical psychology. Journal of Clinical Psychology, 63, 607–705.

Stein, D. M., & Lambert, M. J. (1995). Graduate training in psychotherapy: Are therapyoutcomes enhanced? Journal of Consulting and Clinical Psychology, 63, 182–196.

Steinhelber, J., Patterson, V., Cliffe, K., & LeGoullon, M. (1984). An investigation ofsome relationships between psychotherapy supervision and patient change.Journal of Clinical Psychology, 40, 1346–1353.

Stoltenberg, C. D. (2009). Evidence-based clinical supervision. Living Proof PodcastSeries [Audio Podcast]. Retrieved August 26, 2010, from http://www.socialwork.buffaloedu=podcast=episode.asp?ep¼ 12

Stoltenberg, C. D., & McNeill, B. W. (2009). IDM supervision: An integrated develop-mental model for supervising counselors and therapists (3rd ed.). San Francisco:Jossey-Bass.

Thomason, T. C. (2010). The trend toward evidence-based practice and the future ofpsychotherapy. American Journal of Psychotherapy, 64, 29–38.

Triantafillou, N. (1997). A solution-focused approach to mental health supervision.Journal of Systemic Therapies, 16, 305–328.

Psychotherapy Supervision Outcomes 255

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3

Vallance, K. (2004). Exploring counsellor perceptions of the impact of counsellingsupervision on clients.British Journal of Guidance and Counselling, 32, 559–574.

Wampold, B. E. (2008). The great psychotherapy debate (2nd ed.). Mahwah, NJ:Lawrence Erlbaum.

Wampold, B. E., & Holloway, E. L. (1997). Methodology, design, and evaluation inpsychotherapy supervision research. In C. E. Watkins, Jr. (Ed.), Handbook ofpsychotherapy supervision (pp. 11–27). New York: Wiley.

Watkins, C. E., Jr. (Ed.). (1997). Handbook of psychotherapy supervision. New York:Wiley.

Watkins, C. E., Jr. (1998). Psychotherapy supervision in the 21st century: Some press-ing needs and impressing possibilities. Journal of Psychotherapy Practice andResearch, 7, 93–101.

Watkins, C. E., Jr. (2010). Psychoanalytic developmental psychology and thesupervision of psychotherapy supervisor trainees. Psychodynamic Practice,16, 393–407.

Watkins, C. E., Jr. (2011). Psychotherapy supervision since 1909: Some friendlyobservations about its first century. Journal of Contemporary Psychotherapy,41, 57–67.

Weisz, J. R., & Kazdin, A. E. (Eds.). (2010). Evidence-based psychotherapies forchildren and adolescents (2nd ed.). New York: Guilford.

Westefeld, J. S. (2009). Supervision of psychotherapy: Models, issues, and recom-mendations. The Counseling Psychologist, 37, 296–316.

Wheeler, S., & Richards, K. (2007). The impact of clinical supervision on counselorsand therapists, their practice and their clients: A systematic review of the litera-ture. Counselling and Psychotherapy Research, 7, 54–65.

White, E., & Winstanley, J. (2010). A randomised controlled trial of clinical super-vision: Selected findings from a novel Australian attempt to establish the evi-dence base for causal relationships with quality of care and patient outcomes,as an informed contribution to mental health nursing practice development.Journal of Research in Nursing, 15, 151–167.

256 C. E. Watkins, Jr.

Dow

nloa

ded

by [2

20.2

55.1

.173

] at 1

7:50

27

June

201

3