article · 2012-07-24 · psychiatric rehabilitation journalan update on randomized controlled...

Psychiatrie Rehabilitation Journal2008, Volume 31, No. 4, 280-290

Copyright 2008 Trustees of Boston University

DOl: 10.2975/31.4.2008.280.290

ARTICLE

An Update onRandomized Controlled Trials

of Evidence-BasedSupported Employment

V

Gary R. Bond

Indiana University-Purdue University

Indianapolis

Robert E. Drake and

Deborah R. Becker

Dartmouth Medical School

Background: The Individual Placement and Support (IPS) model of supported em-

ployment for clients with severe mental illness has been described as a standardi-

zation of evidence-based supported employment. Although several reviews on the

literature on its effectiveness have been conducted, the completion of several new

studies suggests an updated review is warranted.

Methods: We conducted a comprehensive literature search for randomized con-

trolled trials of IPS, limiting our review to programs with high-fidelity IPS pro-

grams, locating li studies. We examined the following competitive employment

outcomes: employment rates, days to first job, annuatized weeks worked, and job

tenure in longest job held during the follow-up period.

Findings: Across the 11 studies, the competitive employment rate was 61 % for IPS

compared to 23% for controls. About two-thirds of those who obtained competitive

employment worked 20 hours or more per week. Among those who obtained a

competitive job, IPS participants obtained their first job nearly lo weeks earlier

than did controls. Among IPS participants who obtained competitive work, dura-

tion of employment after the start of the first job averaged 24.2 weeks per year, or

47% of the 52-week year.

Conclusions: The current review is consistent with earlier reviews, although the evi-

dence for high-fidelity IPS appears to be somewhat stronger here than in reviews

evaluating studies with more heterogeneity in the supported employment models

examined. The number, consistency, and effect sizes of studies of evidence-based

supported employment establish it as one of the most robust interventions avail-

able for persons with severe mental illness.

Keywords: supported employment, employment, evidence-based practices, ran-domized controlled trials

280

SPRING 2008—VOLUME 31 NUMBER 4

Introduction

1 he Individual Placement and Support(IPS) model of supported employmentfor clients with severe mental illnesshas been described as a standardiza-tion of evidence-based supported em-ployment (Bond et al., 2001). It hasbeen so identified because it is thebest-described (Becker & Drake, 2003)and most extensively researched modelof supported employment for this popu-lation. The core principles of this modelare (1) a focus on competitive employ-ment, (2) eligibility based on consumerchoice, (3) rapid job search, (4) integra-tion of mental health and employmentservices, (5) attention to consumer pref-erence in the job search, (6) individual-ized job supports and (7) personalizedbenefits counseling (Bond, 2004).

Starting in the late 1990s, the literatureon supported employment for individu-als with severe mental illness has beenreviewed on numerous occasions(Bond, 2004; Bond et al., 2001; Bond,Drake, Mueser, & Becker, 1997; Burnset al., 2007; Crowther, Marshall, Bond,& Huxley, 2001; Twamley, Jeste, &Lehman, 2003). The evidence from ran-domized controlled trials (RCTs) contin-ues to accumulate quickly, and reviewsfrom only a couple years ago are al-ready obsolete. The purpose of the cur-rent review is to provide acomprehensive summary of competi-tive employment outcomes for RCTsevaluating evidence-based supportedemployment for this population. A sec-ond difference from earlier reviews isthat we restricted our review to evalua-tions of programs with documented ad-herence to the aforementioned IPSprinciples. Our rationale was based onthe literature suggesting that fidelity toIPS is associated with better competi-tive employment outcomes (Becker,Smith, Tanzman, Drake, & Tremblay,2001; Becker, Xie, McHugo, Halliday, &Martinez, 2006; Gowdy, Carlson, &

Rapp, 2003; McGrew & Griss, 2005;

McGrew, 2007).

Methods

Study Inclusion Criteria

To be included in this review, a studywas required to be a randomized con-trolled trial design examining longitu-dinal competitive employmentoutcomes for individuals with severemental illness in which participantswere randomly assigned to two or moreconditions, one of which used a high-fidelity IPS supported employmentmodel. IPS has been well described inthe literature (Becker & Drake, 2003),and a psychometricatly validated fideli-ty scale has been developed to deter-mine which programs achieve highfidelity (Bond, Becker, Drake, & Vogler,1997). A further requirement for inclu-sion in the review was that the controlgroup or groups must have received ei-ther services as usual or some otherform of vocational rehabilitation be-sides IPS.

Literature Search Strategies

Three main sources were used to iden-tify studies. The first was to draw onpublished literature reviews (Bond,2004; Bond et al., 2001; Crowther etal., 2001; Twamley et al., 2003). Thesecond was to examine each of thestudies in the Employment InterventionDemonstration Project (EIDP) (Cook etal., 2005). The third was through con-tacting principal investigators and con-tinuous review of the publishedliterature.

Sample

The 11 studies included in the currentreview are shown in Table 1, sequencedaccording to date of publication. Tenstudies have been published in peer-reviewed journals, one is in press(Killackey, Jackson, & McGorry, inpress). One study aimed at youngadults with schizophrenia comparing

IPS to services as usual was not includ-

ed because the results have not yet

been reported (Nuechterlein et al.,

2005; Nuechterlein, Subotnik, Turner,

Ventura, Becker, & Drake, this issue).

Ten of the 11 studies used a 2-groupdesign (IPS versus control); theConnecticut study compared IPS to 2control groups (Mueser et aL, 2004).The number of IPS sites ranged fromone in 8 studies, two in 2 studies(Bond et al., 2007; Drake, McHugo,Becker, Anthony, & Clark, 1996), andsix in 1 study (Burns et aL, 2007). Interms of control groups, two studiesused nonintegrated supported employ-ment control groups (Drake et aL,1996; Mueser et al., 2004). Otherwise,all of the control groups consisted ofeither treatment as usual (typically re-ferral to the state vocational system orequivalent) or alternative vocationalmodels. In every study, high fidelity toIPS was ensured through systematicmonitoring using the IPS Fidelity Scale(Bond, Becker et al., 1997). Many otherdetails of the study protocols, includ-ing sample inclusion criteria and meas-urement batteries, were the same orsimilar across studies. In all the stud-ies, participants were adults who meteach state's or province's criteria forsevere mental illness, typically aDiagnostic arid Statistical Manual ofMental Disorders—4th Edition(American Psychiatric Association,1994) Axis I or II diagnosis plus severeand persistent impairment in psy-chosocial functioning. In most studies,participants were recruited from men-tal health centers. In all the studies,participants were unemployed at thetime of study admission. In all but onestudy, the study inclusion criteria in-cluded an expressed desire to work;the single exception was the Marylandstudy (Lehman et al., 2002). Anothereligibility criterion common acrossmost studies was the absence of signif-icant medical conditions that would

PSYCHIATRIC REHABILITATION JOURNAL An Update on Randomized Controlled Trials of Evidence-Based Supported Employment

TABLE I -RANDOMIZED

WITH SEVERE

study

Drake et al. (1996)

Drake et al. (1999)

Lehman et al. (2002)

Mueser et al. (2004)

Gold et al. (2006)

Latimer et al. (2006)

Burns et al. (2007)

Bond et al. (2007)

Wong et al. (2008)

Twamleyetal.(2008)

Killackey et al. (in press)

CONTROLLED TRIALS OF INDIVIDUAL PLACEMENT AND

MENTAL ILLNESS

Control Condition Study Population

& Salient

Eligibility Criteria

Skills training,

nonintegrated

Sheltered workshop

PSR

(1) Brokered SE; (2) PSR

Sheltered workshop

Traditional vocationalservices


Diversified placementapproach

Stepwise conventionalvocational services

VR referral


CMHC clients

Case management

program clients

CMHC clients, includingthose without voc goals

CMHC clients

CMHC clients

Clients receiving mental

health services

Clients receiving mentalhealth services

New admissions toPSR agency

Outpatients at hospitaloccupational therapyprogram

Middle aged and

older adult

Early psychosis program

SUPPORT FOÜ

study Site

Location

NH

DC

MD

t INDIVIDUALS

Months of

Follow-up

18

18

24

CT 24

SC 24

Québec, Canada 12

6 European

cities

IL

HK

CA

Melbourne,Australia

18

24

18

12

6

Label

96 NH

99 DC

02 MD

04 CT

06 SC

06 QUE

07 EUR

07 IL

08 HK

08 CA

08 AUST

Acronyms: SE = supported employment; IPS = Individual Placement and Support; ACT = assertive community treatment;PSR = psychosocial rehabilitation; VR = State-federal vocational rehabilitation system; CMHC = community mental health center;ICCD = International Center for Clubhouse Development

Note: Two control groups in the Mueser et al. (2004) study combined into a single control group in the tables and figure to follow.

preclude working or participating in as-sessment interviews. In many of theIPS studies, participants were requiredto attend multiple research informationmeetings in which the project was ex-plained before giving consent to partic-ipate in the study.

Excluded Randomized Controlled Trials

Excluded RCTs fall into 3 categories:RCTs evaluating supported employ-ment prior to the development of theIPS model; recent supported employ-ment RCTs evaluating a different model

than IPS or a program for which fidelityof IPS is uncertain; and RCTs compar-ing an enhanced form of IPS with IPSwithout an enhancement.

Five "pre-IPS" RCTs were found. Inevery case they evaluated a supportedemployment model falling short on oneor several IPS criteria. Four of these(Bond, Dietzen, McGrew, & Miller, 1995;Chandler, Meisel, Hu, McGowen, &Madison, 1997; Gervey & Bedell, 1994;McFarlane et al., 2000) were examinedin earlier reviews (Bond, 2004; Bond et

al., 2001). One comparing the Programof Assertive Community Treatment(PACO to usual services has never beenpublished (Test, 1992; Test, Allness, &Knoedler, 1995).

The second category consists of 4 recentRCTs that either have evaluated a differ-ent model of supported employment(Macias et al., 2006; Rogers, Anthony,Lyass, & Penk, 2006) or an approach inwhich the fidelity to the IPS model wasunknown or uncertain (Shafer, 2005;Tsang, 2006). The specifics of these 4studies are as follows:


The Massachusetts EIDP study findingsare reported in two papers (Macias etal., 2oo6; Schonebaum, Boyd, & Dudek,2oo6). The former paper uses the stan-dardized definition of competitive em-ployment and limits the analysis toparticipants enrolling in the study withan avowed interest in working. The lat-ter paper uses the full randomized sam-ple with an expanded definition ofcompetitive employment. The reportedexperimental differences given in twopapers are fairly similar. This study eval-uated a newly established PACT team inwhich a supported employment positionwas created. Although in many respectsadhering to IPS principles, the role de-scription for the supported employmentposition followed the PACT model inwhich the employment specialist hassignificant clinical duties, which com-promises model effectiveness.

Although the "choose-get-keep" modeldeveloped by Anthony and his col-leagues (Danley & Anthony, 1987) wasoriginally viewed as a kind of support-ed employment model, in a recently re-ported randomized controlled trial ofthis model (Rogers et al., 2006), theauthors label their approach "psychi-atric vocational rehabilitation." Thechoose-get-keep model differs in im-portant respects from the IPS model,centering on its emphasis on prevoca-tional career planning.

The Shafer EIDP study, which has notbeen published, compared a support-ed employment approach to servicesas usual (Shafer, 2005). The investiga-tors did not specifically monitor theirservices to achieve high fidelity to IPS.We have been unable to obtain a finalreport of the study.

The study conducted by Tsang et al.(2006) had two experimental conditions:one was IPS only and the second was IPSplus skills training. These two experi-mental conditions were compared to atreatment-as-usual control group.

Although the authors state that they im-plemented IPS with high fidelity, the crit-ical ingredients of the model were notfollowed. In addition, the findings fromthis study are not usable in the currentreview, because their employment out-come measures combined competitiveemployment and what the report labels"partially competitive employment."

The third category of excluded studiesconsists of RCTs comparing IPS to anenhancement of IPS. Two studies fromthe EIDP (Maine and Texas studies)compared supported employment to anenhanced form of supported employ-ment and would be excluded for thisreason (although in neither study didthe investigators closely adhere to theIPS model, so they would be excludedfor that reason as well). Since the com-pletion of the EIDP, there have been agrowing number of studies comparingsupported employment to supportedemployment plus an enhancement.Enhancements have included bothskills training (Mueser et al., 2005;Wallace & Tauber, 2004) and cognitivetraining strategies (Bell, Greig, Zito, &Wexler, 2007; McGurk, Mueser, &Pascaris, 2005). While these studies areanswering important questions aboutsupported employment, they lie out-side the scope of the current review,which compares supported employ-ment to other vocational approaches.

Outcome Measures

This review focuses exclusively on com-petitive employment outcomes, de-fined as jobs paying at least minimumwage in integrated community settings(i.e., employing nondisabled workers)and are jobs that anyone could hold,not just individuals with disabilities. Bythis definition, protected jobs, such astransitional employment (Propst,1992), sheltered employment, andother set-aside jobs (Black, 1988) wereexcluded.

All studies reported competitive em-ployment rates, defined as working acompetitive job at any time during fol-low-up. Several other competitive em-ployment outcome measures were alsoexamined. When reported, we com-piled the following outcomes: days tofirst job (i.e., time from study entry tofirst job start), annualized weeksworked (that is, number of weeksworked per year; dividing total weeksworked by 1.5 for studies with 18-month follow-up and dividing totalweeks work by 2.0 for studies with 2-year follow-up), and job tenure (i.e.,weeks worked) in longest job held dur-ing the follow-up period. Oob tenure ismeasured from job start date. The enddate is determined from either job end-ing date or end of follow-up, whichevercomes first.) As seen below, informa-tion was not available on these vari-ables for all 11 IPS studies.

For the first measure of duration of em-ployment—weeks worked per year—wecalculated the findings for both the fullintent-to-treat sample (everyone en-rolled in the study) and the workersample only (those who obtained atleast one competitive job during fol-low-up). We calculated weeks workedin a third way as well: weeks workedafter obtaining first competitive job.Each of these methods for estimatingduration of employment assesses it ina different way.

Data Analyses

Data were either recorded directly frompublished reports or hand calculatedfrom information presented in these re-ports. As noted above, the Connecticutstudy was the only study included withtwo control groups (Mueser et al.,2004). Campbell (2007) examined ef-fect sizes for the differences on 3 com-petitive employment outcomesbetween the two control groups in thisstudy and found the effect sizes for thedifferences to be small. Based on these


FIGURE I-COMPETITIVE EMPLOYMENT RATES IN l i RANDOMIZED

CONTROLLED TRIALS OF INDIVIDUAL PLACEMENT AND SUPPORT

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

78%7

40%

5% 74%70%

34%28%

6

29%

1

51° 64%

10%

26%

19%

-

5

i1

b%47%

28% 1 27%

1 | i 8 % 1

1 1»1 196 NH 07 IL 04 CT 08 HK 08 AUST 06 SC 99 DC 08 CA 07 EUR o< QUE 03 MD

• IPS nControl •Control2

findings, we collapsed the outcomes

for the two Connecticut control groups

into a single control group for all em-

ployment outcomes.

Effect size for each study for the differ-ence in employment rate between sup-ported employment and controls wascalculated using the arc sine approxi-mation (Lipsey, 1990). An unweightedoverall effect size was calculated as thesimple mean of the individual effectsizes. For all other outcome measures,means are reported without standarddeviations, which are not availablefrom the published studies. Overallmeans were calculated weighting indi-vidual means by sample sizes.

Effect sizes were also estimated forpublished review articles using the arcsine method on overall percentageworking. It should be noted that thismethod of estimation gives slightlydifferent estimates than the unweight-ed overall effect size method de-scribed above.

Results

Competitive employment rates. In all l istudies, the competitive employmentrate was significantly higher for the IPScondition than for controls, as shownin Figure l . Averaging the rates acrossstudies, the competitive employmentrate was 61% (Median = 64%) for IPScompared to 23% (Median = 27%) forcontrols. The average difference in per-centage employed between supportedemployment and controls was 38%,ranging from 20% to 55%. The individ-ual study effect sizes ranged from .56to 1.23. The overall unweighted effectsize was .83.

Using a stem-and-leaf diagram (Tukey,

1977), the Maryland IPS sample

(Lehman et al., 2002) was a statistical

outlier, falling more than the interquar-

tile range (19%) below the lower quar-

tile (55%) in the distribution of

competitive employment rates. With

this outlier removed, the mean compet-

itive employment rate for IPS was 65%.

The next two lowest employment rates

were for two non-U.S. studies in Canada

(Latimer et al., 2006) and Europe (Burns

et al., 2007). However, two other non-

U.S. studies were above the median

(Killackey et al., in press; Wong, Chiu,

Tang, Mak, Liu, & Chiu, 2008).

Other competitive employment out-comes. Four IPS studies reported thefrequencies of participants whoworked 20 hours or more per week, asshown in Table 2. Aggregating acrossthese studies, 134 (43.6%) of 307 IPSparticipants and 53 (14.2%) of 374 con-trols held such jobs, yielding an effectsize of .67.

Days to first competitive job was re-ported in 7 IPS studies, as shown inTable 3. Overall, the average time was50% faster in first competitive job forIPS compared to controls (138 days ver-sus 206 days). Excluding the extremevalues (less than 3 months in theQuébec (Latimer et al., 2006) and HongKong studies (Wong et al., 2008) andover 6 months in the Hartford study(Mueser et al., 2005), the range was

TABLE 2-PERCENTAGE OF PARTICIPANTS EMPLOYED IN

HOURS OR MORE A WEEK IN 4 IPS STUDIES

Study IPS

96 NH

99 DC

07 IL

04 CT

Total

34 (46.6%)

34 (45.9%)

43 (46.7%)

23 (33-8%)

134 (43.6%)

A COMPETITIVE JOB 2 0

(BASED ON TOTAL SAMPLES)

Controls

15 (22.4%)

4 (5-3%)

22 (23.1%)

12 (8.8%)

53 (14.2%)

SPRING 2008—VOLUME 31 NUMBER. 4

between 4 months and a little over 5

months as the average length of time

to first job for IPS participants.

Proportion of time in competitive em-ployment, as measured by annualizedweeks worked, is reported for the same7 IPS studies, as shown in Table 4. Theresults are reported for both all studyparticipants, including those whonever worked, and for the worker sam-ple. The Maryland study (Lehman et al.,2002) reported substantially lowerrates of weeks work in the full sample.Excluding the Maryland study, annual-ized average weeks worked for IPSranged from 10.0 weeks for the SouthCarolina study (Gold et al., 2006) to17.0 weeks for the Québec study(Latimer et al., 2006). Overall, the aver-age weeks worked for IPS was overtwice that for controls. When the sam-ples are limited to participants who ob-tained competitive employment duringfollow-up, the weeks worked are virtu-ally the same for IPS and controls.

Duration in longest held competitive jobfor those who worked at least one suchjob, is shown in Table 5. The average jobtenure for IPS participants in these stud-ies ranged from 37 weeks for the Illinoisstudy (Bond et al., 2007) to 10 weeks in

TABLE 3-MEA14

study

08 HK

06 Qué

99 DC

06 SC

07 IL

02 MD

04 CT

Total

DAYS TO FIRST COMPETITIVE JOB

IPS

72(A/=32)

84 (A/=51)

126 (A/= 45)

133 (A/= 42)

156 (A/= 69)

164 (A/=47)

197 (A/ = 51)

137.6 (A/=337)

IN 7 IPS STUDIES

Control

118 (A/ = 13)

89 (A/= 39)

293 (A/= 7)

322 (A/= 20)

193 (A/ = 32)

287 (A/ = 12)

277(A/=3i)

205.9 (N = 154)

the New Hampshire study (Drake et al.,1996). It should be noted that this meas-ure was influenced by length of follow-up. Except for the Connecticut study(Mueser et al., 2004), which found fargreater job tenure for IPS participants,IPS and control samples looked fairlysimilar on this measure.

Excluded Studies

Although not a focus of the current re-view, some comments on the excludedstudies are warranted. Nine studieswere located comparing a non-lPS sup-ported employment program to some

other vocational model. Four had signifi-cant findings suggesting superior com-petitive employment findings forsupported employment (Bond et al.,1995; Gervey & Bedell, 1994; McFarlaneet al., 2000; Test et al., 1995), while tworeported significantly better employ-ment outcomes for their supported em-ployment intervention but did notclearly differentiate between competi-tive employment and set-aside jobs, sothat the interpretation of their results isclouded (Chandler et al., 1997; Tsang,2006). Two studies found no differencesin employment outcomes for supported

TABLE 4-

06 Qué

07 IL

04 CT

08 HK

99 DC

06 SC

02 MD

Total

-ANNUALIZED WEEKS WORKED IN COMPETITIVE JOBS IN 7

All study Participants

IPS

17.0 (A/ = 75)

16.2 (A/ = 92)

14.9 (A/ = 68)

13.0 (N = 46)

10.1 (A/ = 74)

10.0 (A/= 66)

6.0 (A/ = 113)

12.1 (W=534)

Control

14.1 (A/= 74)

8.2 (A/= 95)

2.3 (A/=136)

7-0 (A/= 46)

0.8 (N = 76)

2.9 (A/= 77)

1.6 (A/ = 106)

4.8 (W=6io)

IPS STUDIES

Participants Working a Competitive Job

Total Follow-up Period

IPS Control

25.0 (A/= 51)

21.6 (A/ = 69)

19.8 (A/=51)

18.6 (A/ = 32)

16.6 (A/= 45)

15.8 (A/= 42)

14.4 (A/ = 47)

19.2 («=337)

26.8 (A/= 39)

24.3 (A/= 32)

9.8 (A/= 31)

24.9 (N = 13)

8.7 (A/= 7)

11.3 (N = 20)

14.1 (A/ = 12)

18.9 (/V=154)

Time Period from

IPS

32.5 (A/ = 51)

25.6(A; = 69)

27.1 (A/=51)

21.4 (A/ = 32)

21.6 (A/= 45)

19.3 (A/ = 42)

18.6 (A/= 47)

24.2 (Ai=337)

ist Job Start

Control

35.4 (A; = 39)

26.1 (A; = 32)

15.8 (A/= 31)

31.7 (A/=13)

iS.7(A/ = 7)

20.2 (A/ = 20)

23.2 {N = 12)

25.5 (Ai=i54)

PSYCHIATRIC REHABILITATION JOURNAL An Update on Randomized Controlled Trials af Evidence-Based Supported Employment

employment and the comparison group

(Rogers et al., 2006; Shafer, 2005).

Finally, one study had mixed results, on

balance favoring the comparison group

over supported employment (Macias et

al., 2006). Combining the employment

rates for 4 early studies (Bond et al.,

1995; Chandler et al., 1997; Gervey &

Bedell, 1994; McFarlane et al., 2000),

we found that the combined employ-

ment rate was 53% for supported em-

ployment and 16% for controls, for an

effect size of .82.

Comparison with Other Reviews

We compared our competitive employ-ment rates with 5 earlier reviews, asshown in Table 6. The aggregated com-petitive employment rate for supportedemployment was highest in the currentreview, while, with the exception of theCooi< (2005) review, the mean competi-tive employment rate for controls wasalso higher than the other 4 reviews.All five reviews concluded that thecompetitive employment rate for sup-ported employment is at least twicethat for clients receiving some otherform of vocational assistance. Usingthe arc sine method, the effect sizes forthe overall IPS/control differenceranged from .43 to .79 in previous re-

TABLE

study

07 IL

04 CT

02 MD

06 SC

06 Qué

96 NH

Total

5-WEEKS WORKED AT LONGEST COMPETITIVE JOB DURINGFOLLOW-UP IN 6 IPS STUDIES (WORKER SAMPLE ONLY)

IPS

36.8 (/V = 69)

25.5 (W= 51)

21.6 (W = 47)

19.0 (W = 42)

14.6 (/V = 51)

10.0 (W = 57)

22.0 («=317)

Control

32.7 (W = 32)

4.4(W = 3i)

23.1 (W = 12)

20.0 (W = 20)

12.7 (A/= 39)

10.0 (W = 27)

16.3 (/V = 161)

views, compared to .79 in the currentreview.

Discussion

This review examined evidence-basedsupported employment in order to esti-mate expected outcomes. We concludethat the majority of iPS participants ob-tain competitive employment, at a farhigher rate than clients enrolled inother vocational services. Most IPSclients work part time; about two-thirdsof those who obtain competitive em-

ployment work 20 hours or more perweek. This could be due to preferences,limited stamina, or fear of losing healthinsurance. Undoubtedly the number ofhours worked per week is influenced bythe rules governing receipt of disabilitypayments and Wledicaid eligibility. Lessthan 1% of IPS participants left disabili-ty rolls during the follow-up period.Consistent with the principle of rapidjob search, the time to first competitivejob for IPS participants is nearly 10weeks less than for controls, althoughthe average for IPS of 20 weeks to first

TABLE 6-AGGREGATED COMPETITIVE EMPLOYMENT RATES IN 6 REVIEWS OF SUPPORTED EMPLOYMENT

Effect Size Effect SizeStudy Supported Reported Using Arc Number of

Employment Control by Authors Sine Studies

Bond (2004) 56%

Burns et al.(2OO7) ^ 58%

Cook et al. (2005) 55%

Crowther et al. (2001) 34%

Twamley et al. (2003) 51%

Current Review 61%

19%

21%

34%

12%

18%

23%

0.85

0.79

0.83

0.79

0.78

0.43

°-54

0.72

0.79

9

6

7

5

5^

11

Number ofStudies of

High Fidelity IPS

5 (56%)

6 (100%)

3 (43%)

2 (40%)

3 (60%)

11

3 Findings aggregated by the current authors for this table

'' Twamley review included 11 RCTs of vocational programs, 5 of which were supported employment studies used in this table.


job is somewhat of a surprise. Time tofirst job strongly affects longitudinalcompetitive employment outcomes; inmost studies, the large majority whowork at all do so in the first six months.Thus, this might be an area for modelimprovement, which we speculatemight require better job developmentstrategies (Carlson & Rapp, 2007). Inaddition, the use of the vocational pro-file to help identify job types and worksettings that match the individual'spreferences, skills, and experiences isanother area that may help improveand speed up the job seeking processand increase job tenure as well (Becker,Drake, Farabaugh, & Bond, 1996).

This review represents an advance overearlier reviews in several respects.First, it has the largest and (for the mo-ment) the most up-to-date collection ofpertinent randomized controlled trials.Second, it is one of only two reviewslimited to rigorous evaluations of IPSprograms. Thus, our review is based ona homogeneous set of studies, givingthe clearest picture of the potential forevidence-based supported employ-ment. None ofthe previous reviews in-cluded more than 6 high-fidelity IPSstudies. Notably, the Cook et al. (2005)analysis included two studies compar-ing enhanced supported employmentto supported employment only, whichconfounds the impact of supported em-ployment. We also note that the currentis a substantial update ofthe Cochranereview on vocational rehabititation(Crowther, Marshall, Bond, & Huxley,2000; Crowther et al., 2001). We attrib-ute our somewhat stronger results togrowing maturation ofthe field withboth higher fidelity supported employ-ment programs and stronger controlgroups. The differential advantage forevidence-based supported employmentremains about the same, but our con-clusions are stronger because the com-parisons appear to be more rigorous.

Could the current review be subject tothe "file drawer" problem - failing to in-clude studies that have not been pub-lished (Rosenthal, 1984)? We can give apartial answer to this question: Of fourunpublished studies, three are knownto have strongly positive results(Nuechterlein et al., 2005; Test et al.,1995; Tsang, 2006), while a fourthstudy, the Arizona study from the EIDP,had null results (Shafer, 2005).Moreover, of these unpublished studies,only the Nuechterlein et al. study (2005)conforms to the sampling criterion ofhigh-fidelity IPS. We deem it implausiblethat any unpublished studies could tipthe balance of evidence present here.

Some comment is warranted about theinclusion of Maryland study (Lehman etal., 2002) as one ofthe 11 IPS high-fi-delity studies. This study was a statisti-cal outlier. In terms of sample inclusioncriteria, it clearly deviated from theother IPS studies in that it was the onlystudy among those reviewed that didnot require participants to have a goalof competitive employment. The poorercompetitive employment outcomes areconsistent with this difference in sam-ple inclusion criteria.

Three widely repeated criticisms ofsupported employment concern exclu-sion of clients who do not have voca-tional goals, high dropout rates (i.e.,the contention that supported employ-ment has a high attrition rate) and briefjob tenure (supported employmenthelps clients get jobs but not to keepthem, i.e., most supported employ-ment jobs are short term). The currentcollection of studies sheds light onthese three issues.

Exclusion of individuals who do nothave competitive employment goals.Some observers have criticized evi-dence-based supported employmentfor its lack of outreach to individualswho do not have competitive employ-ment goals (Macias, DeCarlo, Wang,

Frey, & Barreira, 2001; Roberts & Pratt,2007). This criticism violates the ethi-cal principle of informed choice. Ofcourse we agree that all consumerswith severe mental illness should beencouraged to pursue work, and casemanagers and others within the mentalhealth center should create a culture inwhich work is valued (Gowdy, Carlson,& Rapp, 2004). At the same time, fromthe shared decision-making perspec-tive (O'Connor et al., 2007), the deci-sion to pursue work should be anactive choice based on a clear under-standing of what the decision means.

Early termination rates in iPS studies.Although only a handful of studies re-port the findings regarding program re-tention, the studies that do show astrong advantage for supported employ-ment. Five IPS studies compared ratesfor dropping out of vocational services.Although the criterion for early termina-tion differed across studies, all 5 studiesfound substantially greater program re-tention for IPS. In the NH study, early at-trition (dropping out of services withinfirst two months) was higher in the con-trol condition (38%) than in IPS (0%)(Drake et al., 1996). In the DC study, de-spite the addition of an extra staff per-son to help link clients with providers inthe comparison condition, early attrition(dropping out of services within first twomonths) was greater in the control con-dition (16%) than in IPS (5%) (Drake etal., 1999). In the Hartford study, approxi-mately 50% ofthe participants assignedto both control conditions dropped outwithin a few weeks, whereas less than10% of IPS participants dropped outduring this early period (Mueser et al.,2004). In the Québec study, 9% of IPSparticipants failed to receive at least oneservice contact during each ofthe firstand second 3-month periods in thestudy, compared to 70% of those inusual services (Latimer et al., 2006). Inthe Illinois study, 18% of IPS partici-pants dropped out within 6 months.


compared to 35% of controls (Bond et

al., 2007).

Thus it is time to suspend the concernthat supported employment programshave high dropout rates. This conclusionwas warranted in 1997 based on theavailable data at that time (Bond, Drakeet al., 1997), but the research now showsthat high-fidelity supported employmentprograms have minimal attrition.

Job tenure. Another widely repeated criti-cism of supported employment is that ithelps people get jobs, but not retainthem (Mueser et al., 2005). The currentreview suggests that the averagelongest-held job among IPS participantswho obtain work is 22 weeks. Two fac-tors affect the interpretation of this find-ing. First, there is wide variation acrossstudies. Second, the longest-held job isa biased statistic in a short-term study,because it gives no credit for jobs heldpast the foUow-up. Annualized weeks ofwork is probably a better statistic; on av-erage, IPS participants who obtain atleast one competitive job work 19 weeksout of 52, or about 36% of the availabletime. Moreover, this figure is not adjust-ed for the initial period of unemploymentlooking for a job, which averaged 21weeks for the first job. If we remove theinitial job search period and calculatethe duration of employment over the pe-riod of time after the start of the first job,IPS participants average 24.2 weeks peryear, or 47% of the 52-weekyear.

We found that among those who ob-tained at least one competitive job, IPSand control participants did not differin job tenure and other employmentoutcomes. This finding sometimes hasbeen misinterpreted to suggest that al-though IPS helps individuals get a firstjob sooner, it provides no advantageover usual services in helping themkeep the job. However, these sub-groups are no longer equivalent (e.g.,the IPS subgroup of those with a job in-cludes more than twice the proportion

of the enrolled sample as the compari-

son group). The resulting sampling

bias undoubtedly favors the non-IPS

comparison program because IPS

helps a wider range of participants ob-

tain employment.

To study duration of employment prop-erly, longer-term studies are needed.Two such studies present a clearer pic-ture of the stability of long-term em-ployment than do the studies in thecurrent review. Salyers and colleagues(2004) conducted follow-up interviews10 years after clients had enrolled in anIPS program, finding that 47% werecurrently working and 33% had workedat least 5 years during this period. Evenmore impressive results were obtainedby Becker and colleagues (2007) whointerviewed clients 8 to 12 years afterenrollment in IPS, finding that 71%were currently working and the identi-cal percentage had worked for morethan half of the follow-up period.

One contribution of this review is tohighlight common outcome measuresthat every supported employmentstudy should measure and report.Although the field is making goodprogress in converging on standard-ized measures, the inability to includeall studies in all comparisons is a limi-tation of this review.

Study Limitations

One confound in our comparativeanalysis is the lack of standardizationof follow-up period. Follow-up was 6months in one study, 1 year in 2 stud-ies, 18 months in 4 studies, and 2years in 4 studies. For the competitiveemployment rate variable, participantsin studies with longer follow-up havemore opportunity to obtain work.However, the reality is that the proba-bility of obtaining a first job diminishesover time. As already noted, a secondlimitation of this set of studies is theirrelatively short follow-up period, espe-

cially for measuring job tenure, as

measured by time on longest job held.

This is an inadequate indicator of em-

ployment outcome in short-term fol-

low-up studies, because the measure

does not capture the successful job

tenure for a participant who is em-

ployed at the end of the follow-up peri-

od and continues to work for years

thereafter.

Future Directions

The number, consistency, and effectsizes of studies of evidence-based sup-ported employment establish that it isone of the most robust interventionsavailable for persons with severe mentalillness. Recognizing this, researchershave moved ahead to examine a varietyof enhancements to amplify outcomesthrough early interventions, motivation-al interventions, cognitive interventions,alteration of benefits, and other ap-proaches. We review these efforts at theend of this special section (Drake &Bond, this issue).

In the meantime, research should alsoaddress the myriad issues related todissemination of evidence-based prac-tices (Drake & Skinner, in press). TheScylla and Charybdis of U.S. mentalhealth services continue to be the fail-ure to provide access to evidence-based health care and excessivespending on ineffective health care(Wang, Demler, & Kessler, 2002).

References

American Psychiatric Association. (1994).Diagnostic and statistical manual of men-tal disorders: DSM-IV(iith ed.).Washington, DC: American PsychiatricAssociation.

Becker, D. R., & Drake, R. E. (2003). A workinglife for people with severe mental illness.New York: Oxford Press.

Becker, D. R., Drake, R. E., Farabaugh, A., &Bond, G. R. (1996). Job preferences ofclients with severe psychiatric disordersparticipating in supported employmentprograms. Psychiatric Services, 47,1223-1226.

SPRING 2008—VOLUME 31 NUMBER4

Becker, D. R., Smith, J., Tanzman, B., Drake, R.E., & Tremblay, T. (2001). Fidelity of sup-ported employment programs and em-ployment outcomes. Psychiatric Services,52, 834-836.

Becker, D. R., Whitley, R., Bailey, E. L., &Drake, R. E. (2007). Long-term employ-ment outcomes of supported employmentfor people witii severe mental illness.Psychiatric Services, 58,922-928.

Becker, D. R., Xie, H., McHugo, G. )., Halliday,|., & Martinez, R. A. (2006). What predictssupported employment program out-comes? Community Mental Health Journal,42, 303-313-

Bell, M. D., Greig, T. C, Zito, W., & Wexler, W.(2007). An RCT of neurocognitive enhance-ment therapy with supported employ-ment: Employment outcomes at 24months. Schizophrenia Bulletin, 33,420-421.

Black, B. J. (1988). Work and mental illness:Transitions to employment. Baltimore:Johns Hopkins Press.

Bond, G. R. (2004). Supported employment:Evidence for an evidence-based practice.Psychiatric Rehabilitation Journal, 27,345-359-

Bond, G. R., Becker, D. R., Drake, R. E., Rapp,C. A., Meisler, N., Lehman, A. F., et al.(2001). Implementing supported employ-ment as an evidence-based practice.Psychiatric Services, 52, 313-322.

Bond, G. R., Becker, D. R., Drake, R. E., &Vogler, K. M. (1997). A fidelity scale for theIndividual Placement and Support modelof supported employment. RehabilitationCounseling Bulletin, 40, 265-284.

Bond, G. R., Dietzen, L. L., McGrew, J. H., &Miller, L. D. (1995). Accelerating entry intosupported employment for persons withsevere psychiatric disabilities.Rehabilitation Psychology, 40, 91-111.

Bond, G. R., Drake, R. E., Mueser, K. T., &Becker, D. R. (1997). An update on sup-ported employment for people with severemental illness. Psychiatric Services, 48,335-346.

Bond, G. R., Salyers, M. P., Dincin, J., Drake, R.E., Becker, D. R., Fraser, V. V., et al. (2007).A randomized controlled trial comparingtwo vocational models for persons withsevere mental illness. Journal ofConsulting and Clinical Psychology, 75,968-982.

Burns, T., Catty, |., Becker, T., Drake, R. E.,Fioritti, A., Knapp, M., et al. (2007). The ef-fectiveness of supported employment forpeople with severe mental illness: A ran-domised controlled trial. Lancet, 370,1146-1152.

Campbell, K. (2007). Consumer predictors ofcompetitive employment outcomes in sup-ported employment. Unpublished doctoraldissertation, Indiana University-PurdueUniversity Indianapolis, Indianapolis.

Carlson, L., & Rapp, C. A. (2007). Consumerpreference and individualized job search.American Journal of PsychiatricRehabilitation, 10,123-130.

Chandler, D., Meisel, |., Hu, T., McGowen, M.,& Madison, K. (1997). A capitated modelfor a cross-section of severely mentally illclients: Employment outcomes.Community Mental Health Journal, 33,501-516.

Cook, I. A., Leff, H. S., Blyler, C. R., Gold, P. B.,Goldberg, R. W., Mueser, K. T., et al.(2005). Results of a multisite randomizedtrial of supported employment interven-tions for individuals with severe mental ill-ness. Archives of General Psychiatry, 62,505-512.

Crowther, R., Marshall, M., Bond, G. R., &Huxley, P. (2000). Vocational rehabilitationfor people with severe mental disordersICochrane Review Art. No.: CD003080],Cochrane Library (Issue 1, ISSN 1464-780X); www.cochrane.org/. Oxford,England: Update Software.

Crowther, R. E., Marshall, M., Bond, G. R., &Huxley, P. (2001). Helping people with se-vere mental illness to obtain work:Systematic review. British Medical Journal,322, 204-208.

Danley, K. S., & Anthony, W. A. (1987). TheChoose-Get-Keep Model: Serving severelypsychiatrically disabled people, ̂ menconRehabilitation, 13(4), 6-9, 27-29.

Drake, R. E., & Bond, G. R. (2008). The futureof supported employmentfor people withsevere mental illness. PsychiatricRehabilitation Journal, 31, 367-376.

Drake, R. E., McHugo, G. J., Bebout, R. R.,Becker, D. R., Harris, M., Bond, G. R., et al.(1999). A randomized clinical trial of sup-ported employment for inner-city patientswith severe mental illness. Archives ofGeneral Psychiatry, 56, 627-633.

Drake, R. E., McHugo, G. J., Becker, D. R.,Anthony, W. A., & Clark, R. E. (1996). TheNew Hampshire study of supported em-ployment for people with severe mentalillness: Vocational outcomes. Voumo/ ofConsulting and Clinical Psychology, 64,391-399.

Drake, R. E., & Skinner, ). S. (in press).Technology diffusion in mental health. InH. H. Goldman (Ed.), Mental health in themainstream of public policy: Research is-sues and policy. Chicago: John D. andCatherine T. MacArthur Foundation.

Gervey, R., & Bedell, J. R. (1994). Supportedemployment in vocational rehabilitation.In J. R. Bedell (Ed.), Psychological assess-ment and treatment of persons with severemental disorders (pp. 151-175).Washington, DC: Taylor & Francis.

Gold, P. B., Meisler, N., Santos, A. B.,Carnemolta, M. A., Williams, 0. H., &Kelleher, J. (2006). Randomized trial ofsupported employment integrated withassertive community treatment for ruraladults with severe mental illness.Schizophrenia Bulletin, 32, 378-395.

Gowdy, E. A., Carlson, L. S., & Rapp, C. A.(2003). Practices differentiating high-per-forming from low-performing supportedemployment programs. PsychiatricRehabilitation Journal, 26, 232-239.

Gowdy, E. A., Carlson, L S., & Rapp, C. A.(2004). Organizational factors differentiat-ing high-performing from low-performingsupported employment programs.Psychiatric Rehabilitation Journal, 28,150-156.

Killackey, E., Jackson, H. J., & McGorry, P. D. (inpress).Vocational intervention in first-episode psychosis: A randomised con-trolled trial of Individual Placement andSupport vs. treatment as usual. BritishJournal of Psychiatry.

Latimer, E., Lecomte, T., Becker, D., Drake, R.,Duelos, 1., Piat, M., et al. (2006).Generalisablity of the individual place-ment and support model of supported em-ployment: Results of a Canadianrandomised controlled trial. BritishJournal of Psychiatry, 189, 65-73.

Lehman, A. F., Goldberg, R. W., Dixon, L B.,McNary, S., Postrado, L., Hackman, A., etal. (2002). Improving employment out-comes for persons with severe mental ill-ness. Archives of General Psychiatry, 59,165-172.

Lipsey, M. W. (1990). Design sensitivity.Newbury Park, CA: Sage.

Macias, C, DeCarlo, L. T., Wang, Q., Frey, ]., &Barreira, P. (2001). Work interest as a pre-dictor of competitive employment: Policyimplications for psychiatric rehabilitation.Administration and Policy in MentalHealth, 28, 279-297.

Macias, C, Rodican, C. F., Hargreaves, W. A.,Jones, D. R., Barreira, P. J., & Wang, Q.(2006). Supported employment outcomesof a randomized controlled trial of ACT andclubhouse models. Psychiatric Services,57,1406-1415.


McFarlane, W. R., Dushay, R. A., Deakins, S.M., Stastny, P., Lukens, E. P., Toran, )., etal. (2000). Employment outcomes inFamily-aided Assertive CommunityTreatment. American Journal ofOrthopsychiatry, 70, 203-214.

McGrew, J. H. (2007). IPS fidelity survey of 17supported employment programs inIndiana: Final report to SECT Center.Unpublished report. Indianapolis: IndianaUniversity-Purdue University Indianapolis.

McGrew, J., & Griss, M. (2005). Concurrent andpredictive validity of two scales to assessthe fidelity of implementation of support-ed employment. Psychiatric RehabilitationJournal, 29, i^i-tij.

McGurk, S. R., Mueser, K. T., & Pascaris, A.(2005). Cognitive training and supportedemployment for persons with severe men-tal illness: One year results from a ran-domized controlled trial. SchizophreniaBulletin, 31, 898-909.

Mueser, K. T., Aalto, S., Becker, D. R., Ogden, J.S., Wolfe, R. S., Schiavo, D., et al. (2005).The effectiveness of skills training for im-proving outcomes in supported employ-ment. Psychiatric Services, 56,1254-1260.

Mueser, K. T., Clark, R. E., Haines, M., Drake, R.E., McHugo, G. J., Bond, G. R., et al.(2004). The Hartford study of supportedemployment for persons with severe men-tal illness. Journal of Consulting andClinical Psychology, 72,479-490.

Nuechterlein, K. H., Subotnik, K. L, Turner, L.R., Ventura, J., Becker, D. R., & Drake, R. E.(2008). Individual Placement and Supportfor individuals with recent-onset schizo-phrenia: Integrating supported educationand supported employment. PsychiatricRehabilitation Journal, 31, 340-349.

Nuechterlein, K. H., Subotnik, K. L, Ventura, |.,Gitlin, M. )., Green, M. F., Wallace, C. )., etal. (2005). Advances in improving and pre-dicting work outcome in recent-onsetschizophrenia. Schizophrenia Bulletin, 31,530.

O'Connor, A. M., Wennberg, J. E., Legare, F.,Llewellyn-Thomas, H. A., Moulton, B. W.,Sepucha, K. R., et al. (2007). Toward the"tipping point": Decision aids and in-formed patient choice. Health Affairs, 26,716-725.

Propst, R. N. (1992). Standards for clubhouseprograms: Why and how they were devel-oped. Psychosocial Rehabilitation Journal,16, 25-30.

Roberts, M. M., & Pratt, C. W. (2007). Putativeevidence of employment readiness.Psychiatric Rehabilitation Journal, 30,175-181.

Rogers, E. S., Anthony, W. A., Lyass, A., &Penk, W. E. (2006). A randomized clinicaltrial of vocational rehabilitation for peoplewith psychiatric disabilities. RehabilitationCounseling Bulletin, 49,143-156.

Rosenthal, R. (1984). Meta-analyticproceduresfor the social sciences. Beverly Hills: Sage.

Salyers, M. P., Becker, D. R., Drake, R. E.,Torrey, W. C, & Wyzik, P. F. (2004). Ten-year follow-up of clients in a supportedemployment program. PsychiatricServices, 55, 302-308.

Schonebaum, A. D., Boyd, J. K., & Dudek, K. ).(2006). A comparison of competitive em-ployment outcomes for the clubhouse andPACT models. Psychiatric Services, 57,1416-1420.

Shafer, M. (2005). Employment InterventionDemonstration Project: Arizona site (un-published study).

Test, M. A. (1992). Training in CommunityLiving. In R. P. Liberman (Ed.), Handbookof psychiatric rehabilitation (pp. 153-170).New York: Macmillan.

Test, M. A., Allness, D. )., & Knoedler, W. H.(1995, October). Impact of seven years ofassertive community treatment. Paper pre-sented at the American PsychiatricAssociation Institute on PsychiatricServices, Boston, MA.

Tsang, H. W. (2006). Enhancing employmentopportunities of people with mental illnessthrough an Integrated SupportedEmployment approach (ISE) of IndividualPlacement and Support (IPS) and SocialSkills Training (SST) (HSRC Report #S121014). Hong Kong: Health ServicesResearch Committee DisseminationReport.

Tukey, |. W. (1977). Exploratory data analysis.Reading, MA: Addison-Wesley.

Twamley, E. W., Jeste, D. V., a Lehman, A. F.(2003). Vocational rehabilitation in schizo-phrenia and other psychotic disorders: Aliterature review and meta-analysis of ran-domized controlled trials./ourno/ ofNervous and Mental Disease, 191,515-523-

Twamley, E. W., Narvaez, J. M., Becker, D. R.,Bartels, S. )., & leste, D. V. (2008).Supported employment for middle-agedand older people with schizophrenia.American Journal of PsychiatricRehabilitation, 11, 76-89.

Wallace, C. |., & Tauber, R. (2004).Supplementing supported employmentwith workplace skills training. PsychiatricServices, 55, 513-515.

Wang, P. S., Demler, 0., & Kessler, R. C.(2002). Adequacy of treatment for seriousmental illness in the United States.American Journal of Public Health, 92,92-98.

Wong, K. K., Chiu, R., Tang, B., Mak, D., Liu, J.,& Chiu, S. N. (2008). A randomized con-trolled trial of a supported employmentprogram for persons with long-term men-tal illness in Hong Kong. PsychiatricServices, 59, 84-90.

GARYR. BOND, PHD, IS CHANCELLOR'SPROFESSOR, DEPARTMENT OF PSYCHOLOGY,INDIANA UNIVERSITY-PURDUE UNIVERSITYINDIANAPOLIS, INDIANAPOLIS, IN.

ROBERT E. DRAKE, MD, PHD, IS PROFESSOR OFPSYCHIATRY AND COMMUNITY AND FAMILYMEDICINE, DARTMOUTH MEDICAL SCHOOL,LEBANON, NH.

DEBORAH R. BECKER, M ED, IS ASSISTANTPROFESSOR, DEPARTMENT OF COMMUNITY ANDFAMILY MEDICINE, DARTMOUTH MEDICAL SCHOOLAND DARTMOUTH PSYCHIATRIC RESEARCH CENTER.

CORRESPONDING AUTHOR:

GARYR. BOND

DEPARTMENT OF PSYCHOLOGY

INDIANA UNIVERSITY-PURDUE UNIVERSITYINDIANAPOLIS402 N. BLACKFORD ST., LD124INDIANAPOLIS, IN 46202-3275PHONE: 317-274-6752FAX: 317-274-6756EMAIL: [email protected]

article · 2012-07-24 · psychiatric rehabilitation journalan update on randomized controlled...

Documents