ATIM FINAL IMPACT REPORT BRIEFING

April 18, 2017

1

Deborah Kogan and Hannah Betesh

Today’s Presentation 2

Evaluation Overview and Context

Impact Study Sample

Impact Study Findings on Training

Impact Study Findings on Employment and Earnings

Implications for WIOA

Q & A

Overview of ATIM: Project Tested Evidence-Based Innovations

3

Industry Driven

Regional Manufacturing Partnerships

Accelerated Occupational

Skills Training

Industry

Recognized Credentials

Work-Based Training

Opportunities

Individualized Training Plans

and Case Management

Overview of ATIM: The State Played A Strong Leadership Role

4

Specification of the ATIM Program Model

Establishment of State Agency Partnerships with IDES, IMA and ICCB

Selection and Funding of Regional Consortia (with IDES and ICCB)

Development of centralized MIS for Applications, Intake, and Progress Monitoring

Overview of ATIM: Development of Regional Partnerships

5

Coordination of Activities Between Participating Local Areas

Enhanced Partnerships between Employers, Education Providers, and Workforce System

Coordinated Outreach and Intake of Applicants

Creation of New Accelerated Training Opportunities

ATIM Evaluation Overview 6

Implementation Study

• Implementation Progress and Fidelity: 2013 - 2015 Data from two rounds of site visits Stakeholder survey

Outcome Study

• Participant Training and Employment Outcomes 2013 -2016 Data from ATIM tracking system and

DCEO administrative records Data from Illinois DES

Impact Study

• Differences in outcomes between ATIM participants and the control group 2013 - 2016 Enrollment and completion of

training Completion of employer-

recognized credentials Rates of employment Quarterly earnings during

quarters following RA

Impact Study Sample 7

Enrollment Totals 8

90

204

88 112 20

514

42

93

25 60

4

224

Central Collar Counties North Central Northern Stateline Southwest Total

Study Participants, Overall and By Region

ATIM Participants Control Group

Key Characteristics 9

Demographics

Mostly white and male

Average age: 40 years old

Education and Employment

History

Nearly all had prior employment

history, half in manufacturing

Over half had education beyond

a high school diploma

Barriers to employment

Most did not self-report barriers to

employment

Most common = criminal record

Training 10

ATIM Participants Had Higher Rates of Training Enrollment and Completion

11 95.6%

[VALUE]

23.7% 19.7%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Enrolled in training Completed training

Training Enrollment and Completion

ATIM Participants Control Group

71.9%***

51.9%***

ATIM Participants Earned More Certificates

12

0.5

2.5

Control Group

ATIM Participants

Average Number of Certificates Earned Following Random Assignment

21.0%

76.1%

Control Group

ATIMParticipants

Attained a Certificate Following Random Assignment

55.1%***

2***

Limited Uptake of Work-Based Training 13

2.8%

1.6%

7.0%

Internships

Work Experience

On-the-Job Training(OJT)

Percent of Total ATIM Participants Enrolled in Work-Based Training

Impacts on Employment and Earnings 14

Training-Related Employment 15

According to the ATIM Tracking System…

• 71% of ATIM participants obtained employment at the end of the program

• 63% of those participants found training related employment

ATIM Participants Had Higher Rates of Employment Two Years After RA

16

0

10

20

30

40

50

60

70

80

-8 -7 -6 -5 -4 -3 -2 -1 0 1** 2 3* 4 5** 6** 7** 8

Empl

oym

ent (

%)

Quarter Relative to Random Assignment

Program Group Control Group

ATIM Led to Higher Earnings 17

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

-8 -7 -6 -5 -4 -3 -2 -1 0 1** 2 3 4 5 6*** 7** 8

Ave

rage

Ear

ning

s ($

)

Quarter Relative to Random Assignment

Program Group Control Group

Implications for WIOA 18

ATIM Anticipated Key Features Emphasized in the WIOA Legislation

19

WIOA

Sectoral Focus

Career Pathways to

Jobs of Today and Tomorrow

Employer Demand-

Driven Orientation

Work-Based Learning

Data Driven

Implications of ATIM Experience 20

Key Features of ATIM Model

Fidelity of ATIM Implementation Implications for WIOA

Sectoral Industry-driven Training +

Focus on promising sectors, but be prepared for economic downturns in targeted sectors.

Accelerated Training Schedules

Some regions were more successful than others

Strengthen partnerships with community colleges and increase their commitment to offering short-

term stackable credentials.

Contextualized Basic Skills “On Ramps”

- Not enough time in grant

period for sequenced services

Recognize that individuals with basic skills gaps may need longer to complete occupational skills

training

Case Management by Multiple Partners

- Agency and employer partners not interested

Negotiate partner roles to add value for participating partners

Individualized Employment/ Training Plans

+

Encourage design of services to match individualized assessed customer needs and

interests rather than a one-size-fits-all approach

State-level Participant Tracking System

+ Need time and resources for detailed training of local staff on standardized data system; greatly

increases transparency of results at state, regional and local level

Limitations of ATIM Evaluation 21

Effectiveness of ATIM model for participants with recent employment experience and skills

transferable to manufacturing

Less evidence about whether model works

for individuals with less formal education/ work

experience

Issues for Future Research 22

• What particular features of sector-focused training initiatives make the most difference in bringing about the desired program impacts?

• What program models work best for low-income individuals with serious employment barriers, including limited basic skills?

• What are the most effective models for employer engagement? • To what extent are positive program impacts sustained or

increased over time? • How can programs promote employment retention and further

career advancement, once individuals are placed in their initial jobs after training?

Questions? 23