college access and readiness outcomes of the university of california transcript evaluation service...

College Access and Readiness Outcomes of the University of California Transcript Evaluation Service

2014 NCES STATS-DC Data ConferenceJuly 31, 2014

TRANSCRIPT EVALUATION SERVICE(NOTE: Stock photo. Release for web use is on

file.)

Agenda

TRANSCRIPT EVALUATION SERVICE

• Why transcript data matter

• TES overview

• TES outcomes analysis− Characteristics of participating

schools− Comparison to baseline− Comparison to non-participants

(NOTE: Stock photo. Release for web use is on file.)

2

3TRANSCRIPT EVALUATION SERVICE

Transcript Evaluation Service (TES)

An early and complete sequence of courses matters

• Students who complete 9th grade college preparatory courses begin a clear trajectory that continues through high school

• Students who fall off the college preparatory track early in high school tend to move even further from a complete college preparatory program as they move through high school

• Students who do not get on a college preparatory track never get on

(REL WestEd Policy Brief, 2008)

College Preparation Trajectories3


“a-g” Subject Area Benchmarks4


Educational partnership tool used to improve access to college

Provides programs, schools, and districts with an accurate depiction of course completion patterns to facilitate school improvement and course planning efforts

Provides policymakers and researchers with quality information about student achievement measured against college-going benchmarks

TES Overview5


By providing students, parents, and those who support them with the right data:

• Students (and their counselors) will choose the right courses at the right time, preparing them for college

• High schools will be better able to offer the courses that students need to become college eligible

TES Vision6


7

Features

Individualized transcript evaluations

Courses completed and needed to meetcollege preparatory benchmarks for

every student grades 9-12

Courses completed and needed to meetcollege preparatory benchmarks for

every student grades 9-12

Rosters and reports“a-g” course enrollment patterns

for students at every grade level to aid school planning

“a-g” course enrollment patternsfor students at every grade level to

aid school planning

Training and support Help for schools to make the most of TES as an advising tool and to

develop action plans for access

Help for schools to make the most of TES as an advising tool and to

develop action plans for access

7


TES Benefits

8

For students and families

A plan for life after high school from an accurate, authoritative source

A plan for life after high school from an accurate, authoritative source

For counselors Increased advising efficiency and capacity Increased advising efficiency and capacity

For administrators and policymakers

Optimized school action planning around access and curricular rigor

Optimized school action planning around access and curricular rigor

8


TES Process9


Who participated?• 152 California public high schools• Generally (not exclusively): more disadvantaged (FRL), more ELL, more minority, lower academic performance (API)

TES effect on CSU and UC eligibility?• Eligibility rates improved substantially (up to 30%)• More improvement where better implemented (over 50%)• Some change in total college prep (a-g) courses taken

TES effect on UC applications and admissions?• 10-30% increase over 2-4 years• Possibly more over longer horizons

Outcomes analysis overview10


TES program data

• Participation: Transcript uploads, which schools & when?

• Evaluation: Courses, GPA (for graduates)

• Dosage: Intensity of TES use; effective TES users

Additional data

• CBEDS: Enrollment and graduates

• API: School decile, FRSL, ELL

• UCOP: applicants and admits from each CA high school

Data used in the outcomes analysis11


1 2 3 4 5 6 70

10

20

30

40

50

60

70

80

31

68

50

4045

75 77

TES Cycle

Number of high schools

Number of high schools participating in TES, by submission cycle

2004-05(12/10-9/16)

2005-06 (12/1-11/8)

2006-07(2/6-11/1)

2007-08(4/4-11/3)

2008-09(1/30-5/14)

2009-10(8/24-6/2)

2010-11(9/2-7/20)

12


Number and percentage of TES schools that downloaded TES results

1 2 3 4 5 6 70

1020304050607080

31

68

5040

45

75 77

20

55

41 39 41

75 77

Number of Schools ParticipatingNumber Schools Downloaded

TES Cycle

Nu

mb

er

of

Hig

h S

ch

oo

ls

1 2 3 4 5 6 7

Percent of TES Schools whose data were downloaded 65% 81% 82% 98% 91% 100% 100%

13


1 2 3 4 5 6 70

50,000

100,000

150,000

200,000

64,731

170,921

122,410

85,763101,601

139,857130,849

TES Cycle

Enrollment

Total enrollment in participating high schools, by TES cycle

Source: CBEDS

14


1 2 3 4 5 6 70

10

20

30

40

50

60

70

80

90

100

34

4843

13 11

1 2

Number of TES Cycles in Which School Participated

Number of schools

Number of schools participating in TES according to the number of cycles

15


0

10

20

30

40

50

60

70

80

90

100

67

27

76

5851

10

56

30

TES (All 7 Cycles) CA Public 2010-11

Percent

Selected student characteristics in participating TES high schools, by cycle

16


Year of TES Participation Year 1 to Year 2

1st 2nd 3rd 4th ChangePercen

t Change

Percent of Grads Eligible for CSU 23.8 25.7 1.9 8.0

Percent of Grads Eligible for UC 18.6 20.2 1.6 8.6

Percent of Grads Completing UC Courses

21.9 23.6 1.7 7.8

Mean Number of UC a-g Courses Completed 14.3 14.4 0.1 0.7

Number of Transcripts 28,302 29,576 -- --

Among the 80 TES schools that participated for at least two consecutive cycles…

17


Among the 37 TES schools that participated for at least three consecutive cycles…



t Change

Percent of Grads Eligible for CSU 21.6 23.6 26.1 4.5 20.8

Percent of Grads Eligible for UC 16.8 18.4 19.4 2.6 15.5


19.9 21.7 23.9 4.0 20.1

Mean Number of UC a-g Courses Completed 13.9 14.0 14.2 0.3 2.2

Number of Transcripts 14,902 15,461 15,175 -- --

18




t Change

Percent of Grads Eligible for CSU 16.8 14.4 18.0 22.2 5.4 32.1

Percent of Grads Eligible for UC 13.1 12.5 14.0 15.8 2.7 20.6


15.4 13.8 16.8 20.9 5.5 35.7

Mean Number of UC a-g Courses Completed 12.9 12.6 12.8 13.7 0.8 6.2

Number of Transcripts 3,236 3,664 3,580 3,362 -- --

Among the 10 TES schools that participated for at least four consecutive cycles…

19




t Change

Percent of Grads Eligible for CSU 13.0 14.8 20.4 19.4 6.4 49.2

Percent of Grads Eligible for UC 8.4 13.5 15.8 13.9 5.5 65.5


11.2 14.4 18.7 17.6 6.4 57.1

Mean Number of UC a-g Courses Completed 12.5 12.7 13.9 14.4 1.9 15.2

Number of Transcripts 884 867 702 762 -- --

Among “intensive TES participants” (3 schools) participated for at least four consecutive cycles…

20




t Change

Percent of Grads Eligible for CSU 20.8 31.4 37.1 16.3 78.4

Percent of Grads Eligible for UC 16.4 24.0 26.9 10.5 64.0


19.1 28.8 35.0 15.9 83.2

Mean Number of UC a-g Courses Completed 14.7 15.2 15.7 1.0 6.8

Number of Transcripts 2,635 2,591 2,633 -- --

For “effective implementer” district (6 of 8 schools participating for 3 consecutive years)…

21


Summary: Percent of Grads Eligible for CSU

Year 1 Year 2 Year 3 Year 410

15

20

25

30

35

40

Two consecutive cycles (80)Three consec-utive cycles (37)Four consecu-tive cycles (10)Intensive partic-ipants (3)Effective im-plementer (6)

22


Among the 37 TES schools that participated for at least three consecutive cycles…


1st 2nd 3rd Change Percent Change

a. History/Social Science 2.13 2.13 2.15 0.02 0.9

b. English 3.21 3.19 3.22 0.01 0.3

c. Math 2.57 2.63 2.70 0.13 5.1

d. Laboratory Science 1.74 1.77 1.78 0.04 2.3

e. Language Other Than English 1.95 1.96 2.02 0.07 3.6

f. Visual and Performing Arts 1.32 1.37 1.36 0.04 3.0

g. College-Preparatory Elective 1.02 0.97 1.00 -0.02 -2.0

23


Year of TES Evaluation

2nd 3rd 4th 5th 6th 7th

Number of Schools 74 37 10 5 3 2

Number of Graduates 26,371 13,749 3,464 1,816 1,031 697

UC Application Percent for TES Schools, in Participation Year 15.4 16.6 11.7 11.9 12.7 17.4

UC Application Percent for TES Schools, at Baseline (1st Year) 13.9 13.1 8.2 8.3 7.0 6.9

Growth in UC Application Percent for TES Schools (relative to base year) 1.5 3.5 3.5 3.6 5.7 10.5

Growth in UC Application Percent for Non-TES Schools (relative to base year) 1.3 1.6 2.6 3.0 3.7 5.3

Net Growth in UC Application Percent (relative to non-TES Schools) 0.2 1.9 0.9 0.6 2.0 5.2

Net Growth, as Percent of Baseline 1.5 14.2 10.4 6.6 29.1 76.9

Applications to UC:TES vs. Comparable Non-TES Schools

24


Net Growth in Applicants, as Percent of Baseline


2nd 3rd 4th 5th 6th 7th

All Schools 1.5 14.2 10.4 6.6 29.1 76.9

Intensive TES Participants 0.8 22.2 6.8 17.1 40.0 155.7

Effective TES Implementer 25.4 30.2 -- -- -- --

Schools with TES Partners (cycles 5-7) 4.1 20.3 -- -- -- --

Percent of Graduates Applying to UC:Net Growth, TES vs. Comparable Non-TES Schools

25


2 3 4 5 6 70

10

20

30

40

50

60

70

80

All TES Schools

Intensive TES Participants

Effective TES Implementers

Schools with TES Partners


Net

Gro

wth

in A

pplica

nts

, as P

erc

ent

of

Baseline

Percent of Graduates Applying to UC:Net Growth, TES vs. Comparable Non-TES Schools

26


2 3 4 5 6 7

-10

0

10

20

30

40

50

60

70

80

All TES Schools

Intensive TES Participants

Effective TES Implementers

Schools with TES Partners


Net

Gro

wth

in A

dm

its,

as P

erc

ent

of

Base-

line

Percent of Graduates Admitted to UC:Net Growth, TES vs. Comparable Non-TES Schools

27


TES and CSU/UC eligibility:•Eligibility rates improved substantially (up to 30%)

•More improvement where better implemented (more than 50%)

•Some change on college prep course-taking

TES and UC applications and admits:•10-30% increase over 2-4 years•Possibly more over longer horizons

Summary28

• Broader availability: Available to more schools/districts

• Faster data connections: Pulling in data directly from supported Student Information Systems (SIS)

• More user friendly: Redesign of TES results site and reports

• More support: Additional resources for how and when to use TES

• Ripple effects: More information about SIS data quality and opportunities to improve

Transcript Evaluation Service

Redesigned and Redefined


29

For more information about…

TESReginald [email protected]

www.transcriptevaluationservice.com

TES Outcomes AnalysisRoger [email protected](Photographs by Elena Zhukova, photographer, 415. 852. 8384,

www.elenazhukova.com)


30

college access and readiness outcomes of the university of california transcript evaluation service...

Documents

high school students

college high schools

college access

college preparatory

total college

transcript uploads

school improvement

right courses