inventory of oregon's youth development programs 11.20.14 v2€¦ · contact information...
TRANSCRIPT
ECONorthwestECONOMICS • FINANCE • PLANNING
Inventory of Oregon’s Youth Development Programs
November 2014
Prepared for:
Oregon Youth Development Council in conjunction with the
YDC State Youth-Serving Agencies Committee
With an Executive Summary prepared by the YDC State Youth-Serving Agencies Committee
Contact Information ECONorthwest is solely responsible for the content of this report.
ECONorthwest specializes in economics, planning, and finance. Established in 1974, ECONorthwest has over three decades of experience helping clients make sound decisions based on rigorous economic, planning and financial analysis.
For more information about ECONorthwest, visit our website at www.econw.com.
For more information about this report, please contact:
John Tapogna, President
ECONorthwest 222 SW Columbia Street Portland, OR 97201 503-222-6060
Table of Contents
State Youth-Serving Agencies Committee - Executive Summary ................................. i State Youth-Serving Agencies Committee: A Final Word ..................................................... iv
Identifying State Youth-Serving Programs—The Analysis ............................................. 1 The Initial Data Assembly ...................................................................................................... 1 Inventory Process Assessment ............................................................................................. 1
Inclusion/Exclusion Rules for Evaluating Evidence Base ............................................. 3 Program Categorization ..................................................................................................... 5 Assessing the Evidence Base of State Youth-Serving Programs ................................. 6
Evaluation of Evidence Base, by Category ........................................................................... 9 Findings ................................................................................................................................ 17
Initial Conclusions and Recommendations ................................................................. 19 ECONorthwest Recommendations ..................................................................................... 21
Appendix A: Tables ......................................................................................................... A-1
i
State Youth-Serving Agencies Committee - Executive Summary
The Oregon Legislature established the Youth Development Council (YDC) in 2012 to assist the Oregon Education Investment Board (OEIB) in the assessment and coordination of the state’s investments in programs that help school-‐‑age children and youth succeed and stay out of the juvenile justice system. One of the YDC’s roles in assisting the OEIB consists of assessing youth-‐‑serving programs across the state, as well as making recommendations as to how these programs can be coordinated, consolidated, or, where appropriate, eliminated.
According to the legislation underlying this project, House Bill 4165, the YDC was directed to:
• Assess state programs and services related to youth development and training, and identify methods by which programs and services may be coordinated or consolidated.
The investments that help children and youth succeed in school, gain access to careers and the workforce leading to healthy and productive lives and stay out of the juvenile justice system are delivered by a wide range of state-‐‑funded agencies. The purpose of this report is to assist the OEIB and legislature to understand: 1) which state programs are designed to change the outcomes of interest; 2) how much is spent on those programs and who is served; and 3) the degree to which the program has an evidence base—a track record of having improved outcomes for youth in Oregon or elsewhere.
This initial report seeks to answer the following questions:
• What is the current roster of programs that directly affect the outcomes that the council is charged with improving?
• How much does the state spend on youth serving programs across the state and how much funding do these programs receive from other sources—especially the federal government?
• How many youth are enrolled in the program and what is the cost per participant?
• What degree of evidence exists to link the program to an outcome of interest?
• What trends and issues regarding the ability of youth serving agencies to collect and provide sufficient data to assess program outcomes were found as a result of the inventory process?
ii
Recommendations
Based on responses answering the aforementioned questions, the State Youth-‐‑Serving Agencies Committee of the Youth Development Council makes the following recommendations:
• Require state agencies to coordinate and collect participant and fiscal data for each program site and for each distinct service provided and to monitor the costs and effectiveness of services at each program site. The committee found that in many state agencies, responsibility for youth program information is spread out among several staff and in some instances there is no identified staff to perform data tracking functions or to keep updated summary descriptions of youth programs.
• Require all state agencies funding youth programs to collect a standardized set of data points. These data points should include: tracking population; cost per youth; typical duration of participation, time intensity, risk and protective factors addressed; racial, ethnic and gender composition; budget and expenditures and measurable outcomes.
• Agencies should collect other socioeconomic indicators including education level achieved, household demographics, and whether the youth is in foster care.
• Where possible, and if not already doing so, require state agencies to develop a common set of measurable outcomes that align with education, career and workforce, health, crime prevention and crime reduction.
• Require all state agencies funding programs with state funds to report participant data and outcomes. The committee found that many state agency youth programs did not require outcomes reporting from participants.
• Require state agencies with youth programs to coordinate program efforts across agencies. For example, there are currently three state agencies with programs that address the needs of homeless youth: the Department of Human Services’ Homeless Youth Program; the Oregon Department of Education’s Homeless Education Program; and the Department of Housing and Community Service’s Homelessness Prevention Program.
• Fund state agency youth programs that work and have positive measurable outcomes. Eliminate/de-‐‑fund programs that over time (one biennium) have not produced positive measurable outcomes for children and youth.
• The committee noted that the report identified a heavy reliance on federal funds (subsidy) for programs serving youth at the state level. With the exception of Public
Among the recommendations: ü Require agencies
to coordinate and collect data from each site, monitoring costs and effectiveness.
ü Require agencies to coordinate program efforts across state agencies.
ü Fund programs that work and eliminate those that don’t.
iii
Safety funding, 46% of all funds for state youth programs come from the federal government in the form of grants.
• With the exception of Oregon Youth Authority (OYA) programs, the State of Oregon funds only 12% of non-‐‑basic needs programs serving those ages 5 through 20. The committee recommends that a more balanced approach to the funding of youth programs be put in place, one in which state funding is at least an equal percentage of funding received from the federal government.
Highlighted Observations
The committee wishes to call attention to the following observations made during its review of the data and information collected by ECONorthwest.
In a section entitled Inventory Process Assessment, ECONorthwest made the following assessment of trends:
Through the process of requesting and collecting youth development program data, trends became apparent of what agencies were – and were not – able to provide for the inventory.
• Some agencies reported that data at the level of detail requested are not collected at the state level.
• Most agencies were unable to provide detailed race and ethnicity reporting, as well as the number of youth living in poverty.
• Of the 82 programs considered youth development programs for this report, detailed ethnicity information was provided for only 24 programs. Two programs provided details on the share of youth served who were eligible for free or reduced price lunch, which indicates that the student households have incomes below 185 percent of the federal poverty line.
• In total, the program survey identified a total of $128.4 million devoted to youth development programs in Oregon during the 2013 fiscal year. The state general fund provided about $55.3 million, or 43 percent of this total.
• By category, juvenile justice programs account for the largest share of the $128.4 million total, and also the largest share of the $55.3 million (57.3 percent) in state funding, driven by the $18 million in state general funds allocated to OYA cognitive behavioral treatment (CBT) programs.
• Most identified youth development programs serve relatively few youth and receive relatively little funding. Among included programs, median funding was $269,431 and the median program served 550 youths.
• Federal budget forecasts suggest a significant probability of shrinking discretionary spending for the foreseeable future, a dynamic that the council must recognize in evaluating Oregon’s youth development program portfolio.
• For most programs reviewed, intensity and duration of services were not consistent across all program sites and for all youth served.
iv
• Many programs provided a mix of services, providing complementary services through one program. Other programs provide more basic social services that directly benefit youth and the families of these youth.
• The council should continue to evaluate the evidence base supporting all relevant programs, but may want to focus local evaluation efforts on larger investments.
• Most programs provided details on program outputs, such as survey results and the number of youth participating in and completing the program. Very few programs provided measures of outcomes such as increased student engagement (i.e., higher attendance, lower discipline rates), increased high school graduation rates, increased college enrollment and graduation rates, and recidivism rates.
• The Council should continue to look beyond Oregon’s existing portfolio of programs for emerging or proven practices applicable to the Oregon context. Such “low-‐‑hanging fruit” could include college access programs that provide low cost information on financial aid to populations with historically low rates of college enrollment, dropout prevention programs such as Check & Connect, and others.
The committee is concerned about these omissions in state agency youth program data. The Youth Development Council has on several occasions called attention to the need to address these and other shortfalls with specific examples indicating that data availability and the quality of that data is considerably poorer for Native American youth, homeless youth, and LGBTQ youth, and much work is needed to obtain a more accurate understanding of how these youth are adversely impacted in our education and work systems.
State Youth-Serving Agencies Committee: A Final Word The legislative mandate to begin the review of state youth-‐‑serving agencies, assess program services, and to align and coordinate these programs is a comprehensive task, indeed. However, unless and until there is more collaboration and high-‐‑level discussion concerning investments made in state youth programs, our state systems of support for youth and youth development in the state will not be maximized.
In addition to ECONorthwest’s recommendations, the State Youth-‐‑Serving Agencies Committee believes the state must ensure more cross collaboration in making funding investments, ensure there are standardized collective performance measurements, and that the state fund youth programs that have positive measurable outcomes. The state must also eliminate programs that over time have not produced positive measurable outcomes for children and youth and re-‐‑direct those funds to the many programs that are working to improve outcomes for children and youth across Oregon.
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 1
Identifying State Youth-Serving Programs—The Analysis
The Initial Data Assembly ECONorthwest conducted a comprehensive inventory of the programs provided by the state’s youth-‐‑serving agencies. In an effort to build off of work already performed by YDC, the first step in building this report was to contact each state agency with a request to update the existing list of youth serving programs, as summarized in the Oregon YDC’s Summary of State Youth-‐‑Serving Agencies report.1
Based on the inventory report, ECONorthwest exchanged information with program and budget analysts in 10 state agencies. Through conversations with YDC staff and state analysts we expanded our data gathering to 175 unique programs that ranged in scope from one-‐‑day financial aid workshops to the Temporary Assistance for Needy Families (TANF) program and the Supplemental Nutrition Assistance Program (SNAP).
Agencies were asked to provide the following information when updating their program list:
• General program information (e.g., a summary of program services, characteristics of the target population);
• Program supply and target population (e.g., the number of program slots available, the racial/ethnic composition of participants;
• Budget and expenditures in FY2013 and FY2014;
• Funding sources for FY2013 and FY2014; and
• Program evaluation measures, methods, and findings.
In addition to the list above, agency contacts were asked to provide, if applicable, an explanation of the nationally recognized program model that was the basis for the program in questions, as well as an assessment of the program’s fidelity to the national program model.
Inventory Process Assessment Through the process of requesting and collecting youth development program data, trends became apparent in what agencies were and were not able to provide for the inventory. The following is a summary of these trends:
• Some agencies reported that data at the level of detail requested are not collected at the state level.
1 http://www.oregon.gov/gov/docs/OEIB/1aaYDCreport.pdf 2 Coalition for Evidence-‐‑Based Policy website. Retrieved in October 2013 from http://evidencebasedprograms.org/
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 2
• Most agencies were unable to provide detailed race and ethnicity reporting, as well as reporting on the share of youth in poverty. Of the 82 programs considered youth development programs for this report, detailed ethnicity information was provided for 24 programs. Two programs provided details on the share of youth served who were eligible for free-‐‑ or reduced-‐‑price lunch, which indicates that the students’ households have incomes below 185 percent of the federal poverty line.
• For most programs, intensity and duration of services were not consistent across all program sites and for all youth served.
• Many programs provide a mix of services, providing complementary services through one program. Other programs provide more basic social services that benefit both youth directly and the families of these youth
• Most programs provided details on program outputs, such as survey results and the number of youth participating and completing the program. Very few programs provided measures of outcomes, such as increased student engagement (e.g., higher attendance, lower discipline rates), increased high school graduation rates, increased college enrollment and graduation rates, and recidivism rates.
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 3
Inclusion/Exclusion Rules for Evaluating Evidence Base The initial scan and data collection was intentionally broad. We did not want to preemptively exclude programs that program managers believed were critical to improving youth development related outcomes. But after a review of program missions, we excluded a number of programs designed to address basic needs or family financial stability, as well as a limited number of programs that did not meet program or reporting criteria defined by the YDC or ECONorthwest.
Basic Needs
Basic needs programs address the food, housing, childcare, safety and crisis intervention needs of parents and their children. While these programs create the conditions necessary for positive youth development, they do not explicitly seek to advance youth development.
Four agencies provide basic needs programs: the Oregon Department of Human Services, the Oregon Department of Education, Oregon Housing and Community Services, and Community Action Partnership of Oregon (CAPO). A complete list of programs by agency is included in the in Table 1.
Other Excluded Programs
The following programs are excluded from this analysis because they did not meet the program or reporting criteria set forth by the YDC and ECONorthwest:
• Portland State University Capstone Program. While the program sends college students out into the community to serve a variety of youth-‐‑related needs, the program itself is directed at college seniors who fall outside the age range of this analysis (6-‐‑20). No funding information was reported.
• Two other small Portland Status University programs for developing curriculum for a Russian language program. No funding information was reported.
• The Oregon Youth Conservation Corps, under the Department of Community Colleges and Workforce Development (CCWD), runs a small grant-‐‑funded project with the Oregon State Marine Board. It serves only 10 youth during the summer, and according to the Youth Conservation Corps, nearly all the cost of the program goes to pay for building supplies for a project to improve public boating areas. Total funding for this program amounts to $173,925, including a $98,925 local match and $55,000 of state general funds.
• Career Pathways was included in the Youth Development Council’s 2012 inventory document, but according to CCWD, only two percent of the programs participants are under the age of 20. Total funding for this program amounts to $400,000, all of it from state general funds.
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 4
Table 1. Summary of Funding Sources by Program, Basic Needs Programs, FY 2013
Agency/ProgramTotal of State General Funds
Total of Federal Funds
Total of Other (local govt, tuition, grants, donations, etc.) Total
Unduplicated Participants
DHS $182,042,457 $654,882,709 $62,702,301 $899,627,467 25,898 * Interstate Compacts $137,734 $91,879 $14,862 $244,476 -* Adoption Assistance $18,973,625 $19,246,952 $97,546 $38,318,123 10,794 * Adoption Search $0 $0 $26,626 $26,626 -* Contracted Adoption Services $593,746 $832,142 $93 $1,425,980 -* Domestic Violence Services $127,627 $769,237 $1,004,805 $1,901,669 -* Family Support Teams $614,033 $1,105,050 $15,615 $1,734,698 -* Guardianship Assistance $1,743,499 $2,428,570 $110,558 $4,282,627 1,296 * Independent Adoption Services $0 $0 $73,440 $73,440 -* ISRS (previously Family Based Services) $2,261,196 $3,325,572 $58,303 $5,645,070 -* Nursing Assessments $131,848 $14,055 $16,959 $162,862 -* Other Medical $1,376,816 $361,246 $16,760 $1,754,823 -* Other Tribal $179,236 $593,206 $6,860 $779,302 -* Personal Care $15,920 $82,264 $36,989 $135,173 -* Post Adoption Services $25,329 $229,111 $383 $254,822 -* Private Adoptions $99,501 $122,150 $0 $221,651 -* Recovering Family Mutual Homes $126,655 $180,266 $3,269 $310,189 -* Sexual Assault Victims Fund $23,477 $0 $160,444 $183,922 -* Strengthening & Preserving Families $540,633 $1,737,077 $0 $2,277,710 -* System of Care $1,490,737 $2,137,154 $133,412 $3,761,304 -* Waiver Foster Care $545,395 $888,533 $3,521 $1,437,450 -* Youth Investment Program (Sub Care) $108,018 $2,686,712 $0 $2,794,730 -Child Care Assistance $1,391,256 $0 $55,085,958 $56,477,214 -Family Support and Connections $127,933 $1,927,558 $0 $2,055,491 -Food And Cash Assistance $93,579,750 $542,644,198 $1,005,754 $637,229,702 -Independent Living Services $159,216 $3,964,957 $19,786 $4,143,959 1,450 PC20-Children $437,381 $879,899 $89,658 $1,406,938 1,052 SE #142 Children's Residential $6,989,891 $11,679,689 $0 $18,669,580 188 SE #143 Children's Proctor Foster Care $1,618,707 $2,704,762 $0 $4,323,469 65 SE #145 Children's Intensive In-Home Services (CIIS) $5,279,524 $8,821,768 $0 $14,101,292 422 SE #150 Family Support $958,740 $0 $0 $958,740 1,275 SE #151 Children's Long-Term In-Home Support $1,438,856 $0 $0 $1,438,856 216 SE #44 Crisis-Children $41,066 $39,771 $0 $80,837 63 SE#58 Non-Relative Foster Care-Children $8,947,982 $14,951,543 $0 $23,899,525 683 TANF/Child Care $2,200,000 $3,300,000 $0 $5,500,000 -Temporary Assistance for Domestic Violence Survivors $29,404 $4,784,806 $0 $4,814,210 -x* Regular and Enhanced Foster Care $19,157,528 $12,563,276 $3,716,264 $35,437,068x* Residential Treatment (98% BRS) $5,990,103 $6,986,711 $591,506 $13,568,320x* Shelter Foster Care $810,561 $295,882 $304,928 $1,411,371x* Special Contracts (67% BRS) $2,962,507 $1,303,943 $56,093 $4,322,543x* Target Children (62% BRS) $807,027 $1,202,769 $51,909 $2,061,705xTotal Foster Care 8,394 ODE $59,922 $30,107,847 $0 $30,167,769 60,152 After School Programs $0 $6,104,224 $0 $6,104,224 20,223 After School Snacks $0 $832,728 $0 $832,728 12,084 Child Care Centers $0 $2,570,356 $0 $2,570,356 6,340 Family Day Care Homes $0 $15,383,873 $0 $15,383,873 15,723 Homeless Emergency Shelters $0 $157,473 $0 $157,473 150 Outside-School-Hours Centers $0 $16,268 $0 $16,268 437 Seamless Summer Option $364 $27,062 $0 $27,426 5,195 Summer Food Service Program $59,558 $5,015,863 $0 $5,075,421OHCS local CAP programCommunity Sharing (Lane; SHAP/EHA funds) - - - - -Executive House (KLCAS; SHAP funds) - - - - -Feed the Child (CAPECO; ESG Funds) - - - - -Hearts with a Mission (ACCESS; SHAP & CSBG funds) - - - - -HOME Youth & Resource (MWVCAA; CSBG funds) - - - - -Insights Teen Parent Program/Janus Youth Programs (Multnomah Co; EHA funds) - - - - -Looking Glass (Lane; (SHAP/EHA funds) - - - - -Maslow Project (ACCESS; ESG funds) - - - - -The Inn (CAA of Clackamas Co; EHA & SHAP funds) - - - - -Tonya's House (CAPECO; ESG Funds) - - - - -Grand Total $182,102,379 $684,990,556 $62,702,301 $929,795,236 86,050
FY 2013
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 5
Program Categorization ECONorthwest categorized programs as “Youth Development” when they both served youth directly (rather than serving families or building agency infrastructure) and served goals related to academic and social development, training, crime prevention and decreased juvenile justice involvement, or the prevention of gang violence and involvement. We grouped youth development programs into seven categories:
• Advanced Credit: Programs that provide students with opportunities to take college or postsecondary courses, for credit or otherwise.
• College Access: Programs that expose students to the college environment, provide information and assistance in the college admissions process, or provide academic preparation for postsecondary education.
• Dropout Prevention and Recovery: Programs that target youth who have dropped out of school or have shown signs of being at-‐‑risk of dropping out of school, with the purpose of assisting youth with completing high school or a high school graduation equivalent (e.g., GED).
• Prevention and Intervention for At-‐‑Risk Youth: Programs that support youth at risk of delinquency or who have a history of delinquency, with the purpose of deterring these youth from delinquent behavior and keeping them out of the juvenile justice system.
• Juvenile Justice: Services that support youth with current or past involvement in the juvenile justice.
• Enrichment: Programs that supplement and expand on in-‐‑school education.
• Specialized Educational Services: Programs that support specific populations and address their unique needs, including migrant students, youth with disabilities, blind and visually impaired students, and homeless students.
The categorization of programs can be found in Appendix A of this report.
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 6
Assessing the Evidence Base of State Youth-Serving Programs Evidence-‐‑based policies, evidence-‐‑based programs, and evidence-‐‑based practices, all referred to as EBPs in the literature, are similar terms used to describe activities that satisfy the same underlying concept. To qualify as an EBP, an activity must have been rigorously proven to provide explicitly positive outcomes when faithfully implemented. In the context of this project, ECONorthwest uses “EBP” to refer to evidence-‐‑based programs, rather than to practices that are part of a program or the policies that impact the programs.
As in medical science, the randomized controlled trial (RCT) has become the “gold standard” for proof of effectiveness in social science. However, for a variety of practical reasons, such rigorous studies are rare in the realm of public policy; quasi-‐‑experimental studies with matched control and treatment groups are more common, and these can vary widely in quality. Several public policy organizations provide analysis and recommendations about which public programs have been proven to work, based on a sufficiently rigorous evidence base of well-‐‑designed experimental or quasi-‐‑experimental studies.
Among these organizations is the well-‐‑respected Coalition for Evidence-‐‑Based Policy (CEBP). CEBP’s “Social Programs that Work” website identifies a small group of programs that CEBP deems to have been proven effective through a sufficient number of well-‐‑designed RCTs. To be included in the list, the “social interventions [must be] shown in rigorous studies to produce sizable, sustained benefits to participants and/or society.”2
CEBP defines a study as well-‐‑designed and rigorous if the study:
• used a randomized controlled trial study design; • included a suitably large sample of participants; and • was, preferably, conducted in real-‐‑world settings.
The Washington State Institute for Public Policy (WSIPP) is another organization highly regarded for rigorous research and analysis. For a recent meta-‐‑analysis, WSIPP’s criteria for study inclusion were less rigorous than those of CEBP, but still required that studies demonstrate a reasonable level of scientific rigor. They included studies based on either RCTs or well-‐‑designed quasi-‐‑experimental designs. In particular, quasi-‐‑experimental studies must have included well-‐‑matched treatment and control groups (i.e., no treatment or treatment as usual) to estimate program effects. WSIPP included, but discounted, the results of evaluations with specific weaknesses. For example, if a study was not an RCT or if the studied program was a “model program” run by the program’s developer, WSIPP reduced the estimated program impact by a pre-‐‑determined percentage for the meta-‐‑analysis.
2 Coalition for Evidence-‐‑Based Policy website. Retrieved in October 2013 from http://evidencebasedprograms.org/
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 7
The Institute for Education Sciences, the primary research arm of the US Department of Education, provides analysis of existing research, with similar standards of scientific rigor to those of WSIPP, through the What Works Clearinghouse (WWC). WWC only considers “eligible” studies in their review of an intervention’s effectiveness. WWC initially screens for studies that they consider ineligible for review for any of the following reasons:
• The study is not a primary analysis of the effect of an intervention,
• The study does not have an eligible design (RCT or quasi-‐‑experimental design),
• The study does not use a sample aligned with the review protocol set by WWC,
• The study does not include an outcome within a domain specified in the protocol set by WWC for the topic area, and
• The study was not published in the relevant time frame of 20 years prior to the review.
The results of eligible studies are then designated as falling into one of two measures of quality: Meet WWC Group Design Standards without Reservations and Meet WWC Group Design Standards with Reservations. Only well-‐‑executed RCTs are considered to meet WWC standards without reservations. RCTs with varying probabilities of assignment must consider the differing probabilities of being assigned to the intervention group in order to be considered “well-‐‑executed” and meets WWC Group Design Standards without Reservations. WWC’s assessment of an intervention’s effectiveness is translated into an “improvement index,” which can be interpreted as the expected change in percentile rank for an average comparison group student if the student had received the intervention.
Finally, the U.S. Department of Justice, Office of Juvenile Justice and Delinquency Prevention (OJJDP) maintains an online “Model Programs Guide”3 that identifies effective prevention and intervention programs for youth. OJJDP rates each program according to the strength of empirical evidence supporting each program’s impacts using methods similar to those described for CEBP, WSIPP, and WWC.
In our evaluation of the evidence base of programs within the established program groups, we have relied primarily on these sources of rigorous analysis. We also reviewed other, potentially less stringent research on the inventoried programs or those with similar services and intended outcomes. Programs for which research has been performed, but not up to the standards of the three sources described above, will be considered “research based” programs. Existing literature and research on these programs are also summarized.
3 http://www.ojjdp.gov/mpg/Default.aspx
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 8
We label programs with little or no rigorous evidence of impacts as “Promising” for the purposes of this review. This category includes programs for which the existing evidence is mixed and programs lacking third-‐‑party prepared or reviewed analysis. A lack of evidence does not necessarily imply that a program is ineffective.
The State Youth-‐‑Serving Agencies Committee of the YDC believes the state must not just recognize and appreciate tribal best practices, but it should also design and implement programs that are culturally relevant to the youth being served in youth development programs. Practice-‐‑based solutions allow for communities to identify and implement solutions that often work best for them.
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 9
Evaluation of Evidence Base, by Category
Advanced Credit Programs
All of the surveyed programs under the category of Advanced Credit share the intended outcome of providing students with a successful start on their postsecondary education. These programs include the facilitation or funding of a student’s dual enrollment in college courses while completing high school, as well as partnership programs where high school or college instructors are certified to teach college credit-‐‑eligible courses in participating high schools.
A number of individual studies on dual enrollment and college credit programs suggest positive program impacts, but we find no scientifically rigorous meta-‐‑analyses of these programs. A study performed for a dual enrollment program in California found that, for two years for high school cohorts at six and seven sites (respectively), the “average effect of dual enrollment on graduation remains positive and significant for both graduating classes.” This study also found that dual enrollment programs have shown significant impacts on student performance or GPA in high school courses while enrolled in college credit courses.4
In a study performed by Iowa State University, early college credit was found to have a significant impact on student performance and completion of 2-‐‑ and 4-‐‑year degrees for those students earning any number of college credits while enrolled in high school. The following significant positive outcomes have been found for students entering 4-‐‑year degree programs with college credit:
§ The GPA of students in the first year of college is slightly higher than for those entering a 4-‐‑year program with no college credits.
§ Students in 4-‐‑years colleges with college credits have been found to complete their degree program and graduate sooner than those entering a 4-‐‑year college degree program with no college credits earned while enrolled in high school. The study found that students enrolling at Iowa State in Fall 2005 with 19+ credits had an average time to graduation of 4.1 years compared to 4.5 for their peers without early college credit.
§ Students entering a 4-‐‑year college with early college credits are also more likely to earn a minor or earn a double major. 5
4 http://ccrc.tc.columbia.edu/media/k2/attachments/broadening-‐‑benefits-‐‑dual-‐‑enrollment-‐‑rp.pdf 5 http://www.provost.iastate.edu/sites/default/files/uploads/reports/2011EarlyCreditTaskForceFinalReport.pdf
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 10
College Access Programs
College access programs include a variety of services aimed at increasing the college-‐‑going rates of high school students. Interventions include access to education and information on financial aid, scholarships, admissions requirements, and admissions process. These services are sometimes paired with college visitations, as is the case for the Making College Happen program at Western Oregon University.
In 2009, ECONorthwest conducted a quantitative evaluation of the ASPIRE program in Oregon high schools. Students from ASPIRE sites were found to be, on average, about four percentage points more likely to attend college compared to students from non-‐‑ASPIRE sites, after various underlying differences between these groups are accounted for. Students identified as ASPIRE participants are found to be, on average, about 23 percentage points more likely to attend postsecondary schools than similar students attending the same site who do not participate in ASPIRE. Due to the significant data limitations, these findings do not constitute a strong evidence base.
For the purposes of this review, we categorize GEAR UP programs as “promising” because of significant variation in implementation and the lack of a comprehensive national study. Programs vary considerably even at the school level, limiting the potential for even state level evaluation. On the other hand, the flexibility allowed grantees provides a laboratory in which to test the efficacy of a wide range of interventions aimed at increasing the college enrollment rates of at-‐‑risk youth.6
The preliminary findings of a meta-‐‑analysis, supported by the Institute of Education Sciences, provided mixed impacts of college access programs overall. On average, college access programs were found to increase high school graduation by eight percentage points. However, when only the RCT studies were considered, the estimate of the average impact was not statistically significant. The average impact of college access programs on enrollment in 2-‐‑year or 4-‐‑year college programs is a 12-‐‑percentage point increase. The average impact of RCT-‐‑evaluated programs is also positive, four percentage points, and statistically significant.7
WWC has not performed a full review of interventions aimed at increasing college access, but has reviewed an individual RCT study that examined the impact of offering an online informational video and financial aid materials to high school students. The study found statistically significant impacts on student expectations about attending postsecondary education, with 88 percent of the intervention group expecting to attend postsecondary education, compared to 82 percent in the comparison group. 8
6 https://edsurveys.rti.org/gearup/ls/Muraskin_Final.pdf 7 https://www.sree.org/conferences/2012s/program/downloads/abstracts/520.pdf 8 http://ies.ed.gov/ncee/wwc/SingleStudyReview.aspx?sid=20001
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 11
Dropout Prevention and Recovery Programs
Dropout prevention and recovery programs ultimately seek to assist youth who are either at-‐‑risk of dropping out of school or who have already dropped out of school. These programs provide a broad range of services and include, but are not limited to, programs that provide services directly aimed at the attainment of a high school diploma or GED and programs that provide highly structured environments for at-‐‑risk youth in order to guide them toward successful completion of their secondary education.
The What Works Clearinghouse review of evidence regarding Service and Conservation Corps programs found only one study that met WWC standards. The study included 626 at-‐‑risk youths between the ages of 17 to 26 who participated in community service projects in California, Florida, New York, and Washington State and found no discernable effects of the program on completing school for at-‐‑risk youth. 9 Due to the lack of eligible studies, WWC classified the evidence of this intervention’s effectiveness or ineffectiveness as weak. For this reason, we classify the two Youth Conservation Corps programs as promising.
WWC’s review of National Guard Youth ChalleNGe programs found only one study that met their evidence standards. The study found potentially positive program effects on school completion, but the small evidence base does not provide sufficient evidence to classify the program as evidence based. The study does not examine the effectiveness of the program in keeping at-‐‑risk youth in school or progressing in grade level. 10
The WOU Upward Bound Program, currently managed by the Oregon University System, is a local program funded through the Federal TRIO program. The Pell Institute’s 2009 evaluation of outcomes for TRIO programs across the country found that Upward Bound programs had significant positive impacts on three college-‐‑going outcomes11. Compared to control group participants, Upward Bound students were found to be:
§ 50 percent more likely to attain a bachelor’s degree,
§ 19 percent more likely to attain any postsecondary degree or credential, and
§ 22 percent more likely to apply for student financial aid.
9 http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=457 10 http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=329 11 http://www.pellinstitute.org/downloads/publications-‐‑Studies_Find_TRIO_Programs_Effective_May_2009.pdf
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 12
Prevention and Intervention for At-Risk Youth
The category of prevention and intervention for at-‐‑risk youth consists primarily of programs that attempt to deter youth from delinquent behavior including drug or alcohol use, sexual activity, and other behavior physically harmful to the youth or others.
A WSIPP meta-‐‑analysis examined the effectiveness of pregnancy prevention and other school-‐‑based sexual education programs. WSIPP concluded that these programs had no statistically significant effect on the rate of teen pregnancy (under the age of 18) or on the initiation of sexual activity.12
Another WSIPP meta-‐‑analysis of truancy and dropout prevention programs that pair struggling students with an adult mentor make a small positive impact on school attendance and the dropout rate, but not on achievement. Most of the interventions evaluated employed paid mentors, who may be incentivized to perform better than the more commonly used volunteer mentors.13
The CEBP reviewed the evidence regarding the Big Brothers Big Sisters program. Youth in the intervention group were 46 percent less likely to have started using illegal drugs and 32 percent fewer incidents of hitting someone in the previous 12 months, both statistically significant effects. The program also had a small, marginally significant, positive impact on their GPA.14
We find strong support in the research literature for the effectiveness of parent-‐‑training programs. Specifically, the Office of Juvenile Justice and Delinquency Prevention (OJJDP) reports that several meta-‐‑analyses of Parent Training programs suggest moderate, statistically significant impacts on instances of abuse and other problem behaviors.15
The National Registry of Evidence-‐‑based Programs and Polices, as well as the OJJDP, found that Strengthening Families programs increase parenting competency and reduce student substance use-‐‑related risk factors for youth in the target age group, generally age 10 to 14, impacts which in turn are associated with positive academic performance in 12th grade.16
WSIPP identifies functional family therapy, including therapy for youth on probation, as a positive, research-‐‑based program with high benefit to cost ratio.17 When properly implemented, this program reduces recidivism to a statistically significant degree.18
12 http://www.wsipp.wa.gov/rptfiles/3200.SexEd.pdf 13 http://www.wsipp.wa.gov/rptfiles/09-‐‑06-‐‑2201.pdf 14 http://evidencebasedprograms.org/1366-‐‑2/117-‐‑2 15 http://www.ojjdp.gov/mpg/progTypesParentTrainingPrev.aspx 16 http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=44 17 http://www.wsipp.wa.gov/rptfiles/11-‐‑07-‐‑1201.pdf
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 13
The OJJDP, in its evaluation of the impacts of various program designs, found that gender-‐‑based interventions, such as those employed in the Girls Circle and Boys Council programs, tend to produce positive impacts on program outcomes, but also that overall the available evidence remains weak.19
18 http://www.wsipp.wa.gov/rptfiles/07-‐‑06-‐‑1201.pdf 19 www.ojjdp.gov/mpg/progTypesGenderSpecificProgramming.aspx+&cd=1&hl=en&ct=clnk&gl=us&client=firefox-‐‑a
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 14
Juvenile Justice
Juvenile justice programs as defined here generally involve State or Federal juvenile programming that provide services in direct response to criminal behavior.
Cognitive behavioral therapy or treatment (CBT) is considered the most effective form of psychotherapy. Studies reviewed by the OJJDP found that CBT is associated with significant and clinically meaningful positive changes, including a clear reduction in recidivism rates.20
The OJJDP and WSIPP have found that Drug Courts participants have lower short-‐‑term recidivism rates than participants in standard adjudication systems. OJJDP questions the long-‐‑term effects of drug courts on recidivism rates, especially once juveniles are no longer under court supervision, but underscore the potential of juvenile drug courts to improve long-‐‑term outcomes when closely linked with participation in therapy or other treatment programs.21
A similar analysis also applies to Teen or Peer Courts. The OJJPD found that despite the generally low recidivism re-‐‑referral rates associated with Teen Courts, some studies find that some programs are associated with statistically insignificant or even negative outcomes for youth participants, indicating the need for further research to adequately evaluate the effectiveness Teen Court programs.22
Restorative justice programs emphasize repairing the harm caused by criminal behavior. The holistic restorative justice approaches seek to create more culturally appropriate juvenile justice programming. There is little research conducted on the effectiveness of the two program structures used by Oregon’s restorative justice program.23
20 www.ojjdp.gov/mpg/progTypesCognitivePrev.aspx+&cd=1&hl=en&ct=clnk&gl=us&client=firefox-‐‑a 21 www.ojjdp.gov/mpg/progTypesDrugCourt.aspx+&cd=2&hl=en&ct=clnk&gl=us&client=firefox-‐‑a 22 www.ojjdp.gov/mpg/progTypesTeenYouth.aspx+&cd=1&hl=en&ct=clnk&gl=us&client=firefox-‐‑a 23 www.ojjdp.gov/mpg/progTypesRestorative.aspx+&cd=1&hl=en&ct=clnk&gl=us&client=firefox-‐‑a
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 15
Enrichment
Enrichment programs consist of academically focused services to the state’s youth that operate independently from the public K-‐‑12 system. Many of these programs are after-‐‑school, summer, or alternative education programs with a focus on hands-‐‑on activities. A distinct subset of enrichment programs provides youth with activities focused on science, technology, engineering and mathematics (STEM).
Given the extremely limited available evidence regarding the impacts of enrichment programs, we take a broader look at attributes found to have positive effects on student outcomes. WSIPP reports that alternative education programs, within a traditional school environment, have been found to have statistically significant, but modest, impacts on the reduction of student dropout rates. These programs saw small but significant increases in student achievement in the form of grades and test scores from students participating in these programs. The majority of these programs are focused on serving at-‐‑risk youth, but did not only provide services for these youth. The positive effects can be attributes to career-‐‑focused programming.24 Based on these findings, programs aimed at providing career-‐‑focused interventions will be considered research based, although program-‐‑specific research will likely be needed in order to assess the effectiveness of each program.
Many enrichment programs provide educational activities and instruction paired with college access or college credit programming, similar to those evaluated under the college access and credit category evaluations. Programs providing college tours and college entrance informational sessions as part of their enrichment services are considered research based for the purposes of this review.
The 21 Century Community Learning Center Programs is a long-‐‑standing initiative and is the only federal funding stream that provides dedicated funds for afterschool programs. The 21st Century program is highly flexible, often making it the source of innovative student development programming. Because of the unique structure of this initiative, poor performing programs can be more easily replaced with programs that show promise and positive outcomes. The Harvard Family Research Project provides regular evaluations of 21st Century programs.25 Due to the complexity of this initiative, we are unable to address the evidence base of these programs overall.
The Oregon Department of Education awarded second year funds for all 22 current Title IV-‐‑B grantees for the 21st Century Community Learning Center (CCLC) program. The award from the U.S. Department of Education for 2014-‐‑15 was sufficient for ODE to restore all grantees to
24 http://www.wsipp.wa.gov/rptfiles/09-‐‑06-‐‑2201.pdf 25 http://www.hfrp.org/publications-‐‑resources/publications-‐‑series/research-‐‑updates-‐‑highlights-‐‑from-‐‑the-‐‑out-‐‑of-‐‑school-‐‑time-‐‑database/research-‐‑update-‐‑8-‐‑21st-‐‑century-‐‑community-‐‑learning-‐‑centers-‐‑stable-‐‑funding-‐‑for-‐‑innovation-‐‑and-‐‑continuous-‐‑improvement
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 16
100% of their originally approved proposal and also provided one-‐‑time supplements to each grantee for the support of their proposed work in their originally approved grant. The one-‐‑time supplements were allocated by a base award for all grantees plus equitable amounts based on the number of sites served. A total of 93 of Oregon’s CCLC Cohort #3 grantees from across the state were awarded more than $11.6 million.26
Specialized Educational Services
Specialized educational service programs typically target the barriers faced by youth in a specific high-‐‑needs population, presenting difficulties in generalizing the findings of effectiveness to programs that have not been independently evaluated. As a result, we classify all but one program in this category as promising.
WSIPP-‐‑evaluated studies have found that transitional employment services, such as those provided in the Oregon DHS Vocational Rehabilitation program, have shown a statistically significant increase in the likelihood that a program participant would be employed after two to six years and they would be more likely than the study’s control group to be employed in a non-‐‑training or workshop job.27
26 http://www.ode.state.or.us/news/announcements/announcement.aspx?ID=9984 27 http://www.wsipp.wa.gov/ReportFile/1054
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 17
Findings Understanding the full breadth of the state’s youth-‐‑serving programs is challenging, but also critical to sound decision-‐‑making by the YDC and other agencies as they make recommendations, align, and invest in evidence-‐‑based, age-‐‑ and gender-‐‑appropriate and culturally relevant programs across all youth serving programs.
In total, the program survey identified a total of $132.5 million devoted to youth development programs in Oregon during the 2013 fiscal year. The state general fund provided about $56.1 million, or 42 percent of this total. Currently, less than 11 percent of programs that fall under agencies that are to be assessed by the Youth Development Council can be considered “evidence based.” In Figure 1, we see that about 26 percent of the total program funds identified, and less than 11 percent of the named programs, fit into the “evidence based” category. A significant research base suggests the positive impacts of many more identified youth development programs, supported by a much greater share of the identified funds (see Figure 1), but the available evidence does meet the high standards to be considered evidence-‐‑based programs (see Figure 1).
Figure 1. Funding for Youth Development Programs, by Total Funding FY 2013
Across the three funding sources we collected data on (state general fund, federal funds, and other funding sources), a greater share of evidence based programs funds, $19.6 million, for fiscal year 2013 came from state general funds (see Figure 2). In Figure 2, we see that $35.7 million, or 65 percent of state general funds were not supporting evidence based programs.
27%
21%
51%
Evidence based
Research based
Promising
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 18
Figure 2. Funding by Source and Strength of Evidence Base, FY 2013
By category, juvenile justice programs account for the largest share of the $128.4 million total, and also the largest share of the $55.3 million (57.3 percent) in state funding, driven by the $18 million in state general funds allocated to OYA cognitive behavioral treatment (CBT) programs (see Figure 3). Many programs operated by OYA also have the strongest supporting evidence (e.g., CBT, functional family therapy). Appendix A provides additional detail on the source of funds for individual programs. A separate spreadsheet document contains the complete set of information provided by the programs.
Figure 3. Total Program Funds by Category, by Funding Source, FY 2013
$34.9
$19.6 $14.3
$0.9
$66.0
$31.2 $30.5
$4.3
$27.5
$4.5
$16.1
$6.9
$-
$10.0
$20.0
$30.0
$40.0
$50.0
$60.0
$70.0
Total State Federal Other
Mill
ions
Evidence based Promising Research based
$- $10 $20 $30 $40 $50
Advance Credit
College Access
Dropout Prevention/Recovery
Enrichment
Juvenile Justice
Prevention & Intervention for At-Risk Youth
Specialized Educational Services
Millions State General Funds Federal Funds Other Funding Sources
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 19
Figure 4. Share of State Funding by Category, FY 2013
Initial Conclusions and Recommendations A large share of state general funds (35 percent) spent on youth development programs support initiatives with a strong evidence base. Most prominently, several juvenile justice programs are funded: cognitive behavioral therapy and functional family therapy (see Figure 4). In assessing each youth development program that lacks rigorous evidence of impact, including most of the programs in the enrichment, specialized education services, dropout prevention, college access, and advanced credit categories, the Youth Development Council should begin by considering three important, related program features in addition to local needs: program scale, source of funds, and the feasibility of a local program evaluation.
As illustrated in Figure 5, most identified youth development programs serve relatively few youth and receive relatively little funding. Among included programs, median funding was $269,431 and the median program served 550 youths. For these smaller programs, rigorous evaluation may never make sense. At the same time, the relatively small investments require less rigorous justification to continue. In addition, emerging research may indicate that some of these programs do produce significant, positive outcomes. In other cases, a program may satisfy
Outpatient Services$4.4M
MinorityServices$0.6M
JCP--Gang Prevention inMult. Co.$2.3M
JCP--Diversion$4.6M
JCP--BasicServices$3.9M
Parole/Probation$11.7M
OYAFosterCare$0.8M
Cognitive Behavioral Treatment in OYA Community ResidentialPrograms$6.4M
Cognitive Behavioral Treatment in OYAClose Custody Facilities$12.6M
ReadytoReadGrant$0.6M
OSU 4-H Youth Development$3.2M
Program CategoryAdvance CreditCollege AccessDropout Prevention/RecoveryEnrichmentJuvenile JusticePrevention & Intervention for At-Risk YouthSpecialized Educational Services
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 20
a critical local need. The council should continue to evaluate the evidence base supporting all relevant programs, but may want to focus local evaluation efforts on larger investments.
Figure 5. Median and Average Total Program Funding and Total Unduplicated Program Participants
Federal funding, which comprises 49.9 percent of funding for programs not classified as evidence based, unfortunately comes with two weaknesses. First, federal funding may significantly limit flexibility in program design and implementation. On the other hand, large federally funded initiatives, such as GEAR UP and Workforce Investment Act programs, may also present opportunities to experiment with a range of evidence-‐‑based programs at different sites, even as this limits the possibility of a statewide evaluation. Second, federal budget forecasts suggest a significant probability of shrinking discretionary spending for the foreseeable future, a dynamic that the council must recognize in evaluating Oregon’s youth development program portfolio.
The Council should also continue to look beyond Oregon’s existing portfolio of programs for emerging or proven practices applicable to the Oregon context. Such “low-‐‑hanging fruit” could include college access programs that provide low cost information on financial aid to populations with historically low rates of college enrollment, dropout prevention programs such as Check & Connect, and others.
Check & Connect is a comprehensive intervention designed to enhance student engagement at school and with learning for marginalized, disengaged students in grades K-‐‑12, through relationship building, problem solving and capacity building, and persistence. A goal of Check & Connect is to foster school completion with academic and social competence.
$269
$1,585
$-
$500
$1,000
$1,500
$2,000
Median Average
Thou
sand
s
Total Funding
550
11,461
0
2,500
5,000
7,500
10,000
12,500
Median Average
Total Unduplicated Participants
ECONorthwestECONOMICS • FINANCE • PLANNING
ECONorthwest Inventory of Oregon’s Youth Development Programs 21
Of the dropout prevention interventions reviewed by the U.S. Department of Education'ʹs What Works Clearinghouse, Check & Connect is the only program found to have strong evidence of positive effects on staying in school. 28
ECONorthwest Recommendations ECONorthwest suggests that the Youth Development Council consider the following ways to improve program evaluation and alignment:
• Require programs to report to agencies on their participant data and outcomes. • Educate and inform program coordinators on program design and evaluation methods. • Provide a standard set of program measures that apply to all programs, including
detailed information on of the gender and ethnicity of youth served. • Collect data on “units of service” per program, or the number of hours of service
received per program participant. • Collect participant and fiscal data for each program site and for each distinct service
provided, to monitor the costs and effectiveness of services at each implementation site.
28 http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=78
1
ECON orthwest Appendix A: Tables A-1
Appendix A: Tables Table 2. Summary of Funding Source by Program, Youth Development Programs, FY 2013
Agency/ProgramTotal of State General Funds
Total of Federal Funds
Total of Other (local govt, tuition, grants, donations, etc.) Total of All Funds
Unduplicated Participants
CCWD $0 $10,151,814 $1,375,850 $11,527,664 312,582 Career Information Services, CIS $0 $0 - - 300,000 Community Stewardship Corps $0 $0 $388,274 $388,274 516 GED $0 $0 $350,000 $350,000 8,500 Summer Conservation Corps $0 $250,160 $637,576 $887,736 604 Workforce Investment Act - Title IB Youth $0 $9,901,654 $0 $9,901,654 2,962
DHS $0 $3,927,092 $911,885 $4,838,977 1,446 Pregnancy Prevention/Youth Services $0 $557,825 $0 $557,825 -Vocational Rehabilitation $0 $3,369,267 $911,885 $4,281,152 1,446
ODE $700,000 $22,846,499 $0 $23,546,499 59,967 21st Century Community Learning Center Programs $0 $10,800,000 $0 $10,800,000 21,000 Expanded Core Curriculum Summer Programs (BVIS) $100,000 $0 $0 $100,000 85 Goalball (BVIS) $10,000 $0 $0 $10,000 35 Homeless Education Program $0 $1,300,000 $0 $1,300,000 20,000 Local programs for At-Risk Youth (Part D, Subpart 2) $0 $675,000 $0 $675,000 2,000 Migrant Education $0 $10,071,499 $0 $10,071,499 16,838 Oregon FIRST Robotics grant program $500,000 $0 $0 $500,000 -Salem Summer Work Experience Program (SWEP) (BVIS) $90,000 $0 $0 $90,000 9
OR Military Dept $0 $4,185,000 $1,165,000 $5,350,000 2,312 Oregon National Guard Youth ChalleNGe $0 $3,495,000 $1,165,000 $4,660,000 312 STARBASE-Klamath Falls and Portland $0 $690,000 - $690,000 2,000
OSAC $164,000 $0 $561,000 $725,000 8,000 ASPIRE $164,000 $0 $561,000 $725,000 8,000
OUS $4,172,802 $4,394,240 $6,959,221 $15,526,263 158,198 Building a Bridge - - - - 2,500 Carpe Diem Education Abroad - - - - 98 Central High School and WOU Math Partnership - - - - -Cesar E. Chavez Leadership Conference $35,000 $0 $25,000 $60,000 1,500 Enactus (formerly know as SIFE) $0 $0 $10,000 $10,000 2,000 Financial Aid Nights $1,600 $0 $0 $1,600 -GEAR UP $5,000 $3,000,000 $800,000 $3,805,000 7,908 Healthy Youth Program $0 $0 $13,140 $13,140 687 High School Journalism Institute $0 $0 $14,000 $14,000 20 Making College Happen $5,500 $0 $0 $5,500 500 OIT Advance Credit Program $293,633 $0 $50,435 $344,068 -OIT High School Transition $91,926 $0 $18,725 $110,651 -OIT Rural Outreach for College Access $1,500 $0 $0 $1,500 200 OIT Youth Summer Camp Programs - - - - 97 Oregon Robotics Tournament & Outreach Program $0 $0 $269,431 $269,431 4,536 Oregon Wood Magic $3,000 $0 $6,000 $9,000 -OSU 4-H Youth Development $3,200,000 $800,000 $4,000,000 $8,000,000 116,878 OSU Campus Field Trip Program $0 $0 $10,000 $10,000 5,000 OSU Center for Outreach in Science and Engineering for Youth (COSEY) $0 $0 $25,000 $25,000 160 OSU Chemistry Outreach - - - - -OSU Hatfield Marine Science Center Education Programs $26,740 $26,740 $35,360 $88,840 8,000 OSU Inner-City Youth Initiative $0 $60,000 $0 $60,000 75 OSU Kaleidescope of Colleges and Cultures $4,828 $0 $0 $4,828 168 OSU Science and Math Investigative Learning Experiences Program $295,325 $170,000 $20,000 $485,325 647 OSU STEM Academy/Apprenticeships in Science and Engineering $0 $15,000 $63,000 $78,000 300 OSU Talented And Gifted Programs $0 $0 $185,000 $185,000 600 PSU Senior Inquiry High School Program (Jefferson, Liberty, Roosevelt and Westview) - - - - 350 PSU-College of Liberal Arts and Sciences: Challenge Program $0 $0 $474,828 $474,828 1,250 PSU-College of Liberal Arts and Sciences: Leap Into New Knowledge (LINK) Program/Young Scholars Program $0 $0 $30,000 $30,000 35 PSU-College of Liberal Arts and Sciences: Louis Stokes Alliance for Minority Participation (LSAMP) $37,500 $72,500 $0 $110,000 634 PSU-College of Liberal Arts and Sciences: MA Publishing- Roosevelt High School - - - - -PSU-College of Liberal Arts and Sciences: Summer Academy to Insprire Learning (SAIL) - - - - -SOU Early College Credit (Advanced Southern Credit and Early Entry) $0 $0 $510,254 $510,254 1,260 SOU Youth Programs Underserved Audiences (Konaway, Academia, Cesar Chavez Day) $45,000 $0 $40,000 $85,000 410 UO Youth Enrichment and Talented & Gifted Programs $0 $0 $349,868 $349,868 1,700 WOU Making College Happen $5,000 $0 $0 $5,000 500 WOU Preview 20XX $0 $0 $2,180 $2,180 55 WOU Project $100,000 $0 $0 $100,000 75 WOU Upward Bound Program (TRiO) $21,250 $250,000 $7,000 $278,250 55
FY 2013
2
ECON orthwest Appendix A: Tables A-2
Agency/ProgramTotal of State General Funds
Total of Federal Funds
Total of Other (local govt, tuition, grants, donations, etc.) Total of All Funds
Unduplicated Participants
FY 2013
OYA $47,382,927 $13,242,623 $287,696 $60,913,246 3,381 Cognitive Behavioral Treatment in OYA Close Custody Facilities $12,600,000 $0 $0 $12,600,000 1,137 Cognitive Behavioral Treatment in OYA Community Residential Programs $6,397,507 $10,524,143 $0 $16,921,650 1,113 JCP--Basic Services $3,943,734 $0 $0 $3,943,734 -JCP--Diversion $4,646,642 $0 $0 $4,646,642 -JCP--Gang Prevention in Mult. Co. $2,322,344 $0 $0 $2,322,344 -Minority Services $617,911 $45,752 - $663,663 -Outpatient Services $4,438,513 $198,490 $118,147 $4,755,150 -OYA Foster Care $751,292 $0 $132,149 $883,441 131 Parole/Probation $11,664,985 $2,474,238 $37,400 $14,176,623 1,000
State Library $609,801 $0 $874,415 $1,484,216 193,870 Ready to Read Grant $609,801 $0 $874,415 $1,484,216 193,870
YDC $2,299,734 $2,186,142 $0 $4,485,876 28,195 Behavioral Health Services $541,458 $101,761 $0 $643,219 1,591 Big Brothers Big Sisters $40,746 $112,758 $0 $153,504 136 Functional Family Probation Supervision $0 $110,818 - $110,818 68 Functional Family Therapy $175,351 $84,469 - $259,820 353 Girls Circle/ Boys Council (Motivational Interviewing and Strengths-Based Approach) $40,500 $0 $0 $40,500 135 Juvenile Accountability/Court/Probation Programming $0 $245,180 - $245,180 835 Juvenile Drug Courts $0 $34,780 - $34,780 301 Parent Training $428,280 $0 $0 $428,280 544 Restorative Justice: Diversion Panels and Sanction Courts $148,711 $0 $0 $148,711 399 Runaway and Homeless Youth Services $0 $72,705 $0 $72,705 232 School - Based Services $662,807 $722,690 $0 $1,385,497 9,988 Strengthening Families $0 $508,691 $0 $508,691 12,831 Teen Courts/Peer Courts $111,881 $53,074 - $164,955 360 Title II Formula Grant--Disproportionate Minority Contact $0 $139,216 - $139,216 324 Tribal Best Practices $150,000 $0 $0 $150,000 98
Grand Total $55,329,264 $60,933,410 $12,135,067 $128,397,741 767,951
3
ECON orthwest Appendix A: Tables A-3
Agency/Program Evidence Base Program CategoryCCWD
Career Information Services, CIS Promising EnrichmentCommunity Stewardship Corps Promising Dropout Prevention/RecoveryGED Promising Dropout Prevention/RecoverySummer Conservation Corps Promising Dropout Prevention/RecoveryWorkforce Investment Act - Title IB Youth Promising Dropout Prevention/Recovery
DHSPregnancy Prevention/Youth Services Promising Prevention & Intervention for At-Risk YouthVocational Rehabilitation Evidence based Specialized Educational Services
ODE21st Century Community Learning Center Programs Research based EnrichmentExpanded Core Curriculum Summer Programs (BVIS) Promising Specialized Educational ServicesGoalball (BVIS) Promising Specialized Educational ServicesHomeless Education Program Promising Specialized Educational ServicesLocal programs for At-Risk Youth (Part D, Subpart 2) Promising Dropout Prevention/RecoveryMigrant Education Promising Specialized Educational ServicesOregon FIRST Robotics grant program Promising EnrichmentSalem Summer Work Experience Program (SWEP) (BVIS) Promising Specialized Educational Services
OR Military DeptOregon National Guard Youth ChalleNGe Research based Dropout Prevention/RecoverySTARBASE-Klamath Falls and Portland Promising Enrichment
OSACASPIRE Research based College Access
OUSBuilding a Bridge Promising EnrichmentCarpe Diem Education Abroad Research based EnrichmentCentral High School and WOU Math Partnership Research based EnrichmentCesar E. Chavez Leadership Conference Promising EnrichmentEnactus (formerly know as SIFE) Research based EnrichmentFinancial Aid Nights Research based College AccessGEAR UP Promising College AccessHealthy Youth Program Promising EnrichmentHigh School Journalism Institute Promising EnrichmentMaking College Happen Research based College AccessOIT Advance Credit Program Research based Advance CreditOIT High School Transition Research based Advance CreditOIT Rural Outreach for College Access Research based College AccessOIT Youth Summer Camp Programs Promising EnrichmentOregon Robotics Tournament & Outreach Program Promising EnrichmentOregon Wood Magic Promising EnrichmentOSU 4-H Youth Development Research based EnrichmentOSU Campus Field Trip Program Research based EnrichmentOSU Center for Outreach in Science and Engineering for Youth (COSEY) Research based EnrichmentOSU Chemistry Outreach Promising EnrichmentOSU Hatfield Marine Science Center Education Programs Promising EnrichmentOSU Inner-City Youth Initiative Evidence based Prevention & Intervention for At-Risk YouthOSU Kaleidescope of Colleges and Cultures Research based College AccessOSU Science and Math Investigative Learning Experiences Program Research based EnrichmentOSU STEM Academy/Apprenticeships in Science and Engineering Research based EnrichmentOSU Talented And Gifted Programs Promising EnrichmentPSU Senior Inquiry High School Program (Jefferson, Liberty, Roosevelt and Westview) Research based Advance CreditPSU-College of Liberal Arts and Sciences: Challenge Program Research based Advance CreditPSU-College of Liberal Arts and Sciences: Leap Into New Knowledge (LINK) Program/Young Scholars ProgramResearch based Advance CreditPSU-College of Liberal Arts and Sciences: Louis Stokes Alliance for Minority Participation (LSAMP) Promising EnrichmentPSU-College of Liberal Arts and Sciences: MA Publishing- Roosevelt High School Research based EnrichmentPSU-College of Liberal Arts and Sciences: Summer Academy to Insprire Learning (SAIL) Research based EnrichmentSOU Early College Credit (Advanced Southern Credit and Early Entry) Research based Advance CreditSOU Youth Programs Underserved Audiences (Konaway, Academia, Cesar Chavez Day) Promising EnrichmentUO Youth Enrichment and Talented & Gifted Programs Promising EnrichmentWOU Making College Happen Research based College AccessWOU Preview 20XX Research based College AccessWOU Project Research based College AccessWOU Upward Bound Program (TRiO) Research based Dropout Prevention/Recovery
Evidence Base Assessment
Table 3. Evidence Base Assessment and Program Categorization
4
ECON orthwest Appendix A: Tables A-4
Agency/Program Evidence Base Program CategoryEvidence Base Assessment
OYACognitive Behavioral Treatment in OYA Close Custody Facilities Evidence based Juvenile JusticeCognitive Behavioral Treatment in OYA Community Residential Programs Evidence based Juvenile JusticeJCP--Basic Services Promising Prevention & Intervention for At-Risk YouthJCP--Diversion Promising Prevention & Intervention for At-Risk YouthJCP--Gang Prevention in Mult. Co. Promising Prevention & Intervention for At-Risk YouthMinority Services Promising Prevention & Intervention for At-Risk YouthOutpatient Services Promising Prevention & Intervention for At-Risk YouthOYA Foster Care Promising Juvenile JusticeParole/Probation Promising Juvenile Justice
State LibraryReady to Read Grant Promising Enrichment
YDCBehavioral Health Services Promising Prevention & Intervention for At-Risk YouthBig Brothers Big Sisters Evidence based Prevention & Intervention for At-Risk YouthFunctional Family Probation Supervision Evidence based Prevention & Intervention for At-Risk YouthFunctional Family Therapy Evidence based Prevention & Intervention for At-Risk YouthGirls Circle/ Boys Council (Motivational Interviewing and Strengths-Based Approach) Research Based Prevention & Intervention for At-Risk YouthJuvenile Accountability/Court/Probation Programming Promising Juvenile JusticeJuvenile Drug Courts Evidence based Juvenile JusticeParent Training Evidence based Prevention & Intervention for At-Risk YouthRestorative Justice: Diversion Panels and Sanction Courts Research based Juvenile JusticeRunaway and Homeless Youth Services Promising Prevention & Intervention for At-Risk YouthSchool - Based Services Promising Prevention & Intervention for At-Risk YouthStrengthening Families Research based Prevention & Intervention for At-Risk YouthTeen Courts/Peer Courts Research based Juvenile JusticeTitle II Formula Grant--Disproportionate Minority Contact Promising Prevention & Intervention for At-Risk YouthTribal Best Practices Promising Prevention & Intervention for At-Risk Youth