description: tags: hbcu-eval
Post on 31-May-2018
219 Views
Preview:
TRANSCRIPT
-
8/14/2019 description: tags: hbcu-eval
1/35
7/10/2006 1
Federal Program Evaluation:
Strengthening Historically BlackColleges and Universities
-
8/14/2019 description: tags: hbcu-eval
2/35
7/10/2006 2
Federal Performance
Measurement
How do we know if a federal program is:
Making progress and
Achieving results
?
-
8/14/2019 description: tags: hbcu-eval
3/35
7/10/2006 3
GPRA Drives the Federal
Process
Government Performance and Results Actof 1993
Holds federal agencies accountable forachieving program results
Requires setting goals and public reporting
Seeks to reduce waste and inefficiency
-
8/14/2019 description: tags: hbcu-eval
4/35
7/10/2006 4
Why Was GPRA Enacted?
Waste and inefficiency had underminedtaxpayer confidence and reduced the
government's ability to meet public needs
Inadequate goals and performance data
impeded managers efforts to make
programs work
Congress needed performance information
-
8/14/2019 description: tags: hbcu-eval
5/35
7/10/2006 5
GPRA Requirements
All federal agencies must Prepare:
Strategic Plans: Major long-term goals and objectives
Performance Plans: Annual program strategies andtargets
Performance Reports: Annual performance and
accountability reports
Make program information available to the public
-
8/14/2019 description: tags: hbcu-eval
6/35
7/10/2006 6
The Department of Educations
Mission
To ensure equal access to education and
to promote educational excellence
throughout the nation
-
8/14/2019 description: tags: hbcu-eval
7/35
7/10/2006 7
The Department of EducationsStrategic Plan
Has six major goals
Postsecondary Education Goal:
Enhance the quality of, and access to,
postsecondary and adult education
-
8/14/2019 description: tags: hbcu-eval
8/35
7/10/2006 8
Postsecondary Education
Objectives Reduce gaps in college access and completion among
different student populations while increasing educationalattainment
Strengthen accountability of postsecondary education
institutions Strengthen minority-serving institutions
Establish effective funding mechanisms
Enhance literacy and employment skills of U.S. adults
Increase the capacity of institutions to teach world
languages, area studies, and international issues
-
8/14/2019 description: tags: hbcu-eval
9/35
7/10/2006 9
GPRAs Impact Over Time Over the past 10 years, GPRA results show
steady, but uneven, progress 4 years ago, OMB developed the Program
Assessment Rating Tool (PART) tosystematically assess programs across
federal agencies
-
8/14/2019 description: tags: hbcu-eval
10/35
7/10/2006 10
What Is the PART?
The Program Assessment Rating Tool evaluates
program performance in four main areas:
1. Program Purpose and Design
2. Strategic Planning3. Program Management
4. Program Results and Accountability
-
8/14/2019 description: tags: hbcu-eval
11/35
7/10/2006 11
PART Goals
Increase focus on effectiveness (outcomes)and efficiency
Make program assessment evidence-based
Provide consistent measurement and
evaluation across programs
Rate program performance
Improve public information
-
8/14/2019 description: tags: hbcu-eval
12/35
7/10/2006 12
How Does the PART Evaluate
Performance?
The PART is heavily weighted to focus onquantitative results (50% of score)
Outcome measures
Efficiency measures
Evidence-based program evaluation
-
8/14/2019 description: tags: hbcu-eval
13/35
7/10/2006 13
In Other Words
How well is the program doing what it isintended to do?
What is it costing to achieve the programs
purpose?
Do independent evaluations show that the
program is effective and efficient?
-
8/14/2019 description: tags: hbcu-eval
14/35
7/10/2006 14
Why Does PART Include a
Cost Measure?
Assess a programs ability to achieve
results relative to resources
Demonstrate the ability of a program to use
resources (e.g., time, effort, money)
effectively
-
8/14/2019 description: tags: hbcu-eval
15/35
7/10/2006 15
How Are PART Scores Used?
Identify program strengths and weaknesses
Promote program improvement and
administrative changes
Inform Congress, the public, and other
stakeholders
Make performance-based budgeting decisions
-
8/14/2019 description: tags: hbcu-eval
16/35
7/10/2006 16
What is Performance-Based
Budgeting? Performance results inform resource allocation
The integration of a performanceplan with a
proposed budget
Basically, how dollars are expected to turn into
results--For example, program dollars that fund
day-to-day activities (counseling and tutoring)produce results (student graduation)
-
8/14/2019 description: tags: hbcu-eval
17/35
7/10/2006 17
Overview of PART ScoresFederal Education Postsecondary
Education
Programs assessed 793 64 23
Effective 15% 3% 0%Moderately effective 29% 8% 9%
Adequate 28% 31% 30%
Ineffective 4% 9% 9%
RND 24% 64% 52%
-
8/14/2019 description: tags: hbcu-eval
18/35
7/10/2006 18
Why aResults Not Demonstrated
Rating?
New program measures were set last year Targets were set for the new measures
Waiting for additional data to be collected,analyzed, and compared to targets
Waiting for results of independentevaluation
-
8/14/2019 description: tags: hbcu-eval
19/35
7/10/2006 19
What are the Measures?
Performance outcome measures:
Undergraduate and graduate school enrollment
Postsecondary persistence Completion rate
Cost per (undergraduate and graduate) degree
-
8/14/2019 description: tags: hbcu-eval
20/35
7/10/2006 20
What Will Improve the
PART Score? Performance data must be
made publicly available
used to assess institutions effectiveness and settargets
used to make program improvements
Independent evaluation results
-
8/14/2019 description: tags: hbcu-eval
21/35
7/10/2006 21
Study of the Strengthening
Institutions Programs
Systematic study: Financial Health of TitleIII and Title V Postsecondary Institutions:
Is financial health improving?
Do enrollment, persistence, graduation rates
affect financial health?
Is there a discernible impact of OPE programson financial health?
Findings due in late 2007
-
8/14/2019 description: tags: hbcu-eval
22/35
7/10/2006 22
How Can HBCUs Help?
Provide accurate, reliable, and on-time datareporting toIPEDS
Submit complete, accurate, reliable, and on-time annual and final performance reports
Assess the results of your use of program
funds
Provide feedback to federal managers
-
8/14/2019 description: tags: hbcu-eval
23/35
7/10/2006 23
EDs Next Steps
Use program-wide assessments ofindividual grantees to help identify:
exemplary practices
cost efficient projects
opportunities for training and technical
assistance Improve the quality and completeness of
data reported by grantees
-
8/14/2019 description: tags: hbcu-eval
24/35
7/10/2006 24
Expanding HBCU Performance
Analyses to Improve Feedback
Analyze data from multiple sources
Analyze multiple measures of grantee
performance together
Provide grantees with results
Analyzing A Single Measure
-
8/14/2019 description: tags: hbcu-eval
25/35
7/10/2006 25
Analyzing A Single Measure
at the Institutional LevelCollege Success Rates (n=835)
1023 43
92 92
161
205 209
0
50
100
150
200
250
Percent successful outcomes
Num
berofgra
ntees*
45-64%
65-69%
70-74%
75-79%
80-84%
85-89%
90-94%
95-100%
-
8/14/2019 description: tags: hbcu-eval
26/35
7/10/2006 26
Analyzing Data from Multiple Sources
Public 4-Year Institutions
$0
$500
$1,000
$1,500
$2,000
$2,500
$3,000
$3,500
$4,000
40% 50% 60% 70% 80% 90% 100% 110%
Successful outcome rate
Co
st/SuccessfulOutcom
Analyzing Data from Multiple Sources by
-
8/14/2019 description: tags: hbcu-eval
27/35
7/10/2006 27
Analyzing Data from Multiple Sources by
Multiple Categories
Comparison by Number of Students Served
$0
$500
$1,000
$1,500
$2,000
$2,500
$3,000
$3,500
$4,000
40% 50% 60% 70% 80% 90% 100% 110%
Successful Outcome Rate
Cost/Succ
.Outcome*
No. students served: small No. students served: medium No. of students served: large
-
8/14/2019 description: tags: hbcu-eval
28/35
7/10/2006 28
Providing Grantees with
Performance Analyses
Individual performance and demographic datacan be grouped with similar institutions for
comparisons
For example, success rates can be grouped by: Institutional type and control
Geographic location Numbers of students served
Numbers of students receiving Pell Grants
-
8/14/2019 description: tags: hbcu-eval
29/35
7/10/2006 29
How Can Institutions Use These
Data?
Evaluate your performance relative to
others based on multiple characteristics
Compare your performance to the nationalprogram averages for each category
Identify areas of strengths and weakness Make data-driven decisions
-
8/14/2019 description: tags: hbcu-eval
30/35
7/10/2006 30
Program Data ReviewsCommunity
College A
All
CommunityColleges
Difference
Success Rate 97% 84% +13
Cost Per Success $789 $1,519 -730
Other Factors ? ? ?
-
8/14/2019 description: tags: hbcu-eval
31/35
-
8/14/2019 description: tags: hbcu-eval
32/35
7/10/2006 32
Performance Measurement is
Essential to Make Improvements
At national program level
At project/institutional level
Through systematic studies
-
8/14/2019 description: tags: hbcu-eval
33/35
7/10/2006 33
HBCUs Are Performing in
Measurable Ways
19 percent increase in enrollment from2002-03 to 2005-06
One-year persistence rate (2005-06) is 65
percent
Percentage of entering students in four-year
schools graduating within six years (2004-05): 39 percent
Source: IPEDS, NCES
-
8/14/2019 description: tags: hbcu-eval
34/35
7/10/2006 34
Achieving ResultsPerformance data show HBCUs are a
significant force in minority education... Although HBCUs represent only 3 percent of
the nations postsecondary institutions...
...they graduate nearly one-quarter of Blacks
who earn undergraduate degrees
12% of Black students that attend accredited 2-and 4-year postsecondary institutions attend
HBCUs
-
8/14/2019 description: tags: hbcu-eval
35/35
7/10/2006 35
Resources PART Web site:
http://www.expectmore.gov
IPEDS Web site:
http://nces.ed.gov/ipeds
ED Performance Web site:http://www.ed.gov/about/reports/annual/index.html?src=pn
ED HBCU Program Web site:http://www.ed.gov/programs/iduestitle3b/index.html
http://www.ed.gov/programs/iduestitle3b/index.htmlhttp://www.ed.gov/programs/iduestitle3b/index.html
top related