career and technical education (cte) - amazon...

STATE CAREER AND TECHNICAL EDUCATION (CTE) SELF-ASSESSMENT

Assessing The Progress And Future Planning Of CTE: A Self-Assessment Tool For State Agencies Background and Description The State Career and Technical Education (CTE) Self-Assessment is a comprehensive, voluntary instrument designed to help guide states’ program improvement efforts. Central to this self-assessment tool is the premise that all programs have strengths upon which to build. The instrument identifies many activities, tasks, processes, and collaborations that, if they occur consistently, ensure that CTE programs are being implemented with a high degree of quality. By using this tool in a dynamic, ongoing way, states can identify many existing CTE practices and policies that comprise quality and use them as building blocks for system-wide continuous improvement—from properly administering their basic grants and tech-prep programs to using their accountability data to fund local programs. Moreover, the instrument is a mechanism to share information with all state agencies about what comprises “quality” in the delivery of CTE programs—both from the vantage point of what is currently working in states and where the Administration would like to see CTE programs move in the future. The U.S. Department of Education’s Office of Vocational and Adult Education (OVAE) charged with monitoring state’s compliance with the authorizing legislation for CTE (i.e., the Carl D. Perkins Vocational and Technical Education Act), now seeks to facilitate states’ efforts to move beyond the “letter of the law” toward quality performance. To this end, the CTE Self-Assessment is structured around the Career and Technical Education Quality Indicators which were developed through a comprehensive, collaborative process involving an extensive review of the literature on CTE and related fields, feedback and contributions from 10 state directors of CTE, and discussions with staff from OVAE. Although, the indicators encompass some of the key requirements contained in the Perkins legislation, they go beyond that capturing what might be considered an ideal or model system of career and technical education.

Altogether, 30 quality indicators are included in the instrument. They address six areas of CTE functioning: Administration, Local Application, Tech Prep, Special Populations, Fiscal Responsibility, and Accountability. Indicators are further defined by evidence or specific criteria for what constitutes quality: the more evidence in place, the more confidence a state can have that each indicator has been well-implemented. As with any framework of this type, it is important to remember that the quality indicators are not carved in stone. At the time of development, they were based on the best thinking of CTE State Directors and OVAE staff team. Nevertheless, high quality is a moving target, and continuous improvement can only be maintained if state stakeholders continue to examine what they are doing, explore creative strategies, and share their knowledge and experience. Combined with OVAE monitoring visits, this self-assessment provides a structure for doing that.

1

Purpose The future look of CTE will depend on how well state agencies focus their efforts toward continuous improvement. This requires self-assessment, strategic planning, and commitment. When the key participants in a state use this tool in a dynamic, ongoing way, in combination with the OVAE monitoring visit, available data, reports, and feedback/input from key stakeholders, the results should be a shared understanding of the current degree and quality of statewide CTE program implementation and impact. It is important to note that the use of the State Career and Technical Education (CTE) Self-Assessment is voluntary. It is intended to be used by states as a tool for leveraging coordination and collaboration for program improvement. In addition, it can help to promote conversation/ internal dialogue about

y where states are and what degree of program improvement they perceive they have made;

y what areas states feel they would like to focus on for future improvements;

y what areas states feel would represent models of effective practice; and

y what areas states feel that may need further assistance to improve their practice.

The process of the self-assessment also provides intangible value beyond any written reports or assessments because it

y builds commitment and ownership on the part of the state-level staff who participate in the process;

y promotes team building and consensus among state CTE leaders

y increases the capacity for strategic thinking in the field of CTE; and

y builds an understanding of what the federal government requires of states with respect to quality performance.

Planning teams who engage in CTE self-assessment prior to the OVAE monitoring visit will be better prepared to take full advantage of the exchange of ideas and technical assistance provided. They will have considered the views of key stakeholders, assembled and digested information on the different components of CTE, and come to a consensus on the current status of their statewide efforts.

2

Directions for Use The CTE Self-Assessment asks state agencies to rate their CTE programs according to 30 quality indicators. The ratings should take into account the various pieces of evidence that define each indicator. A five-point rating scale—representing a continuum of implementation progress—has been developed and is described below.

Rating Rubric

1 No Implementation

Our state is not implementing any (or hardly any) of the evidence for this indicator.

2 Minimal

Implementation

Our state is implementing some of the evidence for this indicator, but most of our efforts are in the planning stage. Substantial work is needed to improve our approach.

3 Moderate

Implementation

Our state is implementing most of the evidence for this indicator, but some gaps in implementation exist and improvements could be made.

4 Complete

Implementation

Our state is implementing most of the evidence for this indicator. Our approach is systematic and organized with no major gaps.

5 Exemplary

Implementation

Our state is implementing all of the evidence for this indicator. We have a sound, systematic approach that could serve as a model for other states.

Directions for Use of the Notes Section Each indictor and related evidence are accompanied by a section for Notes. This section may be used to record other information that can expand upon the state’s performance in a given area. For example, a state may use the Notes section to

y provide statements about progress, e.g., “our state is in the early planning phase, but we have a commitment to move forward”; or “we have a planning document and are in the process of developing strategies to implement this indicator”; or “this indicator is well-established”, etc.;

y provide more information and greater detail on accomplishments regarding a given indicator—going beyond the listed evidence;

y describe strengths, weaknesses, and plans for improvement;

3

y elaborate on the numerical ratings to facilitate discussion during OVAE monitoring visits;

y identify information which should be included in its CAR; and/or

y provide an explanation as to why the state is not implementing a specific indicator, e.g., “we have an alternative method for achieving the intent of this indicator that works well in our state”; or why a specific piece of evidence is not applicable for your state, e.g., we have consistently surpassed our enrollment targets for special populations of students and, therefore, have not implemented an outreach/recruitment campaign”.

Recommended Steps The following steps are recommended to conduct the state CTE self-assessment.

Identify and recruit the key CTE stakeholders to complete the self-assessment. A variety of approaches to conducting this step can prove effective. One possibility is to have a representative team of state-level decision-makers complete the instrument as a group. Another approach is to have individual stakeholders fill out the tool separately, and then have the individual results compiled for group discussion and instrument completion. Yet a third way is to have the instrument completed by one or two people who are most knowledgeable of the state’s CTE program. Regardless of the approach used, however, it is important to enlist input from key stakeholder groups.

1.

2.

3.

Gather supporting evidence and data. The instrument should be completed by knowledgeable stakeholders who use as much supporting evidence as possible. Sources of information can include the state plan, reports, minutes of meetings, mission/vision statements, policies, written documentation and data gathered through interviews with stakeholders, student records, program site visits, third-party evaluation evidence, financial records, proposals, local applications, monitoring tools, the state’s professional development plan, progress reports, and so forth. A “point person” should be identified to ensure that supporting materials are collected, organized and distributed; necessary stakeholder meetings are held; and a timeline established to review all information and complete the self-assessment. Examples of information sources for each indicator can be found in the document, State CTE Self-Assessment Sample Data Sources, beginning on page 40.

Complete the self-assessment. Carefully read the evidence for each indicator. If the evidence is in place (i.e., implemented), place a checkmark (√) in the box provided. If you feel that your level of implementation is systematic, without significant weaknesses or gaps, place an asterisk (*) next to the checkbox. Then review these individual assessments and decide on a final rating for the indicator; fill in the appropriate circle: . Use the Notes section to record any explanatory or expanded information about the state’s performance for that indicator. Once you have rated all of the indicators in each major CTE area, transfer your ratings to the Summary Form, beginning on page 36.

4

Provide feedback to CTE stakeholders involved in Step 1. Throughout the process of completing the CTE self-assessment, information should be fed back to the key CTE stakeholders as part of this dynamic process of inquiry and reflection.

4.

Optional Uses The primary use of the CTE self-assessment is to help guide states’ program improvement efforts through careful study of statewide policies, procedures, and activities. However, a number of states have found it helpful to use the tool in other, creative ways:

y as a monitoring tool for reviewing local grantee programs, functions, as well as expenditures and

y as an instructional device for orienting new staff (or re-acquainting veterans) about what comprises “quality” in the delivery of CTE

programs.

Still others have used it

y as a way of communicating the importance of Perkins/CTE to other non-CTE state-level stakeholders.

In Conclusion . . . It is important to remember that high quality is a moving target. States should consider completing the CTE Self-Assessment, along with the summary form, on a regular basis to see the impact of their program improvement efforts.

5

The state has a clearly articulated mission for CTE that is consistent with the State Plan; a consolidated set of policies and procedures exists for translating the mission/vision into action.

Evidence Check ( ) if Implemented

1. The mission statement accurately reflects the purpose of the CTE initiative, who is served, the services offered, and the outcomes expected.

2. The mission communicates the belief that all students—including special populations—can meet high standards of

academic and technical excellence as well as engage in active, productive learning.

3. The state provides leadership for achieving the mission through a coherent set of policies and procedures that govern all areas of program administration, planning, development, and implementation in accordance with the State Plan and Perkins legislation.

4. The state has oriented relevant local- and state-level stakeholders to the CTE mission and policies/procedures (e.g., through training workshops, dissemination of print resources, electronic resources, personal contact, etc.).

5. The state has a process—involving local input—for periodically revising the mission and policies/procedures to ensure their continued relevance; modifications are made to reflect the evolving knowledge base in CTE.

Final Rating: No Minimal Moderate Complete Exemplary

Implementation Implementation Implementation Implementation Implementation

NOTES (evidence of accomplishments, related data/criteria, key stakeholders involved, critical issues, website, etc.):

Quality Indicator 1.1: Mission

1. State Administration

6


The state has established effective working relationships between and among secondary and postsecondary institutions.


1. Secondary and postsecondary state-level offices coordinate activities to administer and provide statewide leadership for the CTE initiative.

2. Interagency agreements/Memorandum of Understanding exist between secondary and postsecondary state-level offices that specify the exact nature of the collaborative relationship for administering/leading the CTE initiative.

3. Areas of state-level collaboration may include…

a) program planning and development,

b) program monitoring and evaluation (including data sharing systems),

c) CTE-related professional development,

d) Tech prep activities including the design of articulation agreements, approval of course sequences, etc.,

e) the alignment of secondary/postsecondary system requirements (e.g., college enrollment, credit transfer, etc.),

f) other __________________________________________________________________________________.

4. The state requires local grantees to set aside funds for the purpose of effectively linking secondary and postsecondary institutions; grantees must include specific goals and targets for this purpose in their local plans.

5. The state provides supportive resources (e.g., technical assistance), events (e.g., meetings, forums, conferences), and incentives to assist local grantees in effectively linking secondary and postsecondary institutions.

6. The state has created a resource (e.g., database, compendium, electronic material) of successful secondary-postsecondary partnerships to share with local grantees and assist them in establishing effective institutional linkages.


Implementation Implementation Implementation Implementation Implementation NOTES (evidence of accomplishments, related data/criteria, key stakeholders involved, critical issues, website, etc.):

Quality Indicator 1.2: Secondary-Postsecondary Collaboration

7


The state has established collaborative linkages with other state-level agencies and programs involved in workforce preparation.


1. Interagency agreements exist among state-level agencies involved in workforce preparation that specify the exact nature of the collaborative relationship including, for example, roles and responsibilities, communication/feedback mechanisms (frequency, methods), and other procedures to meet the needs of youth.

2. The state leverages resources through this collaboration to address the need of students and accomplish its overall mission for CTE.

3. Areas of state-level collaboration may include…

a) cross-agency participation on statewide leadership teams for workforce preparation,

b) cross-agency participation at Perkins III hearings,

c) cross-agency participation in the development of State Plans (e.g., for WIA and Perkins III),

d) identification of activities/efforts that promote coordination at the local level (e.g., Perkins activities offered through one-stop centers, co-location of one-stops at Perkins recipients’ sites, Perkins recipients serving as approved training providers under WIA, coordinated services for youth under WIA and Perkins III, shared activities among agency providers for career exploration, job seeking skills, lifework planning, etc.), and

e) other___________________________________________________________________________________.


Implementation Implementation Implementation Implementation Implementation NOTES (evidence of accomplishments, related data/criteria, key stakeholders involved, critical issues, website, etc.):

Quality Indicator 1.3: Collaboration-Other State Agencies

8


The state uses its reserve funds to foster program improvement.


1. The state uses its reserve funds to reward the performance of high-achieving local grantees (e.g., b owing flexibility in the use of reserve funds).

2. The state uses its reserve funds to improve the performance of low-achieving grantees (e.g., by lim flexibility in the use of reserve funds and targeting funds for activities that directly support the achievement of rmance goals).

3. The state establishes “centers of excellence” by funding groups of high-achieving local grantees to rate innovative programs and activities.

Final Rating: No Minimal Moderate Comp

Implementation Implementation Implementation Implemen

Quality Indicator 1.4: Use of Reserve

NOTES (evidence of accomplishments, related data/criteria, key stakeholders involved, critical issu

y all

itingperfo

ope
lete Exemplary tation Implementation
es, website, etc.):

9


The state monitors local grantees for compliance with Perkins requirements and performance goals.


1. The state has developed—and disseminated to local grantees—monitoring policies and procedures that specify State Educational Agency and local grantee responsibilities for assuring compliance with Perkins requirements and performance goals.

2. The state has developed instruments for the review of local grantee programs, functions, activities, and expenditures.

3. The state uses the local application as a tool for monitoring proposed program activities. 4. The state provides local grantees with examples of acceptable documentation for the monitoring review(s). 5. The state has a process for identifying areas where technical assistance intervention may be necessary to improve

grantee performance.

6. The state monitors each grantee throughout the year rather than relying on a once-a-year audit.

Final Rating: No Minimal Moderate Complete Exemplary Implementation Implementation Implementation Implementation Implementation


Quality Indicator 1.5: Local Monitoring

10


The state plays a significant role in building the capacity of local grantees through professional development and technical assistance activities.


1. The state conducts statewide needs assessment/needs sensing activities to identify issues and needs relevant to local grantees (e.g., through formal/informal surveys; regional meetings; local meetings with administrators, teachers, students, and parents, etc.).

2. Based on current data (including needs assessment), the state organizes professional development and technical assistance (TA) activities to ensure that local grantees and end-users—vocational and technical, academic, guidance, and administrative personnel—can effectively fulfill their roles and responsibilities for CTE program design and implementation.

3. The state uses a variety of strategies to provide professional development/TA to local grantees and end-users including institutes, workshops, mentoring, procedural guidelines, videoconferencing, and web-based conferencing.

4. The state identifies/assigns available technical assistance agents who can provide professional development/TA in general CTE areas as well as within specific program fields (e.g., agribusiness, technical education, consumer science, etc.).

5. The state has a well-publicized mechanism for local grantees and end-users to request and receive information and resources in a timely manner (e.g., via websites, listservs, toll free numbers, and e-mail).

6. The state works with appropriate individuals or entities to ensure that participation in professional development/TA (including preservice) experiences is recognized in state and local teacher licensure and credentialing procedures.

7. The state evaluates its professional development/TA activities periodically using indicators such as satisfaction, acquisition of knowledge and skills, and changes in practices.




Quality Indicator 1.6: Capacity Building

11

The state has effective communication strategies for keeping local grantees and other partners informed of CTE activities and outcomes.


1. The state has a written communication plan that identifies a) relevant stakeholders who need information about the CTE initiative; b) the type of information each stakeholder needs to know and the time frame when it is needed; and c) the type of communication strategies for sharing information with stakeholders.

2. The state uses multiple and ongoing strategies to provide current information about the CTE initiative to stakeholders including newsletters, brochures, fact sheets, websites, listservs, public forums, etc.

3. The state uses multiple and ongoing strategies to increase public awareness of CTE programs and opportunities, including marketing strategies for reaching special and non-traditional populations.

4. The state provides opportunities for two-way exchange of communication so that local grantees, end-users, and other constituents (e.g., parents, local businesses, students, agency stakeholders) can share/provide as well as receive information about the CTE initiative.





Quality Indicator 1.7: Communication

12

The state clearly defines size, scope, and quality of program services/activities in its local application process to ensure a common understanding at state and local levels.


1. The state has developed program criteria to assure that local grantee services and activities are of sufficient size, scope, and quality to be effective.

2. The state criteria for program “quality” include performance indicators that are part of the state’s accountability

system.

3. The local application process includes suggested evidence/documentation for how eligible grantees can meet the established criteria.

4. Local program approval and renewal is based on the program’s capacity to meet the established criteria as determined by previous program performance.

5. The state provides technical assistance to eligible grantees to help them design and implement services and activities that are of sufficient size, scope, and quality to be effective.

6. The state disseminates information on research-based best practices (e.g., High Schools That Work, Project Lead-the-Way) to facilitate local development of services/activities that are of sufficient size, scope, and quality to be effective.




Quality Indicator 2.1: Size, Scope, and Quality

2. Local Application

13


The state integrates accountability results into the local application process, requiring local grantee se performance data to drive goals, strategies, resources, and measures for program improvement.


1. The local application process requires eligible grantees to describe how their planning process f TE design and implementation takes into account student performance data.

2. The state provides technical assistance to eligible grantees to help them use local data to design deliver quality program services that will lead to higher performance results.

3. Local application approval and renewal is based on the program’s capacity to implement strateg hat will lead to higher performance results.

4. The state uses accountability results to direct federal funding (e.g., by requiring low-achieving ees to target projected expenditures to known program shortfalls, and affording high-achieving local grantee ater flexibility in the use of Perkins funds).

Final Rating: No Minimal Moderate Com te Exemplary

Implementation Implementation Implementation Implem

Quality Indicator 2.2: Integration of Accountability

NOTES (evidence of accomplishments, related data/criteria, key stakeholders involved, critical is

s to u

or C

and

ies t

grants gre

ple

entation Implementation

sues, website, etc.):

14


The state’s local application process gathers budget data from an accounting as well as a programmatic perspective.


1. The state has designed streamlined budget tables that require eligible grantees to report local expenditures by general accounting categories (e.g., salaries, travel, equipment) and by program/intended use categories (e.g., professional development, technology applications, career guidance, evaluation, work-related experiences, integration, etc.).

2. The state uses the local budget to determine how identified needs—particularly student performance needs—will be addressed financially.




Quality Indicator 2.3: Budget Data

15


The state has established an automated and/or web-based process for the review and approval of local applications.


1. The automated/online local application contains all elements required in the Perkins law for local usage of funds.

2. The automated/online application…

a) allows for importation of data from existing databases (e.g., student data, financial data, etc.),

b) contains extensive use of help buttons or hot links to clarify terminology, explain procedures, and invite feedback/questions from eligible grantees,

c) allows eligible grantees to reuse application content, and

d) other ___________________________________________________________________________________.

3. The state has procedures for allowing eligible grantees to easily track an application’s process.

4. The state has procedures to encourage open communication/feedback on the application process via e-mail, voice mail, the state website, and other communication methods.

5. The automated/online local application (as well as the standard application) is updated to accommodate changing needs.




Quality Indicator 2.4: Automated Application

16

The state’s Tech Prep program functions as an initiative for advancing systemic educational collaboration.


1. Tech Prep activities are aligned with—and build upon—related reform initiatives to provide youth with comprehensive education, workforce development, and economic development services.

2. Responsibilities for Tech Prep planning and implementation are shared among key groups—secondary and postsecondary educators, students, parents, employers, community agencies, and citizens—to provide a foundation and network for collaboration.

3. The state has created policies and procedures for local Tech Prep consortia that ensure collaboration and systematic articulation among high school programs and diverse postsecondary partners including community and technical colleges, 4-year colleges and universities, apprenticeships, and employer-provided training.

4. The state has created policies and procedures for local Tech Prep consortia that ensure the tangible, meaningful involvement of business, industry, and labor in program development, curriculum, work-based learning experiences, and program improvement.

5. The state has created policies and procedures for local Tech Prep consortia that ensure linkages with local, state, and regional economies.

6. The state disseminates information on successful secondary, postsecondary, and business partnerships; it maintains an electronic database of successful secondary-postsecondary-business partnerships to share with local grantees and assist them in establishing effective institutional linkages.




Quality Indicator 3.1: Systemic Collaboration

3. Tech Prep

17

3. Tech Prep

The state has developed a model statewide articulation agreement for each program area to promote and ensure consistency among Tech Prep consortia and consortium members.


The statewide articulation agreement identifies a common core of sequentially more challenging courses/programs of study eligible for statewide articulation.

The articulation agreement sets common statewide standards for the award of postsecondary credit for content-enhanced high school courses.

The articulation agreement allows credits to be awarded across consortia boundaries.

The articulation agreement may encompass…

a) 2+2, 4+2, and other Tech Prep arrangements (e.g., 2+2+2),

b) academic and technical courses, competencies, and skills,

c) 2-year and 4-year postsecondary institutions, and

d) other __________________________________________________________________________________.

The state articulation agreement has provisions for traditionally underrepresented populations to ensure equity in enrollment and transition to postsecondary programs.

Information on the statewide articulation agreement is disseminated through a variety of publications and activities to reach secondary and postsecondary teachers, counselors, administrators, and, ultimately, parents and students.

The articulation agreement is updated periodically to accommodate emerging technical occupational areas/priorities.



Quality Indicator 3.2: Articulation Agreement

1.

2.

3.

4.

5.

6.

7.


18

3. Tech Prep

The state uses its accountability data to fund and improve local Tech Prep consortia.


The state uses a “pay for performance” funding formula for Tech Prep by targeting funds based on valid and reliable data on student participation and performance indicators including…

a) student progress toward meeting academic standards, technical skills, and career-related learning standards,

b) student retention in and completion of postsecondary programs,

c) student retention in and completion of secondary programs,

d) student participation in career-related learning experiences,

e) student placement in employment, and/or

f) other ___________________________________________________________________________________.

The state provides technical assistance to low-performing consortia in program areas linked to poor performance (i.e., in identified need areas).

The state identifies and disseminates information on “best solutions” to performance problems, including improvement strategies and successful Tech Prep models linked to academic and industry standards.


Quality Indicator 3.3: Accountability Data

1.

2.

3.


19

4. Special Populations

The state uses multiple strategies, including its accountability data and local planning process, to assure equal access and full participation of special populations in CTE programs and services.


The state has developed systematic outreach procedures for increasing the enrollment of special populations in basic CTE and Tech Prep programs.

The state has developed and disseminated information regarding the Perkins requirements for equal access, full participation, and nondiscrimination of special populations in CTE programs.

The state has developed and disseminated guidelines/recommendations to local grantees for promoting equal access, full participation, and nondiscrimination of special populations in CTE programs; guidelines may address…

a) methods of affirmative outreach and recruitment,

b) strategies for identifying and eliminating barriers to program admission,

c) professional development activities for supporting special populations,

d) information for local employers to ensure access and full participation in high skill, high wage careers, and

e) other ___________________________________________________________________________________.

The state requires local grantees to submit goals, strategies, measures, and outcome targets in their local application that assure special populations the same opportunities for program access and full participation afforded to the general student population.

State monitoring instruments (and reports) contain specific criteria/indicators that require local grantees to address access, full-participation, and nondiscrimination for all special populations.

The state’s accountability system is able to track special population categories by enrollment and placement and “flag” discrepancies in access and participation between special and non-special population learners.



1.

2.

3.

4.

5.

6.


Quality Indicator 4.1: Equal Access

20


The state uses multiple strategies, including its accountability data and local planning process, to assess and address the needs of special populations.


The state conducts statewide needs assessment/needs sensing activities to identify issues and needs relevant to special populations (e.g., through formal/informal surveys; regional meetings; local meetings with administrators, teachers, students, parents, employers, etc.).

The state disaggregates accountability data by special population categories to determine the degree to which local grantees are meeting special population learner needs in an integrated manner.

The state requires local grantees to submit goals, strategies, measures, and outcome targets in their local application—as well as the necessary funds—to ensure that the needs of special populations are assessed and addressed in an integrated manner and in accordance with Perkins requirements.

State monitoring instruments (and reports) contain specific criteria/indicators that require local grantees to demonstrate how the needs of special populations have been determined; how needs are being addressed; and what programs, services, and activities have been effective (or ineffective) in meeting needs and raising overall performance levels.

The state provides technical assistance to local grantees to help them use data—and sound assessment tools—in making decisions about the needs and appropriate placement of special populations, as well as the effectiveness of programs, services, and activities for meeting needs and raising overall performance levels.



Quality Indicator 4.2: Assessing and Addressing Needs

1.

2.

3.

4.

5.


21


The state uses its leadership funds for programs/activities and support services aimed at raising the performance of special populations and ensuring that they are prepared for further learning and high skill, high wage careers.


1. The state organizes professional development activities to expand the knowledge and skills of professionals working with special populations (e.g., through workshops on issues of civil rights; recruitment and retention; assessment; the design/implementation of inclusive programs; the identification/elimination of barriers, etc.).

2. The state creates and disseminates resources (e.g., speakers directory, technical assistance providers, job sites, organizations, employers/unions, curricula, assessment strategies, mentoring strategies, effective practices) to promote the development/implementation of programs and support services for special populations.

3. The state develops and/or adopts model comprehensive programs to better serve the future learning/career needs of special populations (e.g., career magnet schools, career academies, model programs, etc.).

4. The state develops transition strategies/programs to ensure the smooth, effective transition of special populations from secondary programs to postsecondary education and the workplace (e.g., transition planning guide, transition policies/procedures, sample enrichment activities, model programs, suggested strategies for placement staff, etc.)

5. The state employs a full- or part-time special populations coordinator to provide leadership, oversight, and direction in addressing the needs of special populations throughout the CTE delivery system (e.g., by providing technical assistance to local grantees, reviewing student participation rates and performance data, publicizing programs, etc.).

6. The state requires local grantees to provide special populations with the necessary support services to enable them to fully participate in and benefit from rigorous CTE programs (e.g., specialized equipment, career counseling, accommodations to learning environments, tutoring or mentoring, linkages with social services and CBO, etc.).

7. The state has developed a comprehensive high school planning guide (e.g., Career Plan, Lifework Plan) to assist special populations in designing an individualized course of study to help them reach their post-high school goals.

8. The state has developed web-based resources (e.g., for accessing data from O*Net) to assist special populations in making informed decisions about careers, employment, and further education.




Quality Indicator 4.3: Preparing for Further Learning High Skill, High Wage Careers

22


The state promotes collaboration within CTE program components and across state-level offices serving special populations to better meet students’ needs.


State personnel responsible for Perkins priority components collaborate to address the needs of special populations (e.g., tech prep and equity personnel work together to create articulation agreements; accountability and equity personnel review performance data and make joint recommendations for improvement strategies, etc.).

The state collaborates with other state-level agencies that work with special populations to leverage funds and avoid duplication of effort in designing and implementing programs, activities, and services for special populations.

The state forms advisory boards—of secondary and postsecondary educators, students, parents, employers, community agencies, citizens, and other state-level agencies—to enhance opportunities for special populations (e.g., in areas of curriculum/program development, work-based learning, transitions, etc.).



Quality Indicator 4.4: Horizontal/Vertical Collaboration

1.

2.

3.


23

5. Fiscal Responsibility

The state has a fiscal management system in place for ensuring the proper expenditure of Perkins funds.


1. The state maintains detailed records of all Perkins expenditures at the state level and has access to local records. The records are able to document all transactions including a) the amount, date, purpose, and category (e.g., formula, leadership, reserve) of each expenditure; and b) reimbursements requested and received.

2. The state is able to demonstrate that expenditures comply with required Perkins funding formulas including…

a) within state allocation formulas,

b) state allocation formulas for distribution of funds to secondary grantees, and

c) state allocation formulas for distribution of funds to postsecondary grantees.

3. The state is able to demonstrate that Perkins funds…

a) are expended according to the financial plan in the approved application,

b) are expended only for allowable activities and services, and

c) are used to supplement and not supplant federal or state funds.

4. The state has detailed procedures for and documentation of alternative funding formulas for secondary and/or postsecondary grantees.

5. The state has the capacity to prepare and submit current and accurate financial reports that provide complete disclosure on the source and application of Perkins funds.

6. The state has effective internal controls to monitor and ensure local grantee award compliance and reporting.



Quality Indicator 5.1: Fiscal Management System

24


The state has a valid and reliable method for calculating maintenance of effort and assuring that this level is met each year.


1. The state has policies and procedures to ensure that fiscal effort is calculated consistently from year to year.

2. The state includes all costs that meet the definition of vocational and technical education in the effort calculation, but excludes tuition revenue, locally generated funds, capital expenditures, special one-time project costs, and the cost of pilot programs.

3. The state has the capacity to disaggregate funding sources and ensure that only state-appropriated dollars are included in the effort calculation.

4. The state has detailed documentation of funding sources for auditing purposes.

5. The state has controls in place to ensure the accuracy of student enrollment data as well as non-duplicated student counts in per-student effort calculations.



Quality Indicator 5.2: Maintenance of Effort


25


The state has a mechanism for ensuring that federal funds for administration are matched with non-federal funds, and that the hold-harmless level for the state administration match is properly maintained.


1. The state maintains detailed records of federal and non-federal expenditures for state administration activities and can document matching allocations from year-to-year.

2. The state carefully monitors administration costs and adjusts activities accordingly to keep within the hold-harmless level.

3. The state can readily and effectively forecast factors that might jeopardize the hold-harmless level in future years, and plans appropriately to keep within the level.



Quality Indicator 5.3: State Administration Matching and Hold Harmless


26

y5. Fiscal Responsibilit

The state has policies and procedures for ensuring that a high percentage of allocated funds are obligated and liquidated each year, and that the level of carryover is limited.


1. The state sets an estimated target level for unobligated funds at the 12-month and 24-month time intervals of the award period.

2. The state carefully monitors the expenditure of funds by category (e.g., formula, leadership, reserve) to identify areas that fall below the unobligated funds target.

3. The state develops an action plan to ensure that a higher percentage of funds in problem areas are obligated and liquidated in subsequent years.

4. The state has a “first-in/first-out” payment system to ensure that older Perkins grant funds are obligated and liquidated prior to the use of more recent Perkins funds.



Quality Indicator 5.4: Funds Obligated and Liquidated


27

6. Accountability

The state has aligned its Perkins and No Child Left Behind (NCLB) accountability systems.


The state has the same method of calculating standards for academic achievement under Perkins and NCLB (i.e., CTE students are held to the same standards as all students in the state).

The state has the same method of calculating standards for graduation under Perkins and NCLB (i.e., CTE students are held to the same standards as all students in the state).

The state is able to disaggregate the performance of CTE students on NCLB assessments in basic skill areas, e.g.,

a) through a K-12 student information system that is capable of providing data elements required for Perkins,

b) by creating data fields in a CTE information system that allows cross-matching with demographic and performance data maintained in the general K-12 information system,

c) through a data transfer utility between CTE information system and general K-12 information system, and

d) other ___________________________________________________________________________________.

The state identifies local grantees (districts and/or schools) with CTE programs that did not make adequate yearly progress under NCLB.

The state has procedures and a system in place to train and update local grantees on the accountability system and its link to NCLB.



1.

2.

3.

4.

5.


Quality Indicator 6.1: Perkins and NCLB Alignment

28

6. Accountability

The state’s accountability system uses a valid and reliable assessment approach to measure academic attainment of secondary and postsecondary CTE students.


The academic attainment assessments include state assessments and/or standardized national assessments aligned with state academic content and performance standards.

The academic attainment assessments…

a) provide a representative coverage of academic content areas, and

b) are administered consistently, according to carefully developed guidelines.

The state uses other data sources to make informed decisions about CTE secondary and postsecondary student performance, e.g.,

a) end-of-course examinations,

b) rigorous coursetaking patterns (e.g., 4 years of math; 4 years of science),

c) SAT/ACT scores,

d) postsecondary entrance examinations,

e) dual placement or advanced placement data, and

f) other _____________________________________________________________________________.

Sec.

P/Sec.



1.

2.

3.


Quality Indicator 6.2: Valid, Reliable Assessment of Academic Attainment

29

6. Accountability

The state’s accountability system uses a valid and reliable assessment approach to measure technical skill attainment of secondary and postsecondary CTE students.


1. The state has a set of state-approved, industry-identified skill standards for all CTE courses.

Sec.

P/Sec.

The state aligns its technical skill standards across major workforce development and training programs.

The technical skill attainment assessments…

a) are aligned to state-approved, industry-identified skill standards—both content and performance standards,

b) provide a representative coverage of state-approved, industry-identified skills standards, and

c) are administered consistently, according to carefully developed guidelines.

Attainment assessments include national/state standards and assessment systems (e.g., tests developed by the National Occupational Competency Testing Institute, industry exams, etc.).

The state uses other data sources to make informed decisions about CTE secondary and postsecondary student performance, e.g.,

a) end-of-course examinations,

b) licensing/certification examinations,

c) assessment tools for probing performance of students in non-traditional career and technical programs, and

d) other ______________________________________________________________________________.



2.

3.

4.

5.


Quality Indicator 6.3: Valid, Reliable Assessment of Technical Skill Attainment

30

6. Accountability

The state has the capacity to aggregate and disaggregate data collected from local grantees.


The state has developed and supports a comprehensive K-12 student information management s that has the ability to aggregate and disaggregate Perkins performance data by various factors including…

a) CTE program area,

b) student subpopulations (including race/ethnicity, gender, and special population status),

c) geographic regions,

d) school/district need clusters, and

e) other __________________________________________________________________ _____________.

The state is able to disaggregate and track K-12 CTE data from year-to-year (trend results) with cus on illuminating achievement gaps (e.g., gaps among student subpopulations).

The state has procedures in place to report aggregated and disaggregated performance results to ublic.

The state works with postsecondary institutions to incorporate Perkins indicator data into existin formation management systems.

The state assists postsecondary institutions to gather and report disaggregated data on special po tions that may not be part of existing reporting systems but are required in Perkins (e.g., data on single parents le pregnant women).

Final Rating: No Minimal Moderate Com

Implementation Implementation Implementation Implem

Quality Indicator 6.4: Capacity to Aggregate and Disaggregate

1.

2.

3.

4.

5.

NOTES (evidence of accomplishments, related data/criteria, key stakeholders involved, critical is

ystem

____

a fo

the p

g in

pula, sing

plete Exemplary

entation Implementation

sues, website, etc.):

31

The state has implemented quality control procedures for ascertaining that data from grantees are valid, reliable, alignedcollected at the same time, measured by the same instrument, and reported for all students.

Evidence

The state has clear definitions of its accountability system including performance criteria; eligible students; vocational participants, concentrators, and completers; eligible courses and course sequences, etc.

The state has developed—and disseminated to local grantees—a data collection manual with relevant definitionsinstructions for data submission.

The state has an automated data collection system with built-in quality controls, edit features, and accuracy and verification checks (e.g., pop-up menus that remind users of critical items to review; data entry features that bar from entering out-of-range values; reminders to complete all data fields, etc).

The automated data collection system…

a) is web-based and/or electronic,

b) allows cross-matching/checking of student records with other administrative databases, and

c) is able to generate error reports that alert users to potential problems in their data (e.g., omissions, inconsistencies, etc.).

The state provides supportive services to assist local grantees in improving data quality and accurate reporting, e

a) training workshops and technical assistance sessions on data collection/reporting topics,

b) a data collection specialist to assist designated local personnel responsible for completing required repor

c) a well-publicized mechanism for local grantees to request and receive technical assistance on data collection/reporting in a timely manner (e.g., via websites, listservs, toll free numbers, and e-mail), and

d) other _______________________________________________________________________________

Data is collected for the current year and reported in that year.

Final Rating: No Minimal Moderate Complete

Implementation Implementation Implementation Implementation

Quality Indicator 6.5: Quality Control

1.

2.

3.

4.

5.

6.

6. Accountability

with state standards,

Check ( ) if Implemented

and

users

.g.,

ts,

____.

Exemplary Implementation

32

NOTES (evidence of accomplishments, related data/criteria, key stakeholders involved, critical issues, website, et

The state has reliable follow-up procedures in place to track students as they enroll, participate in, and then completeducation, and as they enter and persist in employment.

Evidence

The state has an electronic student record data system that allows the tracking of individual students to postsecoeducation, employment, and the military.

The state is able to track students through administrative record exchange (e.g., UI wage records, U.S. DepartmeDefense records, etc.).

The state has a process for linking community, technical, and state colleges with one-stop career centers to facilitracking of students from postsecondary education to employment.

The state has a prescribed methodology (e.g., mail, telephone, online surveys using minimum number or percentrate of return stratified random sampling procedures) and written guidelines for conducting follow-up studies ofsecondary and postsecondary graduates.

The state’s secondary and postsecondary offices collaborate to follow-up on the placement of secondary complein higher education.

The state’s secondary, postsecondary, and labor organizations collaborate to follow-up on the placement and retention of students in employment.

Final Rating: No Minimal Moderate Complete

Implementation Implementation Implementation Implementation

1.

2.

3.

4.

5.

6.

NOTES (evidence of accomplishments, related data/criteria, key stakeholders involved, critical issues, website, et

Quality Indicator 6.6: Student Follow-up

6. Accountability

c.):

e postsecondary

Check ( ) if Implemented

ndary

nt of

tate

age

ters

Exemplary Implementation

c.):

33

6. Accountability

The state uses its accountability data to develop and implement continuous improvement activities for local grantees.


The state compares performance levels on core indicators among schools/districts, colleges, student populations, and CTE programs over time to identify gaps and improvement priorities.

Where feasible, the state employs multiple scientifically based research methods to determine causes for gaps (review of research, surveys, evaluations).

The state has the capacity to transmit detailed performance reports/data sets to local grantees that identify performance gaps and program areas in need of improvement.

The state requires grantees to develop a local improvement plan to address performance gaps.

The state identifies and disseminates information on “best practices” and strategies to address performance problems including improvement strategies with demonstrated effectiveness.

The state rewards high performing grantees (e.g., through waivers, flexibility in use of funds, recognition, etc.).

The state provides assistance and/or sanctions low-performing grantees, (e.g., through targeted professional development, support teams, improvement plans, corrective action, student transfer, etc.).

The state regularly reviews its performance targets and measures and makes adjustments as necessary.



1.

2.

3.

4.

5.

6.

7.

8.


Quality Indicator 6.7: Continuous Improvement

34

6. Accountability

The state provides for the ongoing, systematic, objective, and independent evaluation of Perkins programs.


The evaluation reflects the needs and expectations of key stakeholders: local grantees, end-users, and other constituents (e.g., students, parents, employers, community agencies, etc.).

The evaluation examines multiple program areas including…

a) state and local organizational indicators (e.g., leadership; policies/procedures; frequency/intensity of professional development; size, scope, quality of programs; work-based learning activities; employer involvement; satisfaction of end-users, including educators, employers, labor, parents, and students), and

b) student indicators (e.g., academic achievement; technical skill attainment; course-taking patterns; attendance; graduation; entry and retention in postsecondary education, employment or the military; achievement gaps among sub-population, etc.).

The evaluation uses multiple quantitative (e.g., survey, record review) and qualitative (e.g., interviews, site visits) data collection procedures, including follow-up studies of students.

The evaluation involves multiple stakeholders—local grantees, end-users, students, parents, employers, and community agencies—in the evaluation process.

The state disseminates the evaluation results to stakeholders on a regular basis through multiple dissemination vehicles (e.g., formal reports, newsletters, non-technical research briefs, web bulletins, and oral presentations).

The state uses the evaluation results and its accountability data for continuous system improvement (i.e., as a basis for expanding successes, correcting shortcomings, identifying needed resources, and developing short- and long-term strategies to better meet the needs of students).



1.

2.

3.

4.

5.

6.


Quality Indicator 6.8: External Evaluation

35

1. Program Area/Quality Indicator Final Rating: Level of Implementation


Quality Indicator 1.1: Mission The state has a clearly articulated mission for CTE that is consistent with the State Plan; a consolidated set of policies and procedures exists for translating the mission/vision into action.

None Minimal Moderate Complete Exemplary

Quality Indicator 1.2: Secondary-Postsecondary Collaboration The state has established effective working relationships between and among secondary and postsecondary institutions.


Quality Indicator 1.3: Collaboration-Other State Agencies The state has established collaborative linkages with other state-level agencies and programs involved in workforce preparation.


Quality Indicator 1.4: Use of Reserve The state uses its reserve funds to foster program improvement.


Quality Indicator 1.5: Local Monitoring The state monitors local grantees for compliance with Perkins requirements and performance goals.


Quality Indicator 1.6: Capacity Building The state plays a significant role in building the capacity of local grantees through professional development and technical assistance activities.


Quality Indicator 1.7: Communication The state has effective communication strategies for keeping local grantees and other partners informed of CTE activities and outcomes.



Quality Indicator 2.1: Size, Scope, and Quality The state clearly defines size, scope, and quality of program services/activities in its local application process to ensure a common understanding at state and local levels.


Quality Indicator 2.2: Integration of Accountability The state integrates accountability results into the local application process, requiring local grantees to use performance data to drive goals, strategies, resources, and measures for program improvement.


Final Rating Summary Form

36

Program Area/Quality Indicator Final Rating: Level of Implementation

Quality Indicator 2.3: Budget Data The state’s local application process gathers budget data from an accounting as well as a programmatic perspective.


Quality Indicator 2.4: Automated Application The state has established an automated and/or web-based process for the review and approval of local applications.


3. Tech Prep Quality Indicator 3.1: Systemic Collaboration The state’s Tech Prep program functions as an initiative for advancing systemic educational collaboration.


Quality Indicator 3.2: Articulation Agreement The state has developed a model statewide articulation agreement for each program area to promote and ensure consistency among Tech Prep consortia and consortium members.


Quality Indicator 3.3: Accountability Data The state uses its accountability data to fund and improve local Tech Prep consortia.



Quality Indicator 4.1: Equal Access The state uses multiple strategies, including its accountability data and local planning process, to assure equal access and full participation of special populations in CTE programs and services.


Quality Indicator 4.2: Assessing and Addressing Needs The state uses multiple strategies, including its accountability data and local planning process, to assess and address the needs of special populations.


Quality Indicator 4.3: Preparing for Further Learning High Skill, High Wage Careers The state uses its leadership funds for programs/activities and support services aimed at raising the performance of special populations and ensuring that they are prepared for further learning and high skill, high wage careers.


Quality Indicator 4.4: Horizontal/Vertical Collaboration The state promotes collaboration within CTE program components and across state-level offices serving special populations to better meet students’ needs.


37

Program Area/Quality Indicator Final Rating: Level of Implementation

5. Fiscal Responsibility Quality Indicator 5.1 Fiscal Management System The state has a fiscal management system in place for ensuring the proper expenditure of Perkins funds.


Quality Indicator 5.2: Maintenance of Effort The state has a valid and reliable method for calculating maintenance of effort and assuring that this level is met each year.


Quality Indicator 5.3: State Administration Matching and Hold Harmless The state has a mechanism for ensuring that federal funds for administration are matched with non-federal funds, and that the hold-harmless level for the state administration match is properly maintained.


Quality Indicator 5.4: Funds Obligated and Liquidated The state has policies and procedures for ensuring that a high percentage of allocated funds are obligated and liquidated each year, and that the level of carryover is limited.


6. Accountability

Quality Indicator 6.1: Perkins and NCLB Alignment The state has aligned its Perkins and No Child Left Behind (NCLB) accountability systems.


Quality Indicator 6.2: Valid, Reliable Assessment of Academic Attainment The state’s accountability system uses a valid and reliable assessment approach to measure academic attainment of secondary and postsecondary CTE students.


Quality Indicator 6.3: Valid, Reliable Assessment of Technical Skill Attainment The state’s accountability system uses a valid and reliable assessment approach to measure technical skill attainment of secondary and postsecondary CTE students.


Quality Indicator 6.4: Capacity to Aggregate and Disaggregate The state has the capacity to aggregate and disaggregate data collected from local grantees.


Quality Indicator 6.5: Quality Control The state has implemented quality control procedures for ascertaining that data from grantees are valid, reliable, aligned with state standards, collected at the same time, measured by the same instrument, and reported for all students.


38

Quality Indicator 6.6: Student Follow-up The state has reliable follow-up procedures in place to track students as they enroll, participate in, and then complete postsecondary education, and as they enter and persist in employment.


Quality Indicator 6.7: Continuous Improvement The state uses its accountability data to develop and implement continuous improvement activities for local grantees.


Quality Indicator 6.8: External Evaluation The state provides for the ongoing, systematic, objective, and independent evaluation of Perkins programs.


39

STATE CTE SELF-ASSESSMENT SAMPLE DATA SOURCES

1. State Administration 1.1 Mission: The state has a clearly articulated mission for CTE that is consistent with the State Plan; a consolidated set of policies and

procedures exists for translating the mission/vision into action. Sources

a. State plan for CTE (Note: this data source is relevant for all indicators)

b. Written mission statement and/or set of guiding principles

c. Policies/procedures manual or guidelines (Note: this data source is relevant for all components, e.g., Tech Prep, Accountability, Special Populations)

d. Event calendar/contact log/website identifying workshops, resources, and strategies to inform stakeholders about the CTE mission and policies/procedures

e. Sample of orientation products developed and disseminated

1.2 Secondary-Postsecondary Collaboration: The state has established effective working relationships between and among secondary and postsecondary institutions.

Sources

a. Strategic plan/action plan for joint work

b. Signed interagency agreements or Memorandum of Understanding (MoU)

c. Event calendar/contact log documenting planning sessions, meetings, jointly sponsored activities; minutes of meetings

d. Inventory of products/processes produced collaboratively; sample of products jointly developed

e. Copy of local application re: secondary-postsecondary linkage requirements

f. Event calendar/contact log/website identifying workshops, resources, and strategies to help grantees create effective linkages

g. Sample of products developed and/or disseminated to assist grantees

h. Sample of completed local application review instruments and/or approved applications re: grantee capacity for establishing effective secondary-postsecondary linkages

40

1.3 Collaboration-Other State Agencies: The state has established collaborative linkages with other state-level agencies and programs

involved in workforce preparation.

Sources

a. Unified collaboration plan/action plan

b. Signed interagency agreements or MoU

c. Event calendar/contact log documenting planning sessions, meetings, jointly sponsored activities; minutes of meetings

d. Documentation of resource leveraging (e.g., unified plan for funding allocation)

1.4 Use of Reserve: The state uses its reserve funds to foster program improvement.

Sources

a. Criteria for identifying high-achieving and low-achieving grantees

b. List of rewards/waivers for high-achieving grantees; list of grantees rewarded

c. List of sanctions for low-achieving grantees; list of grantees sanctioned

1.5 Local Monitoring: The state monitors local grantees for compliance with Perkins requirements and performance goals.

Sources

a. Monitoring policies and procedures

b. Criteria for identifying grantees that require follow-up assistance or intervention

c. Monitoring instruments/procedures

d. Schedule of monitoring visits

e. Sample of completed monitoring reports

f. Event calendar/contact log/website identifying assistance/interventions provided to low-achieving grantees; description of sample interventions

41

1.6 Capacity Building: The state plays a significant role in building the capacity of local grantees through professional development and technical assistance activities.

Sources

a. Needs assessment instrument(s); description of needs assessment procedures; needs assessment results

b. Written professional development plan (e.g., activities, resources, time lines); professional development marketing strategy

c. Event calendar/contact log/website identifying workshops, resources, and strategies to build local capacity

d. Inventory of professional development resources (e.g., personnel/designated staff, print, electronic resources, listserv of training opportunities); sample of professional development products and programs developed and/or distributed to assist grantees

e. Sub-state structure for delivering professional development (e.g., regional centers, statewide institutes)

f. Professional development evaluation procedures; evidence of use

g. Credit/credentialing options for professional development

1.7 Communication: The state has effective communication strategies for keeping local grantees and other partners informed of CTE activities and outcomes.

Sources

a. Written communication plan

b. Communication log (e.g., memos, newsletters, e-mails, faxes)

c. Sample of communication/marketing tools (e.g., CTE brochures, fact sheets, websites, listservs, forums) developed and/or disseminated to keep stakeholders informed

42


2.1 Size, Scope, and Quality: The state clearly defines size, scope, and quality of program services/activities in its local application process to ensure a common understanding at state and local levels.

Sources

a. Copy of local application re: definition of size, scope, and quality

b. State standards/criteria for size, scope, and quality

c. Event calendar/contact log/website identifying workshops, resources, and strategies to help grantees develop quality programs of sufficient size and scope

d. Sample of “size-scope-quality” products/programs developed and/or distributed to assist grantees

e. Sample of completed local application review instruments and/or approved local applications re: grantee capacity for meeting size, scope and quality criteria

2.2 Integration of Accountability: The state integrates accountability results into the local application process, requiring local grantees to use performance data to drive goals, strategies, resources, and measures for program improvement. Sources

a. Copy of local application re: accountability requirements for grantees

b. Sample of completed local application review instruments and/or approved local applications re: grantee capacity for reaching higher performance targets

c. Event calendar/contact log/website identifying workshops, resources, and strategies to help grantees use data to design and deliver quality programs

d. Sample of program planning products/programs developed and/or distributed to assist grantees

e. Records/documentation of differentiated funding decisions based on grantee performance (e.g., high-achieving grantees are given greater flexibility in use of funds)

2.3 Budget Data: The state’s local application process gathers budget data from an accounting as well as a programmatic perspective.

Sources

a. Copy of local application re: budget requirements; budget tables

b. Sample of completed local application review instruments and/or approved local applications re: grantee allocation of funds to meet students’ needs

43

2.4 Automated Application: The state has established an automated and/or web-based process for the review and approval of local applications.

Sources

a. Web address for online application

b. Sample of completed web-based applications/automated applications

c. Inventory of resources and strategies to help grantees with the application process; sample of products developed and/or distributed to assist grantees

3. Tech Prep 3.1 Systemic Collaboration: The state’s Tech Prep program functions as an initiative for advancing systemic educational collaboration.

Sources

a. Tech prep strategic plan

b. Tech Prep advisory council; list of advisors (relevant for all Tech Prep indicators)

c. Tech Prep policies and procedures covering articulation, involvement of key groups, important linkages

d. Event calendar/contact log/website identifying workshops, resources, and strategies to help grantees create effective Tech Prep partnerships

e. Sample of Tech Prep partnership products/programs developed and/or distributed to assist grantees

3.2 Articulation Agreement: The state has developed a model statewide articulation agreement for each program area to promote and ensure consistency among Tech Prep consortia and consortium members.

Sources

a. Statewide articulation agreement

b. Event calendar/contact log/website identifying workshops, resources, and strategies to help grantees create effective articulation agreements

c. Sample of articulation products/programs developed and/or distributed to assist grantees

d. Copy of local application re: Tech Prep articulation requirements

e. Sample of completed local application review instruments and/or approved local applications re: adequacy of Tech Prep articulation agreements

44

3.3 Accountability Data: The state uses its accountability data to fund and improve local Tech Prep consortia.

Sources

a. Records/documentation of differentiated funding decisions based on grantee performance relative to student participation and performance indicators

b. Event calendar/contact log/website identifying workshops, resources, and strategies to help grantees improve student participation and performance indicators

c. Sample of Tech Prep accountability products/programs developed and/or distributed to assist grantees


4.1 Equal Access: The state uses multiple strategies, including its accountability data and local planning process, to assure equal access and full participation of special populations in CTE programs and services.

Sources

a. Special populations advisory board; list of advisors (relevant for all special population indicators)

b. Sample outreach/marketing/recruitment tools and activities for increasing enrollment of special populations in CTE programs and services

c. Inventory of resources/guidelines re: equal access, full participation, and nondiscrimination; sample of equal access resources developed and/or disseminated

d. Event calendar/contact log documenting meetings with representatives from business, industry, and labor re: broadening access for special populations; minutes of meetings

e. Copy of local application re: equal access and full participation requirements

f. Sample of completed local application review instruments and/or approved local applications re: grantee capacity for ensuring equal access and full participation

g. Monitoring instruments; sample of completed monitoring reports re: grantee capacity for ensuring equal access and full participation

h. List of data elements in state accountability system; enrollment, retention, and completion figures

45

4.2 Assessing and Addressing Needs: The state uses multiple strategies, including its accountability data and local planning process, to assess and address the needs of special populations. Sources

a. Needs assessment instrument(s); description of needs assessment procedures; needs assessment results

b. List of data elements/fields in state accountability system or student information management system; results of disaggregated analyses for special populations (including achievement data)

c. Copy of local application re: requirements for assessing and meeting the needs of special populations

d. Sample of completed local application review instruments and/or approved local applications re: grantee capacity for assessing and addressing the needs of special populations

e. Monitoring instruments; sample of completed monitoring reports re: grantee capacity for assessing and addressing the needs of special populations

f. Event calendar/contact log/website identifying workshops, resources, and strategies to help grantees assess and address the needs of special populations

g. Sample of products/programs developed and/or distributed to assist grantees in assessing and addressing the needs of special populations

46

4.3 Preparing for Further Learning High Skill, High Wage Careers: The state uses its leadership funds for programs/activities and support services aimed at raising the performance of special populations and ensuring that they are prepared for further learning and high skill, high wage careers. Sources

a. Event calendar/contact log/website identifying workshops, resources, and strategies to assist professionals who work with special populations

b. Inventory of resources including model programs to help grantees develop and implement programs and support services for special populations; sample of products/programs developed and/or distributed

c. Sample of funded projects for special populations that lead to high skill, high wage careers

d. Inventory of transition resources to help grantees address the needs of special populations; sample of resources developed and/or distributed

e. State-level job descriptions; staffing chart (re: special populations coordinator)

f. Copy of local application re: requirements that enable special populations to participate in rigorous CTE programs

g. Sample of completed local application review instruments and/or approved local applications re: grantee capacity for enabling special populations to participate in rigorous CTE programs

h. Monitoring instruments; sample of completed monitoring reports re: grantee capacity for enabling special populations to participate in rigorous CTE programs

i. Inventory of resources (including web-based resources) to help special populations attain their post-high school goals; sample of products developed and/or distributed

4.4 Horizontal/Vertical Collaboration: The state promotes collaboration within CTE program components and across state-level offices serving special populations to better meet students’ needs. Sources

a. Event calendar/contact log documenting planning sessions, meetings, jointly sponsored activities; minutes of meetings

b. Inventory of products/processes produced collaboratively; sample of products/programs jointly developed

47

5. Fiscal Responsibility 5.1 Fiscal Management System: The state has a fiscal management system in place for ensuring the proper expenditure of Perkins funds.

Sources

a. Accounting ledgers/records of all Perkins expenditures/allocations

b. Allocation formulas, including alternative formulas

c. Documentation (e.g., spreadsheets) showing census and/or enrollment data for each grantee/consortia

d. Copy of most recent Interim and Final Financial Status Reports (FSR)

e. Fiscal monitoring/quality control procedures to monitor grantee compliance and reporting

f. Budget for Perkins III

5.2 Maintenance of Effort: The state has a valid and reliable method for calculating maintenance of effort and assuring that this level is met each year.

Sources

a. Accounting ledgers/records of expenditures, disaggregated by funding source, for monitored year and year previous to monitored year

b. Fiscal monitoring/quality control procedures, including procedures for calculating MOE

c. Annual state enrollment reports for monitored year and two years previous to monitored year

d. State appropriations language for monitored year and year previous to monitored year

5.3 State Administration Matching and Hold Harmless: The state has a mechanism for ensuring that federal funds for administration are matched with non-federal funds, and that the hold-harmless level for the state administration match is properly maintained.

Sources

a. Accounting ledgers/records of expenditures for state administration activities including federal and non-federal expenditures for monitored year and year previous to monitored year

b. Fiscal monitoring/quality control procedures

c. State appropriations language for monitored year and year previous to monitored year

48

5.4 Funds Obligated and Liquidated: The state has policies and procedures for ensuring that a high percentage of allocated funds are obligated and liquidated each year, and that the level of carryover is limited.

Sources

a. Copy of most recent Interim and Final FSRs

b. Copy of GAPS (Grant Administration and Payment System) report 6. Accountability

6.1 Perkins and NCLB Alignment: The state has aligned its Perkins and No Child Left Behind (NCLB) accountability systems.

Sources

a. Advisory board; list of advisors re: making recommendations about the link between vocational education and educational reform (relevant to all accountability indicators)

b. Event calendar/contact log documenting planning sessions, meetings, activities to gather input from eligible recipients re: CTE performance measures

c. State standards for academic achievement and high school graduation (CTE students/all students in the state)

d. Curricular frameworks and other resources to assist grantees in aligning vocational and academic coursework

e. List of data elements/fields in state accountability system or student information management system; results of disaggregated analyses on key NCLB indicators for CTE students

f. List of grantees that did not make adequate yearly progress (AYP)

g. Event calendar/contact log/website identifying workshops, resources, and strategies to train grantees on the accountability system

6.2 Valid, Reliable Assessment of Academic Attainment: The state’s accountability system uses a valid and reliable assessment approach to

measure academic attainment of secondary and postsecondary CTE students. Sources

a. Copies of state standards and academic assessments

b. Test item-content grids showing the link between assessments and state standards

c. Inventory of other data sources for measuring the academic performance of CTE students

d. Test accommodations for special populations

49

6.3 Valid, Reliable Assessment of Technical Skill Attainment: The state’s accountability system uses a valid and reliable assessment approach to measure technical skill attainment of secondary and postsecondary CTE students.

Sources

a. Copies of technical skill standards for CTE and/or procedures currently underway to identify and validate technical skills

b. Copies of technical skill assessments

c. Test item-content grids showing the link between assessments and technical skill standards

d. Inventory of other data sources for measuring the technical skill performance of CTE students

e. Test accommodations for special populations

6.4 Capacity to Aggregate and Disaggregate: The state has the capacity to aggregate and disaggregate data collected from local grantees.

Sources

a. List of data elements/fields in state accountability system or student information management system

b. Results of disaggregated analyses by demographic, geographic, and need variables; trends over time

c. Results of disaggregated analyses by student subpopulations, including special populations; trends over time

d. Copy of state report cards; dissemination log

e. Event calendar/contact log documenting planning sessions, meetings with postsecondary institutions re: Perkins indicator data

f. Inventory of incentives to encourage postsecondary institutions to provide data (sanctions for not providing data, e.g., no data-no funding)

g. Sample data from postsecondary institutions

6.5 Quality Control: The state has implemented quality control procedures for ascertaining that data from grantees are valid, reliable, aligned with state standards, collected at the same time, measured by the same instrument, and reported for all students. Sources

a. Copy of local application re: definition of accountability system

b. Data collection manual for grantees with policies and procedures; dissemination log

c. Web address for automated data system

d. Event calendar/contact log/website identifying workshops, resources, and strategies to help grantees improve the quality of data and reporting

e. Sample of “data quality” products and programs developed and/or distributed to assist grantees

f. Sample of completed local application review instruments and/or approved local applications re: core indicator data

50

6.6 Student Follow-up: The state has reliable follow-up procedures in place to track students as they enroll, participate in, and then complete postsecondary education, and as they enter and persist in employment.

Sources

a. List of data elements/fields in state accountability system or student information management system

b. Signed agreements/MoU enabling the tracking of students through administrative record exchange

c. Event calendar/contact log documenting collaborative activities re: tracking students post high school

d. Methodology for conducting follow-up studies

e. Follow-up data for most recent cohort of CTE graduates

6.7 Continuous Improvement: The state uses its accountability data to develop and implement continuous improvement activities for local grantees.

Sources

a. Results of analyses to identify gaps

b. Copies of sample performance reports

c. Copies of local improvement plans

d. Event calendar/contact log/website identifying workshops, resources, and strategies to help grantees address performance gaps

e. Sample of programs/strategies developed and/or distributed to assist grantees

f. Sample of tools to help grantees self-analyze and set program improvement goals

g. Event calendar/contact log documenting state review of performance targets

6.8 External Evaluation: The state provides for the ongoing, systematic, objective, and independent evaluation of Perkins programs. Sources

a. Event calendar/contact log documenting planning sessions, meetings, activities to inform stakeholders about the CTE evaluation process and obtain their input

b. Copy of evaluation RFP; name of external evaluator

c. CTE goals, objectives, expected outcomes

d. Copy of the external evaluation design and instruments

e. Copy of recent evaluation reports and/or summary of results

f. Open-ended comments from students, parents, staff, employers re: program satisfaction, suggestions for improvement, areas of strength, etc.

51

career and technical education (cte) - amazon...

Documents