chair academy perkins iv sandy
DESCRIPTION
This is the powerpoint from the presentation I gave at the Chair Academy in Nashville on Technical Skill Attainment in WisconsinTRANSCRIPT
Perkins IV Perkins IV Assessment of Assessment of Technical Skill Technical Skill
AttainmentAttainment
Wisconsin Technical College System
Sandy SchmitWTCS Foundation – WIDS
Judy Neill
Overview of the Wisconsin Technical College System
What does Carl Perkins 2006 (Perkins IV) require?
How does curriculum fit in? How is WTCS using performance
assessment? What will WTCS require for 3rd Party
Assessments? How can WIDS help?
Overview
The Wisconsin Technical The Wisconsin Technical College System (WTCS)College System (WTCS)
16 Technical Colleges
349 unduplicated Career and Technical Education Programs
1179 Duplicated Career and Technical Education Programs
390,272 enrolled students in 2008, 69,632 full time equivalent
23,617 Degrees granted in 2008
WTCS
How do you know when How do you know when your students “get it”?your students “get it”?
Assessment?Assessment?
Summative Summative Assessment?Assessment?
Carl Perkins IV?Carl Perkins IV?
What does Carl Perkins What does Carl Perkins 2006 (Perkins IV) 2006 (Perkins IV)
require?require?
“Student attainment of challenging career and technical skill proficiencies, including student achievement on technical assessments that are aligned with industry-recognized standards, if available and appropriate.” (U.S. Department of Education, 2006).
The Law
Focuses on industry aligned skill sets.
Students know they are acquiring skills that have value & portability.
Increases odds that students will be successful in their employment.
Ensures that WTCS is educating students to meet the needs of employers.
Big Picture Perspective
Increased accountability to improve teaching and learning
Accreditation (NCA, AQIP, industry)
Carl D. Perkins IV legislation and funding
Drivers to Process
StudentsFeedback about skills needed on the jobAdded value for education
CollegesImprovement of teaching and learningSupports overall mission
EmployersAssurance that WTCS graduates have attained technical skills needed on the job
Benefits
Carl D. Perkins 2006
Newly revised Career and Technical Education (CTE) Law
Key Concept - Demonstrate that program graduates have attained the knowledge and skills that are industry validated career and technical skills
Preparation for postsecondary education and employment
Preparation not only to get a ‘job,’but ‘academic and technical preparation’
Increased emphasis on achievement of a degree, certificate or credential
Perkins IV Areas of Emphasis
Increased Accountability
New accountability provisions are a reminder that:
Funds are not an entitlement
Use of funds is flexible & responsive to the accountability data
Separate secondary and postsecondary measures
Performance levels at state and local levels
Negotiations and redirection of funds
Increased Accountability
Indirect Measures Data that support
inferences about the quality of teaching and learning and give insight into why achievement is high or low:• Enrollment• Retention rates• Course completion• Student/graduate/
employer satisfaction surveys
• Placement statistics
Direct Measures Data that provide
evidence that students have achieved the intended outcomes:• Performance
assessments with rubrics• Portfolios• Artifacts• Performances• Outcome referenced
tests
Increased Accountability for Direct Measurement of Learning
Denominator: number of CTE concentrators who took technical courses during the three year reporting period
Prior accountability required only indirect measurement:
Numerator: Number of CTE concentrators who passed 80% of the technical courses they attempted during the three year reporting period
Measure =
Increased Accountability for Direct Measurement of Learning
•Denominator: Number of CTE concentrators who took technical skill assessments during the reporting year
New OVAE Guidance for Perkins IV Measurement requires direct measures of skill attainment:
Numerator: Number of CTE concentrators who passed technical skill assessments that are aligned with industry-recognized standards, if available and appropriate, during the reporting year
Measure =
Increased Accountability for Direct Measurement of Learning
Department of Education goal is to have end of program outcomes assessments that are:• Direct measures of outcomes• Independent• 3rd-Party• Employer validated
By 2015 most WTCS assessments should be external or approved assessments.
Increased Accountability for Direct Measurement of Learning
WTCS TSA Assessments External TSA Assessment
(ex. NCLEX, Barber/Cosmetology, etc.)
Indirect Assessments
Wisconsin Standards for Assessment - WTCS
Increased Accountability for Direct Measurement of Learning
WTCS Assessment StandardsWTCS approved assessment that objectively measures student attainment of industry recognized skills upon graduation.
May be developed and/or implemented by the state system.
May be developed and/or implemented locally according to system guidelines and with industry (advisory committee) approval.
Must be valid and reliable.
External Assessment Standards Indirect Assessment Standards
Wisconsin Standards for Assessment - WTCS
Increased Accountability for Direct Measurement of Learning
WTCS Assessment Standards External Assessment Standards (ex.
NCLEX, ASE, Barber/Cosmetology)
External, third-party assessment that objectively measures student attainment of industry recognized skills upon graduation.
Indirect Assessment Standards
Wisconsin Standards for Assessment - WTCS
Increased Accountability for Direct Measurement of Learning
WTCS Assessment Standards External Assessment Standards Indirect Assessment Standards
Non-assessment related indicators such as indirect measures.GPA and/or Course CompletionProgram Completion – Graduation
Teacher-developed exams that do not meet WTCS standard (not WTCS approved)
Wisconsin Standards for Assessment - WTCS
Increased Accountability for Direct Measurement of Learning
WTCS will move toward a balance of WTCS TSA Assessments and External TSA Assessments
WTC
S Assessm
ent
External A
ssessment
Indirect
2012-2013
New Accountability for Direct Measurement of Learning
How does curriculum fit How does curriculum fit in?in?
Plan•Establish outcomes•Build assessments•Design learning
DO• Deliver• Assess• Document
CHECK• Analyze assessment
data• Propose
improvements
ACT•Adjust design
•Adjust delivery
•Adjust assessment
Assessment Process and Curriculum
WIDS Model 3 of 4 components answer assessment questions
What do they need to be able to achieve?
Who are the learners?
How will we know and show when they’ve achieved it?
How will they get there?
WHO
WHAT
HOW
Program Outcomes Measurable, observable, and field-
specific skills – major outcomes• Number 5-7 per program (guideline not rule)
Originate from:• Current DACUMs• Accrediting Agencies• National (or other) Skill Standards• Advisory Committees
Threaded through courses Performance verified with
summative assessment of skill performance
Integrating Outcomes
Linked Competenci
es(performance assessments)
Exit Learning Outcomes
Program Outcomes(summative assessment)
Core Abilities
(summative assessment)
Gen Ed Outcomes(summative assessment)
Job Task Analysis(DACUM)
Job Task Analysis(DACUM)
External Standards
(Research-based)
External Standards
(Research-based)
Linked Competenci
es(performance assessments)
Linked Competenci
es(performance assessments)
Formative assessment of program outcome
Summative assessment of learning outcomes
External
Program-Level
Course-Level
Drives Learning Design
Assesses Learning Results
Drives Learning Design
Assesses Learning Results
External
Program-Level
Course-Level
Dental Hygiene Example
Graduates are competent in applying ethical, legal and regulatory concepts to the provision and/or support of oral health care services.
Incorporate into dental hygiene practice professional laws, regulations and policies established by the licensing state and regulatory agencies
Respond to a request to perform a task that is not legally permitted to be delegated to a dental hygienist.
ADA Dental Hygiene Education Standard
Dental Hygiene Program Outcome
Dental Hygiene CompetencyFormative assessment of program outcome
Summative assessment of Learning outcomes
Summative/Formative Assessment
Sound Assessment
Valid Outcomes based on standards
(industry) Measures intended outcomes Measures application and critical
thinking
Reliable Performance assessment based on
consistent rubrics, scoring guides, and rating scales
Consistent process - each learner is assessed in same way as other learners
Fair Valid Reliable Learners informed of expectations
up front! Feedback to learners
Standards for WTCS Standards for WTCS Approved TSA Approved TSA AssessmentsAssessments
WTCS will move toward a balance of WTCS TSA Assessments and External TSA Assessments
WTC
S Assessm
ent
External A
ssessment
Indirect
2012-2013
New Accountability for Direct Measurement of Learning
is valid and reliable;meets WTCS standards/guidelines; is approved by industry (advisory
committee); is developed and/or implemented
collaboratively by the WTCS colleges; ordeveloped and/or implemented locally
according to system guidelines and with industry (advisory committee) approval.
WTCS Assessment Standards
WTCS approved assessment objectively measures student attainment of industry recognized skills upon graduation and:
Available in your handout packet
WTCS Standards forWTCS Standards for33rdrd-Party External -Party External
AssessmentsAssessments
3rd Party TSA Also referred to as:
• External• Direct Method• Credential• License
Generally controls entrance into an occupation
Validity / Reliability Alignment to Standards Availability of data Value to the Student Development /
Administrative Costs Pooling / Sharing Resources
Considerations for selecting 3rd Party External Assessments
Can we obtain the results/data?
Are cost and administration feasible?
Does it add value or control entry to the student/occupation?
Is it valid, reliable, reputable, recognized?
Is it summative, cumulative or a partial measurement of skills?
Considerations for selecting 3rd Party External Assessments
Does the assessment align with program outcomes?
Does the assessment measure what needs to be measured?
Does the data help improve and strengthen our programs?
Considerations for selecting 3rd Party External Assessments
WTCS 3rd Party Assessments Currently in Place
Occupations regulated by the Department of Regulation and Licensing• Barber/Cosmetology • Funeral Director • LPN • Registered Nurse• Dental Hygienist• Massage Therapist• Occupational Therapy Assistant• Physical Therapy Assistant• Respiratory Care Practitioner• Veterinary Technician• Wisconsin Registered Interior Designer
Guidelines for Selecting 3rd Party Assessments
This is a guiding document to assist districts in determining direct technical skill assessment appropriateness.
Districts will work with the state office staff in the determination.
Keep in mind this is for a small number of programs.
Guidelines for Selecting 3rd Party Assessments
Concerns: Only measures part of the program
outcomes-”All Code-No Application” Does not indicate job performance Does it help employers? Focused only on technical skills,
what about academic skills and soft skills?
Guidelines for Selecting 3rd Party Assessments
3rd Party Assessment - Example
Automotive Service Excellence (ASE) Long-standing credential of
automotive technicians A series of 8 certification
areas/assessments Would it work for Perkins?
Logistics Professional CPL/DL-International Society of Logistics
CPIM/CSCP-Association for Operations Management
PLS/CTL/DLP conferred by the American Society of Transportation & Logistics
3rd Party Assessment - Example
Federal Aviation Administration (FAA) Airframe and Powerplant Mechanic Certificate
Administered by the FAA Written, practical and oral examination
Required for entry
3rd Party Assessment - Example
How Can WIDS Help?How Can WIDS Help?
The Learning & Assessment Design Technology Tool Kit
Tra
cDat
B
lack
board
iWebfo
lio
eFo
lio
Quest
ionm
ark
People
Soft
B
anner
Data
tel
Create/maintain Program Outcome Summaries for all new and existing programs.
Develop/maintain Course Outcome Summaries for all courses.
Design Performance Assessment Tasks for course level (formative) and summative assessments.
Use WIDS Analyzer to build Learning Outcome Matrices that document how outcomes are linked with programs and courses.
Present WIDS documentation as a central feature in accreditation.
How can WIDS help?
WIDS Model and Software Tools:
Facilitate collaborative development of:• DACUM or Industry Review Focus
Groups• Program Outcomes• Program Outcome Summaries• Summative Performance
Assessments Produce
• Industry Approval Reports• Analyzer Reports• Updated WIDS files and reports
How can WIDS help?
WIDS Team
How do we communicate WTCS long-standing commitment to industry input and standards?
How do we ensure that we use quality assessments? (That we don’t regress to testing regurgitation of information?)
How do we ensure that we actually are doing what our model and assessment plans imply?
How do we collect and report data?
Challenges
Current ProjectsCurrent Projects
Automotive Technology/Technician NATEF end of program evaluation
Accounting WIDS
Early Childhood WIDS
Criminal Justice WIDS
Welding WIDS
Current Projects