nocti overview

NOCTI Overview

Amie Birdsall and Patricia KelleyFebruary 23, 2012

Discussion Topics

NOCTI Overview

Products and Services

Selecting an Assessment

CTEDDI

Q&A

NOCTI Overview

Over 40 years serving the CTE community Non-profit organization

Owned and operated by the 50 states

Each state is a member of NOCTI’s Consortium of States

Works with both secondary and post-secondary populations

Nocti Business Solutions (formerly The Whitener Group) provides services to industry

NOCTI Overview - Educational Partners

Over 80 Job Ready tests offered

30 Pathway tests in 10 clusters available

Over 90 private industry employment and/or advancement tests delivered

20 teacher-level assessments

Study guides for all Job Ready and Pathway assessments

Countdown Kits available for all Job Ready assessments

Successful Schools-Successful Programs Guide

Supplemental ProductsAssessments

NOCTI Products & Services

Professional Learning Opportunities Also

Offered!

Selecting an Assessment

Determine purpose of assessment

Determine audience

Determine results desired

Research available options

Selecting an Assessment

Selecting an Assessment

www.nocti.org

Assessment blueprints are available on

NOCTI’s website at no cost!

Assessment Blueprints

• Blueprints include:• Competency

List• Content

Distribution• Sample

items/jobs

Assessment Blueprints

Content distribution for written and performance

Sample multiple-choice questions

Sample performance job

Job Ready Assessments

Assess technical skills at the occupation level (e.g., Accounting, Carpentry, Pre-Engineering)

Measure aspects of occupational competence such as factual and theoretical knowledge

Target secondary and post-secondary program completers

NOCTI Job Ready Assessment Blueprint

Plumbing

Safety

Plumbing Math

General Plumbing Practices

Joining Pipes and Fittings

Drainage

Vent Installation

Water Supply and Distribution

Hot Water Systems

Traps

Installing Fixtures

Plumbing Service and Repair

8%

10%

24%

9%

8%

4%

7%

6%

8%

7%

9%

NOCTI Job Ready Assessment Blueprint

Number of Questions: 222 Areas Covered:

Accounting

NOCTI Pathway Assessment Blueprint

Construction Pathway

General Architecture & Construction Technical Skills

Construction

Academic Foundations

Systems

Ethics and Legal Responsibilities

Communication

Information Technology Applications

Problem Solving, Critical Thinking, and Decision Making

Leadership and Teamwork

Safety, Health, and Environment

Employability and Career Development

15%

15%

15%

6%

6%

8%

5%

7%

6%

8%

9%

Assessment Results

NOCTI’s Standard Score Reporting Package: Individual Score Reports

Group Score Reports

Comparative Data

Other Reports Available: Task-Link Reports

Integrated Academic Reports

What data do I receive from NOCTI?Score Reports

Online Test Administration

Immediate Score Reports

Aggregate Reports: 2 business days after user codes are released from Client Services Center

Paper/Pencil Administration

Aggregate Reports: 10 business days after receipt of correctly completed answer sheets

What data do I receive from NOCTI?Immediate Score Reports

Immediate Student Scores

Displayed to student if option is designated

Emailed to Site Coordinator upon completion of testing (can be forwarded to instructor)

NOCTI Standard Score Reports

Individual Score Reports

• Provided for each test taker

• Scores by duty and total

• Pre-/post-test comparisons

• Comparative data

NOCTI Standard Score Reports

Group Score Report

• Provided for the testing group

• Duty and total scores for each participant

NOCTI Standard Score Reports

Analysis of Scores Report

• Provided for the testing group

• Comparative data• Criterion-

referenced cut score

• Other statistical information

NOCTI Additional Reports:Task-Link Report

Task-Link Report

• Includes:• Number of items

presented for each task

• Average correct by group

• Number correct for each test taker

Project Goals

• Understand data and assessments• Use data effectively• Improve learning and skill acquisition for students• Plan for continuous improvement of learning and

instruction• Share ideas and collaborate with other educators

Instructional Improvement Cycle

A Nursing AssistingClass

Classroom Example

Areas on a Nursing Assisting Test

28%

10%

10%13%

10%

13%

8%

8% Basic Nursing Assisting

Infection Control

Communication Skills

Personal Care Skills

Mental Health/Social Needs

Client’s Rights

Basic Restorative Devices

Employability Skills

Nursing Assistant Pretest Areas Pretest % CorrectStudent 1

Posttest % CorrectStudent 1

Basic Nursing Assisting 54.6 --Infection Control 30.0 --

Communication Skills 41.1 --

Personal Care Skills 36.4 --

Mental Health/Social Needs 53.8 --

Client’s Rights 36.4 --

Basic Restorative Devices 50.0 --

Employability Skills 27.3 --

Total 43.4

Individual Nursing Pretest

Nursing Assistant Pretest Areas

Student 1 % Correct

Student 2 % Correct

Student 3 % Correct

Student 4 % Correct

Student 5 %

Correct

Student 6 %

Correct

Class Average % Correct

Basic Nursing Assisting

54.6 45.5 42.7 27.3 27.3 54.5 41.9

Infection Control 30.0 20.0 10.0 20.0 10.0 30.0 20.0

Communication Skills

41.1 17.6 23.5 23.5 13.2 35.3 25.7

Personal Care Skills 36.4 54.5 45.5 45.5 36.4 54.5 45.5

Mental Health/Social Needs

53.8 38.5 42.3 38.5 20.8 53.8 41.3

Client’s Rights 36.4 36.4 38.3 36.4 22.5 45.5 35.9

Basic Restorative Devices

50.0 60.0 50.0 60.0 30.0 80.0 55.0

Employability Skills 27.3 18.2 18.2 27.3 9.1 27.3 21.2

Total 43.4 38.4 35.9 38.5 22.8 48.8 38.0

Nursing Assisting Pretest

Nursing Assisting Posttest

Nursing Assistant Pretest Areas

Student 1 % Correct

Student 3 % Correct

Student 5 % Correct Group Site State Nation

Basic Nursing Assisting

100.0 92.4 76.2 86.4 79.5 72.5 75.5

Infection Control 91.7 94.5 72.4 79.2 78.2 70.7 70.6

Communication Skills

100.0 100.0 55.3 95.0 83.8 64.3 66.4

Personal Care Skills 92.3 54.5 52.5 69.2 65.8 51.6 52.9

Mental Health/Social Needs

97.4 72.5 60.3 71.8 69.8 63.1 64.1

Client’s Rights 84.8 66.4 44.7 64.6 67.6 58.1 57.8

Basic Restorative Devices

87.5 60.2 45.2 53.1 61.9 55.6 56.5

Employability Skills 87.5 78.5 37.4 79.7 75.6 66.6 67.0

Total 93.9 79.4 58.4 76.8 73.7 64.1 65.2

• Achievement gaps• Differences between subgroups• Trends • Progress

What the Data Reveal

Collecting Data – More Than NOCTI

• Includes process of collecting, analyzing, and interpreting information

• Determines the degree to which students are becoming competent and meeting standards

• Occurs at many levels (e.g., classroom and program assessments to state and national assessments)

• School • Program• Teacher

What Data Exist?

• Class• Student • Standards

• National• State • District/System

Icebergs

What lies beneath the data scores that influence those scores?

• What is the data designed to answer?

• What other data may be needed?

• What roadblocks are there to interpreting the data?

What Questions Can the Data Answer?

Plan for Learning Improvement

• Start with results: Analyze the previous year’s goals, comparing against local unit tests and end-of-course tests, at the student level and classroom level.

• Triangulate: Find relationships between test data and attendance and discipline referrals.

Plan for Learning Improvement

Continued: • Disaggregate data by subgroups (e.g.,

socioeconomic status or those receiving free/reduced lunch, or Individualized Education Program/IEP).

• Use information from advisory groups or parents.• Set goals for the year as well as long-term goals.

Plan for Learning Improvement

Continued: • A common denominator is to improve learning

through teamwork.• Build school department and state data teams.• Review annual assessment data with a faculty

team to isolate achievement gaps or groups of students with high and low performance.

Plan for Learning Improvement

Continued: • Schedule school and department data team

meetings to discuss data and review progress. • Discuss and develop strategies, such as curricular

alignment with academic and industry standards, curricular sequence, timing, and depth of instruction.

Monitor Student Progress

• Use a progress monitoring system for all students.

• Graph students’ curricula-based scores.

• Identify effective instructional components.

Thank You

Amie Birdsall Amie.Birdsall@nocti.org

Pat Kelley Patricia.Kelley@nocti.org