caep accreditation reporting measures€¦ · edtpa rubric summary task rubric task 1: planning for...

1

CAEP Accreditation Reporting Measures

The Master of Science in Education degree that leads to dual certifications in Childhood 1-6 and Special Education attained National Recognition

from the Association of Childhood Education (ACEI) in 2015 and National Recognition from the Council for Exceptional Children in 2016.

In 2015, the Selected Improvement Commission of the Council for the Accreditation of Educator Preparation (CAEP) continued MCNY’s NCATE

accreditation, citing no areas for improvement relative to any of the NCATE Standards.

1. Impact on P-12 learning and development: The edTPA performance assessment has been required for certification in New

York State since May 2014. For the past four years, the NYSED has provided a “safety-net” for teacher candidate examinees. If a teacher

candidate submits a technically scored edTPA assessment and fails, for a pathway to approved certification a teacher candidate can

either re-take the edTPA and pass or take and pass the older ATS-W written exam. The edTPA safety net ended on June 30, 2018. With

this choice for the pathway for teacher certification, some MCNY teacher candidates utilized the safety net rather than re-taking and

passing the edTPA.

The following report is a review and analysis of aggregated and disaggregated data from the edTPA performance of a recent cohort of MSED

candidates:

2

edTPA Elementary Education Assessment

Task Rubric

Task 1: Planning for Literacy Instruction and Assessment

Rubric 1: Planning for Literacy Learning

Rubric 2: Planning to Support Varied Student Learning Needs

Rubric 3: Using Knowledge of Students to Inform Teaching and Learning

Rubric 4: Identifying and Supporting Language Demands

Rubric 5: Planning Assessments to Monitor and Support Student Learning

Task 2: Instructing and Engaging Students in Literacy Learning

Rubric 6: Learning Environment

Rubric 7: Engaging Students in Learning

Rubric 8: Deepening Student Learning

Rubric 9: Subject-Specific Pedagogy

Rubric 10: Analyzing Teaching Effectiveness

Task 3: Assessing Students' Literacy Learning

Rubric 11: Analysis of Student Learning

Rubric 12: Providing Feedback to Guide Further Learning

Rubric 13: Student Understanding and Use of Feedback

Rubric 14: Analyzing Students' Language Use and Literacy Learning

Rubric 15: Using Assessment to Inform Instruction

Task 4: Assessing Students' Mathematics Learning

Rubric 19: Analyzing Whole Class Understandings

Rubric 20: Analyzing Individual Student Work Samples

Rubric 21: Using Evidence to Reflect on Teaching

3

Elementary Education edTPA

Table 1: 2019 vs. 2018 results for the Elementary Education test with average and standard error by rubric and task. Student-specific

breakdown is for 2019 only.

4

Special Education edTPA

Table 2: 2019 vs. 2018 results for the Special Education test with average for every rubric and task. Student-specific breakdown is for

2019 only.

5

Results by Rubric and Task for the 2019 Elementary Education Exam

Chart 1: Average and standard error for each rubric based on all attempts at the Elementary Education exam in 2019 (based on Table

1).

1.7

1.9

2.1

2.3

2.5

2.7

2.9

3.1

3.3

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Ave

rage

Sco

re

Rubric #

Average Rubric Scores - Elementary Education

6

Chart 2: Average and standard error for each task based on all attempts at the Elementary Education exam in 2019 (based on Table

1).

1.80

2.00

2.20

2.40

2.60

2.80

3.00

Task 1: Planning Task 2: Instructing Task 3: Assessing Task 4: Math

Ave

rage

Sco

re

Task

Average Task Scores - Elementary Education

7

Year-over-year Comparison of Results by Rubric and Task for the Elementary Education Exam

Chart 3: Average and standard error for all tests vs. passed tests across every rubric in 2019 vs. 2018 (based on Table 1).

1.5

1.7

1.9

2.1

2.3

2.5

2.7

2.9

3.1

3.3

3.5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Ave

rage

Sco

re

Rubric #

Average Rubric Scores - Elementary Education

2019 Results 2018 Results

8

Chart 4: Average and standard error for all tests vs. passed tests across every task in 2019 vs. 2018 (based on Table 1).

1.50

1.70

1.90

2.10

2.30

2.50

2.70

2.90

3.10

Task 1: Planning Task 2: Instructing Task 3: Assessing Task 4: Math

Ave

rage

Sco

re

Task

Average Task Scores - Elementary Education

2019 Results 2018 Results

9

edTPA Rubric Summary

Task Rubric

Task 1: Planning for Literacy Instruction and Assessment

Rubric 1: Planning for Literacy Learning

Rubric 2: Planning to Support Varied Student Learning Needs

Rubric 3: Using Knowledge of Students to Inform Teaching and Learning

Rubric 4: Identifying and Supporting Language Demands

Rubric 5: Planning Assessments to Monitor and Support Student Learning

Task 2: Instructing and Engaging Students in Literacy Learning

Rubric 6: Learning Environment

Rubric 7: Engaging Students in Learning

Rubric 8: Deepening Student Learning

Rubric 9: Subject-Specific Pedagogy

Rubric 10: Analyzing Teaching Effectiveness

Task 3: Assessing Students' Literacy Learning

Rubric 11: Analysis of Student Learning

Rubric 12: Providing Feedback to Guide Further Learning

Rubric 13: Student Understanding and Use of Feedback

Rubric 14: Analyzing Students' Language Use and Literacy Learning

Rubric 15: Using Assessment to Inform Instruction

Task 4: Assessing Students' Mathematics Learning

Rubric 19: Analyzing Whole Class Understandings

Rubric 20: Analyzing Individual Student Work Samples

Rubric 21: Using Evidence to Reflect on Teaching

Table 3: Task and Rubric names for the Elementary Education and Special Education exams. Task 4: Assessing Students’ Mathematics

Learning is excluded from the Special Education exam. Highlighted Rubrics indicate best performance across all students. Best

performance in 2018 was in Rubrics 2, 6, 11 and 19.

10

Analysis

For a student to pass their exam, they must answer every question with minimum sufficiency, gaining at least a score of 1.0 per

rubric. Failure to complete any question at this minimum level results in overall failure of the test. One student had to retake the

Elementary Education once this year given incomplete performance on Task 2 (Instructing and Engaging Students in Literacy

Learning). In 2018, four students had to retake the exam, primarily due to poor performance on Task 4 (Assessing Students’

Mathematics Learning).

Elementary Education Exam

Of the 2019 cohort, students who took the Elementary Education exam performed best in Task 3: Assessing Students’ Literacy

Learning with an average score of 2.73. The one student who took the Special Education exam also performed best on this task,

bringing average performance across all students to 2.80. By contrast, performance was worst in Task 3 in 2018 (average score of

2.18) and best performance was in Task 1: Planning for Literacy Instruction and Assessment with an average score of 2.80.

As in 2018, students performed most consistently in Task 2, where standard error of 0.10 was the lowest across all tasks.

In both years, students performed worst in Task 4: Assessing Students' Mathematics Learning. Students in 2019 showed a marked

improvement compared to the previous year with an average score of 2.17 compared to 1.73 in 2018. However, when only

considering the passing scores in 2018 (as mentioned, four students failed on the first attempt last year), then the average score on

this task was 2.49.

This year, performance was highest in Rubric 12: Providing Feedback to Guide Further Learning, which increased its average score

from 2.3 in 2018 to 3.1 this year. Poorest performance resulted in an average rubric-specific score of 2.0, which occurred in Rubric 5:

Planning Assessments to Monitor and Support Student Learning (compared to 2.6 last year) and Rubric 19: Analyzing Whole Class

Understandings (compared to 1.8 last year).

On Task 4: Assessing Students' Mathematics Learning, students performed best on Rubric 20: Analyzing Individual Student Work

Samples and Rubric 21: Using Evidence to Reflect on Teaching this year. Both these rubrics demonstrated improved performance

11

compared to 2018, when students did worse on these questions than on Rubric 19: Analyzing Whole Class Understandings.

Performance increased on all three rubrics comprising this task.

Special Education Exam

Only one student chose to take the Special Education exam; therefore, there were too few results to conduct any statistical analysis.

Standard Error

Standard error was used in place of standard deviation to determine the distribution of test results across students within each Task.

The use of standard error is recommended in place of standard deviation when conducting such analysis on small samples.

𝑆𝐸 =𝜎

√𝑛

As demonstrated in the calculation above, standard error (SE) is determined as the standard deviation (σ) of the population divided

by the square root of the number of samples (n).

As shown on pages 2 and 3, Charts 1 – 4 display �̅� ± 𝑆𝐸 for every Task or Rubric, which is the average [plus or minus] the standard

error.

The greater the standard error, the larger the variation of performance between grades. A small standard error shows that the

majority of grades are concentrated around the average.

As shown on pages 2 and 3, Charts 1 – 4 display �̅� ± 𝑆𝐸 for every Task or Rubric, which is the average [plus or minus] the standard

error.

The greater the standard error, the larger the variation of performance between grades. A small standard error shows that the

majority of grades are concentrated around the average.

12

2. Indicators of teaching effectiveness

MSED Student Teaching Proficiencies

The Student Teacher Proficiencies Form is completed by the cooperating teacher and by the college supervisor in each placement of student teaching. The means reported in the data table represent the scores from both cooperating teachers and the college supervisors. The rubric indicators are designed to assess candidates in three categories: Knowledge, Skills, and Dispositions/Values.

Proficiencies

Mean Number % Meet or

Above Std.

Knowledge

A. Subject Matter

Candidate knows, understands, and can use general and specialized curricula for teaching

across curricular content areas to individualize learning for students with and without

exceptionalities.

3.7 19 100%

B. Student Learning

Candidate knows and understands similarities and differences in human development and

learning and uses this knowledge to provide meaningful and challenging learning experiences

for students with exceptionalities.

3.7 19 100%

C. Diversity of Learners

Candidate uses differentiated instruction that demonstrates understanding of how language,

culture, and family background influence the learning of students including students with and

without disabilities.

3.5 19 100%

D. Environment for Learning

Through collaboration with colleagues, candidate creates safe, inclusive, culturally responsive

learning environments to engage students with and without exceptionalities.

3.4 19 95%

E. Assessment

Candidate demonstrates the ability to select, adapt, monitor, analyze and use multiple methods

of assessment in making educational decisions for students with and without exceptionalities.

3.2 19 95%

Skills

A. Planning Instruction

Candidate is able to select, adapt, and use a repertoire of evidenced-based instructional

strategies to advance language development and communication skills.

3.7 19 100%

13

B, Strategies/Technologies

Candidate uses a variety of instructional and assistive technologies, and augmentative and

alternative communication systems to enhance language and communication skills of students

with exceptionalities.

3.7 19 100%

C. Learning Environment

Through collaboration with general educators and other colleagues, candidate creates a safe,

inclusive, and culturally responsive learning environment to engage students with and without

exceptionalities, in meaningful learning activities and social interactions.

3.6 19 100%

D. Communication

Candidate effectively uses verbal, non-verbal, written language, and media communication

strategies to support and enhance language development and communication skills of students

with exceptionalities.

3.2 19 89%

E. Assessment

Candidate selects, adapts and modifies formal and informal assessments to guide educational

decisions for students with disabilities.

2.9 19 89%

F. Collaboration and Relationships

Candidate demonstrates across a range of settings, the ability to consult, and collaborate with

school colleagues, families and community personnel, to promote the learning and well-being

of students with exceptionalities.

2.9 19 89%

G. Reflection and Professional Development

Candidate uses knowledge of historical, current and cultural and ethical issues and reflects on

the impact of his/her practice to improve instruction for students with disabilities and to guide

their own professional growth, practice, and significance of participating in professional

activities and learning communities.

2.9 19 89%

Dispositions/Values

A. Diversity/Individual Differences

Candidate demonstrates that he/she can create consistent culturally responsive learning

environments that promote increased independence, self-motivation, self-direction, self-

advocacy and personal empowerment of students with exceptionalities.

3.6 19 100%

B. High Expectations

Candidate demonstrates commitment and upholds high standards of competence and integrity

to developing the highest education and quality-of-life potential of students with and without

disabilities.

3.7 19 100%

14

3. Satisfaction of Employers and Employment Milestones

Even though this report is not available at this time, it is planned to be available in one year.

4. Satisfaction of Completers

Even though this report is not available at this time, it is planned to be available in one year.

15

5. Graduation Rates

The data below presents the completion (graduation) and dropout rates for the MSED program. All of the rates track new students

entering the program.

The Academic Year (AY) starts with the summer, followed by the fall, and lastly the spring semester. Example: 2014-2015 academic

year includes the summer 2014, fall 2014, and spring 2015.

Dropout rates do not include students who are still enrolled at the program. Still enrolled at the program included students who are

were enrolled for the spring 2019 semester or enrolled for the summer 2019 semester as of 4/23/19.

Table 1. Completion Rates for the MSED Program from 2014-2015 AY - 2017-2018 AY

Academic Year Admitted Cohort

Completed

Cohort Percentage Completed

2014-2015 27 23 85%

2015-2016 18 13 72%

2016-2017 23 22 96%

2017-2018 18 12 67%

16

Figure 1. Completion Rates for the MSED Program from 2014-2015 AY - 2017-2018 AY

17

Table 2. Dropout Rates for the MSED Program from 2014-2015 AY - 2017-2018 AY

Academic Year Admitted Cohort

Dropped Out

Cohort Percentage Dropped Out

2014-2015 27 4 15%

2015-2016 18 5 28%

2016-2017 23 1 4%

2017-2018 18 5 28%

18

Figure 2. Dropout Rates for the MSED Program from 2014-2015 AY - 2017-2018 AY

19

6. Ability of Completers to Meet Certification and State Requirements

Besides the edTPA, New York State requires candidates who are seeking certifications in Childhood Education 1-6 and Special

Education (Students with Disabilities) to pass the following NYSTCE assessments:

201- Educating All Students

060 – Students with Disabilities

221/222/245 – Multi-Subject: Teachers of Childhood (Grades 1-6)

Note: The former required 202 – Academic Literacy Skills Test was eliminated as a state requirement for certification. In addition,

the choice of passing either the former “safety net” assessment versions, and/or passing the new reformulated versions were still in

place during 2016-17 but ended on June 30, 2017.

The following are the 2019 Title II Assessment Pass Rates:

20

Assessment Pass Rates

Assessment code - Assessment name Test Company Group

Number taking tests

Avg. scaled score

Number passing tests

Pass rate (%)

202 -ACADEMIC LITERACY SKILLS TEST

Evaluation Systems group of Pearson

All program completers, 2015-16

11 515 7 64

201 -EDUCATING ALL STUDENTS


Other enrolled students

12 509 8 67

21



Number taking tests

Avg. scaled score


Pass rate (%)




16 508 12 75




13 517 13 100




20 521 19 95

22



Number taking tests

Avg. scaled score


Pass rate (%)

002 -MULTI-SUBJECT CST



1

1221 -MULTI-SUBJECT GRADES 1 - 6



3




9

23



Number taking tests

Avg. scaled score


Pass rate (%)




8




13 1631 12 92

902 -SAFETY NET MULTI-SUBJECT



3

24



Number taking tests

Avg. scaled score


Pass rate (%)

902 -SAFETY NET MULTI-SUBJECT



3

960 -SAFETY NET STUDENTS WITH DISABILITIES



2

960 -SAFETY NET STUDENTS WITH DISABILITIES



1

25



Number taking tests

Avg. scaled score


Pass rate (%)

TP012 -SPECIAL EDUCATION



2

TP012 -SPECIAL EDUCATION



1

060.1 -STUDENTS WITH DISABILITIES CST.1



7

26



Number taking tests

Avg. scaled score


Pass rate (%)




15 520 10 67




11 535 9 82




15 535 14 93

27

7. Ability of Completers to Be Hired in Education Positions for Which They Have Prepared

The New York State Department of Education does not provide teacher preparation programs with any data on the teaching

employment of graduates. MCNY was able to determine that 17 out of the 19 program completers in 2017-18 were hired in

positions for which they have prepared. Even with the unknown results for two completers, the rate of employment is 89%.

Completer Employment

A Public District Elementary School

B Public Elementary Charter School

C Unknown

D Public Elementary Charter School

E Public Elementary Charter School

F Public Elementary Charter School

G Public District Elementary School

H Public District Elementary School

I Public District Elementary School

J Public District Elementary School

K Public District Elementary School

L Public District Elementary School

M Public District Elementary School

N Public District Elementary School

O Public District Elementary School

P Public Elementary Charter School

Q Public District Elementary School

R Unknown

S Public Elementary Charter School

28

8. Student Loan Default

MCNY Default Rates (2013-2015)

The USDOE Cohort Default Rate History List provides information on Metropolitan College of New York’s student default rate. Each

USDOE reports listed are for cohorts over a three-year period. The most recent cohort report (2015) indicates a low default rate of

4%.

caep accreditation reporting measures€¦ · edtpa rubric summary task rubric task 1: planning for...

Documents