nce annual program assessment information, template and … special... · nce annual program...

2011-2012 NCE Program Assessment Report Template 4/12 1

2011-2012

NCE Annual Program Assessment Information, Template and Rubric

Purpose: The purpose of annual assessment reporting by programs is to demonstrate how we are monitoring candidate learning in order to demonstrate the extent to which candidates are achieving the goals that we have set for them. Program annual assessment reports provide information that allows faculty to make decisions about candidate learning needs and our own pedagogical effectiveness in order to be able to make appropriate changes. As per its constitutional charge, the NCE Assessment Council’s role in reviewing NCE program assessment reports is to provide feedback and support to programs on their assessment processes and to encourage continuous improvement in moving their candidates and programs forward. Directions: This document includes the 2011-12 Annual NCE program assessment template and rubric that NCE programs will use in completing this year’s report. NCE Program Assessment Report Template (2011-2012): Although the same overall format is used as in previous year’s reports, the language and directions of the template have been revised to provide more detail to better assist you in including comprehensive information and data related to your candidates and your program. It is recommended that you use the template as it is designed as the framework for writing your report. NCE 2011-2012 Rubric for Reviewing Program Assessment Reports: The rubric is used by NCE Assessment Council reviewers in reviewing annual assessment reports as the means for providing written feedback to programs. It is highly recommended that programs use the rubric while writing annual assessment reports as a guide for incorporating an appropriate amount and level of detail and completeness so that outside reviewers can fully understand the program and its assessment system. In addition to revisions to the language of this year’s rubric as a means to ensure a more direct alignment to the report template, there is one new additional category that provides feedback on the overall organization and writing of the report. You will find it as the last component of the rubric – Report Writing and Organization. Due Date: Program assessment reports must be completed and emailed as a .doc or (.docx) file to Daniel Sloyan ([email protected]) no later than June 30, 2012 to assist in our NCATE visit that is scheduled for early fall 2012. Need Further Assistance? If you have questions regarding the process, the template or accompanying rubric, please contact your department’s NCE Assessment Council representative or the co-chairs of the Council for assistance.


NCE Program Assessment Report Template (2011-2012)

Program __________LBS1 ___________________________________________________________ Assessment Report Writers ____Grace, Hjelle, Sloyan__________________________________________________

Section I: Program Overview This section provides an overview into your programs’ candidates and completers. The data needed to complete this chart will be provided to your program’s identified contact person from the Office of Institutional Research by June 15, 2012. CAS SPE had 25 students enrolled and 12 admitted MAT SPE had 307 students enrolled(117 of which were alternative), and 38 admitted, 12 of which were alternative MED SPE had 83 students enrolled and 19 admitted.

2011-2012 (September 1, 2011 – December 31, 2012) Number of Students:

Undergraduate Graduate Alternative Programs (AUSL, TFA, etc.)

Online

Number of Students Admitted to the Program

MAT 26 MED 19 CAS 12

MAT 12

Number of Students Enrolled in the Program

MAT 190 MED 83 CAS 25

MAT 117

If deemed helpful, a brief overview of the program can be provided to assist reviewers in understanding the framework, philosophy and key elements of the program.

Section II: Relationship of Assessments to Program Outcomes and Standards

Please complete the following two charts below. In the first chart, show the alignment of the program’s assessments to the NCE Conceptual Framework/Outcomes. In the second chart, show the alignment of the program’s assessments to the program outcomes and other professional standards. In addition, a narrative description of the alignment in these charts to supply further description can be provided. Alignment of Program Assessments to NCE Conceptual Framework/Outcomes

NCE Conceptual Framework/Outcomes

NCE Candidates:

Program Assessments

Envision, articulate and model democratic and progressive education

6, 5, 4, 3

Design powerful learning environments that integrate appropriate technologies

6, 5, 4, 3

Design powerful learning environments that utilize multiple meaningful assessments

5, 6, 4, 3, 2

Design powerful learning environments that enable self-directed learning

6, 5, 4, 3

Work collaboratively in diverse communities and with diverse learners to achieve learning goals

4. 6. 5. 3

Advocate for democratic values, equity, access and resources to assure educational success for all

6, 5, 4, 3


NCE Conceptual Framework/Outcomes

NCE Candidates:

Program Assessments

Cultivate curiosity and excitement for learning in themselves and others

6, 4

Respect and learn from other peoples, cultures, and points of view 6, 3

Demonstrate a caring attitude in recognizing the needs of others and acting to promote their growth

6, 4, 3

Act with confidence and self-knowledge to assume professional leadership roles and responsibilities

1, 2, 3, 4, 5, 6

Use information from self and others to continuously improve 6, 5

Alignment of Program Assessments to Professional Standards and Program Outcomes

Program Outcomes

Professional Standards Program Assessments

The following CEC Professional Standards are referred to below: 1. Foundations 2. Development and Characteristics of

Learners 3. Individual Learning Differences 4. Instructional Strategies 5. Learning Environments 6. Language 7. Instructional Planning 8. Assessment 9. Professional & Ethical Practice 10. Collaboration

Teaching and Students

1-10 inclusive 6, 5, 4, 3

Teaching and Curriculum

1, 2, 3, 4, 5, 6, 7, 8 6, 5, 4, 3, 2, 1

Teaching and Instruction

2, 4, 3, 5, 7, 8 6, 5, 4, 3, 2, 1

Teaching and the Environment

10, 2, 3, 5, 4, 7 6, 5, 4, 3

Teaching and the Profession

2,10, 9, 3, 5, 6 6, 5, 4, 3


Section III: Key Program Assessments Using the chart below, indicate the name, type, and administration point for each of the assessments that the program uses to assess candidate learning and evaluate program effectiveness. Refer to the specific requirements of the program’s SPA (if any) to ensure that the program is meeting these standards. For non-SPA programs, only one content knowledge assessment domain is necessary. The following table indicates the assessments that are required for the SPA Report, which will be referred to in the subsequent table.

Type and Number of Assessment

Name of Assessment Type or Form of

Assessment

When the Assessment is Administered

Assessment #1: Licensure assessment, or other content-based assessment

Learning Behavior Specialist I Content Test ICTS #155 Illinois State Licensure Test K-12

Illinois State Licensure Test K-12

Prior to admission to SPE 592C Practicum/ Teaching Children And Adolescents With Special Needs

Assessment #2: Assessment of content knowledge in special education

Special Education General Curriculum Test ICTS #163

Illinois State Licensure Test

Completion of Program

Assessment #3: Assessment of candidate ability to plan instruction

SPA Assessment #3: Impact on Student Learning Project – Section 2, Lesson Plan

Project Final term: SPE 592C Practicum/ Teaching Children And Adolescents With Special Needs

Assessment #4: Assessment of student teaching

SPA Assessment #4: Student Teaching Competency Appraisal

Evaluation of Student Teaching

Midpoint during and at the Completion of the Final term: SPE 592C Practicum/ Teaching Children And Adolescents With Special Needs

Assessment #5: Assessment of candidate effect on student learning

SPA Assessment #5: Impact on Student Learning Project – Section 3, Assessment Method Section 4, Analyzing and Reporting the Data Section 5, Reflecting on the Data

Project Final term: SPE 592C Practicum/ Teaching Children And Adolescents With Special Needs

Assessment #6: Additional assessment that meets CEC standards

Learning Behavior Specialist I Portfolio

Portfolio Completion of First term required course – SPE 572 Practicum I: M.A.T. Final Term prior to Student Teaching – SPE 527 Final Term after completion of SPE 593C Practicum/ Teaching Children And Adolescents With Special Needs


NOTE: Assessments are the same for all Graduate programs – traditional and alternative. Name of Assessment Type or

Form of Assessment (Note if assessments are

different for Undergraduate, Graduate, Alternative Programs or Online)

When the Assessment Is Administered

Assessment of Content Knowledge I (i.e., Licensure assessment, or other content-based assessment) *Note: Non-SPA programs do not have state content tests)

Assessment #1 Above ICTS #155

See chart above

Assessment of Professional Knowledge (Pedagogical content knowledge)

Assessment #2 Above ICTS #163

See chart above

Assessment of Field Experiences

Assessment #3, #4, #6 Above See chart above

Assessment of Candidate Impact on Student Learning

Assessment #5, #6 Above See chart above

Assessment of Candidate Dispositions

Particular section of Competency Appraisal (#4) address this.

See chart above

Assessment of Candidate Diversity Proficiencies

Particular aspects of Competency Appraisal and Portfolio (#6) address this, along with exit survey

See chart above

Assessment of Candidate Technology Proficiencies

Exit survey as well as portfolio address this.

Chart above and end of program.


Section IV: Assessment Tools and Data Analysis In this section, to provide a complete picture of each assessment and its findings, include information for the components listed below for each individual assessment. Note: If the program does not use an assessment for any of the required assessment domains in Section III (Content Knowledge, Professional/Pedagogical Knowledge, Field Experience, Impact on Student Learning, Dispositions, Diversity and Technology), in section V below you will be asked to provide a description of how the program is working toward developing an assessment for that category or a rationalization for why it is not applicable to the program. Required components to include for each assessment tool: Program Assessments 1. A narrative description of the assessment including:

why it was developed and what it assesses, how and when it is implemented (i.e. where is it administered in the program, the course in which it is a part

of, etc.) how and when it is evaluated, and who evaluates the assessment.

2. A blank sample of the assessment. 3. A blank sample of the scoring guide/rubric that is used to assess the tool. Assessment Data 1. Annual data collected from the tool. 2. A narrative interpretation of what the data means to your program in terms of candidate learning and performance.


Domain 1: Assessment of Content Knowledge I Assessment 1: Learning Behavior Specialist I Content Test ICTS #155

2011-2012 Test #155 Content Area Test Learning Behavior Specialist I

Assessment Type: Paper Pencil Traditional Online Alternative

Total Test 268 266

Average Sub-scores

1. Foundations and Characteristics 270 276

2. Assessing Students and Developing Individualized Programs 266

270

3. Planning and Delivering Instruction 271 273

4. Managing the Learning Environment and Promoting Students’ Social Interaction and Communication Skills 268

249

5. Working in a Collaborative Learning Community 264 260

6. Professional Conduct, Leadership, and Growth 270 276

Traditional Online Alternative

Number of candidates who took the test 67 49

Number of candidates who passed 66 49

Pass Rate (%) 99 100

Number of candidates who did not pass on the first try 2 0


Domain 2: Assessment of Professional Knowledge Assessment 2: Special Education General Curriculum Test ICTS #163

2011-2012 Test #163 Special Education General Curriculum Test

Assessment Type: Paper Pencil Traditional Online Alternative

Total Test 249 266

Average Sub-scores

1. Reading and Literacy 251 258

2. Mathematics 248 273

3. Natural Sciences 255 277

4. Social Sciences 230 255

Traditional Online Alternative

Number of candidates who took the test 65 52

Number of candidates who passed 53 51

Pass Rate (%) 82 98

Number of candidates who did not pass on the first try 6 1


Domain 3: Assessment of Field Experiences Assessment 3: Impact on Student Learning Project – Section 2, Lesson Plan (data needed by Oct 1)


Domain 3: Assessment of Field Experiences Assessment 4: Student Teaching Competency Appraisal

2011-2012 Competency Appraisal Data (September 1 – June 30)

University Supervisor Cooperating Teacher Teaching Candidate Category Midterm Final Midterm Final Midterm Final

Curriculum 2.36 2.80 2.33 2.77 2.22 2.69

Instruction 2.41 2.84 2.38 2.82 2.32 2.71

Environment 2.50 2.86 2.46 2.83 2.38 2.71

Profession 2.71 2.93 2.73 2.92 2.66 2.87

Dispositions 2.56 2.87 2.55 2.83 2.46 2.76

Total 2.52 2.90 2.58 2.91 2.54 2.83

*Data will be disaggregated (traditional vs. alternative) by October 1, 2012.


Domain 3: Assessment of Field Experiences Assessment 6: Learning Behavior Specialist I Portfolio EG … Pls check last year’s report – regarding which section of the portfolio might apply to this. … Or, maybe you just want to address this in the next “Domain” section below.

LBSI Portfolio Data – Traditional Program (n=58)

Portfolio Category 2011-2012

Average Score

Teaching and Students 2.86

Teaching and Curriculum 2.83

Teaching and Instruction 2.93

Teaching and the Environment 2.95

Teaching and the Profession 2.93

LBSI Portfolio Data – Alternative Programs (n=59)


Average Score







Domain 4: Assessment of Candidate Impact on Student Learning Assessment 5: Impact on Student Learning Project – Section 3, Assessment Method Section 4, Analyzing and Reporting the Data Section 5, Reflecting on the Data (Need to get data)


Domain 4: Assessment of Candidate Impact on Student Learning Assessment 6: Learning Behavior Specialist I Portfolio

LBSI Portfolio Data – Traditional Program (n=58)


Average Score






LBSI Portfolio Data – Alternative Programs (n=59)


Average Score







Domain 5: Assessment of Candidate Dispositions Assessment #4: Competency Appraisal – Specific sections These are the questions in the Competency Appraisal, Dispositional section. Data to follow by October 1st. 47. Teaching and the Profession - The Teacher Candidate respects the privacy of students, confidentiality of

information, and fulfills legal responsibilities. 48. Teaching and the Profession - The Teacher Candidate follows school and University policy and procedures. 49. Teaching and the Profession - The Teacher Candidate projects a professional image in dress, grooming, and

demeanor. 50. Teaching and the Profession - The Teacher Candidate assesses his or her own needs for knowledge and skills

related to teaching students with exceptional learning needs, seeks assistance and resources, and accepts constructive feedback.

51. Teaching and the Profession - The Teacher Candidate participates in professional development activities. 52. Teaching and the Profession - The Teacher Candidate collaborates with classroom teachers, parents, para-

educators, and other school and community personnel in decision-making and problem solving to support student learning and well being.

53. Teaching and the Profession - The Teacher Candidate collaborates in the development of individualized education programs (IEPs), and communicates progress with students, parents, and other team members.

54. Teaching and the Profession - The Teacher Candidate maintains accurate records of student performance. 55. Teaching and the Profession - The Teacher Candidate is punctual for school and class, prepared and focused on

teaching.


Domain 6: Assessment of Candidate Diversity Proficiencies Assessment #4: Competency Appraisal – Specific sections Assessment #6: Portfolio – Specific sections Diversity has also been added to an exit survey – disaggregated #4 and #6 and survey data to follow by October 1st.


Domain 7: Assessment of Candidate Technology Proficiencies Exit Survey Assessment #6: Portfolio – Specific sections Technology exit survey data to follow by October 1st along with disaggregated portfolio data


Section V: Use of Assessment Results to Improve Candidate and Program Performance

1. Describe the steps that program faculty have taken to use information from the key assessments for

improvement of both candidate performance and of the program. Provide information for each tool related to the domains from section III: (1) content knowledge, (2) professional/pedagogical knowledge, (3) field experiences, (4) candidate impact on student learning, (5) candidate dispositions, (6) candidate diversity proficiencies, and (7) candidate technology proficiencies. If the program does not have an assessment in place for any of the required domains, please provide a rationalization for why the component is not applicable to the program or how the program is working toward developing an assessment for that category.

Please see below.

2. Reflect on the previous year’s assessment and recommendations from the NCE Assessment Council review (2010-2011). How did/can the program use the feedback? Were changes to the program’s assessments/ assessment system implemented? What progress has been made? Feedback from the NCE Assessment Council for your program can be located on the I drive. Follow this pathway: Councils and Committees folder, Assessment Council folder, 2011-2012 folder, Program Assessment Reports 2010-11 folder, 2010-11 Assmt. Report Reviews folder.

We added exit surveys to address the areas of data weakness.

3. Describe how the assessment data inform the program of candidate achievement related to the NCE Conceptual Framework/Outcomes.

Please see related chart above

4. Based on the program’s 2011-12 candidate and program data, describe recommendations or changes the

program could/will make related to the program’s assessment system and curriculum.

We intend to create separate instruments using the guidelines provided by Assessment Council, especially in the area of Dispositions.


National College of Education 2011-2012 Rubric for Reviewing Program Assessment Reports

I. Program Overview ___ Student data table from section I is complete Section II. Relationship of Assessments to Program Outcomes and Standards No Evidence Some Evidence Acceptance Evidence

Relationship to Standards

The report does not communicate the relationship between program outcomes, NCE conceptual framework, and all relevant professional standards. The report does not show the alignment between the major program assessments and professional standards.

The report partially communicates the relationship between program outcomes, NCE conceptual framework, and relevant professional standards. The report does not show the full alignment between the major program assessments and professional standards.

The report clearly and fully communicates the relationship between program outcomes, NCE conceptual framework, and relevant professional standards. The report shows full alignment between the major program assessments and professionals standards.

Narrative Feedback: Section III: Key Program Assessments No description Insufficient description Sufficient description Description of key program assessments

The report does not describe the key assessments (name, type of assessment, and administration procedures) that align with each assessment domain.

The report partially describes the key assessments (name, type of assessment, and administration procedures) that align with each assessment domain.

The report fully describes key assessments (name, type of assessment, and administration procedures) that align with each assessment domain.

Narrative Feedback: Section IV. Assessment Tools and Data

Assessment of Content Knowledge (Note: Non SPA programs do not have state licensure tests.) No evidence Some evidence Acceptable evidence Strong evidence

Content Knowledge

There is no evidence for assessing candidates’ content knowledge.

There are some, but insufficient evidence for assessing candidates’ content knowledge.

The assessment tool(s) and data provide acceptable evidence for assessing all aspects of candidates’ content knowledge.

The assessment tool(s) and data show multiple kinds of evidence and/or data collection points for assessing all aspects of candidates’ content knowledge.

Narrative Feedback:


Assessment of Professional Knowledge: Pedagogical Content Knowledge No evidence Some evidence Acceptable evidence Strong evidence

Professional Knowledge

There is no evidence for assessing candidates’ pedagogical content knowledge and professional skills required by the state and national standards.

There are some, but insufficient evidence for assessing candidates’ pedagogical knowledge and professional skills required by the state and national standards.

The assessment tool(s) and data show acceptable evidence for assessing candidates’ mastery of pedagogical content knowledge in subjects and context and professional skills in working with students, families, and professional communities.

The assessment tool(s) and data show multiple kinds of evidence and/or data points for assessing candidates’ mastery of pedagogical content knowledge in subjects and context and professional skills in working with students, families, and professional communities.

Narrative Feedback: Assessment of Field Experiences No evidence Some evidence Acceptable evidence Strong evidence

Field Assessment

There is no evidence for assessing candidates’ field experience and practice.

There is some, but insufficient evidence for assessing candidates’ field experience and practice.

The assessment tool(s) and data show acceptable evidence for assessing candidates’ field experience and their professional skills in designing, assessing, implementing lessons, and self-reflections.

The assessment tool(s) and data show multiple kinds of evidence and/or data points for assessing candidates’ field experience and their professional skills in designing, assessing, implementing lessons, and self-reflections.

Narrative Feedback: Assessment of Candidate Impact on Student Learning No evidence Some evidence Acceptable evidence Strong evidence

Impact on Student Learning

There is no evidence for assessing candidates’ impact on P-12 student learning.

There is some, but insufficient evidence for assessing candidates’ impact on P-12 student learning.

The assessment tool(s) and data show acceptable evidence for assessing candidates’ ability to use assessment techniques and data to document P-12 student learning and use data to differentiate teaching and instructions.

The assessment tool(s) and data show multiple kinds of evidence and/or data points or assessing candidates’ ability to use assessment techniques and data to document P-12 student learning and use data to differentiate teaching and instruction.

Narrative Feedback:


Assessment of Candidate Dispositions No evidence Some evidence Acceptable evidence Strong evidence

Dispositions

There is no evidence for assessing candidates’ professional dispositions.

There is some, but insufficient evidence for assessing candidates’ professional dispositions.

The assessment tool(s) and data show acceptable evidence for assessing candidates’ professional dispositions in various areas and settings.

The assessment tool(s) and data show multiple kinds of evidence and/or data points for assessing candidates’ professional dispositions in various areas and settings.

Narrative Feedback: Assessment of Candidate Diversity Proficiencies No evidence Some evidence Acceptable evidence Strong evidence

Diversity

There is no evidence for assessing candidates’ diversity proficiencies

There is some, but insufficient evidence for assessing candidates’ diversity proficiencies.

The assessment tool(s) and data show acceptable evidence for assessing candidates’ understanding of diverse learners, diverse contexts, as well as proficiency in confronting issues of diversity that affect teaching and student learning.

The assessment tool(s) and data show multiple kinds of evidence and/or data points for assessing candidates’ understanding of diverse learners, diverse contexts, as well as proficiency in confronting issues of diversity that affect teaching and student learning.

Narrative Feedback: Assessment of Candidate Technology Proficiencies No evidence Some evidence Acceptable evidence Strong evidence Technology There is no evidence

for assessing candidates’ technology proficiencies

There is some, but insufficient evidence for assessing candidates’ technology proficiencies.

The assessment tool(s) and data show acceptable evidence for assessing candidates’ use of technologies to support teaching and professional development as well as candidates’ reflection on social, ethical, and legal Issues of using technologies.

The assessment tool(s) and data show multiple kinds of evidence and/or data points for assessing candidates’ use of technologies to support teaching and professional development as well as candidates’ reflection on social, ethical, and legal Issues of using technologies.

Narrative Feedback:


Section IV. Assessment Tools and Data Analysis Quality of Assessment Tools Low quality Medium quality High quality

Quality of Assessment Tools

All or some assessment tools are not reflective of the target program outcomes. The tools lack variety, and there is no information on the assessment reliability and validity.

Most assessment tools are reflective of the target program outcomes. A few different assessment tools are involved and there is some but limited reliability and validity information.

All assessment tools are reflective of the target program outcomes. A variety of assessment tools were used and there are good indications of assessment reliability and validity.

Narrative Feedback: Data Analysis and Presentation Low quality Medium quality High quality

Data Analysis and Presentation

Data analysis is not appropriate or efficient and data presentation is disorganized.

Data analysis is generally appropriate and the data presentation is relatively efficient.

Data analysis is sufficient in understanding all aspects of collected information and data presentation is organized, clean, and easy to follow.

Narrative Feedback: Section V. Use of Assessment Results to Improve Candidate and Program Performance Low quality Medium quality High quality Use of Assessment Data

Interpretation of the assessment data is not included or limited. It is not clear how the program use assessment data to support its improvement in curriculum and services.

Interpretation of the assessment data is relatively efficient and the assessment data has been used somewhat to plan for program improvement in curriculum and services.

Interpretation of assessment data is fairly efficient and the assessment data has been used meaningfully to plan for program improvement in curriculum and services.

Narrative Feedback: Low quality Medium quality High quality Use of Assessment Data Related to NCE Conceptual Framework/ Outcomes

Interpretation of the assessment data related to the NCE Conceptual Framework/Outcomes is not included or limited. It is not clear how the program uses assessment data to assess candidates are meeting the NCE Conceptual Framework/Outcomes.

There is some interpretation of the candidate assessment data related to the NCE Conceptual Framework/ Outcomes that demonstrates how candidates are meeting the NCE Conceptual Framework Outcomes.

The program has meaningfully described how it interprets the candidate assessment data related to the NCE Conceptual Framework/Outcomes. There is a plan for program improvement related to the NCE Conceptual Framework/Outcomes explicitly outlined in the narrative.


Narrative Feedback: Section VI. Report Writing and Organization Low quality Medium quality High quality Writing and Organization

The report’s overall writing needs improvement. The organization of the report is not efficient (e.g., sections are not clearly labeled; the assessment description, data presentation, and data interpretation are not well organized.

The report’s overall writing is clear, but the organization needs further improvement.

The report is well written and organized. All information is clearly specified and labeled.

Narrative Feedback: Notes from Reviewers:

nce annual program assessment information, template and … special... · nce annual program...

Documents