do you have the evidence you need for caep?...assessment ∙accreditation ∙e-portfolios with the...

Carolyn LofferLecturer – Department of Literacy, Language, & Culture

Sherry DismukeClinical Assistant Professor – Department of Curriculum

& Instruction/Foundational Studies

Ben Coulter, Ed.D.Senior Director – Campus Solutions

March 2016

Do You Have the EvidenceYou Need for CAEP?

assessment ∙ accreditation ∙ e-portfolios

With the implementation of Taskstream we are able to spend

much less time managing the data and much more time analyzing it and

figuring out how our students are doing.”

Dr. Fred Freking

University of Southern California

“

About Us…

• Founded in 2000

• HQ: New York City

• 500+ client institutions

• Experienced software developers

• Passionate about education

• Dedicated to client success

Manage the full cycle of assessment

AMS

Accreditation Planning &

Documentation

LAT

Portfolio-based Assessment &

Longitudinal Reporting

Aqua

Streamlined Direct Assessment of

Learning Outcomes

Solutions to support you no matter where you are on your assessment journey…

Today’s Presentation

• Taskstream tools related to CAEP Standards

▫ edTPA™▫ e-Portfolios▫ Clinical Practice

• Implementation highlights: Boise State University

▫ Reflections on recent CAEP onsite visit▫ Impact on engagement▫ Looking ahead…

Centralized Processes in Taskstream

• Assessment/data collection/artifact repository

• Longitudinal performance reporting

• edTPA™

• e-Portfolios

• Clinical Practice

• Integration with LMS

• Document/Evidence Quality Assurance

• Facilitate a community of practice & engagement

Standard 1

Content and Pedagogical Knowledge

Collect Evidence from Candidates

• Key assessments/ outcomes

• Organized by course, transition points, other requirements (SPA, state, standards, etc.)

• LMS integration (course-embedded assignments)

Upload, Create and Edit Rubrics

Attach Standards to Facilitate Reporting

Align Evaluation Methods

Create Custom Evaluation Forms

Create Custom Evaluation Forms

Leverage LMS Integration

• Doctoral Degree Granting University

• Mid-size, Public, Metropolitan University; 18,000 students

• College of Education

• NCATE accredited, early CAEP adopter, anticipating accreditation

• 55 tenure track faculty

• 11 clinical faculty

• 5 departments in College of Education

• 15 departments across University

• 1,409 initial program completers 2009 - 2014

• 53 programs

Institutional Perspective

Collecting Evidence (Building a Culture of Inquiry)

• From designing a better “milk crate”...

standards

evidence/signature assignments

proficiency development over time

developing lines of evidence

• … to re-culturing a college.

Lines of Evidence

Lines of Evidence

Path to Proficiency

See additional resource for full image

Initial Certification Template

edTPA™

2007Taskstream begins working with clients to support the CalTPA and PACT in California

2000Taskstream begins supporting local assessments in teacher prep programs nationwide

2011AACTE names Taskstream the Preferred Technology Partner for the TPAC pilot and field test

2012Taskstream passes certification to become an integrated edTPA™ platform provider with Pearson

Taskstream Experience with edTPA™

• Extensive experience

• Pre-built templates with updated resources

• Access to submitted portfolios

• Monitoring tools

• Support for faculty, candidates, staff

• Ability for local scoring

Benefits of Using Taskstream for edTPA™

Pre-Built Templates

Structured tasks

Pre-Built Templates

official handbooks and resources

Pre-Built Templates

Pre-Built Templates

Pearson Integration

candidates can transfer work directly to Pearson

for official scoring

Candidate Portfolio Transfer Tracking

Monitor progress of the transfer to Pearson

Local Evaluation Rubrics and Reporting

Standard Performance Assessment for Teachers

e-Portfolios

Build and Publish Portfolios

• Program completer

• Teacher Work Sample

• Summative evaluation

• Accreditation requirement

• Employment portfolios

• Personal portfolios

• Tenure/Promotion portfolios

Export or Share with Others

Standard 2

Clinical Partnerships and Practice

With more timely submission and evaluation of key student teaching assignments we have been able to

provide programs with reports about student teacher performance and student teacher satisfaction with their experience

and college supervisor.” Elvani Pennil

Director of Field Experiences and the Professional Development NetworkLehman College, School of Education

“

Manage Data about Partner Schools

Manage Placements

Custom Online Evaluation Forms

Custom Online Evaluation Forms

Custom Online Evaluation Forms

Report Placements by Supervisor

Demonstrate Diverse Placements

Standard 3

Candidate Quality, Recruitment, and Selectivity

Manage Transition Points

• Admission requirements

• Monitor candidate progression

• GPA and test score import

Applications and Interviews

Aggregate Performance Reporting

Disaggregated Results

Reporting on Embedded Standards

Filter Reports by Demographics & Dates

Multiple Evaluators – Inter-Rater Reliability

Comparative Reporting and Distribution

Standard 4

Program Impact

Survey Completers and Employers

Other Approaches…

• Teacher Work Sample

• edTPA™

• In-service field observations

• Promotion and retention data

• Pre/post data sharing (State/LEA)

Other Approaches…

Breakout Session: 3:00 p.m. (Indigo E)

Where Are They Now?Following Up With Program Completers

for Multiple Layers of Inquiry This presentation will share an initial case study of EPP completer

performance in terms of professional community, influencing student learning, and inquiry across the professional life span. Using multiple measures for triangulation, this comprehensive case study collected

evidence on teacher performance and potential with a focus on development

Sherry Dismuke and Carolyn Loffer

Standard 5

Quality Assurance and

Continuous Improvement

5.1 The provider’s quality assurance system is comprised of multiple measures that can monitor candidate progress, completer achievements, and provider operational effectiveness. Evidence demonstrates that the provider satisfies all CAEP standards.

5.2 The provider’s quality assurance system relies on relevant, verifiable, representative, cumulative and actionable measures, and produces empirical evidence that interpretations of data are valid and consistent.

5.5 The provider assures that appropriate stakeholders, including alumni, employers, practitioners, school and community partners, and others defined by the provider, are involved in program evaluation, improvement, and identification of models of excellence.

CAEP Standard 5

Quality Assurance Planning in Taskstream

• Identify EPP Outcomes

• Define Measurement(s)

• Align/Map

• Document Findings/Data

• Evidence Actions

Custom Planning Templates

Standard One-Candidate Knowledge Skills & Prof. Dispositions

Candidates preparing to work in schools as teachers or other school professionals know and demonstrate the content knowledge, pedagogical content

knowledge and skills, pedagogical and professional knowledge and skills, and professional dispositions necessary to help all students' learn. Assessments

indicate that candidates meet professional, state, and institutional standards.

1g. Professional Dispositions for

all Candidates

Candidates work with students, families,

colleagues, and communities in ways that

reflect the professional dispositions

expected of professional educators as

delineated in professional, state, and

institutional standards. Candidates

demonstrate classroom behaviors that

create caring and supportive learning

environments and encourage self-directed

learning by all students. Candidates

recognize when their own professional

dispositions may need to be adjusted and

are able to develop plans to do so.

Measure: Dispositions Evaluation Rubric

Program level; Direct - Other

Details/Description: The purpose of assessing professional dispositions throughout candidates’ preparation is to

bring those affective dimensions (teaching and working, learning, communication, outward attitude, and other

professional dimensions related to education and life) into the candidates’ consciousness so that those

dimensions manifest suitable performances. The unit evaluates for effect with the objective to ensure dispositions

are not just cosmetic or short-lived but ingrained ways of behaving. Assessing professional dispositions and

making them a focus throughout the Unit Assessment System’s five transition points allows time for candidates to

reflect on their dispositional actions, formatively and possibly change what is within their power to change:

themselves.

Ideal Target: 100% of all elementary education candidates at graduation/certification will be at a score of 2.0

(approaching) or higher on all dispositions.

Implementation Plan (timeline): Plans are in place that include interventions and requirements for those

candidates who consistently demonstrate dispositional behavior at an unacceptable scoring level on any particular

disposition.

Key/Responsible Personnel: Evaluators: faculty, cooperating teachers, candidates (self-evaluation).

Aggregation and Collection of Data: Elementary Education Program Director and CAEP Coordinator

Supporting Attachments:

Disposition Evaluation Rubric (Microsoft Word)

Define methods and measures to assess chosen outcomes

Identify standards to be

assessed

Planning Outcomes and Measures

Standard One-Candidate Knowledge Skills & Prof. Dispositions

Candidates preparing to work in schools as teachers or other school professionals know and demonstrate the content knowledge, pedagogical content

knowledge and skills, pedagogical and professional knowledge and skills, and professional dispositions necessary to help all students' learn. Assessments

indicate that candidates meet professional, state, and institutional standards.

1g. Professional Dispositions for

all Candidates

Candidates work with students, families,

colleagues, and communities in ways that

reflect the professional dispositions

expected of professional educators as

delineated in professional, state, and

institutional standards. Candidates

demonstrate classroom behaviors that

create caring and supportive learning

environments and encourage self-directed

learning by all students. Candidates

recognize when their own professional

dispositions may need to be adjusted and

are able to develop plans to do so.

Measure: Dispositions Evaluation Rubric

Program level; Direct - Other

Summary of Findings: At the end of each semester, as well as annually, evaluation scores of candidates’

disposition are aggregated in an effort to find areas of needed improvement in candidates’ observable behavior in

relation to their demonstration of appropriate professional dispositions. Summary data found in the attached

tabled was conducted by multiple raters, and aggregated by program/major and by transition point. Data results

show elementary education candidates are performing at a minimum at the approaching level.

Results: Ideal Target Achievement : Exceeded

Recommendations: Continue with current processes for evaluating dispositions. Data from transition point

chart provides clear evidence that candidates in the elementary program are improving as they move through

their preparation. Scores clearly improve for all dispositions through the transition points.

Reflections/Notes: After review and reflection the Elementary Education Program faculty will continue to

evaluate dispositions using the currently outlined system as they believe the data demonstrates candidates are

graduating and being certified with the appropriate level and understanding of professional dispositions needed to

ensure personal success in the classroom as well as enhanced learning for student in the classroom.

Substantiating Evidence:

2008/2009 Aggregated Dispositions Evaluation Data by Program

This chart demonstrates disposition evaluation mean scores for the academic year.

Findings for Dispositions Evaluation Rubric

Measure

Findings

Outcome

Supporting Evidence

Findings and Results

Quality Assurance Actions

Action Item Details

Current Status

Linking Theory & Content Knowledge to Practice

K-12 Student

LearningIntegration of

Knowledge

Adhere to national, state, &

institutional standards

Practice in a

Multicultural Society

Elementary Education Signature Assignments

Assignment 1

Unit Dispositions Evaluation

Assignment 2

Praxis II Examination

Assignment 3

State/SPA Standards

Reflections

Assignment 4

Educational Autobiography

Assignment 5

Exceptionalities Field

Experience Reflection

Assignment 6

Individual Education Plan

Evaluation

P Partially Addressed A Addressed

A

A PP P

A A A A

P P P

A A A A

A A P

A

Map Alignment

Import CAEP, SPA, INTASC, State Standards, etc.

Identify Signature

Assignments

Curriculum Mapping

Curriculum Mapping

Produce Quality Assurance Plans

outcome

alignment

measure

findings

action

Prepare Evidence for Review

Present Evidence Online

College of EducationTrillium State College

Export to PDF

Quality Assurance

• Data sharing across 21 programs and multiple colleges for continuous improvement

• Exhibit Rooms

• Advanced Programs

Exhibit Rooms – Individual Programs

Exhibit Rooms – Evidence Across Programs

Advanced Programs – Common Lines

Reflections & Looking Ahead

Culture of Inquiry is the new norm, Continuous Improvement is what we do

“We found the reciprocity of our work for CI fascinating – as we improved practice to meet accreditation needs, we found that accreditation needs were helping us improve practice.”

-(Dismuke, et. al., 2015)

A data management system was the invitation that brought everyone to the table

“Working together across roles and program outcomes, we resisted the temptation to uncritically buy-in to data solutions and instead became critical consumers and designers of assessments and tools purposeful in our context.”

- (Dismuke, et. al., 2015)

Reflections & Looking Ahead

TaskstreamBen Coulter, Ed.D.Senior Director – Campus Solutions

For more information about Taskstream or to request a demonstration, contact:events@taskstream.com or 800-311-5656

Boise State UniversityCarolyn LofferLecturercloffer@boisestate.edu

Sherry DismukeClinical Asst. ProfessorCheryleDismuke@boisestate.edu

Contact Us

assessment ∙ accreditation ∙ e-portfolios