Changing the way we prepare teachers: A bridge to excellence

Mary Little, Martha Lue-Stewart,Suzanne Martin ,and Jillian GourwitzUniversity of Central Florida

July, 2013Preservice * Initial Teacher * Professional Development

Recruitment - Mentoring - Technology

BRIDGECollaborative Program Evaluation to Bridge Theory into Practice

AbstractDeveloping data systems that continuously measure student and program success by university and school district educators have the potential to not only bridge the research to practice gap, but also inform programmatic decision-making. Our purpose is to discuss data-based decision making processes among key stakeholders of a 325T project.

Critical QuestionsThe relevant questions posed to the audience include:How do we actualize our action plans and logic models through implementation, evaluation, and continuous improvement?How do we bridge the research-to-practice continuum for evidence-based instruction, assessment, and program evaluation?What procedures, practices, and strategies facilitate continuous, collaborative program evaluation focused on student impact?

To enhance existing, approved initial certification programs to prepare highly-qualified teachers (HQT) of students with high incidence disabilities (HID) within the Masters program. The major content enhancements are: math and science instruction for students with HID including Culturally and Linguistically Diverse (CLD) students in co-taught and resource settings within urban middle schools that have not met Adequate Yearly Progress (AYP) targets.Purpose of 325T Project-University of Central Florida

Improve recruitment to increase the number of highly qualified teachers in ESE, especially from diverse backgrounds;Enhance and implement an effective preparation program focusing on math and science content instruction at the middle school level within the ESE Masters and certification to assure HQT status and skills to successfully teach in urban settings using culturally responsive teaching (CRT) practices;Improve mentoring and induction; and,Implement technology to support overall program effectively.

325T Project Goals-UCFIncrease in the number of HQT, knowledgeable of research-based practicesUCF Faculty, Bridges Program Staff & UCF StudentsMiddle/High schools-DistrictsHQ/NBCT Teachers, Students, CounselorsParentsState and Community Partners FLDOE, CEC, NMTE, NSTE, Industry, VolunteersTechnology -Lab facilities & equipmentFLDOE-ATENUSDOE Cooperative Funding & IRIS, FLDOE and UCF ResourcesAssemble Stakeholders & Solicit InputModel Exceptional Education Graduate Program (multiple pathways-content and delivery)Curriculum modules aligned with state/federal standards for HQTUCF ESE Students prepared to use technology in classroomTechnology: Recruitment, Instruction, Coaching/ Mentoring (especially CLD)Develop Virtual Activities Mentoring, Instruction, CollaborationEstablish Bridge Program TeamReview and Revise ESE Graduate Curriculum Integrate New TechnologiesRecruit ESE Graduate Students & Gather Baseline DataUCF ESE Students prepared to teach ESE students in Math/ScienceIncrease the retention of highly qualified teachers into schoolsEstablish data collection system to analyze/revise program Dissemination& PresentationsIncrease in teachers confidence integrating technology in classroomIncrease in the technology for program communication and improvementIncrease the knowledge and skills in math and science of teachersIncrease in teachers understanding of math/science conceptsImproved Mentoring and Induction ActivitiesIncreased knowledge of math/science contentEffective use of technology to support overall program effectivelyBridges Logic Model

Resources Activities Outputs Outcomes ImpactsACCOMPLISHMENTS TO DATEGOAL1:Improve recruitment:Program applicants increased almost 200% in initial year due to strategic marketing, with increases in diversity.GOAL 2:Enhance and implement an effective preparation program focusing on math and science:M.Ed. includes Mathematics endorsement courses, co-teaching/co-planning experiences in content areas, infusion of evidence-based practices, fully on-line delivery.GOAL 3: Improve mentoring and induction; Currently piloting national e-Mentoring program while collaborating with area school districts to enhance current mentoring activities.GOAL 4: Implement technology to support overall program effectively. Synchronous and asynchronous opportunities enhance recruitment, advisement, course instruction and delivery, mentoring, and communication.Guiding Question How do we enhance, co-create, and sustain a system of school improvement focused on improving P-12 student results by our teacher candidatesthat meet the current policies, mandates, and expectations for use of data, continuous progress monitoring, and evidence-based instructional and intervention practices within schools and classrooms?Changing Focus about DataTHAT WAS THENDecisions were based on judgments of the people in authority.Multiple data sources were not readily available.Assessment and program evaluation measures were inputs to be counted, rather than impacts to be measured.

THIS IS NOWIn this Information Age, there is an exponential increase in assessments and data sources.Accountability is publicly mandated and reported.However, educators often feel woefully underprepared to engage in data-based problem solving.Earl & Katz, 2006

Accountability through Informed Professional Judgment (Barber, 2001)Thinking Differently about DataLeveraging this data-systems investment into sustainable gains in student achievement requires a parallel investment in strengthening the capacity to use data to improve teaching and learning. Without this capacity, student achievement will continue to stagnate, and we can all expect the next flurry of reports indicating that Americas schools are falling short. Slotnik & Orland, 2012, pg.13

The Importance of Good DataPatty12Questions about DataHow do you know it is an issue to address?What do you need to know to tackle the issue? What are good data?What data sources do you have?What else do you need to know?What other data sources do you need? Who has the knowledge and expertise to develop, implementation, evaluation, and continuous improve program evaluation systems?How will the data collection system be sustained?13

Improve recruitment to increase the number of highly qualified teachers in ESE, especially from diverse backgrounds;Enhance and implement an effective preparation program focusing on math and science content instruction at the middle school level within the ESE Masters and certification to assure HQT status and skills to successfully teach in urban settings using culturally responsive teaching (CRT) practices;Improve mentoring and induction; and,Implement technology to support overall program effectively.

325T Project Goals-UCFGoal 1: Improve Recruitment and Retention, especially CLDBARRIERSSOLUTIONSTime to identify all sources of data related to the identified issue and goal.Data systems not coordinated by system, office, questions, etc.Data re: specific question not collected (CLD status)Knowledge re: issues and data request not understood.

Identified all resources, sources of data Collected all data in one uniform set of data for program area.Collaboration among PIs and external evaluator to develop, implement, and report data re: our question.Developed our procedures as not part of system-wide data collection

UCF Exceptional Education Programs

Exceptional Students Education homepage

Improve recruitment to increase the number of highly qualified teachers in ESE, especially from diverse backgrounds;Enhance and implement an effective preparation program focusing on math and science content instruction at the middle school level within the ESE Masters and certification to assure HQT status and skills to successfully teach in urban settings using culturally responsive teaching (CRT) practices;Improve mentoring and induction; and,Implement technology to support overall program effectively.

325T Project Goals-UCFHow do we, in universities, improve P-12 student learning? How do we bridge the research-to-practice continuum for evidence-based instruction, assessment, and program evaluation?.Bridging the gap. 19

Collaborative Partners for Program Evaluation

Liz20Collaborative PartnersIHE-basedExternal EvaluatorsDoctoral ResearchersDoctoral Students (Ph.D. and Ed. D.) in Special Education (Program Planning and Survey Development)Student Services PersonnelAlumni StaffFaculty across CollegesDirector of AccreditationData-Systems Personnel

District/State-basedEducator-in-ResidenceAdvisory CommitteesDoctoral Students (Ed.D.)District AdministratorsOther 325 T projectsFDOE personnelSchool district teachers (former UCF graduates)Goal 2:Implement EBPs to Impact P-12 Student LearningBARRIERSSOLUTIONSTime to identify all sources of data across multiple conditions.Identification of EBPs (isolation; then, integration)P-12 data systems not coordinated / not available by system, office, questions, etc.Innovation Configurations (ICs) developed to Internship

Identified all resources, sources of data NCIPP, other 325T shared EBPs; UCF faculty retreat.Collaboration among PIs, UCF accreditation, and district personnel to develop, implement, and report data re: our question.Developed pilot ICs in mathematics and science and extended to P-12 settings.

Overview of Professional Learning and DevelopmentUCF-Masters of Arts Graduate Program

UCF Conceptual Framework of a Reflective Practitioner is Foundation of Program.Overview of Professional Learning and DevelopmentUCF-Masters of Arts Graduate Program-DRAFT 1.18.12

TASKSTEAM COMPLETIONFOLLOW-UPMentoring and Induction

-Locate/Receive Feedback from Grads/OCPS-Develop Mentoring Site /Determine Activities/Staffing-Dr. Little, Kelly Grillo, July, 2012Share site with graduates for feedback-Evaluation Plan-325TSkillful Application during Internship

-Collect/align preservice authentic assignments with coursesRe-do EEX 6946 site-Dr. Little, Katie, OCPS colleagues, (Ingrid?) Tech AssistanceMay, 2012Share site with Cooperating teachers for feedback-Evaluation Plan-325TPreservice Courses

-1. Updates to M.A. Courses (Advanced CEC Standards, Revised FEAPs, Adv. FEAPs, etc.)-2-Infuse technology tools into courses/modules1. Dr. Martin, Sandy Grider, Doc students (class) with course shepherd

2. Dr. Hines, Doc students (class) with shepherdBy Feb. 20, 2012

By March 30, 2012Content and Module Updates-All courses: M. Kinney and M. LittleFrom Program Evaluation to Significance of ResearchThe goal of this current research project within our 325T grant research is to study the knowledge, beliefs, and implementation of evidence-based instructional practices in mathematics related to culturally responsive teaching by graduates of a re-designed teacher preparation graduate program in a large urban university.

This research is guided by the following questions:-To what degree are preservice teachers beliefs and practices impacted by our work related to CRT?-To what degree and quality are evidence-based practices in mathematics implemented with fidelity in urban middle school classrooms to met instructional goals for all students?-To what degree are graduates knowledgeable in EBP in Response to Intervention within the College of Education?-To what degree are graduates knowledgeable and meeting requirements as a HQ teacher with improved achievement for students with and without disabilities?

Research Questions

This research study is currently conducted in phases:Research and program re-design to include and enhance evidence-based practices in mathematics, Response to Intervention, and culturally-relevant teaching practices within preservice courses;Research and develop logic model for program evaluation, using Concerns-Based Adoption Model (Hord et al., 2006) and Innovation Configuration (Reschly et al., 2009); Complete research using mixed methods research design within middle schools in a large urban school district; andEvaluate and disseminate findings related to the final goal leading to the enhanced learning outcomes of students with racial, cultural, ethnic, and linguistic diversity (RCELD) in mathematics.

Methodology28RESOURCES for Identifying Effective PracticesFrom Technical assistance and dissemination projects; USDOE1. Regional Resource and Federal Centers The Federal Resource Center for Special Education (FRC) http://www.rrfcnetwork.org 2. Northeast Regional Resource Center (NERRC) 3. National Research Center on Learning Disabilities (NRCLD) http://www.nrcld.org4. Technical Assistance Center on Social Emotional Intervention for Young Children (TACSEI) http://challengingbehavior.org 5. Access Center: Improving Outcomes for All Students k-8 http://www.kbaccesscenter.org 6. National Center on Student Progress Monitoringhttp://studentprogress.org 7. Center for Positive Behavioral Interventions and Supports (PBIS) http://www.pbis.org 8. The National Center on Response to Intervention (RTI) http://www.rti4success.org 9. Center on Instruction http://www.centeroninstruction.org10. IRIS Center for Faculty Enhancement http://iris.peabody.vanderbilt.edu

And.11. Current Practice Alerts http://teachingld.org 12. National Comprehensive Center for TeacherQuality Innovation Configurations www.tqsource.org13. Narrative reviews/Literature Reviews/Synthesis Collected evidence for practices in areas of: Math*Content Learning: HistoryContent Learning: Science*Foundational ReadingSecondary ReadingInclusionary PracticesAssessment PracticesBehavior PracticesWriting InstructionExplicitly identify EBPs (Regan, 2010)28Degree of EBP Knowledge and Use: To what degree are the practices/items listed here generally addressed in the identified course?

EBP Rating Scale for Courses, Clinical Experiences:

No knowledge of EBPHave heard of EBPCould define EBPCould describe and model EBPHave implemented EBP at UCF or my classroomHave student impact data from implementation of EBP

2945 Evidence-Based Practices identified across Literacy, Instructional Strategies, Inclusionary Practices, Assessment, and Behavior.EBPs then fit into the relevant course (s) Course leads of the relevant licensure classes rated the degree of coverage for EBPs: NOTE: adapted scale from Innovation Configuration Tools out of http://www.ncctq.org

Methodology-EBP-MathEVIDENCE-BASED PRACTICES-MATHEMATICSAll syllabi and internships have been enhanced: EBP and CRT.Using the meta-analyses of effective programs in mathematics conducted by Slavin, Lake, & Groff (2009) (content) and Innovations Configurations (Hord et al, 2006; Reschly et al., (2009), implementation matrices were developed.Delphi Procedure Analyses were completed for the implementation matrices by representatives from the following groups: parents of students with RCELD, administration, general education teachers, ESE teachers, school psychologists, other instructional support staff.Multiple sources of data collection in process.

InstrumentsPre/post surveys and Outcome measuresThe Levels of Use and Implementation Matrix Survey and Observation Tools will be used with all participants to assess implementation. Both tools have described technical adequacy (e.g., validity and reliability).Results of state certification exams will be collected (mathematics and exceptional student education)Review of instructional artifacts / course will be reviewed for evidence of EBP in mathematics and use of CRT practices.Surveys created to meet goals of Program Evaluations.Data from university and college outcome data sources have been targeted for collection and aggregation to provide data re:Satisfaction surveys (first year teachers and principals)Re-hire surveys (principals)Sample-IC Rubric-Mathematics

Initial Results-201290% of all ESE Graduate interns implementing EBP in content as researched during courses and field placements.100% of all ESE Graduating students were satisfied or very satisfied with preparation in all areas of Floridas Educator Accomplished Practices upon completion of program in year 1 of teaching.Increases of Graduate students passing middle grades math certification test noted.Principal re-hire data reported a 100% rehire rate.Results from RTI Research across COE.What procedures, practices, and strategies facilitate continuous, collaborative program evaluation focused on student impact?

Multiple Opportunities for Performance Learning for Student ImpactApplied action research in multiple coursesEEX 6612: Behavior Change Projects (FBA I)EEX 6107: Action Research (I)EEX 6946: Behavior Change Projects and Action Research within Clinical Experiences (FBA II and AR II)Student impact data collected as part of Graduate Internship Teacher Work Sample (M.A.)Case studies on video dissemination (with permission)Graduate student artifacts included into Livetext accounts (B.S., M.A., and M.Ed.)Externally-funded grants

Candidates' Impact on P 12 Student Learning*Computed percentages based upon project completions As compared with rubric for action research projects*(1) Exceeds Criteria/ At Target(85+)71%(2) Meets Criteria/Acceptable(70-84)25%(3) Below Criteria in Developing(51-69)4%(4) Unacceptable (Below 50) *These assignments were redone*0%Grade distribution percentages 2011

Semester: Fall 2011Number of Candidates: 24

*Grades based upon criteria from rubrics entered into livetext system. The following narratives summarize characteristics from all action research k-12 student impact data including: research question, student grade level(s), Number of students, content Area, description of students, Intervention/assessments, results, and student grades*K-12 Student Impact: Results from Action Research: EEX 6107Action ResearchEEX 6612 Behavior Change Project: 2008 to 2011 N=298 University Students ParticipatingData reflect Percent of Non-Overlapping Data points on Single Case Designs for K-12 ChildrenDisaggregated EffectsThe majority of students demonstrate highly effective outcomes for K-12 children on problem behavior.These data are similar to Cohens D.Goal 3: Improve Retentionusing E-mentoringBARRIERSSOLUTIONSData systems not coordinated by system, office, questions, etc.Data re: specific question not collected (retention, either within our institution or after graduating)Difficult to track graduates as official email discontinued as of graduation.

Identified all resources, sources of data Retention data within programs at UCF now being collected.Collaboration among school district teachers, Technology consultants, and district partners to develop, implement, and report data re: our question.Developed our procedures as not part of system-wide data collection BEFORE graduation.

Goal 4:Use of TechnologyBARRIERSSOLUTIONSTime to identify all sources of data related to the identified issue and goal.Data systems not coordinated by system, office, questions, etc.Data re: specific question not collected (Use of technology)

Collected all data in one uniform set of data for program area.Collaboration among PIs and external evaluator to develop, implement, and report data re: our question.Developed our procedures as not part of system-wide data collection.

Student Preference for Use of TechnologiesNext StepsContinue data collection, analyses, and reporting to multiple accrediting agencies;Continue development of collaborative school sites;Finalize all instrumentation, designs, methods, etc.Complete research and OSEP Final Report;Disseminate findings to multiple stakeholders (e.g., ESE Program faculty, CED, OSEP, and others)Extend process to additional content areas and schools to enhance and sustain process, and Comprehensively align entire Graduate Program to meet multiple accountability requirements and to assure sustainability after grant funding ends.

Comprehensive Program Evaluation

Opportunities for Special EducationSchools require a leveraging system that targets resources toward children at vulnerable moments; giving special educators a major role in this new system can redirect their efforts and create new roles for them as mediators and collaborators, who working with others, facilitate childrens entry (and successes) in the school community.Teacher PreparationRobust teacher preparation for special education teachers in the educational curriculum and working experiential knowledge of the best classroom pedagogies develop special education teachers that are well-prepared to work alongside general educators to assure that general education contexts become places where the majority of our children has strong, successful educational experiences. Their unique, explicit work as special educators would follow from this rich base of teacher preparation.Seidl, B. & Pugach, M. (2009). Support and teaching in the vulnerable moments: Preparing special educators for diversity. Multiple voices, 11(2), Spring, 2009.ReferencesCross, B. (2003). Learning or unlearning racism: Transferring teacher education curriculum to classroom practices. Theory into Practice, 42(3), 203-209. Fieldler, C.R., Chiang, B., Van Haren, B., Jorgensen, J., Halberg, S. & Bereson, L. (2008). Culturally responsive practices in schools: A checklist to address disproportionality in special education. Teaching Exceptional Children (40)5, 52-59. Freire, P. (2007/1997). Pedagogy of the oppressed. New York: The continuum international publishing group, Inc.Freire, P. (2007/1997). Human rights and liberating education. In A. M. A. F. (Ed.), Daring to dream: Toward a Pedagogy of the Unfinished. (pp. 57-71). Boulder: Paradigm Publishers.Garcia, S. & Guerra, C. (2004). Deconstructing deficit thinking: Working with educators to create more equitable learning environments. Education and Urban Society, 36(2), 150-168.Gardner, D. (2007). Confronting the achievement gap in In K. Ryan & J. M. Cooper (eds.) Kaleidoscope: Contemporary and Classic Readings in Education. (pp. 154-159). Belmont, CA: Wadsworth, Cengage Learning. Gay, G. (2000). Culturally Responsive Teaching: Theory, Research, & Practice. New York: Teachers College. Hall, S. M., Stiegelgelbauer, S.M., Hall, G. E., & George, A.A. (2006). Measuring implementation in schools: Innovation configurations. Southwest Educational Development laboratory. Univeristy of Texas at Austin; Author.Regan, K. (2010.) Special Education Pre-Service Training Improvement ProjectImproving Course Content and Syllabi Revision. Office of Special Education Programs. Washington, DC: Author.Reschly, D., Oliver, R., Schumaker, J., Smartt, S., & Holdheide, L. (2009). Innovation configurations: Implementing evidence-based practices in teacher preparation. Retrieved from the National Comprehensive Center for Teacher Quality: http://www.tqsource.orgSiedl, B. & Pugach, M. (2009). Support and Teaching in the Vulnerable Moments: Preparing Special Educators for Diversity, Multiple Voices, 11(2), Spring 2009Slavin, R.E., lake, C., & Groff, C. (2009). Effective programs in middle and high school mathematics: A best-evidence synthesis. Review of Educational Research, 79(2), 839-911.

Chart110095609996100758386861008675821019781969594899972100

Student GradesGrade Distribution 2011

Sheet110095609996100758386861008675821019781969594899972100

Sheet2

Sheet3