9/28/2012

1

Keck/PKAL STEM Education Effectiveness Framework Project :

Webinar I

Susan Elrod

Adrianna Kezar

September 28, 2012

Webinar Goals

• Participants will:

– Receive an update on the feedback we heard from calls with team leaders and how we are addressing issues.

– See the framework as a tool for change management that has been used in other projects successfully.

– Provide feedback on alternatives to existing Framework “grid” that is less linear and for people who like a more narrative approach.

9/28/2012

2

Announcements • RSVP for Regional Meetings TODAY

– LA: October 26

– Bay Area: November 1

– San Diego: November 30

• Upcoming Webinars

– October 25 on Best Practices (Ellen Goldey)

– November 16 on Assessment (Peggy Maki)

• First set of Framework Responses DUE Oct 12

Project Website: http://www.aacu.org/pkal/educationframework/index.cfm Project Contact: pkal@aacu.org (Christina Shute, staff support)

Update on Feedback from Calls • Projects vary in scope – perhaps we can team up campuses with

similar scope. Need completed survey responses!! • Confusion over vision, should it be national or local context? Vision

should be informed by both, we brought in outside context to expand your thinking. Suggestion: Blend challenges on campus with broad recommendations and research.

• Many campuses are supporting on-going work and framework seems to start at beginning. Perhaps we can use questions to consider the implementation and recalibrate existing efforts?

• Single framework constraining to some teams; so we built another tool. More later in the webinar

• Participants want models that may help provide a vision like for STEM reform – also have a webinar and will post resources for teams – October 25 Webinar plus links to published examples, possible book/journal club activity

• IR support differs markedly at campus so may need support around data and assessment – November 16 Webinar

9/28/2012

3

What the framework helps accomplish?

• Articulate a vision or direction for change

• Audit or evaluate where you are now and where you want to go

• Chart a path for reaching the vision and goals

• Create a common language and vision around the change

• Pinpoint needed interventions and strategies

• Foster learning

What the framework helps accomplish?

• Serves as an accountability tool for maintaining momentum on the change

• A catalyst for priority setting

• Ensure collective leadership

• A way to garner resources (financial and human) for the vision

• A way to maintain focus and momentum over the long time period change often takes

9/28/2012

4

Framework: A change management tool

Reference: Henderson and Dancy (2011) Increasing the Impact and Diffusion of STEM Education Innovations, White Paper Commissioned for “Characterizing the Impact and Diffusion of Engineering Education Innovations Forum,” Feb 7‐8, 2011.

Key Strategy for Change

• Develop strategies that shape the system as well as the faculty.

– Structures, policies, environment, culture

It is the systemic study and exploration of the reform process that offers the greatest promise for effective large‐scale research‐based reform. –Henderson and Dancy, 2011

9/28/2012

5

Vision and goals Columns A-B –Vision and goals • Designed to help teams articulate their goals

clearly so when they need to communicate them others to become allies or to get support – others understand.

• Thinking as big as makes sense but try to stretch with goals. Those who are thinking about a broad campus initiative will be better served by looking at the RISE framework in the meeting notebook. What we know is that people that do not fine-tune their direction, can not communicate to others, recruit allies and

support, end up getting lost in the process.

Example of item A/B

• Vision: Improved access, retention and graduation of underrepresented minority and women from STEM

• Goal: Increase majors who are underrepresented minority and woman (also examine within specific majors); increase in retention rate; increase in students who come with a STEM interest from these groups

9/28/2012

6

Measures/benchmarks

Column C & G

• Measures/benchmarks -- If campuses do not think about measures or ways to know achieved, later hard to demonstrate support and get ongoing support

What we know is support will wane if you can not document progress and results.

Example of Benchmark/measures

• Increased major who are underrepresented minority 15% and woman from 30% to 30% minority and 50% women (also examine within specific majors); increase in retention rate from 40% to 65%; increase in students who come with a stem interest from these groups from 10% to 30%

• Have task forces annually report evaluation results; help them work with IR to collect data; plan for a summary of effort after three years

9/28/2012

7

Current status

Column D – Status

• Understand current state on issue trying to make progress – helpful to make sure building off existing efforts; also understand and determine challenges; use data to examine issue – use resources we provided on assessment.

What we know is people tend to move too quickly before understanding what the problem actually is, not reviewing data;

also not anticipating or thinking about challenges, so they get stalled.

Example of Current Status

• Benchmarks above include current performance data;

• Need to work more closely with community colleges and local high schools in aligning curriculum to help improve access and retention;

• Math is a challenge to retention so need to work with math department to rethink math offerings and approach;

9/28/2012

8

Example of Current Status

• Faculty do not have skills in or incentives to engage students who may have potential but not showing it in courses;

• Student support systems like clubs, advising and career preparation not in place

Interventions

Column E – Interventions

• Most of you jumped to these quickly, but think across various dimensions – professional development, incentives, external partners, mission and values, campus culture, policies and procedures, program design, leadership

What we know is that people tend to ignore many

interventions that are key to implementing change.

9/28/2012

9

Example of interventions • Interventions need to be aligned to these challenges

and prioritized. Set up task force on each area. • Examine pedagogy in classes; consider evidence-based

methods. • Develop and strengthen partnerships with high schools

and community colleges. • Work to revise math sequence and approach; align

with math placement test. • Create faculty development around engaging students

in STEM learning. • Build community engagement into programs. • Create student club, peer mentoring programs; work

with advising staff, develop career and internship program

Learning process

Column F -- Learning process

• By having some process in place to learn as change unfolds can better address barriers.

We know that people tend to get comfortable on auto pilot and do not scan for emerging challenges that can slow the process and make them inflexible to change.

9/28/2012

10

Example of learning process

• Have quarterly meetings with task forces groups to communicate and share information;

• Work with them to collect data on their effort and craft as evaluation;

• Have twice yearly forums to troubleshoot problems.

Narrative/Question Framework

• Do we have a vision/goals? Is it articulated in a way that will be clear to internal and external groups who we may need to communicate with and get support? How are these goals related to institutional priorities?

• Do we have measures for achieving our goal(s) that we can collect data about so that we can document success to get on-going support? How can we leverage existing data collection/analysis mechanisms?

• Do we know how we are performing (if at all) on the goals currently so we can build off current efforts and not reinvent the wheel? Do we know about historical challenges or aspects of the current context that might make this challenging?

9/28/2012

11

Narrative/question framework

• Have we examined the interventions needed comprehensively and holistically so that implementation is not thwarted by areas we did not consider that are needed for support? Are the interventions supported by data and directed at solving the issues or challenges we have identified? (Look to framework and notebook from first meeting for dimensions to consider)

• Do we have a plan for checking in from time to time during implementation so that problems that emerge with respect to our change process can be addressed quickly? Is our team working well together? Does it have representation from necessary groups?

http://www.carnegiefoundation.org/elibrary/getting-ideas-action-building-networked-improvement-communities-in-education

Improvement Cycle: Networked Improvement Communities

9/28/2012

12

Upcoming Events -- Feedback

• Fall Regional Meetings: Brainstorm specific goals and information

– Sharing across campuses

– Discussion of campus processes, challenges, strategies

– Issues related to the change process

– Other?

• Upcoming Webinars – Best Practices, Assessment

Announcements

• RSVP for Regional Meetings TODAY

– LA: October 26

– Bay Area: November 1

– San Diego: November 30

• Upcoming Webinars

– October 25 on Best Practices (Ellen Goldey)

– November 16 on Assessment (Peggy Maki)

• First set of Framework Responses DUE Oct 12

Project Website: http://www.aacu.org/pkal/educationframework/index.cfm Project Contact: pkal@aacu.org (Christina Shute, staff support)