1

Ten Shifts in ThinkingSAM Florida – Kim Marshall – January 31, 2015

2

Previous beliefs disrupted by evidence

1. Intelligence and talent are fixed at birth

2. Poverty is destiny

3. Principals can only be managers

4. Great teachers are born, not made

5. Teacher evaluation is a weak lever

6. Students are too young to “vote”

7. Tests are tests

8. Test data can fairly evaluate teachers

9. Curriculum anarchy and myopia are inevitable

10. “Just right” texts accelerate kids

3

1. Intelligence and talent fixed at birth

• The innate ability paradigm– A deep-seated belief among most Americans that

people are born:

VS

SS

KD• And many of us believe

intelligence in unevenly distributed by tribe.

Carol Dweck: Growth vs. fixed mindset• Like building muscles, effective work makes us smarter.

• You’re not just born smart, you can

get smart through strategy and effort.

• The way we talk about success and

failure is crucial.

• Compliment on effort and strategy,

not on being “smart” or “talented.”

• Struggle and mistakes are where the best learning happens.

Jeff Howard, Efficacy Institute

9

And it’s not just “book smarts”

10

• The 20th century mindset was that academic proficiency was the name of the game – test scores

• But non-cognitive factors are coming on strong.

• Grit, curiosity, collaboration, and others

• Ground-breaking work in some schools, 3 book:

– Emotional Intelligence by Daniel Goleman

– How Children Succeed by Paul Tough

– Character Compass by Scott Seider

11

2. Poverty is destiny

• A seemingly inexorable link between children’s socioeconomic status and school achievement

• New York state 7th graders’ ELA achievement

The Matthew Effect

For those who have, more will be given and

they will have an abundance; but from those

who have nothing, even what they have will be

taken away.

Matthew 13:12

13

How this plays out in schools

• Entering kindergarten, the huge vocabulary gap

• Who talks in class – the COPWAKTA syndrome

• Independent reading – access to high-quality books

• Poor classroom management – same dynamic

• Mediocre teaching – who survives, who suffers?

• Homework – choosing your parents well

• Computers – the technology gap

• Vacations, long summer – museums, travel

• Rumors of inferiority – fixed mindset

14

Three possibilities

The Matthew effect – achievement gaps get wider

Parallel progress, but still wide gaps

Everyone makes progress and the gaps get narrower

15

The good news• “We can, whenever and wherever we choose,

successfully teach all children whose schooling is of interest to us. We already know more than we need in order to do this. Whether we do it must finally depend on how we feel about the fact that we haven’t so far.”

Ronald Edmonds, 1979

• “Effective schools” research by Larry

Lezotte, Robert Marzano, and others• Karin Chenoweth’s books• Ed Trust’s Dispelling the Myth website

www.edtrust.org/node/3420

16

Percent of New York State 7th-gradersproficient and above in ELA, and FRPL

17

3. Principals can only be managers

18

H.S.P.S. – Hyperactive Superficial Principal Syndrome

• Constant tug of events, crises, stuff

• “Got a minute?”

• Even the best find instructional leadership hard.

• Reinforced for being reactive, helpful with stuff

• But now we have a better understanding of instructional leadership and time management.

19

The principal as instructional leader

– A shared sense of purpose and possibility

– Positive student and family culture

– Thoughtful hiring

– Professional working conditions (especially common planning time for teacher teams)

– Curriculum alignment, planning, and materials

– Continuous use of assessments

– Praise, coaching, and tough-love feedback

New Leaders’ 15 keys to effectiveness

21

However…

• This work will always be challenging

• Principals need supervision and support (SAMs)

• Supervisors with a manageable span of control

– Newark, Boston, Denver: clusters of 12-15 schools

• Hands-on leadership and support:

– Classroom co-visits

– Surveys and data analysis

– Monthly principals’ meetings used well

22

Pause and discuss

• On the points presented so far:

– You’re not just born smart; you can get smart.

– Effective schools can defy the demographic odds.

– Principals can be instructional leaders.

• Do some people in your school have the old beliefs?

• What do they say or do that makes you think so?

• How can you work to shift their beliefs and actions?

23

4. Great teachers are born, not made

24

It’s a God-given talent

• Americans love a hero – movie portrayals

– Mr. Chips, Jaime Escalante, Erin Gruwell

• Conclusion: it’s all about hiring the “best and the

brightest”

• It’s about being a “natural,” credentials, “smarts”

• Either you have it or you don’t

• Only heroes need apply

25

A shift from teachers to teaching

• There is a knowledge base! We can develop talent.

– Jon Saphier – The Skillful Teacher

– Doug Lemov – Teach Like a Champion

– John Hattie – Visible Learning

– Elizabeth Green – Building a Better Teacher

– Boston Teacher Residency Program

– Relay Graduate School

– Teach Plus

26

Rubrics:breaking teaching

down into its component

parts

Kim’s retrospective ratings circa

1981

Effective

27

28

Kim’s three big improvement goals

• Improve classroom management by implementing Fred Jones plan: routines, limit-setting, and PAT

• Enhance learning stations with technology – more differentiation and systematic follow-up

• Implement interim assessments in reading, writing, and math with grade-level colleagues

Which of these wing designs gets lift?

Yeager and Walton, 2011, Marshall Memo 389

31

It seems like magic, but it’s not

• With airplane wings, it’s basic aerodynamics.

– A specific wing design produces lift.

• Within schools, some teachers lift their students.

• Some schools lift virtually all their students

• It happens because of specific instructional and leadership practices.

32

Teaching and school quality

Students enter at the 50th percentile – after 2 years:

• Least effective teaching, least effective school: 3rd

• Least effective teaching, most effective school: 37th

• Average teaching, average school: 50th

• Most effective teaching, least effective school: 63rd

• Average teaching, most effective school: 78th

• Most effective teaching, most effective school: 96th

Marzano 2003

33

5. Teacher evaluation is a weak lever

34

National Teacher Project’sWidget Effect study

• Chicago teacher ratings 2003-08 on 4-point scale:– Superior – 25,332– Excellent – 9,176– Satisfactory – 2,232– Unsatisfactory – 149

• Elgin teacher ratings 2003-08 on 3-point scale:– Excellent – 2,035– Satisfactory – 264– Unsatisfactory – 11

• Denver ratings 2005-08 on binary scale: – 2,374 Satisfactory– 32 Unsatisfactory

35

The evaluation of America’s school teachers is, with few exceptions, an anemic and impotent enterprise – promising much but producing little.

James Popham, 1988

Evaluation has become a polite, if near-meaningless matter between a beleaguered principal and a nervous teacher. Research has finally told us what many of us suspected all along: that conventional evaluation, the kind the overwhelming majority of American teachers undergo, does not have any measurable impact on the quality of student learning. In most cases, it is a waste of time.

Mike Schmoker, 1992

Except for a few instances, the traditional evaluation process is exhausting and fruitless.

Kathleen Elvin, Brooklyn principal, 2008

Principal evaluation of teachers is a low-leverage strategy for improving schools, particularly in terms of the time it requires of principals.

Richard DuFour & Robert Marzano, 2009

36

900 lessons taught each year

37

No wonder supervision and evaluation are such weak levers!

• A seemingly impossible challenge.

• Many educators have become cynical, given up.

• And yet teacher evaluation takes a huge amount of

administrators’ time.

• What’s wrong with this picture!?

38

Beatriz Vergara, 15

• Testifying in the 2014 Vergara trial in California• Three middle-school teachers described as

apathetic, verbally abusive, and ineffective• Comments on her sixth-grade math teacher:

“It was always loud in there, and [he] would even sleep during class. He didn’t even teach, and he couldn’t control his class. I couldn’t hear anything because of how loud it was.” Time Nov. 3, 2014

• A total failure of supervision and evaluation• How could this happen?

39

Ideally, what supervision andevaluation can accomplish

1. Quality assurance – Honestly telling the public that every child is taught well in every classroom based on an accurate assessment of teachers

2. Praise and improvement – Affirming effective teaching and coaching less-than-effective teaching

3. Motivation – Helping teachers reflect, continuously improve, and bring their ‘A’ game every day

4. Personnel decisions – Making fair judgments for retention, tenure, awards, and dismissal

40

4-point scale: Is you is or is you ain’t?

4 –

3 –

2 –

1 – • Level 3 is solid, effective professional practice• Level 4 is highly effective, Level 2 mediocre, Level 1…• The theory: Level 3 and 4 close the achievement gap…• The goal: All teachers performing at Level 3 and 4• Recognize master teachers, use their skills, keep them!• Work hard to improve mediocre and ineffective teaching.

41

Supervisors can only evaluate accurately and

intervene effectively if they are seeing teachers’

everyday practice, and they can do that only if their

classroom visits are frequent, unannounced,

brief, and systematic.

42

Three essential components

• A system for getting into classrooms frequently and seeing daily reality

• A good “eye” for teaching and learning

• An effective way of sharing feedback with teachers

43

6. Students are too young to “vote”

44

Ron Ferguson, Tripod Project

• Students are very observant and astute.

• Strong correlation with achievement:

students taught by teachers rated high by

students do a full semester better than

students taught by teachers rated low by students.

• Results are more stable over time than other methods.

• The MET project endorsed as 1 of 3 evaluation tools.

• Key factors: high-quality questions, students being assured of anonymity, teacher not in room.

45

Sample Tripod questions• Students in this class treat the teacher with respect.

Totally untrue Mostly untrue Somewhat true Mostly true Totally true

• My teacher explains difficult things clearly.• My teacher pushes everybody to work hard. • My teacher takes the time to summarize what we learn each day.• My teacher wants us to share our thoughts.• I understand what I am supposed to be learning in this class.• My teacher thinks we understand even when we don’t.

46

47

However, Campbell’s Law

The more any quantitative social indicator

is used for social decision-making,

the more subject it will be to corruption pressures

and the more apt it will be to distort and corrupt

the social processes it is intended to monitor.

Donald Campbell, 1976

The solution: Medium stakes, Goldilocks level

48

With high stakes…

• Counting survey data as 10% of teacher evaluations.

• Students sometimes dislike demanding teachers…

– Might ding them in the survey.

• Students sometimes like mediocre teachers…

– Might go easy on them in the survey.

• Teachers might worry about getting negative reviews.

– Assign less challenging work, show more movies.

– Pressure students to give good ratings.

• Surveys can be time-consuming and expensive.

49

How about a medium-stakes approach?

• Research has found a strong correlation between survey results and student achievement.

• Logical conclusion: use surveys to evaluate teachers.

• But given the downsides of that, what about using surveys to cause improvement in teaching and learning?

• Looking at survey data can have a powerful effect.

• We need pilot programs, research.

• Here’s a suggested process…

50

Using surveys for improvement

• All teachers give student surveys twice a year.– Using open-source questions

• Teachers look at the survey data with their supervisor:– What were you most pleased with? – What surprised you? – What are two things you’ll change?

• The focus is on what affirms and improves teaching.• One more data source for supervisors

– Or not, if teachers want to keep them confidential– Or share them only with a colleague

51

Pause and discuss

• On the last three points:

– Effective teaching can be taught and learned.

– Supervision and evaluation can actually work!

– Student input is very helpful.

• Do some people in your school have the old beliefs?

• What do they say or do that makes you think so?

• How can you work to shift their beliefs and actions?

52

7. Tests are tests

For teachers, getting annual test scores several months after taking the test and in most cases long after the students have departed for the summer sends a message: “Here’s the data that would have helped you improve your teaching based on the needs of these students if you would have had it in time, but since it’s late and there’s nothing you can do about it, we’ll just release it to the newspapers so they can editorialize again about how bad our schools are.

Doug Reeves, 1998

53

Summative

Interim

On-the-spot

Three kinds of assessment

When a teacher teaches, no matter how well he or she might design a lesson,what a child learns is unpredictable.Children do not always learn what we teach. That is why the most important assessment does not happen at the end of the learning – it happens during the learning,when there is still timeto do somethingwith the information.

Dylan Wiliam, 201155

Mike Rutherford, The Artisan Teacher, 2013

Effective on-the-spot assessments

• Cold-calling (e.g., popsicle sticks)• Think/pair/share, teacher listens in• Quick-writes, “over the shoulder” checking in• Running records• Dry-erase boards, hold up answers• Clickers, flash-time• Journal-writing• Exit tickets or laying out the classwork• Quizzes (checked quickly, followed up)• Others?

Eric Mazur, an early pioneer

• Troubled by his physics students’ low performance on conceptual understanding, higher-level application

• Began to use clickers to check for understanding• If 30-70% wrong answers, “Convince your neighbor”• Some students irate at having to think!

What happened in Mazur’s classes

• Improved understanding from peer instruction

• Mazur became a better teacher by listening in

• Course learning results improved (especially conceptual)

• Long-term retention improved

• Female students did better – gap closed

• More students pursued STEM majors

• Mazur’s book: Peer Instruction (Prentice Hall, 1997)

(Summarized in Marshall Memo 241)

Retrieval works best when you struggle

• Pushing yourself just beyond your current level

• Making mistakes (I thought I knew that!)

• Trying to fix them in real time

• This is where “stereotype threat” kicks in

– Inner voices of doubt – Am I smart enough?

• But when it works well, “deep practice” is powerful.

• Each time, neural pathways are wrapped in more myelin, work more efficiently, increase bandwidth.

Ongoing, spaced, cumulative review

Interim assessments – a PLC in action

64

8. Test data can fairly evaluate teachers

4

3

2

1

1 2 3 4

Student

achievement

Teacher’s evaluation score

65

Or might it look like this?

4

3

2

1

1 2 3 4

Student

achievement

Teacher’s evaluation score

Test scores as part of teacher evaluation?

• This seems logical and has been popular with politicians.

• Value-added data do tell us something about schools.

• Imperfectly, they provide insights on individual teachers.

• Slightly more accurately than traditional evaluations!

• But there are big problems with accuracy and reliability.

• So dismissing teachers is problematic.

• Plus, no help improving instruction.

• In other words, they don’t produce lift.

• And some individual teachers

will be harmed – “collateral damage”

67

New York City’s value-add formula

• Confidence interval in Math is 35 percentile points.• Confidence interval in ELA is 53 percentile points.

68

Can’t student learning be part of the supervision and evaluation process?

• Using test scores for individual teacher evaluation is highly problematic.

• Even as a small fraction of teacher evaluation, it’s an ineffective practice that can cause real harm.

• But advocates of accountability have a point:

• Student learning should be part of the process.

• How?

By using local measures with medium stakes

• Here are some ways to measure student learning:

– Fountas-Pinnell reading levels

– Writing rubrics

– Diagnostic math assessments

– Portfolios

– Performance tasks

• All teachers K-12 can track learning gains, including art, music, physical education, and others.

• These measures can be part of accountability.

70

What this looks like in a school

• 2nd-grade team meets in September, decides on metrics• Meets with the principal, gets the okay• Does a baseline assessment of all students• Sets SMART goals (e.g., 85% of students at Level M)• Teaches all year, using on-the-spot and interim

assessments, team meetings to continuously improve• Assesses students at the end of the year• Presents value-add data to the principal• Gets a team evaluation• The team rating is given to each individual teacher.

71

The power of this approach

• Uses appropriate assessments that teachers trust • Medium stakes (Campbell’s Law)• Encourages teacher teamwork and sharing• The focus is on student learning• Can be used with all teachers (primary, music, art)• Involves meaningful goal-setting• Measures the learning gains made that year• A premium on using learning data during the year• Pressure on underperforming teachers to shape up• Involves the principal up front, all year, at end• A manageable number of end-of-year meetings

72

9. Curriculum anarchy and myopiaare inevitable

• Phonics vs. whole language

• Hands-on versus traditional math, calculators versus computation

• No agreed-upon core knowledge, literature canon

• General track vs. college

• Uneven state-by-state standards and assessments

• A race to the bottom!

73

74

75

But amazingly, consensus on WHATand a good approach on HOW TO

• We’ve found middle ground on balanced literacy

• A growing consensus on math expectations

• Convergence on high-school graduation standards that prepare students for college and career success

• Core Knowledge is more widely accepted

• Common Core State Standards in 45 states, D.C.!

• Next Generation Science Standards

• Common assessments – PARCC and Smarter B.

76

77

Backwards-planning 4-6-week units

• Grant Wiggins, Jay McTighe – Understanding by Design

• Teacher teams given the time and structure to do this right

• Deeper, more thoughtful, reaching all students

• All in the context of the full-year curriculum

78

A linear UbD planning template

1. State/Common Core standards (written out verbatim)

2. Important facts – Students will know…

3. Skills – Students will be skilled at…

4. Big ideas – Students will understand that…

5. Likely misconceptions – Students may think that…

6. Essential Questions – Students will keep considering…

7. Transfer – Students will be able to use what they learn to...

8. Assessments (pre-, on-the-spot, final, performance task)

9. Lesson-by-lesson calendar of key learning events

79

10. “Just right” texts accelerate kids

• This makes perfect sense!

• Teaching children at their current instructional level:

– Not too easy – spinning their wheels

– Not too hard – frustration

– Just right – the Goldilocks level

• Lots of leveled texts, guided reading, comfortable kids

• “If low-performing fourth graders are to be taught from second-grade books, when do they catch up?”

(Shanahan, 2014)

80

A new synthesis – four levels of rigor, lots and lots of reading

• Teacher reading to students – Texts substantially above their level, teacher doing the print work, teacher and students working together on meaning

• Shared reading – Texts on or a little above grade level, teacher and students doing the print and meaning work together.

• Guided reading – Texts on students’ instructional level, students doing the print and meaning work with teacher support in small groups.

• Independent reading – Student-chosen texts at many different levels, students doing all the work

Burkins and Yaris, 2014

81

Discussion on the last 4 – and your big take-aways

1. You’re not just born smart; you can get smart.

2. Some teachers and schools prove that poverty is not destiny.

3. It’s possible for principals to be effective instructional leaders.

4. Effective teaching can be taught and learned.

5. Supervision and evaluation can be powerful levers for change.

6. Student input is extremely helpful to teachers and leaders.

7. On-the-spot and interim assessments can narrow the gap.

8. Local, medium-stakes assessments used by teams are powerful.

9. There’s a helpful new consensus on the WHAT and HOW TO.

10. Students accelerate with a mix of difficult and easy texts.

82

Contact information

Kim’s e-mail:

kim.marshall48@gmail.com

Marshall Memo website for

rubrics and articles (click on

Kim Publications):

www.marshallmemo.com