strong performers and successful reformers - lessons from pisa for turkey

OECD EMPLOYER BRAND

Playbook

1

Strong performers and suc-cessful reformersLessons from PISA for Turkey

Andreas SchleicherIstanbul, 15 February 2014

2 PISA in brief

• Over half a million students…– representing 28 million 15-year-olds in 65 countries/economies

… took an internationally agreed 2-hour test…– Goes beyond testing whether students can

reproduce what they were taught…… to assess students’ capacity to extrapolate from what they know

and creatively apply their knowledge in novel situations– Mathematics, reading, science, problem-solving, financial literacy– Total of 390 minutes of assessment material

… and responded to questions on…– their personal background, their schools

and their engagement with learning and school• Parents, principals and system leaders provided data on…

– school policies, practices, resources and institutional factors that help explain performance differences .

3

1960 1970 1980 1990 2000 2006 200935

40

45

50

55

60

65

70

Routine manual

Nonroutine manual

Routine cognitive

Nonroutine analytic

Nonroutine interpersonal

Mean task input in percentiles of 1960 task distribution

Changes in the demand for skillsTrends in different tasks in occupations (United States)

Source: Autor, David H. and Brendan M. Price. 2013. "The Changing Task Composition of the US Labor Market: An Update of Autor, Levy, and Murnane (2003)." MIT Mimeograph, June.

What do 15-year-olds know……and what can they do with what they know?

4

410

420

430

440

450

460

470

480

490

500

510

520

530

540

550

560

570

580Mean score

High mathematics performance

Low mathematics performance

… Shanghai-China performs above this line (613)

… 12 countries perform below this line

Average performanceof 15-year-olds in

MathematicsFig I.2.13

Socially equitable distribution of learning

opportunities

High mathematics performance

Low mathematics performance

Average performanceof 15-year-olds in

mathematics

Strong socio-economic impact on student

performance

02468101214161820222426

AustraliaAustria

Belgium Canada

Chile

Czech Rep.

Denmark

Estonia

Finland

France

Germany

Greece

Hungary

IcelandIreland

Israel

Italy

Japan

Korea

Luxembourg

Mexico

Netherlands

New Zealand

Norway

Poland

Portugal

Slovak Rep.

Slovenia

Spain Sweden

Switzerland

Turkey

UK

US

Singapore

Hong Kong-ChinaChinese Taipei

Macao-China

Liechtenstein

Viet Nam

Latvia

Russian Fed.Lithuania

Croatia

SerbiaRomania

Bulgaria United Arab Emirates

KazakhstanThailand

Malaysia

2012Shanghai-China


opportunities


performance

AustraliaAustria

Belgium Canada

Chile

Czech Rep.

Denmark

Estonia

Finland

France

Germany

Greece

Hungary

IcelandIreland

Israel

Italy

Japan

Korea

Luxembourg

Mexico

Netherlands

New Zealand

Norway

Poland

Portugal

Slovak Rep.

Slovenia

Spain Sweden

Switzerland

Turkey

UK

US

AustraliaAustriaBelgiumCanadaChileCzech Rep.DenmarkEstoniaFinlandFranceGermanyGreeceHungaryIcelandIrelandIsraelItalyJapanKoreaLuxembourgMexicoNetherlandsNew ZealandNorwayPolandPortugalSlovak Rep.SloveniaSpainSwedenSwitzerlandTurkeyUKUS

2012


opportunities


performance

AustraliaAustria

Belgium Canada

Chile

Czech Rep.

Denmark

Estonia

Finland

France

Germany

Greece

Hungary

IcelandIreland

Israel

Italy

Japan

Korea

Luxembourg

Mexico

Netherlands

New Zealand

Norway

Poland

Portugal

Slovak Rep.

Slovenia

Spain Sweden

Switzerland

Turkey

UK

US


AustraliaAustria

Belgium Canada

Chile

Czech Rep.

Denmark

Estonia

Finland

France

Germany

Greece

Hungary

IcelandIreland

Israel

Italy

Japan

Korea

Luxembourg

Mexico

Netherlands

New Zealand

Norway

Poland

Portugal

Slovak Rep.

Slovenia

Spain Sweden

Switzerland

Turkey

UK

US


Singapore

Shanghai

Singapore

2003 - 2012 Germany, Turkey and Mexico improved both their mathematics performance and equity levels

Performance of countries in a level playing field

How the world would look if students around the world were living in similar social and economic conditions

11

12

Shang

hai-C

hina

Hong

Kong-

China

Viet N

amKor

ea

Liec

hten

stein

Switzer

land

Nethe

rland

s

Belgi

um

Canad

a

Austri

a

New Z

eala

nd

Franc

e

Irela

nd

OECD ave

rage

Slova

k Rep

ublic

Hunga

ryIta

ly

Unite

d Kin

gdom

Lith

uani

a

Unite

d Sta

tes

Sweden

Roman

ia

Serbi

a

Greec

eChi

le

Mal

aysia

Cypru

s5, 6

Costa

Rica

Brazil

Tunisi

aPer

u

Colom

biaQat

ar340

360

380

400

420

440

460

480

500

520

540

560

580

600

Mean score at the country level before adjusting for socio-economic statusMean score at the country level after adjusting for socio economic status

Me

an

ma

the

ma

tic

s s

co

reMathematics performance in a level playing fieldMean mathematics performance after accounting for socio-economic status

Fig II.3.3

The dream of social mobility

In some countries it is close to a reality

13

14

Shang

hai-C

hina

Mac

ao-C

hina

Singa

pore

Chine

se T

aipe

i

Liec

hten

stein

Estoni

a

Polan

d

Finl

and

Portu

gal

Turk

ey Italy

Latvi

a

Austra

lia

Austri

a

Czech

Rep

ublic

Unite

d Kin

gdom

Fran

ce

Icela

nd

Russia

n Fe

d.

Croat

ia

Sweden

Slova

k Rep

ublic

Serbi

aIsr

ael

Roman

ia

Indo

nesia

Kazak

hsta

n

Brazil

Chile

Mon

tene

gro

Argen

tina

Peru

0

2

4

6

8

10

12

14

16

18

20

%

Percentage of resilient students

More than 10% resilient Between 5%-10% of resilient students Less than 5%

Fig II.2.4

Socio-economically disadvantaged students not only score lower in mathematics, they also report lower levels of engagement, drive, motivation and self-beliefs. Resilient students break this link and share many characteristics of advantaged high-achievers.

A resilient student is situated in the bottom quarter of the PISA index of economic, social and cultural status (ESCS) in the country of assessment and performs in the top quarter of students among all countries, after accounting for socio-economic status.

It is not just about poor kids in poor neighbourhoods…

…but about many kids in many neighbourhoods

15

16

Alban

ia

Icela

nd

Norway

Estoni

a

Spain

Polan

d

Kazak

hsta

n

Mex

ico

Costa

Rica

Mal

aysia

New Z

eala

nd

Greec

e

Unite

d Kin

gdom

Austra

lia

Portu

gal

Chile

Roman

ia

Switzer

land

Urugu

ay

U.A.E

.

Serbi

a

Korea

Singa

pore

Italy

Czech

Rep

ublic

Bulga

riaQat

ar

Germ

any

Slova

k Rep

ublic

Belgi

um

Liec

hten

stein

Chine

se T

aipe

i100

80

60

40

20

0

20

40

60

80

100

Variability in student mathematics performance between and within schools

Vari

ati

on

in

stu

den

t p

erf

orm

an

ce a

s %

of

OEC

D a

vera

ge

vari

ati

on Performance variation of

students within schools

Performance differences between schools

Fig II.2.7

OECD average

OECD average

B

2222Le

sson

s fr

om h

igh

perf

orm

ers

Catching up with the top-performers

Low impact on outcomes

High impact on outcomes

Low feasibility High feasibility

Money pits

Must haves

Low hanging fruits

Quick wins

2323Le

sson

s fr

om h

igh

perf

orm

ers




Money pits

Must haves

Low hanging fruits

Quick wins

Commitment to universal achievement

Gateways, instructional systems

Capacity at point of delivery

Incentive structures and accountability

Resources where they yield most

A learning systemCoherence

2424Le

sson

s fr

om h

igh

perf

orm

ers




Money pits

Must haves

Low hanging fruits

Quick wins







A commitment to education and the belief that competencies can be learned and therefore all children can achieve

Universal educational standards and personalization as the approach to heterogeneity in the student body…

… as opposed to a belief that students have different destinations to be met with different expectations, and selection/stratification as the approach to heterogeneity

Clear articulation who is responsible for ensuring student success and to whom

25

-0.60 -0.40 -0.20 0.00 0.20 0.40 0.60 0.80 1.00 1.20300

350

400

450

500

550

600

650

481.366786279212

517.501096817955

561.241096454551

391.459888954175

499.749902827587

452.973426858907

409.291567937716

493.934230896316

520.545521676786518.750335282979

394.329333356314

471.131460759248

490.571021411359

481.644744006327489.845098037208

513.525055819928

478.823277433358

505.540743249801

498.95788231768

559.824796201498

494.98467432064426.737491293011

536.406918234208

447.984414978954 478.260635903011

477.044455015488504.150766311124

466.48143014931

518.078519433354

501.497460196644438.738259877415

385.595556395556

422.632355405519

538.134494733918

U.A.E.

514.745238582901522.971758192682

484.319297801971

388.431709907139

375.114451681749

500.026756625414

431.798408505078

368.102547127357

406.999866988793

530.931003950397

409.626613284347

387.824629620249

492.795697239492

501.127422390953

376.4483986347

573.468314296641

487.063181343903

489.373070348755

376.488601072821

420.512967619054

413.281466667708

534.96508297892

553.766659143613

448.859130247604

Russian Fed.

444.554242787643

511.338207501182

485.321181012553

612.675536305453

f(x) = 138.160916953927 x + 477.587612682211R² = 0.368631715648504

Mean index of mathematics self-efficacy

Me

an

ma

the

ma

tic

s p

erf

orm

an

ce

OE

CD

av

era

ge

Countries where students have stronger beliefsin their abilities perform better in mathematics

Fig III.4.5

26Perceived self-responsibility for failure in mathematics

Percentage of students who reported "agree" or "strongly agree" with the following statements:

I’m not very good at solving mathematics problems

My teacher did not explain the concepts well this week

This week I made bad guesses on the quiz

Sometimes the course material is too hard

The teacher did not get students interested in the material

Sometimes I am just unlucky

0 10 20 30 40 50 60 70 80 90

Turkey Shanghai-China OECD average

%

Fig III.3.6

3030Le

sson

s fr

om h

igh

perf

orm

ers




Money pits

Must haves

Low hanging fruits

Quick wins







Clear ambitious goals that are shared across the system and aligned with high stakes gateways and instructional systems

Well established delivery chain through which curricular goals translate into instructional systems, instructional practices and student learning (intended, implemented and achieved)

High level of metacognitive content of instruction …

3131Le

sson

s fr

om h

igh

perf

orm

ers




Money pits

Must haves

Low hanging fruits

Quick wins







Capacity at the point of delivery Attracting, developing and retaining high quality

teachers and school leaders and a work organisation in which they can use their potential

Instructional leadership and human resource management in schools

Keeping teaching an attractive profession System-wide career development …

3232Le

sson

s fr

om h

igh

perf

orm

ers

Lu

xem

bo

urg

Th

aila

nd

Sh

an

gh

ai-

Ch

ina

Co

lom

bia

Ch

ile

Me

xico

Vie

t Na

m

Uru

gu

ay

Ka

zakh

sta

n

Be

lgiu

m

Ma

lays

ia

Bra

zil

U.A

.E.

Ne

w Z

ea

lan

d

Sw

itze

rla

nd

Ma

cao

-Ch

ina

OE

CD

ave

rag

e

Arg

en

tina

Au

stri

a

Ire

lan

d

Fra

nce

Un

ited

Kin

gd

om

Alb

an

ia

Ca

na

da

La

tvia

Un

ited

Sta

tes

Cro

atia

Mo

nte

ne

gro

Hu

ng

ary

Slo

ven

ia

Se

rbia

Bu

lga

ria

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

Bottom quarter of this index

Me

an

ind

ex

Teacher shortage Fig IV.3.5

3333Le

sson

s fr

om h

igh

perf

orm

ers




Money pits

Must haves

Low hanging fruits

Quick wins







Incentives, accountability, knowledge management Aligned incentive structures

For students How gateways affect the strength, direction, clarity and nature of the incentives

operating on students at each stage of their education Degree to which students have incentives to take tough courses and study hard Opportunity costs for staying in school and performing well

For teachers Make innovations in pedagogy and/or organisation Improve their own performance

and the performance of their colleagues Pursue professional development opportunities

that lead to stronger pedagogical practices A balance between vertical and lateral accountability Effective instruments to manage and share knowledge and spread

innovation – communication within the system and with stakeholders around it

A capable centre with authority and legitimacy to act

-1.5 -1 -0.5 0 0.5 1 1.5300

350

400

450

500

550

600

650

531.551979302783

414.947431329217

430.53288984921

423.795593172672

484.685067484024

507.375949559565

493.913526079401

557.719613495498

454.493852942216459.674291542381

419.468595641077

488.357558008343

404.86657067849406.81928697245

410.692469685374

455.967032005237

396.468122669645

431.953772561969

416.098738598916

300.849653448456

527.668467891543

404.539944308878

440.111661967012

474.054187560775

464.989161819408

547.743708881437

626.566663790363

452.789179885987

529.511834268283

497.071637137884

453.49524309675

482.577394045123

532.465311188924

506.274697797594

488.818411796174

402.907104971934

498.55233132561486.358212456265

502.809277446549

485.011835724539

525.143096315803

466.514022482625

460.853234111852

488.150072840935484.3703865799

468.514073102546

499.317279833724

438.810335285436

499.440165643771501.844010272146

478.664970193416480.554307802789

498.658254792673

481.116171960251

503.011259906496

490.67709912419

463.432481043829

552.313972933536

478.845972683071R² = 0.133981453407518

Index of school responsibility for curriculum and assessment (index points)

Ma

the

ma

tic

s p

erf

orm

an

ce

(s

co

re p

oin

ts)

Countries that grant schools autonomy over curricula and assessments tend to perform better in mathematics

Fig IV.1.15

Schools with more autonomy perform better than schools with less autonomy in systems with more collaboration

Less school autonomy

More school autonomy

455

460

465

470

475

480

485

Teachers don't participate in management

Teachers participate in management

Score points

School autonomy for resource allocation x System's level of teachers participating in school managementAcross all participating countries and economies

Fig IV.1.17

Schools with more autonomy perform better than schools with less autonomy in systems with more accountability arrangements



464

466

468

470

472

474

476

478

School data not public

School data public

Score points

School autonomy for curriculum and assessment x system's level of posting achievement data publicly

Fig IV.1.16



455

460

465

470

475

480

485

No standardised math policy

Standardised math policy

Schools with more autonomy perform better than schools with less autonomy in systems with standardised math policies

Score points

School autonomy for curriculum and assessment x system's extent of implementing a standardised math policy (e.g. curriculum and instructional materials)

Fig IV.1.16

38

Written specification of the school's curriculum and educational goals

Written specification of student-performance standards

Systematic recording of data, including teacher and student attendance and graduation rates, test results and professional development of teachers

Internal evaluation/self-evaluation

External evaluation

Written feedback from students (e.g. regarding lessons, teachers or resources)

Teacher mentoring

Regular consultation with one or more experts over a period of at least six months with the aim of improving the school

Implementation of a standardised policy for mathematics

0 10 20 30 40 50 60 70 80 90 100

Percentage of students in schools whose principal reported that their schools have the following for quality assurance and improvement:

Singapore OECD average

%

Quality assurance and school improvement Fig IV.4.14

The issue is not how many charter schools a country has…

…but how countries enable every school to assume charter type autonomy

39

4040Le

sson

s fr

om h

igh

perf

orm

ers




Money pits

Must haves

Low hanging fruits

Quick wins







Investing resources where they can make mostof a difference

Alignment of resources with key challenges (e.g. attracting the most talented teachers to the most challenging classrooms)

Effective spending choices that prioritise high quality teachers over smaller classes

Money makes a difference……but only up to a point

41

Spending per student from the age of 6 to 15 and mathematics performance in PISA 2012

0 20 000 40 000 60 000 80 000 100 000 120 000 140 000 160 000 180 000 200 000300

350

400

450

500

550

600

650

511.338208

385.595556

368.102547

426.737491

420.512968

409.291568

447.984415

376.488601

387.824630

413.281467409.626613

391.459889

438.738260

422.632355

471.131461478.823277

490.571021

477.044455

612.675536

481.644744

498.957882520.545522

466.481430

517.501097

553.766659

487.063181

499.749903

518.070400513.525056

484.319298

494.984674

485.321181

573.468314

518.750335

536.406918

501.127422501.497460492.795697

522.971758

478.260636

514.745239

UK

504.150766500.026757

481.366786

505.540743

489.373070

530.931004

489.845098R² = 0.369063315519053R² = 0.00587924272458274

Average spending per student from the age of 6 to 15 (USD, PPPs)

Ma

the

ma

tic

s p

erf

orm

an

ce

(s

co

re p

oin

ts)

Cumulative expenditure per student less than USD 50 000

Cumulative expenditure per student USD 50 000 or more

Fig IV.1.8

-0.500.511.5300

350

400

450

500

550

600

650

700R² = 0

Equity in resource allocation (index points)

Ma

the

ma

tic

s p

erf

orm

an

ce

(s

co

re p

oin

ts)

Countries with better performance in mathematics tend to allocate educational resources more equitably

Greater equity

Less equity

Adjusted by per capita GDP

Fig IV.1.11

SHA

4444Le

sson

s fr

om h

igh

perf

orm

ers




Money pits

Must haves

Low hanging fruits

Quick wins







Coherence of policies and practices Alignment of policies

across all aspects of the system Coherence of policies

over sustained periods of time Consistency of implementation Fidelity of implementation

(without excessive control)

CAN

4545Le

sson

s fr

om h

igh

perf

orm

ers




Money pits

Must haves

Low hanging fruits

Quick wins







4646Le

sson

s fr

om h

igh

perf

orm

ers Some students learn at high levels

All students need to learn at high levels

Student inclusion

Routine cognitive skills, rote learning

Learning to learn, complex ways of thinking, ways

of workingCurriculum, instruction and assessment

Few years more than secondary

High-level professional knowledge workers

Teacher quality

‘Tayloristic’, hierarchical

Flat, collegial

Work organisation

Primarily to authorities

Primarily to peers and stakeholders

Accountability

What it all means

The old bureaucratic system The modern enabling system

Thank you !

Find out more about PISA at www.pisa.oecd.org• All national and international publications• The complete micro-level database

Email: [email protected]: SchleicherEDU

and remember:Without data, you are just another person with an opinion

Do you have an idea on how to use this data to improve education in your

country?

Would you like to work with us to develop that idea?

Apply to the Thomas J. Alexander

fellowship programme!

http://www.oecd.org/edu/thomasjalexanderfellowship.htm



strong performers and successful reformers - lessons from pisa for turkey

Education