Evaluation practice in the Nordic countries: Different national traditions

or a common approach?

Hanne Foss HansenDepartment of Political Science

University of Copenhagen

Structure

• Educational evaluation: Concepts and approaches

• Case 1: Higher education -Brief reviews country by country -Similarities and differences

• Case 2: Primary and secondary education (P/S)

• The effects of all this evaluation? The future?

The concept of evaluation

• Everyday language: Measurement, assessment, judgement

• Evaluation language: ”A careful assessment of the merit and worth of processes, structures, output and outcome of interventions and organizations, intended to play a role in future, practical actions situations”

The concept of educational evaluation

• Testing, student assessment, programme evaluation, personel evaluation, auditing, accreditation, benchmarking, curriculum evaluation and probably even more.

Educational evaluation: Focus on many levels

-Individuals (pupils, students, teachers)-Classrooms/courses-Curriculum/programmes-Organizations (schools, universities)-Fields (all schools in a municipality, all

programmes in a discipline)-The national level (national quality development

and quality assurance systems)-The international level (PISA, EQUIS in the

business school area)

Educational evaluation: Many purposes, many uses

-Documenting

-Controlling

-Learning/improving

-Reforming

-Legitimating

-Symbolizing

Focus today primarely on

• The new forms of evaluation (programme evaluation, auditing, accreditation etc.) not on the classical questions of testing and student assessment

• Meso-evaluation defined as evaluation coupled not only to professional practice but also to educational policy

Higher education I

• Adoption of evaluation in the late 1980´s

• 1992-1999: The Danish Center for Evaluation of Higher Education

• 1999: The center is reorganized into the Danish Evaluation Institute (EVA)

Higher education II

• 1990’s: Programme evaluation • 2002: Accreditation is introduced • 2003: A new university law stresses the

responsibility of the universities themselves to conduct evaluations (EVA unclear role)

• 2004: Auditing is introduced

----• 2005: EVA is made responsible for accreditation

of professional education

Higher education I

• Adoption of evaluation in the late 1960’s and early 1970’s, gaining renewed priority in the mid 1990’s

• 1995 The National Agency for Higher Education (Högskoleverket) is established

Higher education II

• 1999-2002: Auditing is the main task

• 2001-2006: Programme evaluation becomes the main task

• Accreditation is also part of the picture

Higher education I

• Adoption of evaluation in the mid 1990´s

• 1996: The Finnish Higher Education Evaluation Council (Finheec) is etablished

Higher education II

• As law places responsibility for evaluation with the higher educational institutions an important purpose of the council is to help institutions to develop quality assurance and development systems

• The council also initiates evaluations of different types

• Accreditation is important in relation to polytechnics and professional courses

• 2004: Auditing

Higher education I

• Adoption of evaluation in the late 1990´s

• 1998: ”Norgesnettrådet” is established

• 2003: The Norwegian Agency for Quality Assurance in Education is established

Higher education II

• Auditing of all higher educational institutions

• Accreditation of programmes and institutions applying for new programmes and institutional status

• The Ministry of Education initiates evaluations of higher educational reforms (Høgskolereformen, Kvalitetsreformen)

Higher education

• Adoption of evaluation in the mid 1990’s

• 1999: It becomes mandatory for higher educational institutions to develop quality assurance systems

• The Ministry of Education initiates programme evaluation ad hoc

• No formalised accreditation system

Similarities across countries in talk, organisation and focus

• Adoption of meso-evaluation in all countries

• Anchoring evaluation in semi-autonomous organizations specialized in evaluation (not Iceland)

• Educational evaluation is decoupled from evaluation of research

• A turn towards auditing (N, DK, FIN)?

From national imitation to international regulative pressures?

National pressures

International pressures

weak

strong

weak strong

Bologna

1990

2005

Factors explaining convergence

• Public-sector reforms: New Public Management, focus on results and effectiveness

• Internationalization: The Bologna proces and the aim of establishing a European Higher Education Area in 2010

• Networking across agencies at Nordic as well as European level

Differences in institutional processes

• Time span in adoption (from Sweden in the late 1960’s, to Denmark in the late 1980’s and Norway in the late 1990’s)

• Time span in institutionalization (e.g. routinization in Denmark from 1992, in Norway from 2003)

• Norway as the late adopter has constructed the most radical system

Differences in balances between quality development (QD) and

control (C) purposes

• DK: QD more than C (except professional education)

• S: From C more than QD to QD more than C

• FIN: QD more than C (except professional education)

• N: C but also QD• IS: QD more than C

Differences in decision contexts

• From Denmark where there is no direct coupling to sanctioning and rewarding (except in professional education) to Norway where there is a direct coupling to sanctioning and rewarding with Sweden somewhere in between

Differences in evaluation models

- Self-evaluation is an important element in DK, S and FIN but not in N

Differences in composition of evaluation panelsDK N S

Peers + + + + + +

Educational research

- + + +

Educational leadership

- + + -

Students - + +

Other users + + - -

Differences in coordination across individual evaluations

Coordination by:

Denmark Norway

Procedures Strong Strong

Specified criteria

Only used in some evaluations

Strong

Panel members Weak Strong

Board decisions - Strong

Factors explaining divergence

Differences in:

• political-administrative cultures

• strategies in public-sector reforms

• structures and traditions in educational systems

• timing and content of higher educational reforms

P/S education

• Late 1990’s the Ministry of Education introduces a program ”Quality development in public Schooling” (attention and tools )

• 1999: EVA gets responsibility for evalution in P/S

• 2002: A law about transparency and openness makes it compulsory to educational institutions to publish evaluations of the quality of teaching

• 2005: Government proposes to establish a council and an agency for quality development

P/S education

1997: Municipalities have each year to work out written quality reports

2003: The agency for education is split up in the Swedish Agency for Education and an agency for school development

2004-2009: Inspection programme. Inspection reports serve as starting points for improving the quality of municpal schooling.

P/S education

• 2003: A council for educational evaluation is established. The council has to plan and implement external evaluations as well as develop methods and coordinate local evaluation

P/S education

• 2004: The Directorate for Primary and Secondary Education is established. The directorate is responsible for an internet-based quality assessment system ensuring transperency in quality information.

P/S education

• Schools have to do and publish self-evaluations

• Every 5th year The Ministry of Education assesses the evaluation methods used by schools (site-visits)

P/S education: Similarities

• Evaluation adopted in all countries • International studies have put educational

quality and evaluation on the agenda (PISA & TIMMS)

• All countries build national institutional capacity to deal with quality and evaluation (increasing state control)

• Transparency in monitoring is important (strenghtening market forces)

P/S education: Differences

• Balances between quality development and control purposes (S: C control but also QD; DK, N, FIN and IS: more soft approaches)

• ? – Too early to really conclude on the practice of the new agencies

Comparing the two cases

Higher education

-Time span in adoption (from late 1960’s to late 1990’s)

-Policy-driven development

P/S

-Later adoption but no time span

-Problem-driven development (DK, N)

Evaluation practice in the Nordic countries: Different national traditions

or a common approach?

Conclusions• Similarities in talk

• Similarities and important differences in actions

• Evaluation is an elastic concept giving room for national and local constructions

Effects of growth in meso-evaluation I

Two very different ways of thinking:

1) Optimism related to the development of learning organizations and a knowledge society

2) Pessimism related to the development of an audit society based om distrust

Effects of growth in meso-evaluation II

• Are educational institutions transformed into

learning organizations or into ”auditable commodities”?

• Is professional practice part of or de-coupled from evolving evaluation cultures?

Limited empirical knowledge in the Nordic countries

The future

A turn towards:

-Auditing and accreditation?

-Evidence-based professional practice?

-Evidence-based educational policy?