SABER – Systems Approach for Better Education Results Tertiary Education Governance

World Congress of Colleges and Polytechnics

Halifax, May 26, 2012

Quentin Wodon, World Bank

1

Structure of presentation SABER goals, scope, and methodology

Context for tertiary education governance

System-wide and institution-level indicators

Example of data collection for institutions: MENA

Analysis of MENA data on autonomy

Financial autonomy

HR autonomy

Academic autonomy

Conclusion

2

What SABER is New initiative to advance Learning for All

First detailed, disaggregated database of education policies/institutions in core areas

Open data tool for empowering stakeholders

Two key areas:

Maps out policies/institutions

Links to implementation data

3

Goal: Make visible what’s underwater

4

Inputs and (some) outcomes

Everything else: • Policies • Institutions • Implementation • Effects of interventions

Domain development: Key steps What Matters paper

Indicators and scoring rubric

Data-collection instrument

Data collection

Analysis

Data validation

Publication of data & analyses

5

What SABER provides (not just ratings)

Analytical framework for thinking about domain

Descriptive data on policies/institutions

Evaluative judgments (ratings, not rankings) “Latent” (poor performance)

“Emerging” (insufficient performance)

“Established” (adequate performance)

“Advanced” (outstanding performance)

6

7

SABER informs policy choices & diagnoses gaps in implementation in each domain

Catalog & assess quality of policy

framework

POLICIES IMPLEMENTATION + OUTCOMES

Collect & analyze data on policy

execution

• Framework

• Collection instrument

• Rubric

• Manual

• Survey data (e.g. PETS, QSDS, Absence, Household)

• Proxy indicators (e.g. economy-wide metrics, survey data from other countries)

SA

BE

R T

oo

lkit

Country, regional, and policy domain reports with interpretation, including expert

judgment

Tools to benchmark policies Tools to assess implementation based on available data

Online knowledge base

Tertiary education governance Large variation in performance of higher education

institutions – governance as a key determinant

Differences in objectives: research, teaching, contribution to local economy, etc. Not one “best” model

Benchmarking as step towards monitoring performance

Three levels of analysis Institutions level: Identification of Strengths and

weaknesses, base line for comparison overtime

Country level: Identification of models, differences between institutions, correlation between models and performance

International level: Comparisons between models, correlations between models and performance

8

9

System Level Context, Mission

and Goals

Management Autonomy Accountability Participation

Government-

driven

Government-

defined missions

and policies

Government-

appointed

president

Centrally managed

budget

Central control for new

programs and

curriculum

Central HR

management

Central audits

Central QA

National driven

curriculum

Low accountability-no

links between

performance and

rewards

Mainly on

consultation

basis

Autonomous

-

Government-

steered

Mission-oriented

Institutions

Strategic plans

prepared by

Institutions

Governing

boards led

Competitive funds

allocation

Autonomy to introduce

new programs and set

curriculum

HR autonomy

External audits

Independent external

QA

Performance-based

salaries

High

participation

of

stakeholders

throughout

the decision-

making

process

10

Context, Mission

and Goals

Management

Orientation

Autonomy Accountability Participation

Corporate Mission-oriented

Decentralized Results-based

High autonomy

in all three

areas,

academic,

financial and HR

High

accountability in

financial and HR

Academic

Mission-oriented-

Defined in

consultation with

academic staff

High academic

autonomy

High internal

academic

accountability

High

participation of

academic staff

Representational High external

accountability

High

participation of

stakeholders

Trustee

Mission-oriented-

Defined in

consultation with

trustee

Results-based High internal

accountability

System-wide Policy Goals

Goal #1: Vision The country or government has a vision and plan for the tertiary education sector, a willingness to translate its vision into a concrete action plan, and an ability to implement and monitor reforms

Goal #2: Regulatory Framework The tertiary education system is governed by an appropriate regulatory framework including for private providers

Goal #3: Leadership The TEA has an appropriate policy on the role and functions of the boards of tertiary education institutions, as well as for the selection of the leadership of tertiary education institutions, and the respective responsibilities of the Board and leadership

Goal #4: Financial Autonomy and Equity The regulatory framework provides enough financial autonomy to tertiary education institutions while still promoting equity

11

System-wide Policy Goals

Goal #5: Staffing Autonomy The regulatory framework provides enough staffing autonomy to tertiary education institutions

Goal #6: Academic Autonomy The regulatory framework provides enough academic autonomy to tertiary education institutions

Goal #7: Performance-based Funding The TEA negotiates performance targets and uses financing as incentives for institutions to achieve the targets.

Goals #8: Quality assurance and transparency The TEA has an independent quality assurance and accreditation agency for both public and private institutions. Institutions are held to specific standards of transparency around financial health, fraud, student engagement and employment of graduates.

12

A Note on QA and quasi-corruption

13

Question Respons

e Share of household with positive value for annual official school cost 95.7% Share of household with positive value for annual unofficial school cost 44.1% Annual official school cost 1.2M Annual unofficial school cost 0.4M Did you or anyone in your family make unofficial payments to get admission? Yes 27.2% No 72.8% It is common for parents to make some “unofficial payments” to gain admission? Never 1.1% Seldom 2.2% Sometimes 43.0% Frequently 47.3% Always 6.5% When unofficial payments are required, how is it done? A school official indicates or asks for a payment 30.3% The parent or family member offer a payment on his/her own accord 37.1% It is known before hand how to pay and how much to pay, so it is not discussed 32.6%

Institution-level: MENA University Governance Screening Card Tool to assess to what extent Universities are following

good Governance practices aligned with their Institutional Goals, but also to allow Universities monitor their progress and compare themselves with other institutions

Inspiration:

Guidelines and Good Practice Codes that have been revised by OECD

Autonomy Score Card- European University Association

CUC in the UK

Benchmarking guidelines- Australian Universities

West Coast Guidelines, USA

14

Institution-level screening card

DIMENSION 1: CONTEXT, MISSION and GOALS Are the missions of the University formally stated? DIMENSION 2: MANAGEMENT Are the management mechanisms results-based or traditional? DIMENSION 3: AUTONOMY What is the degree of academic, HR Management, and financial

autonomy? DIMENSION 4: ACCOUNTABILITY AND PARTICIPATION How much is the university held responsible vis à vis its stakeholders? Do the stakeholders have a voice in decision making?

15

Example of Autonomy - Financial Financial autonomy - ability of universities to:

set tuition fees

accumulate reserves

keep surplus on state funding

borrow money

invest money in financial or physical assets

own and sell the land and buildings they occupy

deliver contractual services;

attract funds on a competitive basis.

16

Example of Autonomy - Academic Academic autonomy

Responsibility for curriculum design

Extent to which universities are autonomous to introduce or cancel degree programs and to determine academic structure

Overall number of students

Admissions criteria

Admissions per discipline;

Evaluation of programs;

Evaluation of learning outcomes

Teaching methodologies.

17

Example of Autonomy – HR Human Resources autonomy

Recruitment procedures for appointment of academic and other staff – hiring and firing

Status of employees (whether they are considered civil servants or not)

Procedure for determining salary levels, salary incentives, and workloads

Human resources policies

Career development policies

Performance management.

18

MENA Case study: Sample size Egypt: 12 universities 6 Public 6 Private

Morocco: 9 universities 8 Public 1 Private

Palestine: 9 universities 2 Public 7 Private

Tunisia: 10 universities 7 Public 3 Private

Statistical Analysis

MCA for construction of indices of autonomy

Comparisons between countries/types of universities

Assessment of correlation structure & causality

19

MENA Case study: MCA results

20

Categories % Coord.

Financial Autonomy

Has autonomy to define revenue structure of the University (No) 0.08 1.994

Has autonomy to set the level of fees (No) 0.074 1.724

Has autonomy to set the level of fees (Yes) 0.074 -1.724

Has the autonomy to run a deficit (No) 0.07 1.678

Has the autonomy to run a deficit (Yes) 0.07 -1.678

Has autonomy to set bonuses to be paid to private owners (Yes) 0.063 -1.969

Allowed to own Financial Assets (No) 0.056 1.928

Allowed to own Land (No) 0.055 2.497

Has autonomy to define revenue structure of the University (Yes) 0.053 -1.329

Normalized Indices of autonomy

21

Indices of autonomy Country

Egypt Morocco Palestine Tunisia

Academic Autonomy 0.62 0.57 0.74 0.29

Human Resources Autonomy 0.76 0.27 0.75 0.36

Financial Autonomy 0.69 0.57 0.81 0.55

All Dimensions of Autonomy 0.80 0.49 0.86 0.42

Indices of autonomy Status

All Public Private

Academic Autonomy 0.48 0.65 0.55

Human Resources Autonomy 0.30 0.88 0.55

Financial Autonomy 0.52 0.84 0.65

All Dimensions of Autonomy 0.46 0.90 0.65

Correlations between indices

22

-2

-1.5

-1

-0.5

0

0.5

1

1.5

-3 -2 -1 0 1 2

Sta

ffin

g A

uto

no

my

Academic Autonomy

-2

-1.5

-1

-0.5

0

0.5

1

1.5

0 1 2 3 4 5 6

Sta

ffin

g A

uto

no

my

Autonomy Self Assessment

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

-3 -2 -1 0 1 2

Fin

an

cia

l A

uto

no

my

Academic Autonomy

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

-2 -1 0 1 2

Fin

an

cia

l A

uto

no

my

Staffing Autonomy

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

-2 -1 0 1 2

Fin

an

cia

l A

uto

no

my

Staffing Autonomy -2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

0 2 4 6

Fin

an

cia

l A

uto

no

my

Autonomy Self Assessment

Importance of various forms of autonomy

23

VARIABLES Autonomy index Subj. Autonomy

Autonomy

Academic autonomy index 1.72*** 1.17***

Staffing autonomy index 2.69*** 1.27***

Financial Autonomy index 1.86*** 0.86***

Size of the university

Log of the number of students -0.05 0.04

Countries (ref= Egypt)

Morocco -0.19 0.34*

Palestine -0.24 0.19

Tunisia -0.19 0.29*

Type of program (ref=PhD)

Undergraduate -0.21 -0.22

Under & Graduate 0.01 0.09

Status (ref=Public)

Private -0.02 0.13

Constant -2.13*** 1.40***

Observations 40 40

Conclusion SABER: New effort and framework to document and

assess policy frameworks

System-wide data and institutio-level data

Institution-level data helpful for implementation, but also calibration (weights) for system-wide indicators

Institution-level tool available for deployment in case of interest among participants at World Congress

Thank you!

24