session 2 creating hei performance indicators framework by

Topic 2 Creating HEI Performance

Indicators Framework

Indicators for measuring higher education performance

September 8, 2016

Beirut, Lebanon, Dr. prof. Tatjana Volkova

Indicators for measuring higher education performance1

National Erasmus + Office

Lebanon

Higher Education Reform

Experts

In collaboration with the Issam Fares Institute and the

Office of Institutional Research and Assessment

Ministry of Education and Higher Education

Directorate General of Higher Education

Agenda

1. PM Frameworks practices

2. Strategic PM framework and its connection with Quality

Management System

3. Barriers and enablers to the effective use of performance

frameworks


1. Critical review of PI Frameworks

There are no commonly agreed PI

Framework;

It varies across HEIs depending on the

context (e.g. legislation requirements),

mission of the HEI, stage of development,

on the level of development; size of HEIs,

priorities of development, type of HEIs

(research intense, teaching intense),

leadership vision, etc.

It is up to HEIs to select areas of

strategic importance;

http://www.bersin.com/uploadedImages/Starter_Apps/Wiki/Lexicon/PM%20Maturity%20Model.jpg


The primary purpose of the Key Performance Indicator

(KPI) Framework

is to support management decision making within the University.

“…. it seeks to provide Council, the Executive and senior management with informative insights into how UTS is performing as a whole against its strategic planning framework and how it is likely to perform in the future. UTS evaluates its performance against the KPIs biannually”

http://www.pqu.uts.edu.au/tracking-performance/key-performance-indicators.html

http://www.pqu.uts.edu.au/tracking-performance/key-performance-indicators.html


Focus on Results and Creating Value:

An organization’s performance

measurements need to focus on key

results;

Results should be used to create

and balance value for students

and for key stakeholders—the

community, employers, faculty and

staff, suppliers and partners, and the

public thus contributing to improving

overall performance and builds loyalty.


http://www.pqu.uts.edu.au/tracking-performance/_images/kpi-framework-2013-1.jpgIndicators for measuring higher education performance6

htt

p://1

7986-p

ress

cdn-0

-31.p

agely

.netd

na-

cdn.c

om

/wp-

conte

nt/

uplo

ads/

site

s/2/2

016/0

7/c

urt

in-2

015-a

nnual

-report

-

kpis

.pdf


http://researchanalytics.thomsonreuters.com/m/pdfs/higher-ed-exec-

report.pdf


http

s://ww

w.n

ottin

gham

.ac.u

k/h

r/guid

esan

dsu

pport/p

erfo

rman

ceatw

ork

/pdpr/d

ocu

ments/p

dprfr

amew

ork

.pdf


https://www.newcastle.edu.au/?a=185090

The Performance Expectations

Frameworks for Academic and

Professional Staff have been developed to

support the University's vision and

strategic direction. The aim of these

Frameworks is to clearly align staff

performance with the University's goal of

delivering excellence in research and

teaching.


Findings reveal that the most frequently used

KPI include:

a specialized student

survey (used by 94% of

institutions);

overall retention (91%),

employee satisfaction

(85%) and

graduation rate (79%). https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSZQmvPnKKX6F2LP4VWqCfNEA951-

Qdlqq_2p9efrXyNpM4-eOe


The least frequently used KPI include

adult learner measures

(15%),

completion rate (9%),

diversity measures (9%),

and

tuition as compared to

peers (6%).

http://4.bp.blogspot.com/-

v_oY4GDl6zQ/Tc3QqTlPRuI/AAAAAAAAAAM/0RuAJSy8nNc/s1600/adult%252520learners.jpg


9 categories are suggested by researchers to assign PI

related to academic performance (Paul J. Ballard)

1. Helping Students Learn

2. Accomplishing Other Distinctive Objectives

3. Understanding Students’ and Other Stakeholders’ Needs,

4. Valuing People

5. Leading and Communicating

6. Supporting Institutional Operations

7. Measuring Effectiveness

8. Planning and Continuous Improvement

9. Building Collaborative Partnerships


2. Strategic Performance management framework

What we are going to measure?

Why we need to measure?

Are those areas of

measurement are relevant to

particular HEI?

What kind of indicators to

select?

Are those indicators relevant?

http://www.studentpulse.com/imgs/640/scale/article-images/uid-140-1014000958-3649/a673b3.jpg


Strategic PM

Takes a long-term view of HEIs

directions of development :

To select the strategic directions of

development in order to spot new

strategic choices;

To identify the best approaches for

setting goals;

To identify the winning approaches

that successful peers use and apply

them to their own strategic process.

http://www.dumblittleman.com/wp-content/uploads/2014/04/Decision-Making-Skills.jpg


In spite on diverse practices there are 4 main

strategic areas for performance consideration:

financial sustainability,

services delivered to

stakeholders,

quality of processes,

development of HR and

growth.


STRATEGIC PM framework for HEI: Balanced approach

What we are going to achieve? Setting strategic directions: Vision, Mission, Values and Goals:

Financial perspective (financialsustainability)

Services to society (Teaching& Learning, Research, Researchtransfer, Service to society (incl. LLL, regional engagement)

Internal Processes perspective

Learning and Growth perspective


Strategic performance framework development

1. Before you start the process of finding KPIs, you must acquire a very goodunderstanding of strategic directions of HEI (vision, mission, values and goals).

2. Once you have determined your strategic goals, you will select KPIs for each of these goals.


If there are to many KPI, it will be difficult to manage and

measure.

So it needs to select the most important KPIs that have significant

contribution to HEI performance.

In order to class this list in descending order of relevance, the survey could

be conducted by involving parties who are directly involved in HEI activities.

In this case, a questionnaire, in which the responsible persons have to give a

mark to each criteria, could be distributed.

The three-point scale of “not important”, “somewhat important” and “very

important” could be used;

Its result is the selected KPIs according to its degree of importance.


Setting targets and assigning KPIs (sample)


Financial perspective

Aim Strategic target Measurement KPI

To ensure financial

stability

To increase revenue

To decrease costs

To increase income from

tuition fees

To outsource cleaning

services

Increase by 20 % in

2017

Decrease by 5 % in

2017

28

Samples: performance indicators for ensuring

Financial sustainability

operating income ratio,

tuition fees from international students;

resources per student,

resources per faculty,

research income;

debt burden ratio,

viability ratio,

service expenses, etc..


Setting targets and assigning KPIs (sample)


Research activities

Aim Strategic target Measurement KPI

To ensure industry

engagement

To increase industry

research income

Patenting income

Licensing income

Spin-out generation income

Consultation income

Increase by X % in

2017 (each area

needs to be

specified)

30

Tool for selecting appropriate PI


The survey by involving parties who are directly involved in HEIs activities

would be helpful to select relevant indicators;

A questionnaire in which the responsible persons have to give a mark to each

metrics could be designed and distributed;

The 3 point scale of “not important” , “ somehow important” and “very

important” could be used;

As the result those with highest degree of importance could be selected;

The survey of key stakeholders on KPIs focus could be

conducted

http://www.nousgroup.com/images/news_attachments/COAG_Reform_Council_-

_performance_framework_research_project_140411.pdfIndicators for measuring higher education performance32

Strategic PM framework connection with Quality

Management System

PM is a tool to enhance

Quality of HEIs activities;

There is a clear connection

between PM and QM;

PM is aiming at monitoring

the quality results achieved by

delivering HEIs activities;

http://www.newdirectionsconsulting.com/wp-content/uploads/2012/08/ScientistQualityHabit.gif


htt

p://w

ww

2.w

arw

ick.a

c.uk/fac

/sci

/wm

g/ft

msc

/cours

es/

mbe-

hom

e/m

odule

s/efq

m/e

fqm

_m

odel.j

pg


Total Quality Management system


https://www.elsevier.com/__data/assets/image/0005/33359/Snowball-Metrics-landscape.jpgIndicators for measuring higher education performance38

Research performance framework

Research inputs

• Research grants: research applications, research awards;

• Enterprise activities/ economic development: Industrial

income: industry engagement

Post-graduate education

Contributors: People: researchers and research assistants

Research process

• Research income

• Industry research income

• Post-graduate experience: facilities

• Organizations

Research output and outcomes

• Publications & citations, collaboration

• Patenting, licensing income, spin-out generation

• Completion rates

• Themes/schemes


3. Barriers and enablers to the effective use of performance

frameworks

challenge to have all data needs met;

small sample sizes can make it difficult to demonstrate improvements or progress with any statistical certainty;

promoting a culture of evidence-based decision-making, while laudable, could also be a constraining factor;

for the success of major longer term reforms to develop a more sophisticated understanding of what the medium-term targets ought to be;

dilemma of simplicity and complexity;


Enablers to the effective use of performance frameworks

Leadership awareness and communication platform;

PM system in place;

performance information that would be publically shared in an accessible format;

high performance have to be rewarded;

performance management culture;

benchmarking;

PM as ongoing process..

http://wpmu.mah.se/comdev/files/2011/05/WB-cartoon-page-4.jpgIndicators for measuring higher education performance42

Enablers to the effective use of performance frameworks

(cont)


the potential within the public sector to benchmark universities is

almost certain to be considered by governments and funding bodies;

….it is not impossible that the development might occur of compulsory

institutional participation in benchmarking;

Although such an approach might be resisted by both benchmarking

specialists and many of the universities concerned, such a compulsory

approach would be consistent with the imposition of other quality

management and assurance mechanisms in many higher education

systems.

http://science.uniserve.edu.au/courses/benchmarking/benchmarking_manual.pdfIndicators for measuring higher education performance44

Types and ways of benchmarking


There are 3 major types of

benchmarking:

Strategic

Performance

Process

BENCHMARKING TYPES


• Long term view

• Identifying and applyingbest approaches to setstrategic directions

StrategicBenchmark

• Selecting areas of measurement

• Identifying indicators of measurement

• Measuring achievementsfor improvement

PerformanceBenchmark • Finding and application

of best practises in keymanagement processes

Process Benchmark

BENCHMARKING TYPES


Determining what University is doing strategically is sometimes easier than trying to

learn how they deliver individual procedures.


Strategic directions can be found in

annual reports, often publicly

available.

Selecting a partner University to

study makes it easier to approach

peers and ask for advice and

information on how organization

built its success.

Source: https://static.securityintelligence.com/uploads/2015/07/Security-Decision-Making-938x535.jpg

Performance BM


In the selection of indicators care was taken to check whether they:

• measure what they claim to measure;

• focus on the performance of programmes or institutions;

• are defined so as to measure “relative” characteristics and thus avoid

dependence of institutional size;

• are already known as a result of benchmarking or ranking exercises;

• provide comparisons from one situation, system or location to another;

• draw on data which is available in existing databases or at higher education

and research institutions,

or which can be collected without too much difficulty (Federkeil et al., 2012c).

Ways for benchmarking


It is important to choose the optimal

approach as it reduces the costs of

the activity and improves the chances

to find the ‘best practices’ you can rely

on

Internal/external

Functional

Generic

Competitive/ collaborative

Internal BM


comparisons are made of the performance of different departments, campuses

or sites within a university

to identify best practice in the institution, without necessarily having an

external standard against which to compare the results;

This type may be particularly appropriate to universities where a high degree of

devolvement exists to the constituent parts of the institution, where a multi-

campus environment exists, or where extensive franchise arrangements exist

whereby standard programs are taught by a number of partner colleges in

different locations.

External (competitive) BM


Comparisons of performance in key

areas is based upon information

from institutions which are seen as

“best in the class” or «competitors»

May be run by the institutions

themselves on a collective basis;

Consultant may be invited in order

to ensure continuity and sufficient

momentum

Higher possibilities of finding the

best practices

Functional BM


Managers of functional departments

find it useful to analyze how well their

functional area performs compared to

functional areas of other Universities in

a selected number of activities;

It is quite easy to identify the best

strategic management, marketing,

finance, human resource, real estate,

library, research, internationalization or

IT departments in other Universities

that excel in what they do and to apply

their practices to your own functional

area http://www.wiesbadener-kurier.de/fm/819/thumbnails/CON_513989848_54831_M.jpg.21626268.jpg

Estate


AUDE (Association of University

Directors of Estates) states that it

promotes excellence in the strategic

planning, management, operation and

development of Higher Education

estates and facilities

(http://www.aude.ac.uk/home).

AUDE is critically involved in the Estate

Management Statistics (EMS) collection,

now transferring to HESA from IPD

Occupiers; a technical implementation

group has been set up and a member of

the AUDE Executive Committee sits on

the HESA HEI User Group.

In the area of estates and facilities, it is

common to benchmark against

commercial sector standards

https://www.hesa.ac.uk/benchmarking_report_nov10

http://www.uky.edu/iraa/content/tuition-and-fees-benchmarkcomparisonsIndicators for measuring higher education performance56

IT Benchmarking: Case of University of Westminster (UK)


Delivered strategic IT outsourcing

to GOOGLE;

Implemented GOOGLE Apps

accross the whole University

Saved ~1 million pounds to date on

ICT costs

Improved email storage capabilities

Universities UK Efficiency and effectiveness in higher education, 2011, p.55

Functional benchmarking: Research management and

knowledge transfer


Benchmarking research activity is widespread, undertaken at the level of

inputs, outputs and grant awards.

Full economic costing has also provided a potential resource of comparable

data

Some HEIs have Research Management Systems to monitor and manage their

research activity, thus allowing for available management information.

The research‐intensive HEIs share application and award data at institutional

level on a collaborative and confidential basis

Generic benchmarking


Comparing your University to some generic (or ‘gold’) standards, which

would indicate that your University is a learning organization, you would be

using generic benchmarking

‘Implicit benchmarking’ is likely to increase in future years as governments and

central funding agencies seek to apply benchmarking approaches to

universities. F. e. Research excellence initiative in Germany, UK

Benchmarks for Effective University Administration


There is some evidence that:

successful BM is much more likely to take place in those organizations that are already relatively well managed and with a strong emphasis on continuous improvement philosophy and innovation;

are likely to be less effective in individualistic or centralized cultures where the long term commitment for benchmarking is unlikely to be found.

http://www.universitiesuk.ac.uk/highereducation/Documents/2011/EfficiencyinHigherEducation.pdf


«.. the development of a common

framework for efficiency benchmarking

in higher education should also seek to

provide guidance on the link between

costs and outputs, so that institutions

can measure the services delivered

against the costs of delivery»

Source: Efficiency and effectiveness in higher education Universities UK

Benchmarks for Effective University Administration


Not only are organizational cultures and structures important in determining

the appropriateness of benchmarking and related quality approaches.

The meanings attached to benchmarking processes, the organizational

assumptions on which they rest, and the consequent success of any

benchmarking initiatives are likely to be significantly affected by the

social culture of the university system concerned

Developing performance management culture


However, developing a positive PM

culture that adds value will not

happen overnight rather it will evolve

over time to fit an organizations

requirements and to be accepted

and embraced by all employees.

http://pwc.blogs.com/milton-keynes/2016/01/unlocking-the-

value-of-performance-management-in-higher-education.html

http://www.michaelbarata.com/wp-content/uploads/2013/03/communication-.jpg

Building PM culture
