QA POLICY
from VISION to KPIs
Lucien Bollaert
International independent QA expert
Vice-Chair EQAR Register Committee
Member of boards of AEQES, QQI, EQ-Arts
Visiting professor & lecturer & researcher
EURASHE – ARES Seminar on QA
6 & 7 February 2017
Brussels
future QA in HE contents
Intro: no KPI without vision
Development of a vision
From vision to mission
Turning mission into strategic objectives
Realizing strategy via action plans
Measuring realisation of strategy with KPIs
Conclusions: pitfalls & recommendations
E
X
E
R
C
I
S
E
Look critically to the global context
with your external & internal stakeholders
Renew your vision
Use a SWOT looking from outside to inside
Engage all your stakeholders
Renew your mission
Identify your existing organisational (Q)C
Identify your wished organisational QC
Engage all your external & internal stakeholders
Renew your (strategic) policy
Use SMART quantitative& qualitative KPIs & indicators
Communicate top-down & bottom-up
(Re)design your programmes & other processes
(re)design your organisation its governance &structures
Create independent facili- tating QA units
“DO” of PDCA
WHY?
this future of HE & Q(A)?
this vision of HE & Q(A)?
WHAT?
do we stand for in HE & its Q?is our mission in Q of HE?
HOW?Are we going to realize our mission?
is our mission in Q of HE?
WHICH?People (leaders & staff)?
Governance QA structure?
HOW?Which (new) strategic policy?
Which action plans?
HOW TO DO?
REAL PRACTICE
from vision to KPIs contents
QA POLICY
from VISION to KPIs
Intro: no KPIs without vision
The Future of HE in Europe :
“Currently, these policies and tools might not represent
the best answer to the question “are we ready for the
future?” This situation makes a fundamental re-thinking
of how a European common space for higher ecuation
could continue necessary. For this, we could build on
the positive experiences and achievements to date.”
“For that, however, a new vision is needed, not just
technical adjustments.”
“(...) inform decision-making and practical action,
coordinated or individually, in order to address new
and emerging national challenges, first, and then also
European challenges.”
“It is important to acknowledge that a European
common space for higher education can continue to
exist and play a positive role in the future even though
the European integration process might be stalled or even in the same ways reversed.”
from vision to KPIs intro: no KPIs without vision
from vision to KPIs intro: no KPIs without vision
Standard I.1:
“Institutions should have a policy for QA that is made public and forms part of their strategic management. Internal stakeholders should develop and implement this policy through appropriate structures and processes, while involving external stakeholders.”
In a changing world characterized by …
Globalization of economy & society;
Climate change;
Informatics turning into ‘internet in internet’ automatization
& robots …
… making use of big data;
Technological innovation;
Demographic changes turning the population older;
A widening gap between rich & poor;
The need of transferable competences for future jobs we
don’t know yet;
International tensions causing massive immigration;
Changing (social) communication, values and beliefs.
from vision to KPIs intro: no KPIs without vision
How does the world look like in 2030?
(Between 3 disasters & 2 utopia)
Automation (computers & robots) & ageing : no
new jobs
Climate change & hunger : new conflicts
Financial (bank) crisis (bis) : local economy vs
protectionism
Sustainability, ecological engineering,
cybersecurity : new high-education jobs
Creative society : new, local, ecological balance
between (part-time) work & life
OECD (2015), Securing livelihoods for all8
from vision to KPIs intro: no KPIs without vision
Challenges of HE in EHEA DIVERSITY in (historical & national) context, in vision & mission, in strategy,
in teaching & research, in practice & culture vs globalisation of knowledge society;
In majority PUBLIC, but with fixed 3 means of income: state/region + tuition + contracts (in competition) vs prolonged crisis & lump sum;
Academic AUTONOMY vs NATIONAL policy;
National external quality (EQA) system inspired by revised ESG andswinging to institutional level;
STAKEHOLDER MODEL, internal (students & staff), external (society, employers, parents, …) vs world of work;
Teaching & research & social impact vs learning & co-creation;
Democratically elected leadership with “blended” new public management vs CEO directives;
In need of new “transversal” competences vs immediate employment;
With student-centred learning strategies & processes vs massification of HE and exponential growth of knowledge;
Equal opportunities based on talents vs selection for excellence;
Underpinned by a culture of shared values & beliefs vs rankings:
individual development to citizenship,
key role in human, social and economic development via innovation;
knowledge co-creation & sharing;
from vision to KPIs intro: no KPIs without vision
Underlying principles & values
Leading with vision, inspiration and integrity by leaders acting as a role model for
their values and ethics (responsibility, people management…)
Managing through structured and strategically aligned processes using fact-
based decision making
Integrated system interconnecting all processes and measuring theirperformances;
Trust in continuous improvement;
Succeeding by people who are valued in a culture of empowerment for
balanced achievement of organisational and personal goals (democracy)
future QA in HE quality culture
QA POLICY
from VISION to KPIs
Development of a vision
Look critically to the global context
with your external & internal stakeholders
Renew your vision
WHY?
this future of HE & Q(A)?
this vision of HE & Q(A)?
from vision to KPIs development of a vision
START WITH THE QUESTION “WHY ?”
THEN ASK “HOW ?”
FINALLY ASK “WHAT ?”
Simon Sinek (2009)
13
from vision to KPIs development of a vision
Why are we in (higher) education?
Why do we want a better society?
Why do we need a better world?
Why do we want a better life?
Why do we exist? What is the added
value we provide to life on the world?
Why do we need quality (assurance)
management?
Why do we want quality? 14
From vision to KPIs development of a vision
New vision, mission & (strategic) policy of HE(I)
• Traditionally :
Education : gone global with new stress on global
competences fit for global knowledge society of the 21st
century
Research : more global than ever through rankings new
continuum from fundamental to applied fit for global
challenges & innovation
Social services/impact : from regional to global
from vision to KPIs development of a vision
“We prepare the leaders of tomorrow.”
“We nurture lifelong learners.”
“We aim to have a global impact, while serving our local
community.”
Gallup (2015) found that more than 50% of vision or mission statements of HEI share striking similarities, regardless of size, public or private, land-grant status or religious affiliation, or for-profit or not-for-profit.
They may accurately represent the broad views andaspirations of education leaders and their institutions, andthey probably differentiate the institutions from financial services and retail companies…
BUT THEY OFFER LITTLE GUIDANCE TO CURRENT AND FUTURE STUDENTS (and staff).
from vision to KPIs development of a vision
GALLUP’s RECOMMENDATIONS :
Establish a clear and differentiated purpose by answering
the questions: “Why do we exist?” and “What value do we
provide to the world?”.
Align the brand by telling the outside world what the
institution is and what it will deliver. (see mission)
Support identity with engaged culture primarily including
the student experiences that should support the HEI’s
purpose and brand.
Gallup (2015)
www.gallup.com/businessjournal/184538/hard-differentiate-one-higher-brand.aspx
from vision to KPIs development of a vision
New vision, mission & (strategic) policy of HE(I)
• Traditionally :
Education : gone global with new stress on global
competences fit for global knowledge society of the 21st
century
Research : more global than ever through rankings new
continuum from fundamental to applied fit for global
challenges & innovation
Social services/impact : from regional to global
• New vision & mission?
HE(I) as a open & global community of co-creation with
all the stakeholders underpinned by a quality culture
from vision to KPIs development of a vision
Possible classification of HEIs
1. Research driven university;2. Profession-oriented HEI;3. Civic college to become critical citizen;4. Open and Distance Learning;5. “Humboldt” model developing character;6. Liberal arts (and sciences) college;7. Technical university;8. Business/arts school;9. International;10. Excellent world-class university;11. Or any topic you think is essential for the future of HE in
your country/in the world.
from vision to KPIs development of a vision
Dirk Van Damme (OECD)
Q
U
A
L
I
T
Y
EDUCATIONAL STRATEGIC CHOICE and REALITY
secondary
secondary
QF minimum standard/LO
excellence
HE a
HE b
secondary
HE c
Secondary minimum leaving standards/LO
from vision to KPIs development of a vision
I
N
T
E
R
N
A
L
O
W
N
E
R
S
H
I
P
Reactive Quality Culture: sees quality as something that is
‘imposed’ from the outside environment and, thus, focuses on
individual aspects of quality and avoiding external threats;
Reproductive Quality Culture: emphasisthe maintenance of the
status quo, because changes lead to internal resistance;
Responsive Quality Culture: primarily evaluates its own practice
in the light of external quality requirements and contributes to
an improvement agenda;
Reproductive Quality Culture: typical of a ‘learning
organisation’ in which quality consciously is embedded in daily
operations;
4 types of quality culture
Harvey & Stensaker (2008)
from vision to KPIs development of a vision
identification of Quality Culture (existing & desired)
dr. Dries Berings (2011)
from vision to KPIs development of a vision
After the meeting they realized the new manager
suffered from severe adaptive difficulty
from vision to KPIs development of a vision
from vision to KPIs development of a vision
Why do we measure ?
To understand
To monitor
To manage
To improve
a best practice
How do you create strategic objectives?
NEW
STR
ATE
GY
CU
RR
EN
T
REA
LITY
vision misionStrategic
objective
LLL
competen
ces
Local-
oriented
Tknowledge
transfer
KPI
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Actions
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Local-
oriented
from vision to KPIs development of a vision
Tpeople-oriented
va
lue
Tcreative
QA POLICY
from VISION to KPIs
From vision to mission
Look critically to the global context
with your external & internal stakeholders
Renew your vision
Use a SWOT looking from outside to inside
Engage all your stakeholders
Renew your mission
WHY?
this future of HE & Q(A)?
this vision of HE & Q(A)?
WHAT?
do we stand for in HE & its Q?is our mission in Q of HE?
from vision to KPIs contents
from vision to mission
vision mission
• WHY do we exist? WHERE do we want to be?
WHY should we exist in HOW you will get there?
the (desired) future
• the FUTURE purposes PRIMARY purposes &
& values objectives related to your
STAKEHOLDERS’ needs &
desires
• Where do we AIM What do we DO?
to be? What makes us different?
from vision to KPIs from vision to mission
from vision to mission
vision mission
• Answers the WHY? & WHAT? Answers the HOW?
• Deals only with the FUTURE Deals with PRESENT leading to
the FUTURE
Answers WHEN? (10 y) Answers WHEN? (5 y)
• Should be INSPIRING Lists the broad GOALS
• For the leadership, all staff & Prime audience is the
potential clients leadership & stakeholders
• Tells what you stand for in Ties back to stakeholders’
the FUTURE (10 y) needs & desires
• WHERE do we want to go to? WHAT do we do today?
from vision to KPIs from vision to mission
desired future vision = answer to the question ‘why?’
to want
mission mission
future needed achievable future
to need to be able
Why?
What is our added
value?
What do we want
to?
What does our
(political) context
wish?
How will we do it?
What do we need
Which means do
we need?
How will we do it?
from vision to KPIs from vision to mission
from vision to KPIs development of a vision
Why do we measure ?
To understand
To monitor
To manage
To improve
a best practice
How do you create strategic objectives?
NEW
STR
ATE
GY
CU
RR
EN
T
REA
LITY
vision misionStrategic
objective
LLL
competen
ces
Local-
oriented
Tknowledge
transfer
SCL co-
creation
International
TC lectures
Locally
involved
KPI
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Actions
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Local-
oriented
from vision to KPIs from vision to mission
Tpeople-oriented
va
lue
Tcreative
QA POLICY
from VISION to KPIs
Turning mission into strategic objectives
Look critically to the global context
with your external & internal stakeholders
Renew your vision
Use a SWOT looking from outside to inside
Engage all your stakeholders
Renew your mission
Identify your existing organisational (Q)C
Identify your wished organisational QC
Engage all your external & internal stakeholders
Renew your (strategic) policy
WHY?
this future of HE & Q(A)?
this vision of HE & Q(A)?
WHAT?
do we stand for in HE & its Q?is our mission in Q of HE?
HOW?Are we going to realize our mission?
is our mission in Q of HE?
from vision to KPIs contents
from vision & mission to (strategic) policy future
WHY
vision WHY?
Mission WHY? & WHAT?
WHOare we?
do we want to be?
WHEREdo we want
to be active?
Which
ACTIVITIES?
WHATdo we want to reach
for our stakeholders?
HOWdo we want to reach
our objectives?
OBJECTIVES &
STRATEGY
HOWdo we want to deal
with our situation?
HOWdo we want to deal
with ourselves?
VALUES, BELIEFS &
CULTURE
Have our
activities
changed?
Do we
need a
new
strategy?
Has our
situation &
culture
changed?
From vision to KPIs turning mission into strategy
SWOT analysis
From vision to KPIs turning mission into strategy
future
turn W into S
turn T into O
From vision to KPIs turning mission into strategy
from vision to KPIs turning mission into strategy
Why do we measure ?
To understand
To monitor
To manage
To improve
a best practice
from vision to KPIs turn the mission into a strategy
Why do we measure ?
To understand
To monitor
To manage
To improve
From vision to KPIs: an example
How do you create strategic objectives?
NEW
STR
ATE
GY
CU
RR
EN
T
REA
LITY
vision misionStrategic
objective
LLL
competen
ces
Local-
oriented
Tknowledge
transfer
SCL co-
creation
International
TC lecturesBuild more
auditoria
Locally
involved
Recruit more
locals
Move from
local to
international
Move from
TCL to SCL
KPI
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Actions
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Local-
oriented
from vision to KPIs turn the mission into a strategy
Tpeople-oriented
va
lue
Tcreative
VERB+ ADJECTIVE +NOUN
QA POLICY
from VISION to KPIs
Realizing strategy via action plans
Look critically to the global context
with your external & internal stakeholders
Renew your vision
Use a SWOT looking from outside to inside
Engage all your stakeholders
Renew your mission
Identify your existing organisational (Q)C
Identify your wished organisational QC
Engage all your external & internal stakeholders
Renew your (strategic) policy
Use SMART quantitative& qualitative KPIs & indicators
Communicate top-down & bottom-up
(Re)design your programmes & other processes
WHY?
this future of HE & Q(A)?
this vision of HE & Q(A)?
WHAT?
do we stand for in HE & its Q?is our mission in Q of HE?
HOW?Are we going to realize our mission?
is our mission in Q of HE?
HOW?Which (new) strategic policy?
Which action plans?
from vision to KPIs realizing strategy via action
plans
intended LO
LO new students study programme achieved LO
mission strategy & policies results
input outputprocesses
transformation = value added
From vision to KPIs realizing strategy via action plans
Nine out of 10
organizations fail
to execute
strategy !
Building KPI Tree
1st level
Total score of HEI
2nd level
The criteria (e.g.) teaching | research
supporting
3rd level
The rating scale
contains KPIs related to each criteria, and its rating scale
Suryadi, Kadarsah (2007), “Key Performance Indicators Measurement Model Based on Analytic Hierarchy Process and Trend-Comparative Dimension in Higher Education Institution”, Bandung Institute of Technology, Chile
The Balaced Scorecard Tree
From vision to KPIs realizing strategy via action plans
Building KPI Tree
1st level
Total score of HEI
2nd level
The criteria (e.g.) teaching | research
supporting
3rd level
The rating scale
contains KPIs related to each criteria, and its rating scale
Suryadi, Kadarsah (2007), “Key Performance Indicators Measurement Model Based on Analytic Hierarchy Process and Trend-Comparative Dimension in Higher Education Institution”, Bandung Institute of Technology, Chile
The Balaced Scorecard Tree
The Balanced Scorecard is the
framework that links your purpose
statement to the execution of your
strategy.
From vision to KPIs realizing strategy via action plans
Is the objective precise and well-defined?
Is it clear? Specific S
Can everybody understand it?
How will the individual know when the task
has been completed? Measurable M
How will that be judged?
What evidence is needed to confirm it?
Is the objective achievable?
Is it within their capabilities? Action-related A
Are there sufficient resources?
Is it possible for the individual(s) to perform?
How sensible is the objective in the current Realistic R
Does it fit?
Is there a deadline? Is it feasible to meet?
Are there review dates? Time-related T
Is it appropriate to undertake the work now?
From vision to KPIs realizing strategy via action plans
intended LO
LO new students study programme achieved LO
mission strategy & policies results
influenced by
development phase
input outputprocesses
(Internal) Quality Assurance (QA)
transformation = value added
methodology
tool tool tool tool tool
“QA is a management approach to focus on the quality of the
organisation and is based on participation of all stakeholders in order to
satisfy their expectations and aims as long as possible” (ISO)
From vision to KPIs realizing strategy via action plans
input outputprocesses
Internal Quality Assurance (QA)
transformation = value added
methodology
tool tool tool tool tool
“QA is a management approach to focus on the quality of the
organisation and is based on participation of all stakeholders in order to
satisfy their expectations and aims as long as possible” (ISO)
underlying principles
underlying (hidden) values
intended LO
LO new students study programme achieved LO
mission strategy & policies results
influenced by
development phase
influence on
development
phase
(Internal) Quality Assurance (QA)
From vision to KPIs realizing strategy via action plans
From vision to KPIs realizing strategy via action plans
Why do we measure ?
To understand
To monitor
To manage
To improve
From vision to KPIs: an example
From vision to KPIs realizing strategy via action plans
How do you create strategic objectives?
NEW
STR
ATE
GY
CU
RR
EN
T
REA
LITY
vision misionStrategic
objective
LLL
competen
ces
Local-
oriented
Tknowledge
transfer
SCL co-
creation
International
TC lecturesBuild more
auditoria
Locally
involved
Recruit more
locals
Move from
local to
international
Move from
TCL to SCL
KPI
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Actions
-
-
-
-
-
-
-
-
-
- Build new building
- Renovate old buil-
dings with auditoria
- Go to high schools
- Recruit local profs
-
Local-
oriented
Tpeople-oriented
va
lue
Tcreative
VERB+ ADJECTIVE +NOUN
QA POLICY
from VISION to KPIs
Measuring realisation of strategy with
KPIs
Look critically to the global context
with your external & internal stakeholders
Renew your vision
Use a SWOT looking from outside to inside
Engage all your stakeholders
Renew your mission
Identify your existing organisational (Q)C
Identify your wished organisational QC
Engage all your external & internal stakeholders
Renew your (strategic) policy
Use SMART quantitative& qualitative KPIs & indicators
Communicate top-down & bottom-up
(Re)design your programmes & other processes
(re)design your organisation its governance &structures
Create independent facili- tating QA units
“DO” of PDCA
WHY?
this future of HE & Q(A)?
this vision of HE & Q(A)?
WHAT?
do we stand for in HE & its Q?is our mission in Q of HE?
HOW?Are we going to realize our mission?
is our mission in Q of HE?
WHICH?People (leaders & staff)?
Governance QA structure?
HOW?Which (new) strategic policy?
Which action plans?
HOW TO DO?
REAL PRACTICE
from vision to KPIs measuring strategy with KPIs
intended LO
LO new students study programme achieved LO
mission strategy & policies results
use standards
& indicators
influenced by
development phase
input outputprocesses
(Internal) Quality Assurance (QA)
transformation = value added
methodology
tool tool tool tool tool
“QA is a management approach to focus on the quality of the
organisation and is based on participation of all stakeholders in order to
satisfy their expectations and aims as long as possible” (ISO)
influence on
development
phaseunderlying (hidden) values
underlying principles
from vision to KPIs measuring strategy with KPIs
Standards & indicators
Standard : a (kind of) description that describes
how the organisation and its people should
develop and manage materials, products, services,
processes & systems in order to satisfy the needs &
expectations of its stakeholders
Threshold or excellence standards?
Guideline : helps to explain how processes &
procedures should/can help in order to meet the
standard (good practice)
from vision to KPIs measuring strategy with KPIs
Indicator : breaks up standards into measurable and
appropriate units in order to measure how much the
organisation has achieved or is compliant with the
standards
Indicators should give information on the
organisation’s performance as far as quality of its
requirements are concerned and help the
organisation to improve its performance
3 functions of indicators : measure, signal &
communicate (internal & external)
from vision to KPIs measuring strategy with KPIs
Why do we measure ?
To understand
To monitor
To manage
To improve
from vision to KPIs measuring strategy with KPIs
(sub-)categories of indicators
Quantitative indicators with a number
Qualitative indicators can’t be presented as a number
Input indicators measure the situation at the start of a process or the
amount of resources consumed during the generation of the outcome
Process indicators represent the efficiency or productivity of the addedvalue during the process
Output indicators reflect the outcome of results of the process activities
Directional indicators indicate whether the organization is getting
better
Actionable indicators measure the organization’s control to effect
change
(non-)Financial indicators (don’t) measure the finances of operations
Strategic indicators measure the realisation of strategic goals
from vision to KPIs measuring strategy with KPIs
Admission
Advancement
Alumni
Athletics
Business connections
Community connections
Course measures
Connections with other
educational institutions
Employee & HR
Enrollment
Facilities
Financial Aid
Graduation Measures
Grants & Research
Library
Other
Peer comparisons
Retention
Satisfaction
Strategic planning
Student engagement
Student success
Technology
Areas Measured
Ballard, J. Paul (2013), Measuring Performance Excellence:
Key Performance Indicators for Institutions Accepted into the Academic Quality Improvement
Program (AQIP), Western Michigan University dissertation
from vision to KPIs measuring strategy with KPIs
Building KPI Tree
1st level
Total score of HEI
2nd level
The criteria (e.g.) teaching | research
supporting
3rd level
The rating scale
contains KPIs related to each criteria, and its rating scale
Suryadi, Kadarsah (2007), “Key Performance Indicators Measurement Model Based on Analytic Hierarchy Process and Trend-Comparative Dimension in Higher Education Institution”, Bandung Institute of Technology, Chile
The Balaced Scorecard Tree
from vision to KPIs measuring strategy with KPIs
Building KPI Tree
1st level
Total score of HEI
2nd level
The criteria (e.g.) teaching | research
supporting
3rd level
The rating scale
contains KPIs related to each criteria, and its rating scale
Suryadi, Kadarsah (2007), “Key Performance Indicators Measurement Model Based on Analytic Hierarchy Process and Trend-Comparative Dimension in Higher Education Institution”, Bandung Institute of Technology, Chile
The Balaced Scorecard TreeBuilding KPI Tree
1st level
Total score of HEI
2nd level
The criteria (e.g. teaching | research
supporting
3rd level
The rating scale
contains KPIs related to each criteria, and its rating scale
Suryadi, Kadarsah (2007), “Key Performance Indicators Measurement Model Based on Analytic Hierarchy Process and Trend-Comparative Dimension in Higher Education Institution”, Bandung Institute of Technology, Chile
from vision to KPIs measuring strategy with KPIs
Building KPI Tree
1st level
Total score of HEI
2nd level
The criteria (e.g.) teaching | research
supporting
3rd level
The rating scale
contains KPIs related to each criteria, and its rating scale
Suryadi, Kadarsah (2007), “Key Performance Indicators Measurement Model Based on Analytic Hierarchy Process and Trend-Comparative Dimension in Higher Education Institution”, Bandung Institute of Technology, Chile
The Balaced Scorecard TreeBuilding KPI Tree
1st level
Total score of HEI
2nd level
The criteria (e.g. teaching | research
supporting
3rd level
The rating scale
contains KPIs related to each criteria, and its rating scale
Suryadi, Kadarsah (2007), “Key Performance Indicators Measurement Model Based on Analytic Hierarchy Process and Trend-Comparative Dimension in Higher Education Institution”, Bandung Institute of Technology, Chile
from vision to KPIs measuring strategy with KPIs
KPIs are
performance indicators
to measure the realisation
of an organization’s
particular activity or
strategic goal.
From strategy goals to KPIs: an example
prio
ritie
so
bje
ctiv
es
KP
Is
Enhance
quality
academic
provision
Improve
student
satisfaction
Grow
income
across all
areas
Improve
productivity
Develop teaching portfolio with stronger emphasis on science &
technology
Enhance learning & working environment
Build productive partnerships
Improve marketing effectiveness
Implement research, enterprise & endowment
strategies
QA Audits Student surveys RAE outcomes Overheads
% staff cost
Property cost
per student
FTE
Market share
Research
income
Business
income
Proportion of 1st & 200
Retention Student
employment
from vision to KPIs measuring strategy with KPIs
from vision to KPIs measuring strategy with KPIs
Why do we measure ?
To understand
To monitor
To manage
To improve
From vision to KPIs: an example
from vision to KPIs measuring strategy with KPIs
Why do we measure ?
To understand
To monitor
To manage
To improve
From vision to KPIs: an example
from vision to KPIs measuring strategy with KPIs
Why do we measure ?
To understand
To monitor
To manage
To improve
From vision to KPIs: an example
from vision to KPIs measuring strategy with KPIs
Why do we measure ?
To understand
To monitor
To manage
To improve
From vision to KPIs: an example
from vision to KPIs measuring strategy with KPIs
Why do we measure ?
To understand
To monitor
To manage
To improve
From vision to KPIs: an example
from vision to KPIs measuring strategy with KPIs
How do you create strategic objectives?
NEW
STR
ATE
GY
CU
RR
EN
T
REA
LITY
vision misionStrategic
objective
LLL
competen
ces
Local-
oriented
Tknowledge
transfer
SCL co-
creation
International
TC lecturesBuild more
auditoria
Locally
involved
Recruit more
locals
Move from
local to
international
Move from
TCL to SCL
KPI
-
-
-
-
-
-
-
-
-
- # buildings
- # auditoria
-
- Ratio local
vs non-local
student/prof
Actions
-
-
-
-
-
-
-
-
-
- Build new building
- Renovate old buil-
dings with auditoria
- Go to high schools
- Recruit local profs
-
Local-
oriented
Tpeople-oriented
va
lue
Tcreative
VERB+ ADJECTIVE +NOUN
from vision to KPIs measuring strategy with KPIs
How do you create strategic objectives?
NEW
STR
ATE
GY
CU
RR
EN
T
REA
LITY
vision misionStrategic
objectiveKPI
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Actions
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
va
lue
Strategic monitoring and follow-up form
NEW
STR
ATE
GY
TIP
S F
OR
FU
TUR
E
IMP
RO
VEM
EN
T
Strategic
objective
-
-
-
-
-
-
-
-
-
KPI
-
-
-
Actions
-
-
-
-
-
-
-
-
-
-
-
-
va
lue
By whom?When?
(deadline)
realisa
tionComment
(lesson learnt)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
from vision to KPIs measuring strategy with KPIs
QA POLICY
from VISION to KPIs
Conclusions: pitfalls & recommendations
KPIs and metrics
Metrics are nationally imposed quantitave indicators that
are supposed to indicate the degree of quality or of
performance of a HEI
They are mostly linked to policies on subsidy from the
state/government
Sometimes linked with risk-based approach in national
EQA
Do not take into account the differences in mission,
strategy, resources, locations, etc of HEI
“Don’t track traditional metrics. Instead of worrying about typical customer
satisfaction measures (…), organizations should look at the number of value-
adding service ideas put into practice. It’s not that conventional metrics are
unimported, the researchers say, but because they are “lagging indicators”,
they can bog down efforts to achieve rapid, dramatic change.”
Mark Pernice, “Revolutionizing Customer Service”, HBR April 2016, p.27
from vision to KPIs conclusions: pitfalls & recommendations
from vision to KPIs conclusions: pitfalls & recommendations
“IN GOD
WE TRUST,
ALL
OTHERS
BRING
DATA.”
W. Edwards
Deming
7 deadly sins of
measuring and how
to avoid themMichael Hammer (2007)
1. Vanity: only measure to look
good.
2. Provincialism: organizational
boundaries and concerns
dictate the metrics.3. Narcissism: measure from
one’s own point of view rather
than from the customer.
4. Laziness: assuming one knows
what is important to measure
without giving it adequate
thought or effort.
5. Pettiness: measure only a small
component of what matters.
6. Inanity: not thinking about the
consequences on human
behaviour & performance.
7. Frivolity: not being seriousabout measurement in the first
place.
from vision to KPIs conclusions: pitfalls & recommendations
“IN GOD
WE TRUST,
ALL
OTHERS
BRING
DATA.”
W. Edwards
Deming
7 deadly sins of
measuring and how
to avoid themMichael Hammer (2007)
1. Vanity: only measure to look
good.
2. Provincialism: organizational
boundaries and concerns
dictate the metrics.3. Narcissism: measure from
one’s own point of view rather
than from the customer.
4. Laziness: assuming one knows
what is important to measure
without giving it adequate
thought or effort.
5. Pettiness: measure only a small
component of what matters.
6. Inanity: not thinking about the
consequences on human
behaviour & performance.
7. Frivolity: not being seriousabout measurement in the first
place.
Look critically to the global context
with your external & internal stakeholders
Renew your vision
Use a SWOT looking from outside to inside
Engage all your stakeholders
Renew your mission
Identify your existing organisational (Q)C
Identify your wished organisational QC
Engage all your external & internal stakeholders
Renew your (strategic) policy
Use SMART quantitative& qualitative KPIs & indicators
Communicate top-down & bottom-up
(Re)design your programmes & other processes
(re)design your organisation its governance &structures
Create independent facili- tating QA units
“DO” of PDCA
WHY?
this future of HE & Q(A)?
this vision of HE & Q(A)?
WHAT?
do we stand for in HE & its Q?is our mission in Q of HE?
HOW?Are we going to realize our mission?
is our mission in Q of HE?
WHICH?People (leaders & staff)?
Governance QA structure?
HOW?Which (new) strategic policy?
Which action plans?
HOW TO DO?
REAL PRACTICE
from vision to KPIs conclusions: pitfalls & recommendations
from vision to KPIs conclusions: pitfalls & recommendations
Improving through measuring (IQA & EQA)
By setting goals
measurable
through KPIs
By measuring
the realisation
using KPIs
By setting new
measurable goals
having learnt from
the previous
measurement
Measuring with wisdom
Knowing what needs to be measured is just the first
step …
finding the right way to do so is the next.
Deciding what needs to be measured is something
of a science;
deciding how to measure remains an art.
Organizations often construct complex mechanisms
for calculating a metric, when, in many cases, a far
simpler one would suffice.
from vision to KPIs conclusions: pitfalls & recommendations
Recommendations (1)
Develop a future-oriented vision with all your stakeholders
Translate in into an inspiring but achievable mission
Turn that mission into a strategy developed by all
stakeholders
Formulate (strategic) objectives in relation to your mission in
SMART terms
Use KPIs and indicators that are related to your own vision,
mission and strategy
Use KPIs and indicators that really tell something about the
degree of quality or of performance of a HEI
Use both quantitative and qualitative indicators
from vision to KPIs conclusions: pitfalls & recommendations
Recommendations (2)
Communicate them, their resulting measurements, their
analyses and lessons with all your stakeholders
Use indicators to monitor and to improve
Some indicators of some rankings (primarily on research)
may be inspiring or useful
(Inter)national metrics can only be used in connection with
your own indicators linked to your mission and strategy
Learn lessons by the results of the realisation and give tips
(out of the box) to change for the better
from vision to KPIs conclusions: pitfalls & recommendations
from vision to KPIs conclusions: pitfalls & recommendations
from vision to KPIs conclusions: pitfalls & recommendations
THANKS again
Q & A