assuring quality at an international level level chea 28 january 2016
TRANSCRIPT
ASSURING QUALITY AT AN INTERNATIONAL LEVEL:
VIEWS FROM THE OECD
Dirk Van DammeOECD/EDU/IMEP
• Consolidation and institutionalisation of external QA in many countries
• Increasing international collaboration, exchange, mutual recognition, shared practices
• Networks: ENQA, INQAAHE, CIQG• Frameworks, guidelines: ESG, OECD
Guidelines• Register: EQAR
International quality assurance: lot of progress
• The world of higher education is changing fast• With changes that are sometimes disruptive• While QA is also criticised on many fronts
Is the global quality assurance and accreditation system, based on self-regulation, on values, assumptions, concepts, and methodologies built up in the past, capable of catching up and continue to lead higher education into the future?
BUT…
TREND 1.CONTINUED EXPLOSION OF
DEMAND
Global expansion & redistribution of qualificationsGlobal distribution of tertiary educated 25-34 y-olds in 2013 and 2030
6
United States, 13.7%
China, 17.8% Russian Federation,
10.9%
Japan, 6.9%
India, 11.4%
Korea, 3.9%Mexico, 3.0%France, 2.6%Germany, 2.0%United Kingdom,
2.9%
Indonesia, 4.3%
Spain, 2.2%
Canada, 2.1%
Brazil, 3.0%
Turkey, 1.7%
Other, 11.7%
Share in academic graduates 2010 Share in academic excellence 2012
United States43.2%
United Kingdom13.8%
Nether-lands6.0%
Germany4.3%
Canada4.3%
Australia4.3%
Switzer-land3.5%
France3.0%
Japan2.5%
Sweden2.6%
Korea2.2%
Hong Kong2.0% Other
8.4%
Share in academic excellence THEWUR 2012
Global distribution of academic graduates and academic excellence
• Explosion of demand is taking place in parts of the world where quality assurance systems are much younger and their capacity challenged– Continued need for capacity building
• Demand will increase in other parts of the world than where (perceived) high-quality institutions are located– Disequilibria in global higher education system
Challenges for international quality assurance
TREND 2.CONTINUED INCREASE
INTERNATIONAL STUDENTS
Distribution of foreign and international students in tertiary education, by country of destination (2013)
Russian Federation 3%
Canada 3%
China 2%
Italy 2%
Saudi Arabia 2%
Korea 1%Turkey 1%
4.5 M students
Distribution of foreign and international students in tertiary education, by region of origin (2013)
Asia53%
Europe25%
Africa8%
Latin America and the Caribbean
5%
North America3%
Oceania1%
Not specified5%
• Students do not frequently rely on quality assurance systems to provide them with the information needed to choose international studies
• Rankings, based on research excellence and reputation, are used as “reliable” sources for information
• How can quality assurance systems provide more transparency on teaching and learning to international students?
• Has the internationalisation of quality assurance, its standards, principles, methodologies, etc. been fully achieved to support international mobility?
Challenges for international quality assurance
TREND 3.DIVERSIFICATION OF DELIVERY AND PARTICIPATION MODES –
ONLINE LEARNING
• Delivery– Online education, MOOCs– New providers
• Participation– Flexible routes– Part-time study
• Qualifications – New credentials, badges, nanodegrees, etc.
Diversity of routes
MOOCs are rapidly becoming part of the higher education system
400+ universities. 2400+ courses. 16-18 million students
MOOCs are extending benefits of higher education to under-served learners
Quality challenges of MOOCs
Quality challenges of MOOCs
• How will MOOCs be integrated in accreditation, credit accumulation and credit transfer systems in higher education?
• Are quality assurance arrangements ready to implement specific evaluation instruments and procedures for MOOCs?
• How is institutional quality assurance and accreditation dealing with institutions with multiple delivery modes?
TREND 4.RISING COST OF HIGHER
EDUCATION
19
• Total (public & private) financial investment grew– Between 2005 and 2012 on average across
OECD increase of 10% in per student expenditure and 27% in total expenditure
– With huge differences between countries, increases higher in countries with below-average expenditure, catching up
– Yearly per student expenditure is now 14 KUS$– Total expenditure increased from 1.3% GDP in
2000 to 1.6% GDP in 2011
Financial inputs in higher education increasing
20
• Private expenditure has increased a lot– 31% of total expenditure (0.5% GDP) comes from
private sources, mainly tuition fees– Increase from 25% in 2000– Total private expenditure increased with 32%
since 2005– >50% in Israel, US, Australia, Japan, UK, Korea
and Chile
Financial inputs in higher education increasing
21
• Increase in total per student expenditure slows down since crisis– Negative growth in almost half of countries
between 2008 and 2011– Expenditure cannot catch up with increasing
student numbers• Increasing concerns about levels of private
expenditure, student debt
But strong signs of stagnating funding
22
• Efficiency and value-for-money become very important policy considerations– Both for governments and students/families– Cost of higher education becoming political issue
in many countries• What are students actually ‘buying’?
– Very weak relationship between cost and actual ‘product’, benefits and outcomes
– Value-for-money depends enormously on institution and field of study
But strong signs of stagnating funding
• Can quality assurance ‘reassure’ students and families that higher education is worth the money?
• Is ‘quality’ an absolute concept or relative to the resources invested? “Added-value”?
Challenges for quality assurance and accreditation
24
TREND 5.EMERGING EVIDENCE ON INTERNATIONAL QUALITY
DIFFERENTIALS
SpainEngland (UK)
England/N. Ireland (UK)Ireland
ItalyKorea
CanadaPoland
United StatesNorthern Ireland (UK)
AustraliaEstonia
AverageFrance
DenmarkNorway
Slovak RepublicGermany
JapanSweden
AustriaNetherlands
Flanders (Belgium)Czech Republic
Finland
200 220 240 260 280 300 320 340 360 380
95th percentile mean score tertiary 25-34y
Numeracy scores of tertiary educated adults of 25-34y old
27
Literacy equivalent of tertiary qualifications
28
Numeracy equivalent of tertiary qualificationsProportion of 25-64 year-olds scoring at PIAAC numeracy level 4 and 5, by educational attainment of the population (2012)Ja
pan
Finl
and
Net
herla
nds
Sw
eden
Aus
tralia
Nor
way
Flan
ders
(Bel
gium
)
Eng
land
(UK
)
Eng
land
/N. I
rela
nd (U
K)
Uni
ted
Sta
tes
Cze
ch R
epub
lic
OE
CD
ave
rage
Pol
and
Can
ada
Nor
ther
n Ire
land
(UK
)
Aus
tria
Ger
man
y
Irela
nd
Fran
ce
Den
mar
k
Est
onia
Slo
vak
Rep
ublic
Kor
ea
Rus
sian
Fed
erat
ion
Spa
in
Italy
0
5
10
15
20
25
30
35
40
Below upper secondary education Upper secondary or post-secondary non-tertiary education Tertiary education
Higher education does not prevent low skills
Decreasing learning outcomes over time?
• Is ‘assured quality’ a guarantee for students meeting certain minimal learning outcomes?
• Has quality assurance an answer to the –perceived or real – grade inflation?
• Has the expansion of quality assurance prevented a (possible) decrease in quality of learning outcomes?
Challenges for quality assurance and accreditation
TREND 6.STRENGTHS AND WEAKNESSES
OF CREDENTIALISM
Qualifications, not skills are rewarded
Source: Survey of Adult Skills (PIAAC) (2012)
34
• Is graduate output higher than the economy’s need for high-skilled labour?– Graduate unemployment– Filtering-down effect?– Over-qualification and over-skilling– Huge field-of-study mismatches
• Is polarization in labour markets, with high employment/high earnings because of skill-biased technological change, going to last?
Concerns about over-qualification
35
Concerns about over-qualification
36
• Concerns about the quality and added-value of a university experience– Academically Adrift: limited improvement in
academic skills– What is the relative contribution of selection
versus teaching and learning in the production of high-quality graduates; what is the actual ‘learning gain’
– Doubts on the quality of the teaching and learning experience at universities
Concerns about quality and value of qualifications
37
Erosion of degrees?
• Should quality assurance protect the higher education system and wider society from credentialism, over-qualification and grade inflation?
Challenges for quality assurance and accreditation
These questions and challenges cannot be addressed without changing the focus of what we mean by quality
from input and process to what students actually learn, to learning
outcomes
What does this mean?
THE FUTURE OF QUALITY ASSURANCE LIES IN
ASSESSING LEARNING OUTCOMES OF STUDENTS
THE FUTURE OF INTERNATIONAL QUALITY
ASSURANCE LIES IN INTERNATIONAL ASSESSMENT
OF STUDENTS’ LEARNING OUTCOMES
42
• What do we know about how quality of teaching and learning results in high-quality output, and socially interesting outcomes?
• Information asymmetry: both public and private actors (and financers) of higher education have very little understanding of what they actually are spending money for
• Increase of investment has not been accompanied by an empowerment of the input side to make smart choices through better information
• In a diversifying system what matters is the output: what have students learned?
Information asymmetry and lack of transparency are critical issues
43
• Sound metrics of learning are very much needed– To reassure governments and families about the
value-for-money of investments– To reward institutions who invest in improving
teaching and learning and are not compensated through other measures
– To value institutional diversification– To reward and foster quality improvement through
mutual learning– To compensate for the over-reliance of rankings on
research and reputation metrics
More and better transparency is a much-needed necessity for higher education
44
• Erosion of the symbolic power of degrees, the only monopoly of the higher education sector– Employers turning to alternative modes of selection– Emergence of alternative modes of qualification (employer credentials,
badges, recognition of prior learning, etc.)• Decreasing trust of governments, employers, families and wider
society in the value of higher education– Degree and grade inflation– Concerns about sub-optimal standards in some countries
• Gradual erosion of the financial health of higher education institutions if value-for-money concerns are left unanswered– Financial bubbles of student debt
• Markets no longer accept non-transparency– Cfr Volkswagen
Neglecting transparency on learning outcomes can come at huge cost
• Is it possible?– To improve our understanding of what students actually ‘learn’ in
higher education– To exchange reputations with empirically grounded observations
of quality of teaching & learning– To gradually transform the field on which credentials are traded
into a more level playing field– To provide better information to students and employers about the
quality of teaching & learning experiences– To develop feedback loops to improve teaching and learning– To reward and incentivise institutions that significantly improve
their teaching & learning environments– To re-confirm the value of teaching as part of the university’s
mission next to research
What is the value-propositions of assessing learning outcomes
46
• Comparative assessment of learning outcomes of graduates is the most promising approach to measure teaching and learning excellence– OECD’s AHELO project– National research projects in Germany, UK, Italy– CLA and various other initiatives in US– OECD-CEA partnership to implement CLA+ in
countries– European Commission supported CALOHEE
project in Tuning framework
Opening the black box: assessing students’ learning outcomes
47
• Strong resistance by parts of the academic community, but do they have a strong case?– No consensus on academic skills that matter– Risk of standardization– Institutional diversity too large to use limited number of
metrics– Methodological concerns– Cost and burden
• …exactly the same arguments used 20 years ago when the PISA programme was born
• …and very similar to arguments used 15 years ago against measuring research excellence
Opening the black box: assessing students’ learning outcomes
48
• Developing reliable metrics of teaching and learning excellence in universities is the next big systemic challenge in the development of higher education worldwide
• In the short term universities might think it’s not in their interest and that non-transparency is the better option
• But in the longer term that might be a very risky approach, in which the costs largely exceed the short-term profits
Opening the black box: assessing students’ learning outcomes
