international trend on university rankings and its impact on higher education
DESCRIPTION
Angela Yung Chi Hou , 19 Dec, 2012 Fu Jen Catholic University. INTERNATIONAL TREND ON UNIVERSITY RANKINGS AND ITS IMPACT ON HIGHER EDUCATION. Presentation Outline. Introduction Development of Rankings Limitations of Rankings Impact of Rankings Outcomes of Ranking Use of Rankings - PowerPoint PPT PresentationTRANSCRIPT
INTERNATIONAL TREND ON UNIVERSITY RANKINGS AND ITS IMPACT ON HIGHER EDUCATION
Angela Yung Chi Hou , 19 Dec, 2012 Fu Jen Catholic University
1
Presentation Outline
I. IntroductionII. Development of Rankings III. Limitations of RankingsIV. Impact of RankingsV. Outcomes of RankingVI. Use of RankingsVII. Conclusion
2
I. Introduction
1. Three major concerns2. Meeting challenges to higher
education in the 21st century3. Recipe for higher education4. World class university5. Global rankings and world class
university6. Rational of college ranking
3
1. Three Major Concerns for Development of Higher Education in Global Times
AccessibilityHigher education enrollment rate has been increasing in the past decade
AffordabilityFinance need is not an impediment for eligible students to attend a college
AccountabilityEnormous resources and talent are available in higher education institutionsHelp students be ready for college, and be equipped to graduate from college
4
2. Challenges to Higher Educationin 21st Century
Over expansion of higher education2305 post secondary higher education institutions in China 726 universities and 488 junior colleges in Japan 201 universities in Korea160 universities in Taiwan 15 in Hong Kong (9/6)12 in Macau(4/8)
Private universities outnumber public universities (except China) Declining birth rate
three are below 1.5 Declining financial governmental support
5
3. Recipe for Higher Education - quality assurance and global competitiveness
Set up Internal and External Quality Assurance System
Establishment of National Accrediting agency Establish Quality Culture on Campus
Compulsory auditEnhance international academic competitiveness
Launching several Excellence Programs Brain21 (Korea) Center for Excellence COE (Japan)5 year-50 Billion Program (Taiwan)China 985 project
Aiming at establishing world class universities 6
4. What does a world class university look like?
In terminologyworld class universities: top universities striving for “excellence”
quality must surpass the expectation of stakeholders
Philip Altbachexcellence in research, top professors, academic freedom and an atmosphere of intellectual excitement, governance, adequate facilities and funding
7
4. What does a world class university look like?
Jamil Salmi (World Bank) based on two rankings (Shanghai and QS)
a high concentration of talent (faculty and students)abundant resources to offer a rich learning environment and conduct advanced research favorable governance
features that encourage strategic vision, innovation and flexibility, and enable institutions to make decisions and manage resources without being encumbered by bureaucracy
8
5. Relevance between Global Rankings and World Class University
characteristics of world class universities are strongly correlated to most indicators used by global rankingsnations use global rankings as a basis of building world class universities despite methodological flaws
9
5. Relevance between Global Rankings and World Class University
top administrators at leading universities use global rankings to achieve the short term and long term strategic plans, not just to boycott them
Minnesota’s initiative become one of the top three research institutions in the worldTaiwan National University “Moving into the top 100” at its 80th anniversaryBaylor University one of the U.S. News Top 50 by 2012
10
6. Rational of College Ranking
Higher education expansion Resources allocation Accountability BenchmarkingMarketization in higher education
11
II. Development of Rankings
1. National College Ranking2. Global Ranking3. Types of Rankings4. Scoring Methods5. Five Global Rankings6. Criteria & Indicators of Rankings
12
1. National College Ranking
U.S. News & World ReportThe most influential college ranking – “American Best Colleges” published by U.S. News & world Report in 1983Maclean’s , The Times, CHE, etc.
13
2. Global Ranking
intense international competitionsglobal college rankings have drawn international attention
14
3. Types of Ranking
By region by country/continent/worldwide
By field / programsuch as Engineering/social science/humanities
By subjecti.e., Biology/ Math
College guidePrinceton review
15
4. Scoring Methods
The indicators were weighted at a certain ratio and the scores were aggregated to rank each college. The top one university received highest points while the scores for the remaining schools descended accordingly.
16
5. Five Global Rankings
Academic Ranking of World Universities Shanghai Jiao Tong University, 2003
World University RankingQS, 2004
Webometrics Rankings of World UniversitiesSpanish National Research Council, 2004
Performance ranking of scientific papers of world class universities
HEEACT, 2007World University Ranking
The Times Higher Education, 2010
17
6. Criteria & Indicators of Ranking
2011 ARWU Ranking2011 QS Ranking2011 Webometrics Ranking2011 THE Ranking2011 HEEACT Ranking
18
6. 2011 ARWU Ranking 分類指標 分項指標 權重 操作型定義
分類指標 分項指標 權重 操作型定義
教育品質 畢業校友(Alumni)
10%
獲得諾貝爾及菲爾茲獎校友人數。得分高低依畢業年度,每往前推十年,降低 10% 權重。如 2001 年以後每人以 1 人計算; 1991-2000 年每人以 0.9 人計算 ,依次遞減。
教師品質獲得重要獎項人數
(Award)20%
獲得諾貝爾及菲爾茲獎教師人數。得分高低如上。如 2001 以後獲獎者每人以 1 人計算; 1991-2000 年每人以 0.9 人計算,依次遞減。獲獎人同時署名兩單位,各計 .05 人。諾貝爾共同得獎者,以獎金比例分配權重。
論文高度被引用研究人數(HiCi)
20% 在 21 個科學領域中,論文名列 ISIHighlyCited.com高度引用之教師人數。
研究成果《自然》與《科學》兩期刊論
文發表篇數 (N&S)20% 2006 至 2010 年間,一所大學在《自然》與《科
學》兩期刊論文發表篇數,不包含評論與快訊。SCI 與 SSCI 論文數 (PUB) 20% 2010 年,一所大學發表論文被 SCI 及 SSCI 收錄的
數量。機構規模 機構規模(專任教師數)
(PCP)10% 以上五項總分 / 專任教師數
6. 2011 QS Ranking
評比內容 權重 分項指標
研究(Research)
60%
學術聲譽(Academic Reputation)
教師平均論文被引用數(Citations per Faculty)
就業力(Employability)
10%就業聲譽
(Employer Reputation)
教學(Teaching)
20%師生比
(Faculty/Student Ratio)
國際化(Internationalization)
10%
國際教師比例(International Faculty Ratio)
國際學生比例(International Student Ratio)
6. 2011 Webometrics Ranking
21
Indicators Definition Covergae Sources Weight
IMPACT Number of backlines Number of back domains
Current Majestic SEO and ahrefs
50%
Presence Number of webp ages
Current Google 20%
openness Number of papers (pdf, doc, docx, ppt)
2007-2010 Google scholar
15%
Excellence 10% most cited papers in their respective scientific fields
2003-2010 Scimago 15%
6. 2011 THE Ranking
評比內容 權重 分項指標 分項指標權重
教學-學習環境
(Teaching -
the learning environment
)
30%
教學聲譽 (Reputational survey-Teaching) 15%
教師平均指導博士畢業生數 (PhD awards per academic) 6%
教師平均教授大學生數 (Undergraduates admitted per academic) 4.5%
授予博、學士學位比 (PhD-to-bachelor’s ratio) 2.25%
教師平均分配院校收入 (Income per academic) 2.25%
研究-數量、收入與聲譽 30%
研究聲譽 (Reputational survey-Research) 18%
研究收入 (Research Income) 6%
教師與研究員平均論文篇數 (Papers per academic and research staff) 6%
論文引用-研究影響 30% 平均論文引用數 (Average citations per paper) 30%
國際化-職員與學生(International outlook
-staff and students)
7.5%
國際對本地生比 (Ratio of international to domestic students) 2.5%
國際對本地教職員比 (Ratio of international to domestic staff) 2.5%
國際合作論文比例 (Proportion of internationally coauthored research papers) 2.5%
業界收入-創新(Industry income -
innovation)
2.5% 研究人員自業界之平均研究收入(Research income from industry per academic staff)
2.5%
Criteria Indicator Weight
Researchproductivity
Number of articles in the last 11 years (2000-2010) 10%
20%Number of articles in the current year (2010) 10%
Researchimpact
Number of citations in the last 11 years (2000-2010) 10%
30%Number of citations in the last 2 years (2009-
2010) 10%
Average number of citations in the last 11 years
(2000-2010) 10%
Researchexcellence
h-index of the last 2 years (2009-2010) 20%
50%Number of Highly Cited Papers (2000-2010) 15%
Number of articles in high-impact journals in the current year (2010) 15%
6. 2011 HEEACT Ranking
23
III. Limitations of Rankings
1. Characteristics of 5 major Global Rankings
2. Methodological Limitations
24
1. Characteristics of 5 Major Global Rankings
ARWU QS Webometrics HEEACT THEEstablished
year 2003 2004 2004 2007 2009
Institution Academicinstitution
Massmedia/PrivateEducationconsulting firm
Governmental research unit
QA Agency Massmedia
Goal Academiccompetition
Profit making Academicsharing
Benchmarking Academic Competition
Number ofindicators
6 6 4 8 13
Indicatorcategory
Researchoutput/ learning input
Research output /Reputation survey
/learning input
Web size/research
output/reputation
Research output
Teaching/ResearchOutput
/survey Data sources Database Survey/
database/institution
database Database Survey/database/institution
Outcomes Presentation
Only Top 100 of500 institutions are shown innumerical orders
Top 400 are shown
in numerical orders
Top 1000 innumerical
order
Top 500 innumerical
order
Top 200 areShown innumerical
ordersTransparency Highly medium Medium Highly
medium Highly medium Medium
25
2. Methodological Limitations of GlobalRankings
Reductionism / Simplicity Research focus Unfair for humanities, arts and social science fields English dominationArbitrary selection of indicators and weightingsData quality
26
IV. Impact of Rankings
1. Outcomes of Rankings2. Popular Use by Stakeholders3. Two Major Reports4. Performance in Asia
27
1. Outcomes of Rankings
US and UK institutions are on the top Asia is on rise, particularly those with Excellence policy
28
2. Popular Use of Global Rankings byStakeholders
Studentsto decide where to study
Governmentsto know where to invest
Scientiststo know where to work
Institutionsto know where they stand and whom they can partner with
29
3. Two Major Reports
OECD survey in 2007 showed: over 50 % of respondents: rankings has a positive impact on the institution’s reputation (student recruitment, academic partnerships and collaborations and staff morale)majority of the institutions incorporated the outcomes of rankings into strategic planning processes at all levels 70 % wanted to be in the top 25 internationally
30
4. Two Major Reports
OECD survey in 2007 showed: an on–line UK study focused on English Universities ‘ attitudes toward rankings
rankings reflect the views of what properties a good university should develop that influenced the institutional and governmental polices a high level of agreement that the reputation of an institution might be affected by rankings
many institutions further down in the rankings do not care too much about global rankings
31
5. Average Number of top 500 Universities of Japan, China, South Korea and Taiwan by three rankings
Country ARWU(2004~2012)
HEEACT(2007~2011)
QS(2007~2012) Ave No
Japan 29.56 29.60 25.67 28.27
China 16.44 14.20 12.33 14.33
South Korea 9.00 9.20 11.50 9.90
Taiwan 6.22 5.40 8.50 6.71
Hong Kong 5.00 5.00 5.67 5.22
India 2.00 2.00 8.00 4.00
32
V. Ranking Outcomes
1. ARWU / QS/ HEEACT/ THE Ranking2. Number of papers and
internationalization in China, Korea, Taiwan and Japan
33
Top 20 Universities in the Four Rankings (2/2)
University Name HEEACT(2011)
ARWU(2012)
THE(2012)
QS(2012)
University of California - San Diego 15 15 --- ---The University of Tokyo 16 20 --- ---University of London - University College London 17 --- 17 4
Duke University 18 --- --- 20Yale University 19 11 11 7Imperial College London 20 --- 8 6California Institute of Technology --- 6 1 10Princeton University --- 7 5 9University of Chicago --- 9 9 8Cornell University --- 13 20 14University of Wisconsin - Madison --- 19 --- ---ETH Zürich - Swiss Federal Institute of Technology Zürich --- --- 15 13
McGill University --- --- --- 1834
Top 20 Universities in the Four Rankings (2/2)
University Name HEEACT(2011)
ARWU(2012)
THE(2012)
QS(2012)
University of California - San Diego 15 15 --- ---The University of Tokyo 16 20 --- ---University of London - University College London 17 --- 17 4
Duke University 18 --- --- 20Yale University 19 11 11 7Imperial College London 20 --- 8 6California Institute of Technology --- 6 1 10Princeton University --- 7 5 9University of Chicago --- 9 9 8Cornell University --- 13 20 14University of Wisconsin - Madison --- 19 --- ---ETH Zürich - Swiss Federal Institute of Technology Zürich --- --- 15 13
McGill University --- --- --- 1835
Figure 1: The number of the paper published on SCI and SSCI journals in China, Taiwan, Japan, South Korea
36
Figure 2: Number of papers in Nature & Science in seven yearsin China, Taiwan, Japan, South Korea
37
Figure 3: Number of international students in China, Taiwan , Japan, South Korea
38
VI. Use of Rankings
1. Findings in rank Mobility2. Rank Differences and moving UP in 4
Global Rankings3. Berlin Principles and Ranking Audit4. Future Development5. CHE Excellence Ranking and Research
Ranking6. College Navigator in Taiwan
39
1. Some Findings in Rank Mobility of Global Rankings
Hou, Yung-chi, & M. Robert (2011) . An Analysis of Positions Mobility in Global Rankings: Making Institutional Strategic Plans and Positioning for Building World Class Universities. Higher Education Research & Development (SSCI).
To explore the major factors of rank mobility in 4 major rankings.
40
2. Rank Differences and moving Up in 4 Global Rankings
Comparison among 4 Global Rankings by positions risingImplication of 4 Global Ranking on making institutional strategic plans
41
2. Comparison among 4 Global Rankings by Positions Rising
ARWUQS Webmetrics HEEACT
Cluster one 1-171-30
1-39 1-19
Cluster two 20-45Over 30
40-99 20-45
Cluster three Over 46X
Over 100 Over 46
total number of positionsmoving ups 218(500)
170 (400) 242 (500) 231(500)
Highest ranks moving up 94125 212 82
42
2. Flow Chart of Implication of 4 Global Ranking on Making Institutional Strategic Plans
Webometrics Ranking QS Rankings ARWU/Shanghai Ranking
HEEACT Ranking: Used to inspect the quality and quantity of FACUTLY publications annually
Short term(3-5 years) Mid-term 5-15 years Long-term(15~30years)
Technology/Internet International Reputation Academic Excellence
43
3. Berlin Principles and Ranking Audit
International Ranking Expert Group (IREG)founded in 2004 by the UNESCO European Centre for Higher Education (UNESCO-CEPES) in Bucharest and the Institute for Higher Education Policy in Washington, DC.
44
3. Berlin Principles and Ranking Audit
It is upon this initiative that IREG’s second meeting (Berlin, 18 to 20 May, 2006) has been convened to consider a set of principles of quality and good practice in HEI rankings—the Berlin Principles on Ranking of Higher Education Institutions.IREG-5 in Oct, 5-7, 2010 proposed “Ranking Audit”
45
3. Berlin Principles: What Rankings and League Tables should Consider
To ensure the quality of rankings: research methods, indicators, data quality, transparency, varying user’s interests, etc. 4 major principles
Purposes and Goals of RankingsDesign and Weighting of IndicatorsCollection and Processing of DataPresentation of Ranking Results
19 criteria of audit published by IREG
46
4. Future Development for Rankings
Field/subject based rankingVarying ranking providersMore interactive, multi-dimensional, personalized
CHE European Excellence Ranking College Navigator in Taiwan
Web-based Benefit student mobility Student survey
47
5. CHE Excellence Ranking and Research Ranking
Multi-dimensional global ranking that will be based on the CHE ranking approach.Results of the feasibility study will be available in mid-2011. One aspect will be the development of a concept to introduce a web-based tool for personalized rankings for particular target groups on a global scale.
48
49
5050
College Navigator in Taiwan Published in 2009
Goal lead to a match between the student and the institution or the program that they’re most interested
Selection of Institutions69 4-year colleges and universities evaluated by HEEACT from 2006 to 2010. 77 University of Technology and Technical College
50
Research Team
51
3 professors/ one international consultant / one full time staff/ one IT designer / 7 graduate students
6. College Navigator in Taiwan-Home
Click here to start
52
College Navigator in Taiwan- Step 1: Indicators
53
Step 1: Choose the indicators
College Navigator in Taiwan- Step 1: Indicators
54
Step 1: Choose the indicators
College Navigator in Taiwan- Step 2: Weighting
55
Step 2: Give each indicator
a weight
College Navigator in Taiwan- Step 3: Preference
56
Step 3: Decide the
Universities you want to
compare
Way 1: Narrow down your choices
Way 2:Choose Universities
College Navigator in Taiwan- Step 4: Result
57
The indicators you choose
Our system will analyze the rank of the universities according to the indicators and weights you decide
The performance of each university
College Navigator in Taiwan- Step 5: General l information
58
59
Hou, Angela Yung-chi, Morse, R., & Shao, Y. J. E. ( 2012 ) . Is There a Gap between Students’ Preference and University Presidents’ Concern over College Ranking Indicators? : A Case Study of “College Navigator in Taiwan”, Higher Education ( in press) (SSCI) (2010 Impact Factor 0.823). ISSN: 0018-1560
60
Table 6 Mean and STD by items Figure 2: Scatter Plot of level of Importance over Criteria and Indicators
61
Categories Mean
STD
95% Confidence Interval
Upper Lower
Institutional policy making 3.95 0.61 4.14 3.76
Staff and faculty recruitment 3.93 0.67 4.14 3.73
Research output 4.06 0.76 4.30 3.83
Resources allocation 3.82 0.69 4.04 3.61
Student services and learning environment 3.86 0.7
2 4.09 3.62
System operation 3.93 0.75 4.16 3.69
62
Focus group’ opinions Most popular indictors Institutions’ attitude Graduate rate Academic survey * (higher) Institutional policy making
( rank 2) Total amount of equipment per student Expenditure per student Resources allocation
(Rank 6) Expenditure per student Enrollment rate Student services and learning
environment (Rank 5) Number of international academic awards earned by students within last 3 years students within last 3 years
Faculty-student ratio* (higher) Student services and learning environment(Rank 5)
Total library holdings per student/Total grant from collaborations between of University and industry
Graduation rate* (lower)Student services and learning environment(Rank 5)
Number of national academic awards by students (higher)
Student services and learning environment(Rank 5)
Total NSC Research grants Total library holdings per student** (lower)
Resources allocation (Rank 6)
Proportion of faculty members with Ph.D. Total NSC grants per faculty** (lower) Research output (Rank 1)
Number of national academic awards earned by students within last 3 years Proportion of full-time faculty
Faculty Resource (Rank 3)Average piece of collaborations between of University and industry per full-time faculty Proportion of professors with Ph.D.
Table 8 Comparison of rank of importance of the indicators among focus students, general users and institutions
VII. Conclusion
1. Conclusion2. Final Questions3. Yes and No Answers4. How Ecosystem Influences Top Research
Universities
63
1. Conclusion
To achieve a good rankings is becoming more and more important. Global rankings are increasingly being used as a tool for building world class universities and pursuing academic excellence .
64
2. Final question raised by the presentation
To what extent can a world class university be replicated by using the factors highlighted in a ranking model? How can it be done?
The answer is both “yes,” it can be replicated, and “no,” it can’t be.
65
3. Yes and No Answers
Rankings can only provide very rough guidance and clues to institutions on which road to take to achieve academic excellence. Not taken into account, but very crucial:
a clear visioninstitutional featuresfavourable governancesufficient resources
66
4. Understanding How the Ecosystem Influences the Performance of Top Research Universities by Jamil Salmi
67
“THERE IS NO SINGLE ROAD TO EXCELLENCE”
by Jamil Salmi (2010) Coordinator,
Tertiary Higher Education, World Bank
68
Thank you for your attention!
Questions and Comments
69
References *Hou, Angela Yung-chi (2012). Mutual Recognition of Quality Assurance Decisions on Higher Education Institutions in Three Regions-A Lesson for Asia. Higher Education, 64:911-926. (SSCI) (2011 Impact Factor 1.016). ISSN: 0018-1560*Hou, Yung-chi, Morse, R., & Shao, Y. J. E. ( 2012 ) . Is There a Gap between Students’ Preference and University Presidents’ Concern over College Ranking Indicators? : A Case Study of “College Navigator in Taiwan”, Higher Education, 64:767–787. (2011 Impact Factor 1.016)Hou, Angela Yung-chi (2012 Quality in Cross-Border Higher Education and Challenges for the Internationalization of National Quality Assurance Agencies in the Asia-Pacific Region –Taiwan Experience. Studies in Higher Education. (online) (SSCI)*Hou, Yung-chi., Ince, M., & Chiang, C.L. (2012). A Reassessment of Asian Excellence Programs in Higher Education – the Taiwan Experience. Scientometrics. 92, 23-42 (SSCI). Print ISSN: 0138-9130. (2011 Impact Factor 1.966)*Hou, Yung-chi. ( 2011 ) . An Analysis of Positions Mobility in Global Rankings: Making Institutional Strategic Plans and Positioning for Building World Class Universities. Higher Education Research & Development (SSCI ) (online) ISSN: 0729-4360(2011 Impact Factor 0.901)*Hou, Yung-chi. ( 2011 ) . Impact of excellence programs on Taiwan higher education in terms of quality assurance and academic excellence, examining the conflicting role of Taiwan’s accrediting agencies. Asian Pacific Educational Review, 13, 77-88 (SSCI). ISSN: 1598-1037*Hou, Yung-chi. ( 2011 ) . Quality Assurance at a Distance : International Accreditation in Taiwan Higher Education, Higher Education, 61(2), 179–191 (SSCI)