procedures and experience in verification of data on higher education institutions in romania
DESCRIPTION
Procedures and experience in verification of data on higher education institutions in Romania. Paul Serban Agachi Member of the Executive Committee of IREG Observatory University Babes- Bolyai , Cluj-Napoca, Romania. Outline. Why necessary verify HE data? - PowerPoint PPT PresentationTRANSCRIPT
1
PROCEDURES AND EXPERIENCE IN VERIFICATION OF DATA ON HIGHER EDUCATION INSTITUTIONS IN ROMANIA
Paul Serban Agachi
Member of the Executive Committee of IREG Observatory
University Babes-Bolyai, Cluj-Napoca, Romania
2
OUTLINE
Why necessary verify HE data? Verification at the level of the university Verification at the level of the
system/Ministry Procedures of verification Results
3
WHY NECESSARY VERIFY HE DATA?
Since the impact of the rankings on university and national policies is increasing
Since the complexity of rankings is increasing Since the aims of using the rankings
diversified
it is a problem of responsibility of the ranking agencies to give the most reliable rankings
U-MAP AND U-MULTIRANK
4
Activity profiles of institutions
Multi-dimensional global university ranking
ClassificationU-Map
Multidimenisonal rankings
Profile A
Profile B
...
Teaching and learningResearch
involvementKnowledge exchange
Regional engagement
Dimen-sion 1
Dimen-sion 2
Dimen-sion 3
Dimen-
sion ...
Multiple excellences
International orientation
Student profile
5
Selection of criteria and their relative importance
Research Education Costs Services Social aspects National context, legislation Financial resources
Selection of indicators Validation of data
Criticism of existing rankings (I)(Vincke, Dubrovnik, 2009, Bucharest 2012)
6
Bibliometrics Quality of data Discrimination between scientific fields Different traditions (journals, books, proceedings,
number of authors, duration of research validation)
Supremacy of publishing in English Which indicators? (IF, citation index, h index, …)
Experts Are there ranking experts? How are they selected? How are the questionnaires structured? How are
the answers handled?
Criticism of existing rankings (II)(Vincke, Dubrovnik, 2009, Bucharest 2012)
7
SUPPOSE 6 UNIVERSITIES
C1 C2 C3 C4 C5 C6
A 1,92 10,0 14 36,0 4350 0,59
B 6,00 6,0 7 54,0 3150 0,76
C 0,72 5,0 14 57,0 5000 0,85
D 1,44 6,5 20 38,25 2750 1,00
E 4,80 1,8 11 49,5 4800 0,87
F 3,60 4,3 11 75 3400 0,73
8
RESULT OF RANKINGC D A F H E B
9
CHANGE OF ONE DATA
C1 C2 C3 C4 C5 C6
A 1,92 10,0 14 36,0 4350 0,59
B 8,00 6,0 7 54,0 3150 0,76
C 0,72 5,0 14 57,0 5000 0,85
D 1,44 6,5 20 38,25 2750 1,00
E 4,80 1,8 11 49,5 4800 0,87
F 3,60 4,3 11 75 3400 0,73
10
RESULTB C D A F E
11
TWO CONCLUSIONS
Robust algorithms for calculating the positions of the universities in rankings
Reliable data
similar problems with the allocation of funding for the universities in Romania
12
PROPOSAL OF A RANKING AGENCY ROMANIA
Steering Committee/Comitet Director
Executive director
WG 1Methodology
WG 2Elaboration benchmark, data
collection and validation
WG 3IT
WG 4Program monitoring
13
VERIFICATION AT THE LEVEL OF THE UNIVERSITY
Database with the Human Resource of the university (teaching, research, technical, administrative staff) – HR compartment
International staff and their status – HR and International offices
International students and their status – Registrar and International office
National Student Enrollment Registry - Registrar Individual verification of each registration (consistency,
nature, values, intervals etc.) Students with the situation incomplete at the end of the
academic year Coherence of he output and input data from different
consecutive years Unique position in the NSER
14
VERIFICATION AT THE LEVEL OF THE UNIVERSITY
IT conditions – IT department Databases with researchers and doctoral
students – Research and Doctoral studies compartment
Research data- Research compartment Scientometric compartment Internal Database of research activity Citations, IF Comparison with the International Data Bases
(Thomson, Scopus) Consistency with the two databases
15
VERIFICATION AT THE LEVEL OF THE UNIVERSITY
Database with alumni – Alumni office Databases with alumni at the level of
departments/faculty Database with employers at the level of
faculties and university Employment situation – in UBB – Center for
University Development
one of the major problems is that all these data are not correlated and their consistency is not verified by one unique body
16
VERIFICATION AT THE LEVEL OF THE SYSTEM
Ministry (National Student Enrollment Registry) Individual data for each student Data concerning the university
University structure University teaching programs Data concerning schooling capacity figures approved Titles and diplomas
17
VERIFICATION AT THE LEVEL OF THE SYSTEM
CNFIS Allocation of funds based on the equivalent
student number Allocation of funds based on quality:
Teaching staff and its quality Scientific performance: publications, invention patents
etc. Extra-budgetary funds and their allocation Quality of social services Internal management of the allocations
18
VERIFICATION AT THE LEVEL OF THE SYSTEM
CNFIS Analysis of the students’ number reported on
October 1 of the current academic year relative to the report of the same number on January 1 of the previous academic year Consistency of the statistical data based on the
correlation between their value and their significance number of students participants to the pedagogical
seminar < number of BA students
19
VERIFICATION AT THE LEVEL OF THE SYSTEM CNFIS
Analysis of the students’ number reported on October 1 of the current academic year relative to the report of the same number on January 1 of the previous academic year
Consistency of the statistical data, comparing the enrollment figures on January 1 of the previous academic year with the schooling capacity:
variations larger than +/-10%; data referring to the continuing education for teaching staff in the pre-university system – not considered because of the annual fluctuations
Variations larger than +/-3%; data referring to period 1 Oct – 1 Jan of the same academic year
Identify situations of discordance between situations from Jan 1 of the two consecutive academic years
“surplus” of the declared number of students and schooling capacity
20
VERIFICATION AT THE LEVEL OF THE SYSTEM CNFIS
Analysis of the academic staff number reported on October 1 of the current academic year Primary analysis of the consistency
Number of academic staff should be < or = to the total number of professors of all grades (full, associate, lecturers etc.)
Number of young staff should be < or = to the total number of lecturers, assistants
Secondary analysis of the consistency Comparison with the situation of a reference (2008)
for the difference larger that +/- 10% Academic tenure staff change; difference in the number
of professors of all grades; teaching staff under 35; teaching staff with PhD degree; teaching positions number; vacancies; auxiliary staff change
21
VERIFICATION AT THE LEVEL OF THE SYSTEM CNFIS
Analysis of the number of students enrolled and schooling capacity Comparison of the number of students enrolled at
October 1 of the academic year and the allocated figure from the Ministry (OMECTS) in the year verified
Schooling capacity compared with first year students reported at October 1 of the academic year
Comparison between the number of students enrolled in the university, reported in the previous years with the schooling capacity in the same year (e.g. II-nd year students)
22
VERIFICATION AT THE LEVEL OF THE SYSTEMCNCS Analysis of the overall scientific contribution
(articles, books, proceedings, patents, prizes, conferences etc.)
Analysis and comparison of the data reported by the universities with international databases: ISI Thomson, SCOPUS Result: report for Ministry, ANCS and CNFIS
23
VERIFICATION AT THE LEVEL OF THE SYSTEM ARACIS
Analysis of the number of students enrolled and schooling capacity for each program
Analysis of the number of academic staff for each program
Analysis of the state of the properties of the universities
Analysis of the teaching/research space
24
CONCLUSION
HE data are verified at three levels University Ministry Councils
Bad experience with the validation of the data at the last classification
New classification/ ranking
25
Thank you for your attention