examining quality culture: part 1 – quality assurance ... quality culture part i quality... ·...

E U A P U B L I C A T I O N S 2 0 1 0

Examining Qual i ty Cul ture:

Part 1 – Qual i ty Assurance Processes

in Higher Educat ion Inst i tut ions

By Tia Loukkola and Thérèse Zhang

Copyright©bytheEuropeanUniversityAssociation2010

All rights reserved. This informationmaybe freelyusedand copied fornon-commercialpurposes,provided that the source is

acknowledged(©EuropeanUniversityAssociation).

Additionalcopiesofthispublicationareavailablefor10Europercopy.

European University Association asbl

Avenuedel’Yser24

1040Brussels,Belgium

Tel:+32-22305544

Fax:+32-22305751

Afreeelectronicversionofthisreportisavailablethroughwww.eua.be.

Thisprojecthasbeen fundedwith the support fromtheEuropeanCommission.Thispublication reflects theviewsonlyof the

authors,andtheCommissioncannotbeheldresponsibleforanyusewhichmaybemadeoftheinformationcontainedtherein.

ISBN:9789078997214

By T i a Loukko l a and Thé rè se Zhang

Examining Qual i ty Cul ture:Part 1 – Qual i ty Assurance Processes in Higher Educat ion Inst i tut ions

E U A P U B L I C A T I O N S 2 0 1 0

E X A M I N I N G Q U A L I T Y C U L T U R E : P A R T 1 – Q U A L I T Y A S S U R A N C E P R O C E S S E S I N H I G H E R E D U C A T I O N I N S T I T U T I O N S

4

List of figures and tables

Figures

Figure 1: RatiooftypesofinstitutionsFigure 2: DistributionofrespondentspercountryFigure 3: ElementsofqualitycultureFigure 4: StructuressupportingtheinternalqualityassuranceprocessesFigure 5: Introductionofaqualityassurancesystem(orequivalent)Figure 6: Introductionofaqualityassurancesystem(orequivalent)–BreakdownpercountryFigure 7: Howtheinternalqualityassurancesystem(orequivalent)wasintroduced withintheinstitutionFigure 8: TheinvolvementofstakeholdersinformalqualityassuranceprocessesFigure 9: DesigningcurriculumandprogrammeswithintheinstitutionFigure 10: MonitoringcurriculumandprogrammesFigure 11: CharacteristicsofstudentassessmentproceduresFigure 12: Institutionsoffering,monitoring,evaluatingandimprovingtheirlearningresources

Tables

Table 1: ActivitiescoveredbyinstitutionalqualityassuranceprocessesTable 2: Informationincludedintheinformationsystem(s)regardingstudyprogrammesTable 3: InformationprovidedbytheinstitutiononitsstudyprogrammesTable 4: InternalevaluationprocessesprovidingfeedbacktothestrategicplanningTable 5: KeyfindingscorrespondingtoESGspart1

14151720212223

2429303132

1926272833

5


Foreword

Acknowledgements

Executive summary

1. Setting the stage

1.1.Background

1.2.TheExaminingQualityCultureproject:aimsandmethodology

2. Quality assurance as a component of quality culture

2.1.Theelementsofqualityculture

2.2.Qualityassuranceprocessesasunderstoodinthisproject

3. Mapping internal quality assurance processes: survey results

3.1.Qualityassurancestructures

3.2.Participationofstakeholders

3.3.Theuseofinformation

3.4.Qualityassuranceinteachingandlearning

3.5.ImplementationoftheESGs

4. Key trends and further reflection

4.1.Trendsandperceptions

4.2.Areasforfurtherdevelopment

5. Concluding remarks

Annex: Questionnaire

References

Table of contents

6

7

9

12

12

13

16

16

17

19

19

23

25

28

33

35

35

38

40

41

49


6

ThequalityofEuropeaneducation,andofhighereducationinparticular,hasbeenidentifiedasoneofthekeyfactorswhichwillallowEuropetosucceedinaglobalcompetition.In2003inBerlintheMinistersforhighereducationstatedthattheprimaryresponsibilityforqualityassurancelieswithhighereducationinstitutions.Further,variouspoliciesandactionlineshavebeendevelopedtoimprovequality;amongotherinitiatives,Europeandegreestructureshavebeenrevised,mobilityofstudentsandteachersisencouraged,andtransparencyandcomparabilityofqualificationsispromoted.

Inparallel,EUAhassupporteditsmembersinpromotinganinstitutionalqualityculturethatisfitforpurposeandthattakesaccountofthesignificantinstitutionaldiversitywhichexistsinEurope.Afteradecadeofworkinthisfield,andgiventheofficiallaunchoftheEuropeanHigherEducationAreain2010,wefeltthatthetimehadcometotakeamomenttoanalyseprogressmadeinthisrespect.

Theaimofthe“ExaminingQualityCultureinEuropeanHigherEducationInstitutions”projecthasbeen toask the institutionshow,and throughwhichactivities, theyare responding to thechallengeofassuringandenhancingthequalityof theirprovision.This report focusesontheactivitiesdevelopedbyuniversitiestoenhancetheirinternalquality,toimprovetheiraccountabilityandtherebyalsoimplementingtheEuropeanStandardsandGuidelines(ESGs)inpractice,andisbasedontheresultsofasurveycompletedby222institutionsacrossEurope.

Nevertheless,truehighqualityeducationcannotresultonlyfromformalqualityassuranceprocesses,but rather is a consequence of the emergence of a quality culture shared by all members of a highereducationcommunity.Hence,theworkofthisprojectwillcontinue.Theintentionistocomplementthisreport by another one which will address the complex relationship that exists between formal qualityprocessesandtheexistenceofanoverallinstitutional‘qualityculture’.Itwillalsopresentsomecasestudyexamplesofthisinteraction.

Wehopethatthispublicationwillbeofinteresttoallourmembersaswellastopolicymakers.Astheresultsdemonstrate,whileconsiderableprogresshasbeenmadethereremainsmuchtobedone.Inthatcontext,wehopethatthispublicationwillinvitereflectionsthatwillcontributetothiswork.

Foreword

Jean-Marc RappEUAPresident

7


Firstandforemost,onbehalfoftheprojectconsortium,wewouldliketoexpressourgratitudetoall thehighereducation institutionsthattookthetimetoanswerthequestionnaire.Thisreport isbasedon those repliesandwouldnothave seen the lightofdaywithout thecontributionofeveryonewithintheseinstitutions.Wearewellawarethatthetimespentrespondingtothesurveywastakenawayfromthedailytasksoftheinstitutionsandverymuchhopethattheresultsofthisprojectwillbenefitusallinthelongrun.Specialthanksgotothose14institutionsacrossEuropethattookpartinthetestingphaseofthequestionnaire,providinguswithinvaluableinputonthedesignofthequestionsanditstechnicalrealisation.

Secondly,wearegratefultothemembersoftheprojectSteeringCommittee(seelistbelow),chairedby Professor Henrik Toft Jensen, former President of Roskilde University in Denmark. Ever since the firstmeetingoftheSteeringCommitteewehavebeenimpressedbythesharedcommitmenttothisprojectandtopromotingaqualitycultureapproachwithinhighereducationinstitutions.Thiscommitmenthasbeendemonstratedbytheuntiringwillingnesstocommentondraftsofboththequestionnaireandthisreportaswellastocontributetotheanalysisoftheresults.

Furthermore, EUA would like to thank its consortium partners, the German Rectors’ Conference(HRK)andQAAScotlandforagreeingtoembarkonthischallengingadventureofmappingqualityassuranceprocessesandaddressingqualityculture.OurthanksalsogototheEuropeanCommission’sLifelongLearningProgrammeforco-fundingtheproject.

Finally, the authors wish to thank Barbara Michalk for acting as our sounding board during thewritingprocessandcontributingtothetext.Andlastbutbynomeansleast,thanksgotoJoanneByrne,whosehelphasbeeninvaluableinrespecttoallaspectsofthepracticalarrangementsrelatedtotheprojectandwhohasscrupulouslyprovidedfeedbackonvariousdraftversionsofthisreport.

TiaLoukkolaHeadofUnit

ThérèseZhangProjectOfficer

Steering CommitteeHenrik Toft Jensen,Chair,formerPresidentofRoskildeUniversity,DenmarkAndrea Blättler,representative,EuropeanStudents’UnionDavid Bottomley,AssistantHead,QAAScotland,UnitedKingdomKarl Dittrich,representative,EuropeanAssociationforQualityAssuranceinHigherEducation(ENQA)Tia Loukkola,HeadofUnit,EUABarbara Michalk,HeadofQualityManagementProject,GermanRectors’Conference(HRK),GermanyOliver Vettori,HeadofQualityManagementandProgrammeDelivery,ViennaUniversityofEconomicsandBusiness,Austria

Acknowledgements


8

9


1. Setting the stage1.HighqualityofprovisionhasbeenoneofthekeyaimsofthecurrentreformsinEuropeanhigher

education,andhasledtotheincreasingdemandforqualityassurance(QA).OfalltheworkcarriedoutinthisregardatEuropeanlevel,theEuropeanStandardsandGuidelinesforQualityAssuranceintheEuropeanHigherEducationArea(ESGs),adoptedin2005,areconsideredasacornerstone,reinforcingtheimportanceofinstitutionalautonomyandresponsibilityinQA.

2.When working on QA processes, higher education institutions (HEIs) are ideally expected todevelopinternalqualitycultureswhichtakeintoaccounttheirinstitutionalrealitiesandarerelatedtotheirorganisationalculture.

3.Theproject“ExaminingQualityCultureinHigherEducationInstitutions”(EQC)aimstoidentifyinstitutionalprocessesandstructuresthatsupportthedevelopmentofaninternalqualityculture.ThefirstphaseoftheprojectfocusedonmappingtheexistingQAprocessesthroughasurvey,whilethesecondphasewillprovideaqualitativeapproachandembracetheculturalandmoreinformalelementsofqualityculture.

4.Thereport,whichresultsfromthefirstphase,seekstoexaminehowhighereducationinstitutions(HEIs) respond in their activities to thedevelopments inQAatpolicy level. It is basedon thequantitativeresultsofasurveythatwasconductedduringspring2010.Atotalof222institutionsfrom36countriesacrossEuroperesponded.


5.Thestudybasesitsunderstandingof“qualityculture”onthedefinitionprovidedbyEUA’sQualityCultureproject (2006),which sees it as referring toanorganisational culture characterisedbyacultural/psychologicalelementontheonehand,andastructural/managerialelementontheotherhand.Itiscrucial,intheauthors’minds,todistinguishqualityculturefromqualityassuranceprocesses,whicharepartofthestructuralelement.

6.Thedefinitionofqualityassurancevariesfromonecountryandinstitutiontoanother.ThestudyusesQAinitsbroadestsense,includingallactivitiesrelatedtodefining,assuringandenhancingthequalityofanHEI,thusarguinginfavourofadoptinganall-encompassingapproachderivedfrominstitutions’ownstrategicgoals,fittingintotheirinternalqualityculture,whilealsofulfillingtheexternalrequirementsforQA.Theseactivitiesshouldinclude,butarenotlimitedto,activitiesmentionedbytheESGs.TheapproachadoptedbythesurveyisthereforenotlimitedtocheckingcompliancewiththeESGs,butalsoincludeselementsofinstitutionalstrategicmanagement.

Executive summary


10

3. Mapping internal quality assurance processes: survey resultsQuality assurance structures

7.Remarkable progress has been made in QA in recent years, and most of the responding HEIshavefundamentalpolicies,structuresandprocessesinplaceinthisregard–althoughinstitutionstend not to systematically identify or call all QA practices in place as such. A large variety oforganisationalstructuresexistwhenitcomestosupportingtheimplementationofQAprocesses.HEIswitha longerhistory inQAaremore likely tohavedevelopedsupport structures suchaspedagogicalinnovationandstaffdevelopment.

8.In terms of policy and associated procedures, most HEIs have a strategic document either atinstitutional(forthemajorityofcases)oratfacultylevel.

Participation of stakeholders

9.ThecrucialroleofinstitutionalleadershipindemonstratingcommitmenttoqualityhasbeentakenonboardbymostHEIs,whichhavetheirseniorleadershipinvolvedinonewayoranotherinQAprocesses.

10.WhereastheparticipationofstaffandstudentsisoneofthekeyprinciplesindevelopingbothaqualitycultureandQAprocesses,nearlyhalfoftherespondentsdonothaveacommitteeresponsibleforQA.CommitteesaremorelikelytobefoundinHEIshavingworkedforlongerinQA.HEIswithalongerhistoryinQAarealsomorelikelytogiveimportancetotheinfluenceofstudentsurveysaswelltheimportanceofafeedbackloopandinformingthestudentsaboutthefollow-upofQAactivitiestheyparticipatedin.Whiletheinvolvementofacademicstaffseemstobesystematicandcommoninallstages,fromcurriculumdesigntoinvolvementinformalQAprocesses,studentinvolvementisnotaswidespread.

11.InmostHEIs,externalstakeholders(employers,experts,alumni...)areinvolvedinQAprocessesinvariousways,butthelevelandthenatureoftheirparticipationvaries,fromsittingongovernancebodiestobeingconsultedassourcesofinformation–thislatterseemingtobethemorecommon.

The use of information

12.Practically all responding HEIs have an information system for monitoring their activities.Institutionstendtocollectinformationabouttheirprofileandwhattheyoffer,buttheinformationrelatedtoresourcesavailabletothestudents(suchaslibraryservices,computerfacilities...)ismorelimited.Moreover,theinformationcollectedisnotnecessarilytheonemadepublic.Usuallytheinformationmadepublicisthatonstudyprogrammes,althougheventhisdoesnotoftenincludeinformationongraduateemployment.

13.The link between collecting information and informing the staff or students involved in thisdatacollectionisnotobvious,assomeinformation(suchasteachers’performances)istypicallyconsideredasconfidentialoraccessibleonlyatleadershiplevel.Studentswhoprovidefeedbackthroughsurveysareinformedabouttheresultsandfollow-upactionsinabouthalfoftheHEIs,although a significantly higher percentage of institutions do actually conduct student surveys.Dataalsoshowthatinstitutionsthathaveprocessesinplacetoobligeateachertoimprovehis/herperformancegivemoreconsiderationtotheresultsofstudentsurveys.Theseinstitutions,again,arethosewithalongerhistoryinQA.

14.With regard to strategic management, in about two thirds of HEIs the institutional leadershipconductsanannualevaluation to reviewthegoals.However,onlya littleoverhalfof theHEIsreportedhavingformulatedkeyperformanceindicatorstomonitortheirprogress.

11


Quality assurance in teaching and learning

15.AbouttwothirdsofHEIshavedesignedtheirQAframeworkforteachingandlearningasinstitution-specific but following national frameworks and guidelines. Few HEIs chose to adopt externalmodels(CAF,ISO...)assuch.

16.Thecurriculumistypicallydesignedbyacommitteeoraworkinggroup.Afteraprogrammeisupandrunning,avarietyofprocessesformonitoringitexist.MostHEIsconductsomekindofinternalevaluationinadditiontoanexternalone,shouldtherebeone.

17.ThepercentageofHEIsthatreportedlyhavedevelopedlearningoutcomesishigherthan90%,buttheydonotallmakethempubliclyavailable.Lessthanhalfmeasurethestudentworkloadneededtoreachthedescribedlearningoutcomesthroughstudentsurveys.

18.However,where institutionshavedeveloped learningoutcomes, student assessment is directlyrelatedtothem.StudentassessmentcombinesavarietyofcharacteristicsacrossEurope.Assessmentmethodsandcriteriaareusuallymadetransparenttostudents.

19.HEIsofferlearningresources,buttheydonotallsystematicallymonitororevaluatethem.Studentsupportservicesaremore likelytobe inplaceandmonitoredin institutionshavingintroducedtheirQAsystembefore2000.

Implementation of the ESGs

20.ThelastpartofChapter3offersanoverviewontheimplementationoftheESGsinthelightofresultscollectedthroughthesurvey.

4. Key trends and further reflectionTrends and perceptions

21.The report argues that QA systems are largely in place, although their development in theircurrentformatisarecentphenomenon.Yet,developingaqualityculturetakestimeandeffort,asitiscloselyrelatedtovalues,beliefsandaculturalelementwhichcannotbechangedquickly.ParticipationofallstakeholdersintheimplementationofQAprocessesandstrivingforastrongerqualitycultureappearstobeessential,butstilldemandsattention.Moreover,HEIsseemtohavemoreinformationavailableontheinputandonwhatisoffered,thanontheoutput.Finally,HEIstendtobegoodatcollectinginformation,butpromotingabetterandmoreefficientuseofitmaybettercontributetostrategicplanningandfostercontinuousimprovement.

Areas for further development

22.SeveralkeyareasforfurtherdevelopmentininternalQAprocessesemergefromthisstudy.Amongthese, an all-encompassing approach to QA, the development of explicit feedback loops, theparticipationofallrelevantstakeholders,andtherelationbetweeninformationonstrategicgoalsandcommunicationstrategyshouldbeunderlined.Finally,thecomplexityoftheframework inwhichinternalQAprocessesoperateshouldnotbeunderestimated:otherdevelopmentsinhighereducation,externalregulations,financialconstraints,andpotentialreluctancefromtheinstitution’scommunityitselfaretobecarefullytakenintoaccountwhenfurtherdevelopingaqualityculture.


12

1.1. BackgroundTheriseofdemandforqualityassurance(QA)processes–bothinternalandexternal–hasusually

beenlinkedtothemassificationofhighereducation,totheincreaseofinvestmentanddoubtsconcerningthe possibility of maintaining quality in the resulting new circumstances, as well as to the belief in theimportanceofhighereducationinthenewknowledgesociety.

ThehighqualityofprovisionhasbeenoneofthekeyaimsoftheBolognaProcessandtheLisbonStrategyasameanstopromotetheattractivenessandcompetitivenessofEuropeanhighereducation.TheMinisterialmeetingswithintheBolognaProcesshaveshapedtheEuropeanqualityassuranceframework.In2003,theBerlinCommuniquéstatedthat“consistentwiththeprincipleofinstitutionalautonomy,theprimaryresponsibilityforqualityassuranceinhighereducationlieswitheachinstitutionitself”(BPMC2003).

TwoyearslatertheStandardsandGuidelinesforQualityAssuranceintheEuropeanHigherEducationArea(ESGs)wereadopted,basedonaproposalpreparedbytheE4Group(ENQA,ESU,EUAandEURASHE1).TheESGsyetagainreinforcedthecentralimportanceofinstitutionalautonomy,whichbringswithitheavyresponsibilitiesforHEIs(ENQA2005:11).In2007,theMinistersendorsedtheproposalofthesameE4GrouptosetupaEuropeanRegisterofQualityAssuranceAgencies(EQAR),thusconsolidatingtheframework.(EUA2010a:61-62)

Whiletheabove-mentionedEuropeanqualityassuranceframeworkwasbeingdeveloped,theworkonqualityassurancecontinuedatthegrass-rootslevel.HEIsareconstantlydevelopingandimplementingqualityassuranceprocessesandconsequently fosteringtheirqualityculture.Nevertheless,whereastherehavebeenvarious reportspreparedon theprogressmade in thefieldofexternalqualityassurance (forexample,ENQA2008)andotherreportsthathaveaimedatcoveringboth internalandexternalqualityprocesses(forexample,Rauhvargerset al.2009,EC2009;Westerheijdenet al.2010;ESU2009and2010),therehasnotbeenaEuropeanlevelreportspecificallyaimedatexamininghowhighereducationinstitutionshaverespondedtothedevelopmentsatpolicylevelintheirdailyactivities.

EUA’sTrendsseriestouchedbrieflyonthetopicofinternalQAaspartofthelargerBolognaProcessframework.TheTrends 2010resultsdemonstrateclearlythatinstitutionsfindqualityassurancereformstohavebeenamongthemostimportantdevelopmentsthathaveshapedtheirstrategyinrecentyears,since63%oftherespondentstoTrendsmentioneditamongthetopthreedevelopments.Furthermore,qualityassurancealsocontinuestobethesecondmostimportantdevelopmentexpectedtoimpactHEIsinthenextfiveyears,with21%havingmentioneditassuch.(EUA2010a:73)

IdeallyHEIsarenotmerelyworkingon theirqualityassuranceprocesses,butdeveloping internalqualityculturesadaptedto theirown institutional realities,which isamuchmorechallengingtask thanthat of simply settingupprocesses requiredby external parties. For quality culture is closely related toorganisationalcultureandfirmlybasedonsharedvalues,beliefs,expectationsandacommitmenttowardsquality,dimensionswhichmakeitadifficultconcepttomanage.

ItisinthiscontextthatEUAlaunchedthe“ExaminingQualityCultureinEuropeanHigherEducationInstitutions(EQC)”projectwithitspartners,theGermanRectors’Conference(HRK)andQAAScotland,withthegoalofexhibitingtheprocessesandstructuresthroughwhichthehighereducationinstitutionssupportthedevelopmentoftheirinternalqualityculturethusbothenhancingtheirqualitylevelsandrespondingtothedemandsofaccountability.

1. Setting the stage

1TheEuropeanAssociationforQualityAssuranceinHigherEducation(ENQA),theEuropeanStudents’Union(ESU),theEuropeanUniversityAssociation(EUA)andtheEuropeanAssociationofInstitutionsinHigherEducation(EURASHE).

13


1.2. The Examining Quality Culture project: aims and methodology

The project “Examining Quality Culture in European Higher Education Institutions” (EQC) waslaunchedinOctober2009andwilllastuntilautumn2011.Itbuildsuponthepartners’long-termworkondevelopingtheconceptofqualitycultureandexaminingitsnature,driversandobstacles(EUA2005andEUA2006inparticular).

Theaimoftheprojectistwofold:

1)IdentifyinternalqualityassuranceprocessesinplaceinHEIs,payingparticularinteresttohowtheinstitutionshaveimplementedthepartoftheESGsdedicatedtointernalQAwithinHEIs.

2)Discuss the dynamics between the development of institutional quality culture and qualityassuranceprocesseswhileidentifyingandpresentingcasepracticesinafinalreportinordertodisseminatethem.

Inthefirstphaseoftheprojectthefocushasbeenonmappingtheexistingqualityassuranceprocessesthroughasurveythatwaslaunchedtogatherquantitativeevidencetoreachthefirstaim.Chapter3ofthisreportisbasedontheanalysisofthedatagatheredviathesurvey,andChapter4includesfurtherreflections.

Thenextphaseoftheprojectwillfocusonthelatteraim,andwillcompriseofinterviewswhichwilltakeplaceduringthefirsthalfof2011.Intervieweeswillconsistofasampleofinstitutionrepresentativeswhohaveansweredthesurveyandindicatedtheirwillingnesstoparticipateinsuchaninterview.Theaimofthesecondphaseistocontributetointerpretingthequantitativedataprovidedduringthefirstphase.

This twofold approach was adopted so as to allow proper data collection, and to embrace aqualitativeapproachthatwasneededtocapturetheculturalandmoreinformalelements,whichcouldnotbedocumentedassuchbyaquantitativeapproachonly.

Keyquestions,towhichtheprojectaimstofindanswers,canbesummarisedasfollows:

•HowareEuropeanHEIsimplementingtheESGspart1inpractice?Whatkindofqualityassuranceprocessescanbeidentified?

•Atwhatstageistheimplementationofthequalityassuranceprocesses?WhileaEuropeanframeworkandprincipleshavebeendeveloped,howhavetheyinfluencedinstitutionalreality?Whatkindofmainchallengescanbeidentified?

•What is therelationshipbetweenformalqualityassuranceprocessesandan institutionalqualityculture? Does having one necessarily permit or preclude having the other? How do the twointeract?Howcananinstitutionensurethatthereissynergybetweenthemratherthandiscord?

•What kind of examples can be showcased to disseminate good practice or to demonstratechallengestoaqualityculture?

Thefirsttwoquestionsaboveareofprimaryinterestinthisreportandthesecondreportwilltakeacloserlookatthetwolatterpointsidentified.Therefore,itisworthunderliningthatthe main focus of this particular report is on the formal quality assurance processes and the progress HEIs have made in this regard in the light of the survey conducted.


14

The final project publication – which is to follow – will take the discussion one step furtherconsidering the relationshipbetweenqualitycultureand formalqualityassuranceprocessesaimingatabetterunderstandingoftheseconcepts(theirsynergies,similaritiesanddifferences)whilepresentingsomecaseexamplesandcitingobservationsmadeinthecourseoftheproject.

The survey

TheEQCsurveywasdesignedinordertocapturethedevelopmentsinqualityassuranceprocesseswithinEuropeanhighereducationinstitutions(HEIs)inaquantitativemanner.Thequestionnaire(seeannex)consistsoftwomainparts:onefocussingonquestionsdealingwiththedevelopmentoftheinstitutionalqualityassurancesystemorconcept,andthemainbodyofthequestionnairefocussingonquestionsonqualityassuranceinteachingandlearningwhileadaptingthelogicoftheESGs.

Priortothelaunchofthequestionnaire,theprojectpartnersinvited14HEIsacrossEuropetotestitandprovidefeedback.TheonlinesurveywaslaunchedatthebeginningofFebruary2010andadvertisedbythethreeprojectpartners.Aninvitationtoparticipate,alongwithreminders,wasaddressed,interalia,toallEUAmemberinstitutions.AnumberofpartnerorganisationsactiveinEuropeanhighereducationwerealsoaskedtoencouragetheirmemberstorespond.Theresponseswereanalysedintermsofgeneraltrendsandrespecting,atthisstage,theprincipleofconfidentiality:thereforenoinstitutionorindividualisidentifiedinthisreport.TheanalysisofresultsasstatedunderChapter3issolelybasedonresponsesreceivedfromthesurvey.

Atotalof222institutionsfrom36countriesansweredthequestionnairebythedeadlineof30April2010.Whereastherewasoneanswerfromeachofsevensmallercountries,itappearsthatnoneofthelargercountriesisoverrepresentedinthesample.Countriesfromwhichmorethan10answerswerereceivedareFinland,Germany,Ireland,Portugal,SpainandUnitedKingdom(SeeFigure2).

Thevastmajorityofrespondentswereuniversities(176),followedbyuniversitiesofappliedsciences(32),andotherhighereducationinstitutions(14)–thelatterbeingcomposedofdiscipline-basedinstitutionsanduniversitycolleges(seeFigure1).Mostoftherespondinginstitutions(195),includingalltypes,traintheirstudentsuptothedoctoralorthird-cycleequivalentlevel.

About59.5%(132)oftherespondinginstitutionscountmorethan10,000students,whereas35.1%(78)aremiddle-sized(between1,000and10,000students),and5.4%(12)small(lessthan1,000students).

72,78%

Figure 1: Ratio of types of institutions

14%

79%

6%

University of AppliedSciences, Polytechnic,Fachhochschule orequivalent

University

Other higher educationinstitution

15


Figure 2: Distribution of respondents per country

Countries with no answer

Countries with 10 or more answersCountries with 1 to 4 answers

Countries with 5 to 9 answers

2

104

5

1

11 8

8

2

26

21

4

4

8

1

99

1212

169

3

4

10

11

8

8

3

16

20

1

1

1


16

Thenameoftheproject–ExaminingQualityCulture–containsapromiseofexposingthetruestateandnatureofqualitycultureinEuropeanhighereducationinstitutions.Asdescribedinthepreviouschapter,thispublicationfocusesonthequantitativeresultsofthesurveyexaminingthequalityassuranceprocessesinplaceinhighereducationinstitutions.Beforescrutinisingtheseresultsitis,nevertheless,worthtouchinguponhowwehaveunderstoodbothqualitycultureandqualityassuranceprocessesintheframeworkofthisparticularprojectfortheresearchpurposes.Duringthecourseofthisproject,weexpecttoreachabetterunderstandingoftheseconceptsandthedynamicsbetweenthemandwillfurtherelaborateonthisinthefinalpublicationoftheproject.

2.1. The elements of quality cultureEUA’sQualityCultureproject’s (which ran in three rounds from2002 to2006)mainaimwas to

“increaseawarenessfortheneedtodevelopaninternalqualitycultureininstitutionsandtopromotetheintroductionofinternalqualitymanagementtoimprovequalitylevels”.(EUA2006:6-7)

As EQC continues the long-standing work of EUA in promoting quality culture, the followingdefinitionofqualityculturedevelopedbytheaforementionedprojectwaschosenasthestartingpointforourwork.Thus,inthecontextofthisproject:

[q]uality culture refers to an organisational culture that intends to enhance quality permanently and is characterised by two distinct elements: on the one hand, a cultural/psychological element of shared values, beliefs, expectations and commitment towards quality and, on the other hand, a structural/managerial element with defi ned processes that enhance quality and aim at coordinating individual efforts.(EUA2006:10)

The recentdiscussions in thefieldofquality andquality assurance inhighereducationarequiteunanimousinadvocatingpromotingqualityculture,somuchso,thatoftenqualitycultureisconsideredtobeasynonymfor“thedevelopmentof,andcompliancewith,processesofinternalqualityassurance”.(Harvey2009:1)

However,asdemonstratedbyFigure3,itiscrucialtodistinguishthesetwoconcepts:qualitycultureandqualityassurance.Whereasqualityassuranceprocessesare something tangibleandmanageablebyinstitutional decisions, the cultural aspect of quality culture – shared values, beliefs, expectations andcommitment–isfarmoredifficulttochange.(Ehlers2009)


“There needs to be a perceived value of quality assurance. Quality culture and quality assurance are not the same thing. You can have good QA in place but not necessarily a quality culture. The challenge is linking the outcomes of QA to the development of a quality culture that enhances the student experience.” - Respondent to the survey

“Much of the quality is dependent on the informal nature of staff/student relationships. The increasing calibration of quality indicators has led to a concern that this relationship will become formalised and thus less productive.” - Respondent to the survey

17


Thebasicassumptioninourprojecthasbeenthatqualitycultureandqualityassuranceprocessesareinterrelatedandthatqualityculturecanbeenforcedbystructuraldecisionswhichstimulatesharedvaluesandbeliefs.(Harvey&Stensaker2008:434)Furthermore,weacceptthatnoone(correct)qualitycultureexistsasacultureisalwayscloselylinkedtotheenvironmentandwithinoneHEItheremightevenbeseveralsub-culturesofquality.(Harvey&Stensaker2008;Harvey2009;Ehlers2009)

WhendescribinghowaHEIcouldgoaboutdevelopingitsqualityculture,Lanarèswrote:

There are at least two ways of seeing this. In some cases, the institution will introduce quality assurance. This will imply new values which will have to be integrated in the organisational culture. In other ones, the creation of quality assurance will start from the existing quality culture. Once finalised, quality assurance will in turn influence and modify the quality culture [...]. This second option may be preferable, considering that some continuity will facilitate change.(Lanarès2008:13)

ThismakesthechallengethatHEIsarefacing,andthusthetopicofourproject,evenmoreintriguing.

2.2. Quality assurance processes as understood in this project

Having established that quality assurance processes form one key component of quality culturealthoughtheydonotequalqualityculture,anddefinedthatthefocusofthisreportisonqualityassuranceprocesses,thequestionthatnaturallyfollowsis:whatdoweunderstandthesetobe?Indeed,itisclearthatthedefinitionvariesconsiderablyfromonecountryandinstitutiontoanother–nottomentionfromonepractitioner toanother.As theBolognaProcessStocktakingreport,whichaskedgovernmentsabout theprogressmadeinimplementingvariousBolognaProcessactionlines,observed:

It should be noted that the answers of some countries suggest that they think internal quality assurance within higher education institutions means only preparing self-assessment reports, without any reference to learning outcomes-based and improvement-oriented internal quality assurance systems. In addition, some HEIs have established a management system and they claim that it is a quality assurance system. However, some of these systems focus on measuring the performance of staff and/or units rather than on implementing ESG. This suggests there is a need to increase the focus on internal quality assurance within the EHEA.(Rauhvargerset al.2009:51)

Quality Culture

CommunicationParticipation

Trust

Formal qualityassurance processes

Tools and processes todefine, measure, evaluate,assure, and enhance quality

Quality commitmentCultural element

Individual level: personalcommitment to strive forquality

Collective level: individualattitudes and awarenessadd up to culture

Figure 3: Elements of quality culture (adapted from EUA 2006: 20 and EUA 2005: 18)


18

It iscorrect that, insomecontexts, internalqualityassuranceprocessesareseenastheprocessesthataimtoprepare the institutionor theprogramme foranexternalevaluation (i.e.preparing theself-evaluationprocess)orthemonitoringtaskassignedtoaspecificallyestablishedqualityunit. Ifthis isthewayqualityassuranceprocessesareregarded,itinvolvesrisks.WherearecentENQAsurveytoitsmemberagencies noted that three-quarters of respondents have recently changed or are about to change theirqualityassuranceapproach(ENQA2008:26),EUA’smostrecentTrends reportconcluded:

that the introduction of new national external evaluation procedures has caused some institutions to pay much less attention to their own internal accountability procedures, thus leading to a compliance culture. This seems to be particularly true when the external agency is perceived as being formalistic and bureaucratic.(EUA2010a:63)

Intheframeworkofthisprojectwehaveoptedto:

use quality assurance in the broad meaning of the term, including in practice all elements of a strong quality culture of a HEI. Internal QA in the context of this report should not be understood merely as a specific quality monitoring (such as process descriptions, data collection and analysis) or evaluation processes often carried out by a specific quality unit, but including all activities related to defining, assuring and enhancing the quality of an HEI. (EUA2009:13)

Furthermore, going back to our previous quote from the Stocktaking report, we agree withRauhvargerset al. thattheremaybeaneedtofocusorclarifywhat isunderstoodbyqualityassurance,but,inthelightofwhatwasdiscussedpreviouslyregardingcultureofcompliance,donotthinkthatqualityassuranceshouldbelimitedtocoveringonlytheactivitiesmentionedbytheESGs.

Wewouldevengoso faras toencourage institutionstoadoptanall-encompassingapproachtodevelopingtheirinternalqualityassuranceprocesses:anapproach,thus,thatwouldbetailor-madefortheinstitutionandderivedfromitsownstrategicgoalsandfitintothestate-of-artoftheinstitution’sinternalqualityculture,whilefulfillingtheexternalrequirementsintheprocess.

ItisinthiscontextthatwehaveintentionallychosennottolimitourselvesonlytomeasurescoveredbytheESGsasitwouldprovideaveryone-dimensionalviewonqualityassuranceprocesseswhichreallycan–andoftendo–takevariousformsandshapes.Also,itshouldbehighlightedthattheESGsonlydealwithteachingandlearningactivitywithinaninstitution,whereas,whenplanningtheiractivities,institutionstypicallyviewtheiractivitiesintheirentirety.Forexample,inaninstitutionwhichhasastrongresearchfocusandbasesteachingonresearch,canonecompletelyseparatethequalityassuranceofteachingandlearningfrom research? Furthermore, the strategic management and the quality of governance will inevitablyinfluencethequalityofteachingasOECD’sProgrammeonInstitutionalManagementinHigherEducation(IMHE)arguesintheirup-comingpublication(IMHE2010:2).

Also, EUA’s work over the years has stressed the importance of creating a link between qualityprocessesandinstitutionalstrategicplanning(EUA2006;EUA2010a).Aninstitution,havingclearlydefinedits mission and strategic goals and knowing what quality means in the light of its own goals, lays thegroundworkforawell-functioningqualityassurancesystem.Thenextstepinvolvessettingupprocessestoensurethatthisqualityisreachedandtomonitorprogressinthisregard(asaddressedbyEGSspart1)andfinally,beingabletoreactwhenallisnotwelloreven,ifallseemstobeworkingrelativelywell,beingableandwillingtoimprovecontinuously.Inthisregard,aninstitutionneedstodevelopafeedbackloopthatwouldstrengthenthelinkbetweentheresultsofmonitoringactivitiesandstrategicplanning.

Asaresult,weincludedinoursurveysomequestionsrelatedtotheinstitutionalqualityassuranceframework in general as well as questions related to quality assurance in research and to strategicmanagement.However,themainbodyofthesurveyremainsfocussedonqualityassuranceofteachingandlearningfollowingthelogicoftheESGsandaspirestomapthatstateofplayintheimplementationoftheESGstoprovidequantitativedataforthebasisofanypolicydiscussionsontheprogressmadeinthatregard.

19


3.1. Quality assurance structures

Data collected through the survey shows that most of the responding HEIs have fundamental quality assurance structures and processes inplaceandremarkableprogresshasbeenmadeinrecentyears,althoughanumberofchallengesremain.

Qualityassuranceprocessesmostcommonlycoverteachingandlearningactivities,whichisquiteunderstandable,asthecreationoftheEuropeanHigherEducationArea–andtheESGsasanintegralpartofit–hasfocussedonthispartofHEIs’mission.Thus,98.2%oftherespondentstooursurveyansweredthattheirqualityassuranceprocessescoverteachingandlearning,whilestudentsupportservices,whicharecloselyrelatedtoteachingandlearning,arecoveredinonly75.7%ofHEIs(seeTable1).Theotheractivitiesmentionedby6.7%(15)oftherespondentscoverareasasdiverseasartisticactivitiesandequalopportunitypolicies.

Table 1: Activities covered by institutional quality assurance processes

OnecommonphenomenonnotedwasthetendencynottorecogniseallQArelatedprocesseswithinaHEIassuch.Forexample,while79.3%oftherespondentsrepliedthattheirinstitutionalqualityassurancesprocessescoverresearchactivities,atotalof97.3%ofallrespondents–whenaskedwhetherornottheyhave specific processes in place with regard to QA in research – recognised that they do indeed haveindividualprocessesinplace.Themostcommonprocessesareinternalseminarswhereresearchprojectsandideasarediscussed(65.3%),preparingstatisticsonpublishedarticles(64.9%)andexternalpeerreviewofresearchprojectsinrelationtograntapplications(53.6%).Afurtherexampleofthisinabilitytodiscernthe

3. Mapping internal quality assurance processes: survey results

Which activities do your institutional quality assurance processes cover?Please choose all applicable options.

Teachingandlearning 98.2%

Research 79.3%

Servicetosociety 47.7%

Studentsupportservices 75.7%

Governanceandadministrationoftheinstitution 65.8%

Other 6.7%

“Quality culture must articulate Teaching, Research and Service to society. It should respect the diversity in specifi c domains and fi elds and rely on accurate objectives established according to the various institutional, political and cultural situations. Those objectives should be adapted to the broad context. Quality culture should aim at continuous improvement instead of ad hoc and instrumentalising evaluations.” - Respondent to the survey

“Some aspects of quality culture are consolidated in the university community, such as administrative services, but others are still “on paper” and not in “minds”. This is clear in all aspects of evaluating teaching and learning skills, and this is a big handicap we need to overcome to transfer from simply evaluating results to real effective decisions and political commitments.”

- Respondent to the survey


20

presenceofcertainQAprocesseswaspickedupthroughcross-checkinganswersregardingQAprocessesin services to society:whilstonly47.7%of the institutionsdeclaredcovering services to society in theirQAarchitecture,95.9%of the respondents,whenasked to specifywhichprocesses theyhave inplace,pinpointedoneorseveralprocessesrelatedtoservicestosociety.

AsEUA’sQualityCultureprojectidentifiedstrategicplanningascrucialinembeddingqualityculture(EUA2006:13)andthefirstguidelineforinternalqualityassurancementionedintheESGsstatesthatHEIs“shouldhaveapolicyandassociatedproceduresfortheassuranceofquality”,oursurveyaskedHEIsiftheyhadan institutional strategic plan or equivalent document.Anoverwhelmingmajority(92.8%)repliedthattheydohavesuchadocument.WhereassomeHEIsreportedthattheirstrategicplanshavebeendrawnupatfacultylevel,onlythreeinstitutionsadmittednothavingastrategicdocumentatall.Ourdatafurtherindicatesthatifaninstitutionhasastrategicplan,itusuallyalsohaseitheraseparateinstitutionalQApolicystatement(68.1%)oritsqualitystatementisincludedinthestrategicplan(25.5%).

FurthertoQApolicy,HEIshavealarge variety of organisational structuresinplacetosupporttheimplementationofQAprocessesbutnotypicalsolutiononhowtoarrangetheresponsibilitiesforQAwithinaHEIseemstoexist.Figure4demonstratesthedistributionofdifferentstructuresamongtherespondents.ItisnoteworthythatnearlyhalfoftherespondingHEIs(103)donothaveanycommittees(neitheratfacultynor institutional level)responsible forQAprocesses2. Inaddition,thereare80 institutionsthatansweredthatarectororvice-rectorisnotresponsibleforQAissues.Eveninthesecasestherewasnotypicalsolutiononhowtoorganisetheinstitutionallevelresponsibilitieswherethereisalackofinstitutionalleadership.Asexamplesoforganisationalfeatures,outofthese80institutions,andincombinationwithallsortsofotherfeatureswithnotypicalscheme,27institutionshaveacentralunitspecialisedinQA(amongotherthings),12haveapersoninchargeofQA(amongotherthings)withintherectorate,and13haveboth.

ItcanbenotedthatcentralisedunitsforQA,pedagogicaldevelopmentandstaffdevelopmentaremorelikelytobeinplaceinthoseuniversitiesthatworkedontheirQAsystembefore2000:respectively88.2%and55.8%ofinstitutionsthatintroducedtheirsystembeforeandinthe1990’shaveaunitresponsiblefor staffdevelopment,whereas thispercentagedropsandnever exceeds30% for institutions thathaveintroducedtheirsystemwithinthelastdecade.Theexistenceofaunitinchargeofpedagogicalinnovationfollowsthesametrend:respectively76.5and60.5%ofthosewhointroducedtheirsystembefore1990andinthe1990’shaveone,whilethepercentagedropsbelow50%forinstitutionsthatintroducedtheirsystemafter2000.

72,78%

Figure 4: Structures supporting the internal quality assurance processes

0 20 40 60 80 100 120 140 160

The rector or specially assigned vice-rector is in charge of QA issues.

There is a centralised QA unit, with specialised staff.

There is an institutional level quality committee or equivalent.

There is a unit responsible for pedagogical innovation (or equivalent) that offerssupport to the teachers in developing teaching methods.

There are contact persons or persons in charge of QA within their unit, who alsohave other responsibilities.

There are faculty level and/or department and/or programme level qualitycommittees or equivalent.

There is a unit responsible for staff development.

There is a person in charge of QA within the rectorate.

Other

There are QA units in each faculty with specialised staff.

64.0%

62.2%

53.6%

47.7%

45.0%

40.5%

38.3%

36.5%

9.9%

9%

Number of answers

2SeealsoChapter3.2.inthisregard.

21


Thus,HEIshavethefundamentalqualityassuranceprocessesandstructuresinplace,butourdataalsodemonstratesthatthe progress made in this regard is very recent.Alittleoverhalfoftherespondinginstitutions(52%)reportedthattheyintroducedtheirQAsystemin2005orlater,aconsiderableportionstillbeingintheplanningorintroductionphase(Figure5).

Thetimingofintroductiondoesnotappeartodependonthetypeorsizeofaninstitution,noraretheredistinctivedifferencesbetweencountries,withtheexceptionoftheUK.WhileinallothercountriesthereareHEIsthathavestartedworkingonQAprocessesonasystematicbasisatdifferenttimes,allUKinstitutionsreplyingtoourquestionnairealreadyhadtheirprocesses inplacebytheturnofthecentury(Figure6).

Figure 5: Introduction of a quality assurance system (or equivalent)

16%

36%

8%

19%

21%

In the 1990's

Between 2000 and 2005


Before 1990

We are currently designingand/or planning it.


22

Althoughthere isacleartrendthattheinstitutionsthat introducedaQAframeworkbefore1990are more inclined to have adopted an institutional approach, ranging from a policy statement to theinvolvementofinternalandexternalstakeholders,theseveryinstitutionsseemtograntlessimportancetomakinginformationandassessmentspublic.However,consideringthatavastmajorityoftheseinstitutionsisfromtheUK,thismaywelljustbeanationalfeatureandnofar-reachingconclusionscanbedrawnfromit.

In terms of how the institutional QA system was introduced, just over half (51.8%) of therespondentsbasedtheirconceptontherequirementsofthenationalQAagency,while40.5%saidthattheinstitutionalleadershipdecidedontheconceptandprovidedinstructions,trainingandsupporttotheunit to implement it. Inaddition,43.7%ofHEIsansweredthattheirQAconcept is theresultofvariousconsultationroundsamongtheacademicandadministrativestaffaswellasstudents,whileonethird(32%)usedpilotprojectsconductedbyselectedunitsintheintroductionphaseanddisseminatedgoodpracticesidentifiedthroughthem(Figure7).

Figure 6: Introduction of a quality assurance system (or equivalent) – Breakdown per country

No data

Before 2000


Currently designing/planning

23


3.2. Participation of Stakeholders

Institutional leadership

AsEUA’sQualityCultureproject’sonekeymessagewastounderlinethecrucialroleofinstitutionalleadership in demonstrating an institutional commitment to quality, thus laying a groundwork for theimplementationofqualityculture(EUA2006:20),itisworthwhiletakingacloserlookathowitsroleplaysinthelightofsurveyresults.

66.2%of the respondentsdefined the roleof senior leadership (rector, vice-rector) inbuildingaqualityculturewithintheirinstitutionastakingtheleadintheprocess.Inthosecaseswheretheleadershipwasnotleadingtheprocess, itwasusuallymentionedthatitmonitors,makesdecisionsorfacilitatestheprocessorsomecombinationofthese.Inuniversities,theleadershipismorelikelytoleadtheprocessandmakedecisionsthaninothertypesofinstitution:indeed,outoftheinstitutionswheretheleadershipdrivestheprocess,82.3%areuniversities–whichisslightlyhigherthantheratioofuniversitiesinthesampleofrespondentstothewholesurvey(79%).

Asmentionedabove(Figure7),in40.5%ofthecases,theinstitutionalleadershipdecidedontheQAconceptoftheHEI.WhenitcomestooperationalresponsibilityforQAprocesses,thepercentageofHEIswheretherectororspeciallyassignedvice-rectorisinchargeofQAissuesgrowsto64%(Figure4,Chapter3.1).Nevertheless,thedataindicatedthattheroleoftherector(orvice-rector)ismoredominantinthoseHEIsthathaveworkedontheirQAsystemformorethan10years.

Figure 7: How the internal quality assurance system (or equivalent) was introduced within the institution

64,0%

0 20 40 60 80 100 120

The concept is based on the requirement of the national QA agency whichdeveloped the standards and guidelines for this.

The concept is the result of various consultation rounds among the academicand administrative staff as well as students.

The institutional leadership decided on the concept, provided instructions,training and support to the units to implement it.

The concept was introduced through pilot projects conducted by some units.Good practices were disseminated based on these experiences.

The concept is the result of various consultation rounds among the academicstaff of the institution.

The concept is the result of various consultation rounds among the academicand administrative staff.

Other

51.8%

43.7%

40.5%

32.0%

14.4%

14.0%

13.1%

Number of answers

“The only way to achieve a functional quality culture is by convincing the members of the HEI that they have something to gain by analysing the qualitative processes of their day-to-day work.”


“The biggest challenge for quality culture implementation is to combine the top-down leadership and managerial approach with the bottom-up approach, while creating favourable learning environments for academic staff and students to be actively involved in quality culture implementation activities via their own initiatives and responsibilities.” - Respondent to the survey


24

Staff and students

Oneofthekeyprinciplesindevelopingbothqualityculture(EUA:2006,Harvey&Stensaker2008)and quality assurance processes (see various ministerial communiqués) is the participation of staff andstudents.AsindicatedbyFigure7(Chapter3.1),inalmost44%ofthecases,studentsandstafftookpartintheplanningoftheinstitutionalqualityassurancesystem.Inaddition,28.4%oftherespondentsinvolvedtheiracademicstaffintheconsultations(whichmakesatotalof72.1%involvingtheiracademicstaff),while14%involvedtheiradministrationstaff(totalof57.7%).

Settingupcommittee(s)forqualityassurancemightseemalogicalwayofensuringtheparticipationofstaffandstudents.However,aftercross-checkingtheresponsesonQAstructures(Figure4,Chapter3.1),itappears that theHEIswithcommitteesat institutional levelaremostly those thathavecommitteesatfaculty/departmentallevel.Thisleavesuswith46.4%oftherespondentswithnocommitteeresponsibleforqualityassurance.However,datadoesshowthatthelongertheHEIhasworkedonQA,themorelikelyitwillbetohaveinstitutionalorfacultylevelcommittees.

Figure8 illustrates theextent towhichvarious stakeholdersare involved in thequalityassuranceprocesses of the responding HEIs. This shows that the level of participation is relatively high, with theacademic staff being the most commonly involved. Furthermore, when HEIs reported that they have

Figure 8: The involvement of stakeholders in formal quality assurance processes

Through formal participationin consultation bodies

Through formal participationin governance bodies(with voting right)

They are not involved

By responding to the surveyson a regular basis (e.g.end ofeach course, academic year...)

By informally providinginformation on the issues atstake

Through formal involvementin self-evaluations or otherevaluation activities

Academic staff

Administrative staff

Leadership,institutional level

Leadership, faculty/department level

Students

Externalstakeholders

Alumni

0% 20% 40% 60% 80% 100%

81.5%74.8%

90.5%65.8%

53.6%

9.9%19.4%

25.7%41.4%

34.7%11.7%

41.9%41.0%

36.9%

21.2%9.0%

49.5%

0.5%

76.6%68.0%

74.3%63.1%

79.7%

77.5%64.9%

59.5%27.9%

76.1%

61.3%62.6%

58.6%68.5%

26.6%2.7%

85.1%66.7%66.2%

54.5%15.8%

25


formedworkinggroups toprepare curricula, academic staff is almost always (99.5%) included in theseworkinggroups,with37HEIsreportingthattheirworkinggroupsaresolelycomposedofacademicstaff.Itisnoteworthythatonlyinhalfofthecases(50.8%)arestudentsinvolved.

With regard to student participation in quality assurance, some previous studies (Rauhvargers et al. 2009:14; ESU200948-49)have shown that, althoughprogresshasbeenmade, there is still roomfor improvement,pointingout inparticular the fact that students arenotusually involved in follow-upmeasures.

Tosomeextent,ourdataconfirmthis.Thefeedbackloopbetweenstudentsurveysandthefollow-upactivitiesisdiscussedinfurtherdetailunderChapter3.3,butfromtheperspectiveofpromotingstudentparticipation,itcanbenotedthattheinfluenceofstudentsurveysandtheimportanceofthefeedbacklooparemuchmoredevelopedatinstitutionsthatintroducedtheirsystemearlyon,i.e.before2000.Indeed,94.1%oftheseHEIstakethesurveysintoaccountinthecurriculumdesign,discusstheminternally,andinformstudentsaboutthefollow-up.

Finally, with regard to the involvement of members of the institutional community, it is worthreferring to thefindings indicatedbyTable4 (Chapter3.3) thatconducting regular surveysamong themembersoftheinstitutionalcommunity(staffandstudents)toanalysetheirperceptionoftheinstitutionalstrategyand its implementationatgrass-roots leveldoesnotseemtobeacommonpractice (withonly27.9%reportingthattheydoso).Chapter3.3willfurtheranalysethefeedbackloopandcommunicationamongtheinstitutionalcommunity.

External stakeholders

External stakeholders (e.g. employers, experts) have been involved in QA processes from thebeginning.However,theirparticipationlevelvaries:around80%oftheHEIsincludethemindifferentwaysin their decisionmaking: throughgovernancebodies, consultationbodies or as sources of information,amongothers,whileroughly20%oftheHEIsdonotconsultorinvolvethematall.

External stakeholders are more likely to be involved in preparing a curriculum (for instance asmembersofworkinggroups)whentheultimatedecisiononthecurriculumismadebyabodyexternaltotheHEI.However,theresultsindicatethattheirroleisusuallythatofinformationprovidersratherthandecisionmakers.Furthermore,thealumniareseldominvolvedassuch(Figure8).

3.3. The use of informationThe purpose of this chapter is to discuss what kind of information HEIs collect and store about

themselves,howtheycommunicateonthebasisofthisinformation,andhowtheinformationfeedsintotheinternaldiscussionsanddecisionmakingprocesses.

Types and sources of information

93.2%of theresponding institutions reportedthat theyhaveacentralised informationsystem inplace.Inaddition,5.9%ofrespondents,whentheydonothaveacentralisedsystem,dohaveinformationsystemsthatexistat faculty level.Therefore,thenumberof institutionsthatdonotuseany informationsystemformonitoringtheiractivitiesisextremelylimited.


26

Mostcommonly,theseinformationsystemsincludestudentprogressionandsuccessrates(87.7%of respondents),aswell as theprofileof the studentpopulation (83.2%).The teacher-student ratiopereducationalunit also scoreshigh (65.5%),asdoes student satisfactionwith theirprogrammes (53.6%).However,this53.6%seemslowcomparedtothe71.6%ofrespondentswhoansweredthattheyconductstudentsurveysassessingtheteacher’sperformancesandcompetences.Table2givesanoverviewofthedifferentinformationincludedininstitutions’informationsystems.

Table 2: Information included in the information system(s) regarding study programmes

Whereas institutions collect information about their profile and what they offer, the informationbecomes scarcewhen it comes to resourcesavailable to the students. Indeed,whilemore than80%ofallrespondentsconfirmedthattheyofferlibraryservices,computerfacilities,laboratories,humansupport(tutors,counsellorsorotheradvisers)andotherkindsoflearningfacilities(seealsoFigure12,Chapter3.4.),only44.1%feedtheinformationontheavailablelearningresourcesand,wherepossible,theircosts,intotheirinformationsystem.

Interestingly,whattheinstitutionscollectmostoften(profileofstudentpopulation,studentprogressionandsuccess rates)doesnotnecessarilycorrespondwith the informationmadepublic (seeTable3).Thedatamosttypicallyfeaturedisinformationonstudyprogrammes,includinginformationonqualificationsgrantedbytheprogramme(86.9%)andonteaching,learningandassessmentprocedures(82%).Whereasqualificationsaredocumentedbymostinstitutions,theyarenotalwayspartoftheinformationonalumniemployment:only40.5%ofrespondentstrackgraduateemploymentandincludethisintheirinformationsystem,butTable3showsthat45.5%providesomekindofdetailonalumniemploymentintheinformationmadeavailableontheirstudyprogrammes.

Which of the following does the information system or systems include? Please choose all applicable options.

Studentprogressionandsuccessrates 87.7%

Teacher-studentratioperfaculty/department/instituteorintherespectivefaculty/department/institute 65.5%

Trackinggraduates’employment 40.5%

Students’satisfactionwiththeirprogrammes 53.6%

Profileofthestudentpopulation(e.g.,age,gender,educationbackground,socio-culturalbackground...) 83.2%

Availablelearningresourcesand,whenapplicable,theircosts 44.1%

Noneoftheabove 0.9%

Other(suchastheinstitution’sownperformanceindicators) 10.0%

27


Table 3: Information provided by the institution on its study programmes

Intermsofpublic informationonQA,institutionsmosttypicallymakepubliclyavailable(throughwebsites,publicationsorothersourcesofinformation)theresultsoftheexternalevaluations(61.3%),butaremorereluctanttomakepubliclyavailabletheresultsoftheirinternalevaluations(38.7%).Consideringthattheinternalevaluations–atleastideally–areimprovement-led,includingpointingoutshortcomingsandsuggestingremedies,thedesiretokeepthemasinternalworkingdocumentsisperhapsunderstandable.

Feedback loop and communication

HavingnotedthatHEIscollectdataontheirperformance,itshouldbeaddedthatthelinkbetweencollecting the information and informing the community involved in this data collection is not all thatobvious.Asanexampletoillustratethispoint,thegeneraltrendwithregardtoteachers’performancesistokeepthemconfidentialandavailableatinstitutionalorfacultyleadershiplevelonly:59%oftherespondentsreportedthatthisisthecaseintheirrespectiveHEIs.18.9%makethemavailableatinternallevel(forthoseinvolvedinQA,andfortheteachingcommunityingeneral),andonly6.3%(14institutions)makethempubliclyavailable.

Moreover,whileexploring further the feedback loop,wecansee that71.6%of respondentsusestudentsurveysasoneofthemeanstomonitorstudents’perceptionsoftheteachingtheyreceive.Amongthoseconductingstudentsurveys,92.4%taketheresultsintoaccountintheassessmentofteachingstaff.Butonly58.5%ofthemstatethatstudentswhoparticipatedinasurveyareinformedabouttheoutcomesandtheresultingactionstaken;andthepercentagedropsto6.3%whenitcomestomakingtheinformationonteachers’aptitudesandperformancepubliclyavailable.

In thecaseof22.3%of the respondents, the legal frameworkdoesnot foresee thepossibilityoftheremovalofanineffectiveteacher.ThisisoneoftheareaswherethestandardssetbytheESGsattheEuropeanlevelclashwithnationalregulations.Asourdata indicates,thefeedback loopclearlyfunctionsbetterwithintheinstitutionswithprocessesinplacetoobligeateacherimprovehis/herperformance.All61institutionsthathaveprocessesinplacetoremoveanineffectiveteacher,conductstudentsurveys,with49ofthemtakingtheresultsintoaccountintheassessmentofteachingstaff,and44oftheminformingthestudentsontheoutcomesandactiontakenasaresult.56oftheseinstitutionsusethestudentsurveysasonesourceofinformationwhendesigningand/orrevisingprogrammes.

The information on your institution’s study programmes include: Please choose all applicable options.

Numberofstudentscurrentlyinvolvedintheprogramme 76.1%

Numberofacademicstaffinvolvedintheprogramme 70.3%

Teacher-studentratiointherespectivefaculty/department/institute 44.6%

Informationontheintendedlearningoutcomesoftheprogramme 81.5%

Informationonqualificationsgrantedbytheprogramme 86.9%

Informationontheteaching,learningandassessmentproceduresusedwithintheprogramme 82.0%

Informationonthelearningopportunities(e.g.,traineeships,exchangeprogrammes,mobilitypossibilities,scholarships...)availabletothestudentsoftheprogramme

78.8%

Informationonalumniemployment 45.5%

Profileofthecurrentstudentpopulation 43.2%

Specificinformationtargetinginternationalstudents 64.0%

Accessibilityand/orpossibilitiesofferedtodisabledstudents 49.5%

Other 5.0%


28

TheresultsofthesurveyleadustobelievethattheQAofteachingstaffmaybecloselyconnectedtothetimeframeoftheintroductionofQA.Whilealmosttwothirds(63.4%)ofallinstitutionssurveyedhavespecifiedtheirownrequirementsforhiringteachingstaff,76.8%ofthosewhohadintroducedtheirQAsystembefore2000havedoneso.

Link to the strategic management

Further to thequestionsof involving thecommunity in the follow-upand the impactof studentsurveys in particular that was discussed in Chapter 3.2., the feedback loop between the results of QAprocessesandstrategicmanagement isoneofthekeysuccessfactors insustainingqualityculture.(EUA2006:13,18)

The most common way of ensuring the feedback loop to strategic management appears to betheconductingofannualevaluationsby the institutional leadership to review thegoals thathavebeenset (65.3%). In addition to the65.3%who chose this option, therewereotherswhodescribed similarprocesses,althoughunderliningthattheevaluationmadebytheleadershiphappensthroughdiscussionswithotherkeystakeholders(deans,headsofunitsetc.).However,thenatureofthisannualevaluationwasnotexaminedinfurtherdetailinthequestionnaireandthereforethedepthandeffectivenessofthisprocesscannotbeexploredhere.Annualevaluationsmaytakevariousshapesandfollowdifferentrationales–formalprocessesrequestedbyaMinistryorotherexternalbodies.

Table 4: Internal evaluation processes providing feedback to the strategic planning

Decisionson future strategicdirectionare ideallybasedon solidevidence,andQAprocessesaretypicallyconsideredtobeoneofthekeyinformationsources.Nevertheless,only55%oftherespondingHEIshaveformulatedkeyperformanceindicators(KPIs)tofollowtheirprogressbasedonthem(seeTable4).Itislogical,however,thatthosewhoarestillworkingontheprinciplesoftheirQAsystemarelesslikelytohaveKPIsinplace.

3.4. Quality assurance in teaching and learningWhenaskedaboutdesigningaqualityassuranceframeworkinteachingandlearninginparticular,

amajorityofHEIs(64.9%)indicatedthatitisinstitution-specificbutfollowsnationalQAframeworksandguidelines.Onequarter (25.7%)considered it tobe tailor-madeto the institution’sneedsanddoesnotapplyanyready-mademodel,whereaslessthan9.5%mentionedthatitappliesaready-mademodelsuchasISO,EFQM,andCAF.

Do you have an internal evaluation process that provides feedback to the strategic planning in place? Please choose all applicable options.

Theinstitutionalleadershipevaluatesannuallytheprogressmadeintermsofachievingthegoalssetbytheinstitution. 65.3%

Thefaculties(and/orrelevantunits)conductregularself-evaluationstoanalysetheircontributiontotheachievementofinstitutionalstrategicgoals. 55.9%

Theinstitutionconductsregularsurveysamongthemembersoftheinstitutionalcommunity(staffandstudents)toanalysetheirperceptionoftheinstitutionalstrategyanditsimplementationatgrass-rootslevel.

27.9%

Theinstitutionhasdefinedasetofkeyperformanceindicatorsandfollowsitsprogressbasedonthem. 55.0%

Theinstitutionalstrategyandtheachievementofthegoalssetinitarerevisitedwhenthedocumentisrevised(every3,5orNyears). 52.7%

Other 11.3%

29


Hence,qualityassuranceinteachingandlearningdoeshaveitsspecificitiesandfewHEIshavechosentoadoptexternalQAsystemmodelsas such.The fact that thenumberofHEIs following theirnationalQAframeworksisnothigherisaninterestingphenomenon,anditmayreflectthefactthatguidelinesforinstitutionallevelQAframeworkdonotexistineverycountry.IncountrieswithanexternalQAsystembasedoninstitutionallevel,amajority,ifnotallofHEIstendtoanswerthattheyfollownationalguidelines.Thisis,forexample,thecaseforFinland,France,IrelandandtheUK,wherenationalguidelinestargetingspecificallyinstitutionallevelQAarrangementdoexist.However,nationalguidelinesoninstitutionalQAprovisionsexistinsomecountrieswhereaccreditationisprogramme-based:forexample,inGermany,Spain,PolandandSweden,whereamajorityofrespondentsansweredthattheyfollownationalguidelinesinthisregard.

Thischapterpresentsselectedkeyresults regardingqualityassurance in teachingand learning inparticular,butitisworthnotingalsothatsomethemesintegrallyrelatedtotheteachingandlearningarealsodiscussedinotherchapters.

Curriculum design

Fromtheanswerstothequestionsrelatedtocurriculumdesign,wecandrawtheconclusionthatadelicatebalanceneedstobekeptduringthisprocess.Programmedesign–ingeneralconsideredbyacommittee(85.1%)-whoseobjectivesandexpectedlearningoutcomesarenecessarilyledatfacultylevel,ideallyhastocorrespondwiththestrategyoftheinstitutionasdefinedbytheHEIleadership.Mediationandcommunicationareanessentialpartofthediscussions.10.8%ofrespondentsalsoreportedthattheircurriculumisdesignedbytheMinistryorotherexternalbodyandthismakesthebalancingactallthemorechallenging.

Furthermore,thetraditions,internalstructures,proceduresaswellasthetypeofgovernancewithintheHEIleadtohighlyindividualisedprocessesinthisfield,whichmayvarydependingonthefacultieswithinoneinstitution.

0

50

100

150

200

Figure 9: Designing curriculum and programmes within the institution

Programme directoror equivalent person

prepares thecurriculum after which

staff members maycomment the draft.

Working group,committee or

equivalent preparesthe curriculum.

OtherThe curriculum isdesigned by theministry or otherexternal bodies.

Each staff memberproposes what

they find essentialfor the programme

and the curriculum iscreated on this basis.

36.0%

85.1%

28.4%

5.4%10.8%

Num

ber

of a

nsw

ers


30

Monitoring and improving an established programme

Onceaprogrammeisupandrunning,thefrequencyandmeansformonitoringitalsovaryfromoneinstitutiontoanother.Inaddition,mostinstitutionsseemtoconductavarietyofprocesses(seeFigure10)inavarietyofcombinations,leadingtotheconclusionthatthereisnotonetypicalprocessformonitoring.

Among the two thirds of respondents who answered that the evaluation of curriculum andprogrammes is part of an external accreditation process, only 12 HEIs (without significant geographicconcentration)answeredthattheydonothaveanyotherformofmonitoring.Thismeansthat, incaseswhensuchexternalprocessesarereported,mostofinstitutionsconduct,inaddition,somekindofinternalevaluationinordertomonitorandimprovetheircurriculumandprogrammes.

Learning outcomes and workload

Whereas95.6%ofinstitutionsstatedthattheyhavedevelopedexplicitlearningoutcomes–eitherforallprogrammes(67.9%)orforsomeprogrammes(27.7%)–theydonotallmakethempubliclyavailablethroughtheirwebsite,studyguidesorequivalent:71.7%havedoneso.Thenumberof institutionsthatinclude information on the intended learning outcomes in the information provided about their studyprogrammes is however higher (81.5%). Further studies and research may address this area further byexploring,forexample,howinstitutionsthatsaytheydevelopedthemmakeuseoflearningoutcomesintheirteachingpractices.

Whilst95.6%ofrespondentsdesignlearningoutcomes,only40.6%actuallymeasurethestudentworkloadthroughsurveysaddressedtostudentsinordertoreachthedescribedlearningoutcomes.ThiswouldbeanecessarystepinordertoimplementECTScorrectlysincethesebuildequallyuponlearningoutcomes and student workload as elements. In 44.8% of cases the teachers are solely responsible forindicatingthestudentworkloadandin4.3%noworkloadisindicatedatall.

Figure 10: Monitoring curriculum and programmes

The curriculum and programme contents, pedagogicalapproaches and intended learning outcomes are

evaluated as part of an external accreditationprocess or equivalent.

The curriculum and programme contents, pedagogicalapproaches and intended learning outcomes areevaluated regularly (every N years/semesters...).

The curriculum and programme contents are evaluatedcontinuously on an informal level (discussions between

staff members, staff and students...).

The curriculum and programme contents are evaluatedoccasionally (at the occasion of a self-evaluation

exercise, for an external evaluation body...).

Curriculum and programme design processes as such(that is, the effectiveness and comprehensiveness

of the processes) are evaluated regularly(every N years/semesters...).

Other

0 20 40 60 80 100 120 140 160

66.6%

57.2%

54.5%

36.5%

32.9%

6.8%

Number of answers

31


Student assessment

Figure11 summarises themain featuresof student assessmentprocedures.Whencross-checkingdata,itisclearthatthoseinstitutionswhoseassessmentisdirectlyrelatedtointendedlearningoutcomesarethesameonesthathavedevelopedexplicitlearningoutcomes,eitherinallorinsomeprogrammes.Thelinkbetweenhavinglearningoutcomesandtheirinfluenceinstudentassessmentproceduresisthereforeestablished.

Finally,onlytwoinstitutionsindicatedthattheyeitherhavenoneofthecharacteristicsmentioned,orthattheydonothaveassessmentprocedures.Inconclusion,althoughnoneofthesefeaturesscoredmorethan75%,mostinstitutionshaveacombinationofseveralcharacteristicsasmentionedintheESGs.

Withregardtoinformingstudentsabouttheassessmentprocedures,82.6%ofrespondentsmaketheirassessmentmethodsandcriteriapubliclyavailable,and83%relyontheteacherstoinformorremindstudentsaboutthisat thebeginningof thecourse.Thus,aclearmajorityof the institutionsseemtobetransparentaboutassessmentproceduresasrecommendedbytheEGSs.

Learning resources

AlmostallHEIsparticipatinginthesurveyofferbasiclearningresources,butwhenaskedspecificallyaboutmonitoringandevaluatingthem,itturnsoutthattheyoftendonotdoitsystematically(Figure12).However,interestinglyenough,thenumberofHEIsreportingthattheyregularlyimprovetheservicestheyofferishigherthanthosewhomonitororevaluatethem.Theoverviewseemstoendorsethefindingfromthe Trends series that regular evaluations of student learning services remain relatively rare while studyprogrammes,teachingstaffandresearchactivitiesareevaluatedmorefrequently.(EUA2010:86)

Figure 11: Characteristics of student assessment procedures

They have clear, pre-defined examinationor other assessment methods in place.

They are designed to measure the achievement of theintended learning outcomes and other

programme objectives.

They have clear regulations covering student absence,illness and other mitigating circumstances.

They have clear and publicly availablecriteria for marking/giving grades.

They ensure that assessments are conducted securely inaccordance with the institution's stated procedures.

The administration checks that the assessmentprocedures are followed.

Other

0 50 100 150 200

75.7%

69.8%

66.7%

65.3%

60.8%

48.2%

4.1%

Number of answers


32

Further, with regard to student support services there seems to be a trend that those HEIs whointroducedaQAsystemafter2000arelesslikelytohaveQAofstudentsupportservicesinplacethanthoseinstitutionsthathavehadaQAsystemsincebefore2000.

Figure 12: Institutions offering, monitoring, evaluating and improving their learning resources

Library

Computing facilities(including email account

and internet access)

Human support in the formof tutors, counsellors and

other advisers (in additionto teaching staff)

Laboratories

Learning facilities(language labs, musical

instruments, materialused for classes...)

0 50 100 150 200 250

93.2%

71.2%

69.4%

77.0%

91.3%

68.0%

63.9%

59.8%

55.3%

60.8%

75.8%

92.0%

91.7%

94.2%

66.3%

61.5%

67.3%

61.1%

55.8%

64.9%

Evaluated

Offered

Improved

Monitored

Number of answers

33


Corresponding ESG Summary findings based on the survey

1.1Policy and procedures for quality assurance:Institutionsshouldhaveapolicyandassociatedproceduresfortheassuranceofthequalityandstandardsoftheirprogrammesandawards.Theyshouldalsocommitthemselvesexplicitlytothedevelopmentofaculturewhichrecognisestheimportanceofquality,andqualityassurance,intheirwork.Toachievethis,institutionsshoulddevelopandimplementastrategyforthecontinuousenhancementofquality.Thestrategy,policyandproceduresshouldhaveaformalstatusandbepubliclyavailable.Theyshouldalsoincludearoleforstudentsandotherstakeholders.

Two-thirdsoftherespondentshadaseparateinstitutionalQApolicystatement(67.1%)andinaquarter(24.8%)ofthecasesthequalitystatementwasincludedinanotherinstitutionalpolicydocument.4.5%donothaveanyQApolicydocument.

98.2%oftherespondentsansweredthattheirqualityassuranceprocessescoverteachingandlearning.

1.2 Approval, monitoring and periodic review of programmes and awards:Institutionsshouldhaveformalmechanismsfortheapproval,periodicreviewandmonitoringoftheirprogrammesandawards.

95.5%oftherespondentshavedefinedexplicitlearningoutcomesforallorsomeofthestudyprogrammesand71.7%havemadethempubliclyavailable.

Inmostcases(85.1%)thecurriculumisdesignedbyaworkinggroupconsistingofvariousstakeholdersandultimatelyapprovedatinstitutionallevel(41%)orbyanexternalbody(30.6%).

Processesforreviewingandmonitoringprogrammesvarygreatlyandmostinstitutionsusecombinationsofvariousprocesses.

Involvementofstakeholdersisnotalwaystransparentorstructured.Studentsareinvolvedin40.5%ofHEIsinmeasuringstudentworkloadand,whenaworkinggrouppreparesthecurriculum,50.8%ofHEIsreportthatstudentsarepartofthegroup.

3.5. Implementation of the ESGsTheESGsprovideaEuropeanframeworkforacommonunderstandingofthegenericprinciplesof

qualityassuranceinteachingandlearningandassuchtheyareusuallyexpectedtoguidebothinternalandexternalqualityassurancearrangementsofHEIs.Thus,oneoftheaimsofthissurveyhasbeentogatherquantitativedataonstocktakingoftheimplementationofpart1oftheESGsinHEIs.Table5summarisesthekeyfindingswithregardtotheimplementationoftheESGs.

Table 5: Key findings corresponding to ESGs part 1


34

1.3Assessment of students:Studentsshouldbeassessedusingpublishedcriteria,regulationsandprocedureswhichareappliedconsistently.

75.7%ofinstitutionshaveclear,pre-definedexaminationorotherassessmentmethodsinplace,including,for66.7%,regulationscoveringstudentabsence,illnessorothermitigatingcircumstances.Mostinstitutionshaveamixofseveralfeaturesasmentionedintheguidelines.

82.4%ofinstitutionsmaketheassessmentmethodsandcriteriapubliclyavailablethroughtheirwebsite,studyguidesorequivalent.Inaboutthesamenumberofinstitutions,teachersinformthestudentsaboutthesemethodsandcriteriaatthebeginningofthecourse.

60.8%ofinstitutionsensurethatassessmentsareconductedsecurelyinaccordancewiththeinstitution’sstatedprocedures,and48.2%ofthemhavetheiradministrationcheckingthattheproceduresarefollowed.

1.4Quality assurance of teaching staff:Institutionsshouldhavewaysofsatisfyingthemselvesthatstaffinvolvedwiththeteachingofstudentsarequalifiedandcompetenttodoso.Theyshouldbeavailabletothoseundertakingexternalreviews,andcommenteduponinreports.

71.8%ofinstitutionsconductstudentsurveys,63.1%havespecifiedtheirownrequirementsforcompetenciesofpermanentteachingstaffwhenhiringthem.61.7%offeroptionalpedagogicaltrainingforteacherswhilst26.1%organisecompulsorytraining.

59%ofinstitutionskeeptheinformationonteachers’aptitudesandperformanceconfidentialandavailableonlyattheleadershiplevel(institutionand/orfacultyand/ordepartment).

Inthecaseof22.3%,thelegalframeworkdoesnotforeseethepossibilityofremovinganineffectiveteacher.

1.5Learning resources and student support:Institutionsshouldensurethattheresourcesavailableforthesupportofstudentlearningareadequateandappropriateforeachprogrammeoffered.

Learningresourcesarequitecommonlyoffered,themostcommonbeinglibrary(93.2%)andcomputerservices(90.1%).However,theirregularmonitoringandevaluationisnotquiteascommon.

1.6Information systems:Institutionsshouldensurethattheycollect,analyseanduserelevantinformationfortheeffectivemanagementoftheirprogrammesofstudyandotheractivities.

93.2%oftheinstitutionshavecentralisedinformationsystemsinplacethatincludeinformationontheirteachingmission.Mostcommonlythisinformationincludes:studentprogressionandsuccessrates(87.7%),profileofthestudentpopulation(83.2%)andteacher-studentratioperfaculty/department/institute(65.5%).

1.7Public information:Institutionsshouldregularlypublishup-to-date,impartialandobjectiveinformation,bothquantitativeandqualitative,abouttheprogrammesandawardstheyareoffering.

Themostcommonlypublishedinformationisonqualificationsgrantedbytheprogramme(86.9%),theteaching,learningandassessmentproceduresusedwithintheprogramme(82.0%)andtheintendedlearningoutcomesoftheprogramme(81.5%).Allinstitutionsoffer,withavarietyoffeatures,somesortofinformationontheirprogrammes.

35


4.1. Trends and perceptionsQuality assurance systems largely in place

DatafromtheEQCsurveyshowsthatqualityassurancedevelopments intheircurrentformatareveryrecentphenomena,andimpressiveprogresshasbeenmadeduringthelastdecade.Thisdevelopmentcoincideswith the European levelpolicydevelopments andwould indicate thatHEIshavemobilised torespondtotheQAframeworkputinplaceatthepolicylevel.In2003,theMinistersinhighereducationagreedthateachcountryshouldsetupanationalqualityassuranceframeworkandthatHEIsshouldhavetheirowninstitutionalQAsystemsandtwoyearslatertheESGswereadoptedprovidinggenericprinciplesforbothinternalandexternalQAprocesses.Consequently,since2005,HEIshavebeenincreasinglyworkingontheirQAsystems.

Hence, the Bologna Process and subsequent changes in the national frameworks have clearlypromotedagrowingawarenessandinsightintotheneedforQAinteachingandlearning.WhileMinisterialCommuniquéshavenotdefined indetailhowqualityassurance isunderstood inteachingand learning,the ESGs, adopted and published in 2005, have done so. In addition, various European level projects,organisations3aswellasevents4,havefurtherendorsedacommonperceptionoftheconcept.WhiletheESGsseemtohaveestablishedthis frameworkforteachingandlearning,wearetemptedtoconclude–basedontheresponsestoquestionsdealingwithotheractivitiesofaHEI–thatthedevelopmentofasharedunderstandingofqualityassuranceingeneralissomethingtobeworkedon.

ThefindingsdoindicatethatwhenitcomestotheactualprocessesandsupportstructuresinQA,agreatvarietyofcombinationsexists,whichendorsestheviewthattherearenoone-size-fits-allsolutionswhendecidingwhatan institutionalQAsystemshouldconsistof.AsvariedasstructuressupportingQAsystemsmaybe–suggestingthattheremaybeoverlapsandadditionalstructures,maybeeveninefficientdoublingsandcouplings–theyallareproofthatHEIsareworkingtowardstheirQAsystems.

Trends 2010notedthatrecentdevelopmentsininternalqualityprocessesinteachingandlearninghave not necessarily been linked to European QA developments, particularly to the ESGs. HEIs seemedto be mostly responding to the external QA requirements imposed on them, and these requirementsdidnot includethepartoftheESGsthatappliestoHEIs,norwasthe linkbetweenthemandEuropeanQA developments explicit. (EUA 2010: 63) The EQC survey responses indicate that a clear majority ofrespondents started implementing a quality assurance framework within their institution as from 2000.WhereasthisprobablyhashappenedasaconsequenceofthediscussionsanddecisionsatEuropeanlevelandnewlegislationstobeadoptedatnationallevels,itappears,indeed,thatmostinstitutionsdonotapplytheESGsasanintegratedwhole,buttendtoshowinterestinoneorseveralaspectsofthem.

4. Key trends and further reflection

3OrganisationssuchastheEuropeanAssociationforQualityAssuranceinHigherEducation(ENQA)atagencylevel,theEuropeanUniversityAssociation(EUA)andtheEuropeanAssociationofInstitutionsinHigherEducation(EURASHE)atinstitutionallevel,aswellasvariousdiscipline-basedassociationsatEuropeanlevel.

4SuchastheEuropeanQualityAssuranceForum,ayearlyeventgatheringtogethervariousstakeholdersinvolvedinqualityassuranceinhighereducation.

“Quality was introduced a long time ago even if it was not felt and stated as quality. The development and formalisation of quality processes is done step by step. It is diffi cult to measure the reality of the quality in a big institution.” - Respondent to the survey


36

Yet developing quality culture takes time and effort

AsreferredtoinChapter2,qualitycultureiscloselyrelatedtotheorganisationalcultureofrespectiveHEIs and, as such, always exists in the form of values, beliefs etc. Although based on the quantitativeresults related toqualityassuranceprocesses,no far-reachingconclusionscanbemadeon thestatusofqualitycultureintherespondingHEIs;numerousopenfieldcommentsindicatethattheultimategoalinmanoeuvringQAprocessesispursuinganimprovedqualityculture.

Hence, the responses demonstrate that institutions are building quality-related processes andstructures,butthatinsomecases,sometimesbecauseofthelegalframeworkencasingthem,institutionsmaynothaveyetachievedthekindofqualitycultureforwhichtheyarestriving.Evidently,thegoalofHEIsisagenuinequalityculturesupportedbywell-functioningQAprocesses.Nevertheless,thestateofplayatthemomentlooksasifHEIshavefoundawayofimplementingQAprocesses,butthatprocessesmaynothavenecessarilyquestioned,challengedorchangedHEIs’wayofcarryingouttheiractivities.Theeffectivenessandefficiencyoftheexistingqualityassuranceprocessesneedfurtherqualitativeexaminationthatasurveydoesnotallowtocapture.

Ensuring the participation of all stakeholders

Inthelightofourresults,itappearsthattheinstitutionalleadershipisoftenatleastformallyinvolvedintheQAprocessesandalsointhedevelopmentofqualitycultureinthemajorityoftherespondingHEIs.Nevertheless,onecouldarguethatthepercentageofrectorsorvice-rectorsinchargeofQAissuesshouldbehigherinordertodemonstrateatrueinstitutionalcommitmenttotheenhancementofquality.RelyingsolelyonorganisationalstructurestosustaincommitmentcancauseQAprocessestolosetouchwiththereality of academic work and thus, reduce the processes to bureaucratic exercises that take place in avacuumwithnofollow-upactivities.

Butatop-downapproachaloneisnotsufficient;thenotionofsharedvalues,beliefsandcommitmentsimpliestheparticipationofthewholecommunity.Furthermore,consideringthatoneoftheunderpinningprinciplesoftheEuropeanQAframeworkistheparticipationofstakeholders–bothinternal,suchasstudentsandstaff,andexternal–inthequalityassuranceprocesses,thissurveyclearlyshowsthatmoreworkstillremainstobedoneinthisrespect.

Although,theresponsesindicatethatacademicstaff–supportstafftoalesserextent–andstudentsarequiteoftenformallyinvolvedinQAprocesses(seeChapter3.2),theresults,whichatthisstagearearesultofaquantitativeanalysis,donotindicatehowactivetheinvolvementisandhowinfluentialitis.Inthiscontext,withregardtostudentparticipationweshalltherefore,refertotheconclusionsoftheEuropeanStudents’Union:

Overall student participation in QA has progressed since 2007. However, the analysis of the answers shows the serious gaps in terms of formal participation in decision-making processes and a rather un equal rate of participation in the different processes associated with QA across different countries. We can conclude that, in spite of students being accepted as a part of the follow up rather than technical processes, they still face reluctance towards their involvement in the decision-making process. This statement is valid on a case-by-case basis, and it is not currently possible to establish a general trend on the location or scale of the phenomenon.(ESU2009:49)

“Implementation takes a lot of time, especially when, traditionally, faculties are free to develop teaching and research. To implement a quality culture a number of people engaged in developing concepts and discussing the benefi ts and gains of QA are needed.” - Respondent to the survey

37


Furthermore,withregardtostudentparticipationinparticular,wehavetoconsiderthatstudentsgivetheirfeedbackon,forexample,coursesandmoduleswiththenextgenerationofstudentsinmind:theythemselveswillprobablynotprofitfromtheirresponsesandsuggestions.Studentidealismisthus,oftenthebasisforstrategicfacultyandcurriculumdevelopment.Therefore,shouldthatbekeptup,afollow-uphastotakeplaceinduetime,andacommunicationstrategywithregardtothefollow-upactivitiesshouldbedeveloped.

Thus, thekindand intensityofparticipationcannotbe linkedproportionally toa timelinebut tothe recognition of the fact that QA has to integrate staff, students and different levels of leadership informulatingstrategicgoalstobeachievedbybuildingupaqualityculturewithintheHEI.

We know more about offer and input than outcomes

InthelightofthesurveyresultsdealingwithinformationHEIscollectontheiractivities,itisclearthatHEIsareinpossessionofalotofinformation,muchofwhichisstoredin-oftenwelldeveloped–informationsystems.Somequestions remain. Is the information the rightkindand is it accessible?Moreover, is theinformationusedtopromotecontinuousimprovement?

AsregardsthenatureoftheinformationtherespondingHEIscollectandstore,onecanconcludethatitseemstobeparticularlyfocussedonthenatureoftheiractivities,whatkindofprogrammesareavailable,andwhatkindof research iscarriedout.Less information isavailableontheresultsand impactof theiractivities.WhileHEIsknowhowmanypeoplegraduate,notsomanyofthemtrackgraduateemployment.Ingeneral,neitherstudentsatisfactionnortheresultsofstudentsurveysaregatheredtogetheronasystematicbasis,oftenbecausethisiscarriedoutatfacultyorprogrammelevel.AsTrends 2010noted,QAprocessesandtheireffectivenessarestilltoooftendependentontheinterestofindividualteachers(EUA2010:85).And, after all, in order to know how to improve, one should know how successful or well-functioningexistingactivitiesare.

Thesecondquestiondealswiththeaccessibilityofexistinginformation.Fortheinformationtobeusedinthedecision-makingprocesses–whetheratinstitutional,faculty,departmentalorindividuallevel–itshouldbeaccessibletothosewhoneedit.Forexample,prospectivestudentswouldbeinterestedingraduateemploymentinaspecificprogrammeaswellascurrentstudentsatisfactionrates;students,intheimpactoftheirfeedbackgiventhroughquestionnaires;headsofdepartmentordeans,intheperformanceofteachersetc.Alsotheexternalstakeholders,andinparticularthegovernmentswhoareusuallythemainfundersofHEIsinmostcountries,areinterestedinthekindofinformationthatwillallowthemtojudgewhetheraHEIisfulfillingitspublicmission.

Boththenatureofinformationavailableandhowaccessibleitis,arecloselyrelatedtothecurrenton-goingdiscussionontheneedforincreasedtransparency.ShouldHEIsimprovethequalityandaccessibilityoftheinformationtheyprovideontheirperformance,theymightensuretheavailabilityofsatisfyingsourcesofinformationontheirperformance,whichcouldmaketherankingofHEIslesssignificantforstakeholders.

Good at collecting information, but not always using it?

Asalreadysuggested,simplycollectinginformationisnotsufficient:whathappensasaresultoftheinformationandhowtheinformationisusedismorecrucial.Willtheinformationmakeadifference?Willitbeanalysedandusedwhendecisionsaboutthefuturearemade?Iftheinformationdemonstratesthatthereistheneedtomakechangesorimprovements,doesitleadtoconcretemeasures?Andifthereisevidenceofgoodperformance,willitberewardedorfurtherdisseminatedasexemplary?

Theimportanceoffollow-upactivitiestoQAprocesseshasbeendocumentedonseveraloccasions.Iftherearenosignsofimpact,itwillbeextremelydifficulttobuild,andevenmoreso,tosustainqualityculture.AstheQualityCultureprojectputit:“[I]tisimportanttopointoutthatiftheacademiccommunity,


38

including the students, do not see positive results from internal quality processes, discouragement andcynicismwillsetinandleadtoanerosionofthequalityculturethatwillbedifficulttoputrightagain.”(EUA2006:18)

The resultsofour survey–as reported inChapter3.3– lead to theconclusion that the internalfeedbackloopswithinHEIscouldbeimproved:forexample,inteachingandlearning,thesituationnowseemstobethat,althoughstudentsjudgeteachers’performances,theydonotoftenseetheresultsofthisjudgement,nordotheyseehowtheseresultsareusedinteachers’assessments.Oftenstudentscanonlyindirectlydeducethattheirfeedbackhasledtoadditionaltrainingfortheteachersortofartherreachingconsequences.Thislackoftransparencycanthen,inturn,contributetoaviciouscircleofstudentsnotfillingoutnewevaluationquestionnairesandthusleavingtheHEIwithoutvaluablefeedbackinformation.

AsnotedinChapter3.3,justoverhalftheHEIstakingpartinoursurveyhavedefinedkeyperformanceindicators(KPIs).WhileHEIsshouldbeencouragedtodevelopKPIstomonitortheirprogressinstrategickeyareas,wewouldliketodrawattentiontothefactthatagreatdealofconsiderationistobepaidtotheprocessofdefiningthesesothatthattheyalsoreflectqualitativecharacteristics,andnotjustquantitative.Furthermore,experiencehasshownthatwhileHEIshavealotofinformationavailable,itmaynotalwaysbeusefuland is sometimesunstructured.Greatcare shouldbe takenwhenconsidering thisquestionatinstitutionallevel.

Inconclusion,somedoubtremainsregardingtheefficiencyoftheinformationgatheredbyHEIsandwhetheritisusedtofostercontinuousimprovementandstrategicplanninginHEIs.Inthelong-run,HEIswouldfindadvantagesin:1)clearlydefiningtheirstrategicgoals;2)definingthewaystoachievethem;3)analysingcarefullywhatkindof information they–or theirkey stakeholders–need tomonitor theirperformance;4)limitingthecollectionofdatatoinformationthatcantrulybeutilised;5)payingparticularattentiontothetransparencyofthisdataaswellasto6)theinvolvementofinternalstakeholdersinthefollow-upprocedurestosustaintheircommitmentandmotivation.

4.2. Areas for further development

Taking account of EUA’s Quality Culture project’s recommendations, the survey results and theguidelinesframedbytheESGsforinternalqualityassuranceprocesses,andwhilerecognisingthatHEIsareeachinaverydifferentphaseofdevelopment,wewouldliketohighlightthefollowingkeyareaswhichrequirefurtherdevelopmentintermsofinternalQAprocesses:

•HEIsshouldbeencouragedtoadoptorreinforceanall-encompassingapproachtoqualityassurance,derivedfromtheirinstitutionalstrategiesratherthanresortingtoacultureofcomplianceaimingtosatisfyexternalrequirements.

•Inthiscontext,weurgeHEIstodevelopexplicitfeedbackloopsbetweenstrategicmanagementandqualityassuranceprocesses,includingwaystomonitor,inparticular,theresultsandqualityofactivitieswiththeaimoffeedingtheinformationintothestrategicplanningprocess.

•Althougha lothasbeendone inthis respect,promotingtheparticipationofall stakeholders inquality assurance processes in order to enhance the commitment to quality continues to be achallenge.Participationofstudents,inparticular,needstobefurtherstrengthenedinthefollow-

“The real challenge is to build Quality into every stage of our work so that the impact on working lives is spread more evenly than can sometimes be the case at present. There is always the need to refl ect on the common complaint from academic staff that we are killing the geese that lay the golden eggs, and whether new QA processes duplicate existing processes and add to workloads rather than adding value to the student experience or detracting from it by using up scarce staff time.”


39


upactivitiesofQAprocessesandinmeasuringstudentworkload.Moregenerally,furtherattentionneedstobepaidtoinformingpeopleabouttheimpactandfollow-upoftheQAprocesses.

•Andlastbutnotleast,wewouldencourageHEIstoreviewtheinformationontheirstrategicgoalsandonhowthesegoalsaremet,withtheaimofdecidingonanefficientandtransparentcommunicationstrategy,inparticular,vis-à-vistheexternalstakeholders.

ThelistisfarfrombeingexhaustiveandneitherisitapplicableassuchtoanysingleHEI.ButthesurveyresultsindicatethatthesearetheareaswheremanyHEIsarestillstrugglingintheirworktodevelopaqualityassurancesystemthatwilladvancethekindofqualityculturetheyareaimingtowards.

Toconclude,oneshouldunderlinethedangerofthinkingthatinternalqualityassurancedevelopmentsexistinavacuumwheretheireffectivenessorimpactareindependentofotherdevelopmentsinthehighereducation systems or institutional contexts. Quite the contrary, the ultimate quality of a HEI’s work andoutcomesaswellasthenatureofinstitutionalqualitycultureareverymuchsubjecttoavarietyoffactorswhicharehighlyinterdependentandcomplex,qualityassuranceprocessesbeingonlyonepieceofthepuzzle.

Themainchallengesidentifiedbytherespondentsinthisregardwere:frequentlychangingexternalregulatory frameworks; financial constraints that HEIs are facing, the recent data demonstrating that theeconomicdownturninEuropeisaffectingEuropeanuniversitiesinamoredramaticwaythanforeseenevenjustafewmonthsago(EUA2010b);andfinally,thereluctanceofsomestaffmembers,fearing,amongotherthings,additionalbureaucracyorlosingtheiracademicfreedom.


40

AstheTrends 2010reportindicatedandthissurveyhasconfirmed,qualityassuranceprocessesaretruly on the agenda of European HEIs and show no signs of fading away as the expectations towardsincreased quality, mobility and transparency in the European Higher Education Area only seem to bestrengthening.ThroughtheresultsofthissurveywehavehopefullybeenabletodemonstratethatHEIsaredevelopingtheirqualityassuranceprocessesinaseriousmanner,whiletheresultsalsoclearlydemonstratethatthisisworkinprogress.

WhilstthecaseofeachHEIisdifferent–someareverywelldevelopedincertainareaswhileothersareonlyjustatthebeginningofthejourney–andHEIsworkwithinverydifferentnationalcontexts,anumberofchallenges,difficultiesorpitfallscouldbeidentifiedascommontoalmostall.Byunderliningthesewehopetobeabletohelpthosewhoareworkingwiththesematterstounderstandtheirownrespectivesituationsbetterandtolookbeyondtofindsolutions.

Asoutlinedinthebeginning,themainfocusofthispublicationwasthequalityassuranceprocessesinteachingandlearningasframedbytheBolognaProcess.Wearewellaware–asaresomeoftheHEIsthatrespondedtothequestionnaire–thatthequalityassuranceprocessesinotheractivitiesofHEIshavenotreceivedthesameamountofattentionatthisstage.This,however,doesnotunderminetheirimportanceandthe linksbetweentheseactivitiesshouldalwaysbekept inmindwhendevelopingandfosteringaninstitutionalqualityculture.

The ESGs – which have worked as one of the reference points for our survey along with EUA’sQualityCultureproject–weredevelopedasasetofgenericprinciplesinQA:theydefinetheareaswhichshould be covered by institutional QA arrangements in teaching and learning, but do not define howtheseactivitiesshouldbeimplemented.Astheresultsofoursurveydemonstrate,thewaysandlevelsofimplementationvary.HEIshaveoptedforvariouscombinationsofprocessestosuittheirowninstitutionalcultureandstructures.TakingaccountofthediversityofEuropeanhighereducationingeneral,thediversityofQAprocessesshouldbeseen,infact,asaverylogicalandwelcomefeature.ButourdataalsoindicatesthatthereisstillworktobedonebeforewecansaythatpartoneoftheESGsisimplementedinatrulycomprehensivemanner.

Recently,therehasbeendiscussionaboutthepossiblerevisionoftheESGs(seee.g.EC2009)andweareconfidentthattheresultsofthissurveycancontributetothediscussion.Inthelightoftheseresults,wewoulddefinitelyarguethattheESGseffectivelyincorporateandreflecttheEuropeandimensionofqualityassurance inhighereducation.HEIshaveworked,orareworkingtowardshavingasetofcommon,keyprocessesinplace,butatthesametimethedifferencesbetweenthemnotonlydependonthenationalcontextorthesizeofaHEI,butalsoontheirorganisationalstructuresandcultures,allofwhichshouldberespected.

Inaddition,westronglybelievethatpromotinggoodpracticesandsharingexperiencesisimportantforthefuturedevelopmentofqualityassurance.AsarecentEUAprojectrecommended:

We encourage the creation of platforms for both horizontal and vertical dialogue at various levels: within institution between departments, within a country between institutions, at European level between both HEIs and QA agencies, etc. While encouraging this dialogue, it should not be forgotten that when learning from others’ experiences, whether good or bad, one should never aim at merely copying successful practices, but at critically analysing which components of the practice might be applicable to one’s own context.(EUA2009:18)

Andfinally,theworkofourproject“ExaminingQualityCulture”willcontinue.Havingnowmappedthestateofaffairsasfarasqualityassuranceprocessesareconcerned,itistimetomovetowardsthemorequalitativepartofourstudy. Inthenextstagewewillaimtodeepenourunderstandingofthesynergy,similaritiesaswellasdivergenceofqualityassuranceprocessesandqualityculture.Inthatcontextwewillalsoshowcasesomeexamplesofinstitutionalpractices.

5. Concluding remarks

41


Annex: Questionnaire A. General information

1.Nameoftheinstitutioninoriginallanguage:

2.NameoftheinstitutioninEnglish:

3.Countrywheretheinstitutionislocated.Pleasechooseonefromthedrop-downmenu.

4.Representativesoftheinstitution Nameoftherepresentativeoftheinstitution: Name of the quality assurance (QA) manager or equivalent who can be contacted for further

information:

5.Whatisthetypeofyourinstitutionaccordingtothenationalstatutes?Please choose one. University UniversityofAppliedSciences,Polytechnic,Fachhochschuleorequivalent Otherhighereducationinstitution(pleasespecify)

6.Whichisthehighestlevel(orequivalent)towhichyourinstitutioneducatesstudents?Please choose one. Bachelor Master Doctorate(or3rdcycleequivalent)

7.Howmanystudentsdoyouhaveintotal?(full-timeequivalent) Upto1.000 Between1.000and5.000 Between5.000and10.000 Between10.000and30.000 Morethan30.000(pleasegiveanapproximatefigure):

8.Howmanystaffdoyouhaveintotal?(full-timeequivalent,allcategoriesincluded) Upto100 Between100and300 Between300and500 Between500and1.000 Morethan1.000(pleasegiveanapproximatefigure):

9.Wouldyoube interested inparticipating in interviews regardingyour institution’squalitycultureandqualityassurancearrangements?

Yes No

Ifyes,pleaseenterheretheemailoftheQAmanager(orequivalent)whocanbecontacted:


42

B. Institutional QA Framework

10.Doyouhaveaninstitutionalstrategicplanorequivalentdocument?Please choose one. Yes,wehaveastrategicplan(orequivalent)whichincludesaninstitutionalmission,goalsandpriorities Yes,wehavestrategicplans(orequivalent)atthelevelofthefaculties No Other(pleasespecify):

11.Doyouhaveaninstitutionalqualityassurance(QA)policystatement?Please choose one. Yes,wehaveaninstitutionalQApolicystatement Yes,wehaveaninstitutionalQApolicystatement,andinadditionotherdocument(s)alsoaddressthe

QApolicy We do not have a separate QA policy statement, but it is included in another document (e.g.,

institutionalmissionstatement,strategicplan,workplanorequivalent) No,butallormostofthefaculties/departmentshavetheirownQApolicystatements No,wedonothaveaspecificQApolicystatementanditisnotaddressedinotherdocuments Other(pleasespecify):

12.Whendidyourinstitutionstartintroducingaqualityassurancesystem(orequivalent)?Please choose one. Before1990 Inthe1990s Between2000and2005 Between2005and2009 Wearecurrentlydesigningand/orplanningit

13.Howwouldyoudefine the roleof senior leadership (rector,vice-rector) inbuildingaqualityculturewithinyourinstitution?Please choose all applicable options.

Theseniorleadershiptakestheleadintheprocess Theseniorleadershipmonitorstheprocess Theseniorleadershipservesasafacilitatorforabettercommunicationamongdifferentlevelsofthe

institutions Theseniorleadershipisthedecisionmaker Other(pleasespecify):

14.Howdidyouintroduceaqualityassurancesystem(orequivalent)?Please choose all applicable options. Theinstitutionalleadershipdecidedontheconcept,providedinstructions,trainingandsupporttothe

unitstoimplementit Theconceptisaresultofvariousconsultationroundsamongtheacademicstaffoftheinstitution Theconceptisaresultofvariousconsultationroundsamongtheacademicandadministrativestaff Theconceptisaresultofvariousconsultationroundsamongtheacademicandadministrativestaffas

wellasstudents Theconceptwasintroducedthroughpilotprojectsconductedbysomeunits.Goodpracticeswere

disseminatedbasedontheseexperiences TheconceptisbasedonrequirementofthenationalQAagencywhichdevelopedthestandardsand

guidelinesforthis Other(pleasespecify):

15.Whatkindofstructuredoyouhaveinplacetosupporttheinternalqualityassuranceprocesses?Please choose all applicable options.

Therectororspeciallyassignedvice-rectorisinchargeofQAissues ThereisapersoninchargeofQAwithintherectorate ThereisacentralisedQAunit,withspecialisedstaff ThereareQAunitsineachfacultywithspecialisedstaff There are contact persons or persons in charge of QA within their unit, who have also other

responsibilities Thereisaunitresponsibleforstaffdevelopment

43


Thereisaunitresponsibleforpedagogicalinnovation(orequivalent)thatofferssupporttotheteachersindevelopingteachingmethods

Thereisaninstitutionallevelqualitycommitteeorequivalent ThereareFacultyleveland/orDepartmentand/orprogrammelevelqualitycommitteesorequivalent Other(pleasespecify):

16.Doyouhaveaninternalevaluationprocessthatprovidesfeedbacktothestrategicplanninginplace?Please choose all applicable options.

Theinstitutionalleadershipevaluatesannuallytheprogressmadeintermsofachievingthegoalssetbytheinstitution

Thefaculties(and/orrelevantunits)conductregularself-evaluationstoanalysetheircontributiontotheachievementofinstitutionalstrategicgoals

Theinstitutionconductsregularsurveysamongthemembersoftheinstitutionalcommunity(staffandstudents)toanalysetheirperceptionoftheinstitutionalstrategyanditsimplementationatgrass-rootslevel

Theinstitutionhasdefinedasetofkeyperformanceindicatorsandfollowsitsprogressbasedonthem Theinstitutionalstrategyandtheachievementofthegoalssetinitarerevisitedwhenthedocument

isrevised(every3,5orNyears) Other(pleasespecify):

17.Whichactivitiesdoyourinstitutionalqualityassuranceprocessescover? Please choose all applicable options. Teachingandlearning Research Servicestosociety Studentsupportservices Governanceandadministrativeservicesoftheinstitution Other(pleasespecify):

18.Whichofthefollowingprocessesdoesyourinstitutionhaveinplaceinordertoensurethequalityofresearchactivities?Please choose all applicable options.

Internalseminarswhereresearchprojectsandideasarediscussed Internalpeerreviewofresearchprojects External peer review of research projects organised by the institution (inviting external peers and

preparingareport) Externalpeerreviewofresearchprojectsinrelationtograntapplications(evaluationorganisedbyan

externalbodysuchastheEuropeanCommission,fundingcouncils,etc.) Pre-checkingofscientificarticlestobesenttothescientificjournals Preparingstatisticsonpublishedarticles Monitoringtheimpactfactorsofpublishedarticles Keyperformanceindicatorsdefinedforeachresearchgroup,departmentorfaculty Other(pleasespecify):

19.Whichofthefollowingprocessesdoesyourinstitutionhaveinplaceinordertoensurethequalityofitsservicestosociety?Please choose all applicable options.

Keyperformanceindicatorsdefinedforeachoftheservices Monitoringthenumberofpatents,technologiestransferagreements,etc. Monitoringthenumberofco-operationagreements Monitoringtheinteractionswithexternalstakeholders Questionnairestokeystakeholders Forums(stakeholdergroupsorequivalent)toensurethattheinstitutionreceivesfeedbackfromsociety

andrespondstothat Processdescriptionsofactivities(guidelinesorotherdescriptiveformats) Pre-selectionprocesses inplace for initiatives taken in thisfield (for instance rector’sorpresident’s

approval) Alumnifeedbackthroughsurveysorotheractivities Other(pleasespecify):


44

C. Quality assurance processes in teaching and learning

20.HowisyourQAarchitectureinteachingandlearningdesigned?Please choose one. Itistailor-madetotheinstitution’sneedsanddoesnotapplyanyready-mademodel Itisinstitution-specificbutfollowsnationalQAframeworksandguidelines Itappliesaready-mademodelsuchasISO,EFQM,CAF...(pleasespecifywhichoftheabovementioned

models,ormentionanyothermodelused)

21.Whichof thesecategoriesofpeople(seehorizontal row)doyour formalqualityassuranceprocessesinvolveandhow?Please choose all applicable options for each category of people.

22.Howaretheresultsofthestudentsurveysfollowedup?Please choose all applicable options. Theyaretakenintoconsiderationinthedesignandrevisionofstudyprogrammes(includingteaching

methods) Theyaretakenintoconsiderationintheassessmentofteachingstaff Theyarearchivedinordertoinformfutureassessmentsoftheprogramme/institution Theyarediscussedinmeetingsattendedbystaffmembersandstudentsorganisedspecificallyforthis

purpose Studentswhohaveparticipatedinasurveyareinformedabouttheresultsandactionstakenonthe

basisoftheresults Notapplicable(wedonotconductstudentsurveys) Other(pleasespecify):

a. Approval, monitoring and periodic review of programmes and awards

23.Hasyourinstitutiondevelopedexplicitlearningoutcomes?Please choose one. Yes,forallprogrammes Yes,forsomeoftheprogrammes No

Academicstaff

Adminis-trativestaff

Leadership,institutional

level

Leadership,faculty/

departmentlevel

Students

Externalstakeholders

(e.g.,employers,experts...)

Alumni

Throughformalparticipationingovernancebodies(wheremembersareentitledtovote)

Throughformalparticipationinconsultationbodies

Throughformalinvolvementinself-evaluationsorotherevaluationactivities

Byinformallyprovidinginformationontheissuesatstake

Byrespondingtothesurveysonaregularbasis(e.g.attheendofeachcourse,academicyear…)

Theyarenotinvolved

45


24.Aretheselearningoutcomespubliclyavailable?Please choose one. Yes,theyarepubliclyavailableontheweb-site,studyguidesorequivalent Theyareavailableuponrequest Theyareavailableforthestudentsinvolvedineachspecificcourse Other(pleasespecify):

25.Howdoesyourinstitutionknowthestudentworkloadneededinordertoreachthedescribedlearningoutcomes?Please choose one.

Allstudentsareaskedinsurveysabouttheworkloadtheyhavefortheircourses Asampleofstudentsisaskedinsurveysabouttheworkloadtheyhavefortheircourses Theteacherresponsibleforthemoduleestimatestheworkload Thereisnostudentworkloadindicationinthecoursedescription Other(pleasespecify):

26.Howdoestheprocessfordesigningcurriculumandprogrammesworkwithinyourinstitution?Pleasechoose one. If there are several kinds of processes in place in your institutions, please choose the most commonly used.

Programmedirectoror equivalentpersonprepares the curriculumafterwhich staffmembersmaycommentthedraft

Working group, committee or equivalent prepares the curriculum (possibly based on proposalspreparedbyothers)

Each staff member proposes what they find essential for the programme and the curriculum is acombinationoftheseproposals

Thecurriculumisdesignedbytheministryorotherexternalbodies Other(pleasespecify):

27.A working group, committee or equivalent prepares the curriculum and programmes within yourinstitution.Suchagroupconsistsof:Please choose all applicable options.

Students Administrativestaffmembers Academicstaffmembers Externalstakeholders(employers,corporatepartners...) Alumni Other(pleasespecify):

28.Whatkindofprocessesdoyouhaveinplaceformonitoringcurriculumandprogrammedesign?Please choose all applicable options.

Thecurriculumandprogrammecontents,pedagogicalapproachesandintendedlearningoutcomesareevaluatedonaregularbasis(everyNyears/semesters...)

Thecurriculumandprogrammecontents,pedagogicalapproachesandintendedlearningoutcomesareevaluatedaspartofanexternalaccreditationprocessorequivalent

Curriculumandprogrammedesignprocessesassuch–thatis,theeffectivenessandcomprehensivenessoftheprocesses–areevaluatedonaregularbasis(everyNyears/semesters...)

Thecurriculumandprogrammecontentsareevaluatedoccasionally(attheoccasionofaself-evaluationexercise,foranexternalevaluationbody...)

Thecurriculumandprogrammecontentsareevaluatedcontinuouslyonaninformallevel(discussionsbetweenstaffmembers,staffandstudents...)

Other(pleasespecify):

29.Aretheprogrammecontentsorcurriculumultimatelyapproved:Pleasechooseone. Attheleveloftheinstitution Atthefacultylevel Atthedepartmentallevel Byanexternalbody(agencyorother) Byagovernmentalbody Other(pleasespecify):


46

30.Do quality assurance processes within your institution include doctoral studies? Please choose all applicable options.

Yes,attheleveloftheinstitutionasawhole Yes,forthedoctoral/graduate/researchschool(s) Yes,fortheindividualdoctoralprogrammes Yes,aspartofqualityassuranceforteaching Yes,aspartofresearchassessment No Notapplicable(wedonotofferdoctoralstudies) Other(pleasespecify):

b. Student assessment

31.Which of the following characteristics do your student assessment procedures (i.e., examinations)currentlyhave?Please choose all applicable options.

designed to measure the achievement of the intended learning outcomes and other programmeobjectives

haveclearandpubliclyavailablecriteriaformarking/givinggrades haveclear,pre-definedexaminationsorotherassessmentmethodsinplace haveclearregulationscoveringstudentabsence,illnessandothermitigatingcircumstances ensurethatassessmentsareconductedsecurelyinaccordancewiththeinstitution’sstatedprocedures theadministrationchecksthattheassessmentproceduresarefollowed Other(pleasespecify):

32.Arestudentsinformedoftheassessmentprocedures?Please choose all applicable options. Theassessmentmethodsandcriteriaappliedarepubliclyavailableforexampleviastudyguides,web-

site Theteacherinformsthestudentsabouttheassessmentmethodsandcriteriaappliedatthebeginning

ofthecourse Other(pleasespecify):

c. Quality assurance of teaching staff

33.How does your institution ensure that teaching staff is qualified and competent? Please choose all applicable options.

Thereareformalnationalrequirementsforthecompetenceofteachingstaffwhenhiringthem Theinstitutionhasspecifieditsownrequirementsforcompetenciesofpermanentteachingstaffwhen

hiringthem Allteachersareexpectedtohavecertainresearchqualifications Thereareresearchperformanceevaluationsforallpermanentacademicstaffmembers Teachingqualificationsarepartofthequalificationsprofessorcandidatesareexpectedtodemonstrate Weconductstudentsurveys Compulsorypedagogicaltrainingisorganisedforteachers Optionalpedagogicaltrainingisorganisedforteachers Thereisanexternalaccreditationprocessoftheteachers(conducted,forinstance,byaQAagencyor

anationalbody). Theinstitutionhasinplaceapeerfeedbacksystem(teachersgivingfeedbacktoeachotheronteaching) Therearecertainprocesses inplacetoremoveateacherfromhis/herduties if theycontinuetobe

demonstrablyineffective Thelegalframeworkdoesnotforeseethepossibilityofremovinganineffectiveteacher Other(pleasespecify):

34.Isinformationonteachers’aptitudesandperformance(resultsofstudentsurveys,evaluationofhis/herteachingaptitudes...)publiclyavailable?Please choose one.

Yes,itispubliclyavailable

47


Yes,itisavailableforallthoseinvolvedinQAproceduresforteaching(includingstudents) Yes,itisavailablefortheteachingcommunityingeneral No,itiskeptconfidentialandavailableonlyattheleadershiplevel(institutionand/orfacultyand/or

department) Other(pleasespecify):

d. Learning resources and student support

35.Arethelearning resourceslistedbelowregularlyoffered,monitored,evaluatedand/orimproved:

36.Isthereaprocessinplaceformonitoringindividualstudents’progression(i.e.informationrelevanttotheprogressionofparticularstudentsduringtheirstudies)throughanentiredegreecycle?Please choose one.

Yes,andtheinternalQAprocedureregardingthisisstandardisedattheleveloftheinstitution Yes,andtheinternalQAprocedureregardingthisdependsonfaculty/department/institute No

e. Information systems

37.Does your institution have an overarching information system (i.e. database) used for the effectivemanagementofitsactivities?Please choose one.

Yes,theinstitutionhasacentralisedinformationsystemthatcoversallkeyactivities No,buttheinstitutionhasacentralised,non-integratedinformationsystem(dataondifferentactivities

arenotgatheredinonedatawarehouse) No,butseveralinformationsystemsexistatfacultylevel Notapplicable,thereisnoinformationsystem

38.Whichofthefollowingdoesthesystemorsystemsinclude?Please choose all applicable options. Studentprogressionandsuccessrates Teacher-studentratioperfaculty/department/instituteorintherespectivefaculty/department/institute Trackinggraduates’employment Students’satisfactionwiththeirprogrammes Profileofthestudentpopulation(age,gender,educationalbackground,socio-culturalbackground,

etc.) Availablelearningresourcesand,whenapplicable,theircosts Noneoftheabove Other(suchastheinstitution’sownperformanceindicators).Pleasespecify:

Offered Monitored Evaluated Improved

Library Computingfacilities(includingemailaccountandinternetaccess)

Humansupportintheformoftutors,counsellors,andotheradvisers(inadditiontoteachingstaff)

Laboratories

Learningfacilities(languagelabs,musicalinstruments,anyothermaterialusedforclasses…)


48

f. Public information

39.Theinformationthatispubliclyavailableonyourinstitution’sstudyprogrammesincludes:Please choose all applicable options.

Numberofstudentscurrentlyinvolvedintheprogramme Numberofacademicstaffinvolvedintheprogramme Teacher-studentratiointherespectivefaculty/department/institute Informationontheintendedlearningoutcomesoftheprogramme Informationofqualificationsgrantedbytheprogramme Informationontheteaching,learningandassessmentproceduresusedwithintheprogramme Information on the learning opportunities (e.g. traineeships, exchange programmes, mobility

possibilities,scholarships...)availabletothestudentsoftheprogramme Informationonalumniemployment Profileofthecurrentstudentpopulation Specificinformationtargetinginternationalstudents Accessibilityandpossibilitiesofferedtodisabledstudents Other(pleasespecify):

40.Doyouinformthepublicabouttheresultsofevaluationscarriedout?

D. Comments

41.Pleaseusethefieldbelowforanyfurthercommentonhowyouperceivetheimplementationofaqualitycultureandinternalqualityassuranceprocesseswithinyourinstitution(Openfield,300words).

42.Pleaseusethefieldbelowforanyfurthercommentonwhat,toyourmind,arethefuturechallengestobefacedregardingtheimplementationofaqualitycultureandinternalqualityassuranceprocesses(Openfield,300words).

43.Doyouthinkthatthisquestionnairewasabletohandlethemainquestionsrelatedtoyourinstitutionaldailypracticeinqualityassurance?Pleasecomment.(Openfield,100words)

Resultsofinternalevaluations Resultsofexternalevaluations

Yes,viaweb-sites,publicationsorotherinformationmaterial

Yes,whenasked No

49


ReferencesBolognaProcessMinisterialConference(BPMC),19.9.2003,Realising the European Higher Education Area, CommuniquéoftheConferenceofMinistersresponsibleforHigherEducation,http://www.bologna-berlin2003.de/pdf/Communique1.pdf(lastaccessed26.05.2010).

Ehlers,U.D.,2009,“UnderstandingQualityCulture”, inQuality Assurance in Education, vol.17,n°4,pp.343-363.

EuropeanAssociation forQualityAssurance inHigher Education (ENQA),2005, European Standards and Guidelines for Quality Assurance in the European Higher Education Area(Helsinki,ENQA),http://www.enqa.eu/pubs_esg.lasso(lastaccessed01.09.2010).

EuropeanAssociation forQualityAssurance inHigherEducation (ENQA),2008,Quality Procedures in the European Higher Education Area and Beyond – Second ENQA Survey (Helsinki,ENQA),http://www.enqa.eu/pubs_occasional.lasso(lastaccessed01.09.2010).

EuropeanCommission(EC),21.09.2009,Report on Progress in quality assurance in higher education,COM(2009) 487 final, http://ec.europa.eu/education/higher-education/doc/report09_en.pdf (last accessed01.09.2010).

EuropeanStudents’Union(ESU),2009,Bologna with Student Eyes 2009(Leuven,ESU),http://www.esib.org/index.php/Publications/official-publications(lastaccessed01.09.2010).

EuropeanStudents’Union(ESU),2010,Bologna at the Finish Line. An account of ten years of European higher education reform (Brussels, ESU), http://www.esib.org/index.php/Publications/official-publications (lastaccessed01.09.2010)

EuropeanUniversityAssociation(EUA),2005,Developing an internal quality culture in European universities: Report on the Quality Culture project 2002 – 2003 (Brussels, EUA), http://www.eua.be/publications (lastaccessed01.09.2010).

EuropeanUniversityAssociation(EUA),2006,Quality culture in European universities: a bottom-up approach. Report on the three rounds of the Quality Culture project 2002 – 2006(Brussels,EUA),http://www.eua.be/publications(lastaccessed01.09.2010).

European University Association (EUA), 2009, Improving Quality, Enhancing Creativity: Change Processes in European Higher Education Institutions (Brussels, EUA), http://www.eua.be/publications (last accessed01.09.2010).

EuropeanUniversityAssociation(EUA),2010a,Trends2010:A decade of change in European Higher Education,Sursock,A.andSmidt,H(Brussels,EUA),http://www.eua.be/fileadmin/user_upload/files/Publications/Trends_2010.pdf(lastaccessed01.09.2010).

EuropeanUniversityAssociation(EUA),2010b,Impact of the Economic Crisis on European Universities (May 2010),http://www.eua.be/fileadmin/user_upload/files/Newsletter_new/economic_crisis_19052010_FINAL.pdf(lastaccessed01.09.2010).

Harvey,L.&StensakerB.,2008,“QualityCulture:understandings,boundariesandlinkages”,inEuropean Journal of Education,vol.43,n°4,pp.427-442.

Harvey,L.,2009,Deconstructing quality culture,EAIRConferenceinVilnius2009,http://www.qualityresearchinternational.com/Harvey%20Vilnius.pdf(lastaccessed24.08.2010).

Lanarès,J.,March2008,“DevelopingaQualityCulture”,inEUA Bologna Handbook,C2.1-1(Brussels/Berlin,EUA/Raabe).


50

Rauhvargers, A., Deane, C. & Pauwels, W., 2009, Bologna Process Stocktaking Report (Brussels, FlemishMinistryofEducationandTraining),http://www.ond.vlaanderen.be/hogeronderwijs/bologna/conference/documents/Stocktaking_report_2009_FINAL.pdf(lastaccessed01.09.2010).

Westerheijden,D.F.,2010,The Bologna Process Independent Assessment. The first decade of working on the European Higher Education Area(Twente/Kassel,CHEPS/INCHER-Kassel/ECOTEC),www.utwente.nl/cheps/publications(lastaccessed01.09.2010).

ProgrammeonInstitutionalManagementinHigherEducation(IMHE),April2010,IMHE Info,http://www.oecd.org/dataoecd/46/44/44893105.pdf(lastaccessed25.08.2010).

E U A C A S E S T U D I E S 2 0 10

The European University Association (EUA) is the representative organisation

of universities and national rectors’ conferences in 46 European countries.

EUA plays a crucial role in the Bologna Process and in influencing EU policies

on higher education, research and innovation. Thanks to its interaction with

a range of other European and international organisations EUA ensures that

the independent voice of European universities is heard wherever decisions are

being taken that will impact on their activities.

The Associat ion provides a unique expert ise in higher educat ion and research

as wel l as a forum for exchange of ideas and good pract ice among universi t ies.

The results of EUA’s work are made avai lable to members and stakeholders

through conferences, seminars, website and publ icat ions.

European University Association asblAvenuedel’Yser241040Brussels,BelgiumTel:+32-22305544Fax:+32-22305751www.eua.be

examining quality culture: part 1 – quality assurance ... quality culture part i quality... ·...

Documents