examining quality culture: part 1 – quality assurance ... quality culture part i quality... ·...

E U A P U B L I C A T I O N S 2 0 1 0

Examining Qual i ty Cul ture:

Part 1 – Qual i ty Assurance Processes

in Higher Educat ion Inst i tut ions

By Tia Loukkola and Thérèse Zhang

Copyright©bytheEuropeanUniversityAssociation2010

All rights reserved. This informationmaybe freelyusedand copied fornon-commercialpurposes,provided that the source is

acknowledged(©EuropeanUniversityAssociation).

Additionalcopiesofthispublicationareavailablefor10Europercopy.

European University Association asbl

Avenuedel’Yser24

1040Brussels,Belgium

Tel:+32-22305544

Fax:+32-22305751

Afreeelectronicversionofthisreportisavailablethroughwww.eua.be.

Thisprojecthasbeen fundedwith the support fromtheEuropeanCommission.Thispublication reflects theviewsonlyof the

authors,andtheCommissioncannotbeheldresponsibleforanyusewhichmaybemadeoftheinformationcontainedtherein.

ISBN:9789078997214

By T i a Loukko l a and Thé rè se Zhang

Examining Qual i ty Cul ture:Part 1 – Qual i ty Assurance Processes in Higher Educat ion Inst i tut ions

E U A P U B L I C A T I O N S 2 0 1 0

E X A M I N I N G Q U A L I T Y C U L T U R E : P A R T 1 – Q U A L I T Y A S S U R A N C E P R O C E S S E S I N H I G H E R E D U C A T I O N I N S T I T U T I O N S

4

List of figures and tables

Figures

Figure 1: RatiooftypesofinstitutionsFigure 2: DistributionofrespondentspercountryFigure 3: ElementsofqualitycultureFigure 4: StructuressupportingtheinternalqualityassuranceprocessesFigure 5: Introductionofaqualityassurancesystem(orequivalent)Figure 6: Introductionofaqualityassurancesystem(orequivalent)–BreakdownpercountryFigure 7: Howtheinternalqualityassurancesystem(orequivalent)wasintroduced withintheinstitutionFigure 8: TheinvolvementofstakeholdersinformalqualityassuranceprocessesFigure 9: DesigningcurriculumandprogrammeswithintheinstitutionFigure 10: MonitoringcurriculumandprogrammesFigure 11: CharacteristicsofstudentassessmentproceduresFigure 12: Institutionsoffering,monitoring,evaluatingandimprovingtheirlearningresources

Tables

Table 1: ActivitiescoveredbyinstitutionalqualityassuranceprocessesTable 2: Informationincludedintheinformationsystem(s)regardingstudyprogrammesTable 3: InformationprovidedbytheinstitutiononitsstudyprogrammesTable 4: InternalevaluationprocessesprovidingfeedbacktothestrategicplanningTable 5: KeyfindingscorrespondingtoESGspart1

14151720212223

2429303132

1926272833

5


Foreword

Acknowledgements

Executive summary

1. Setting the stage

1.1.Background

1.2.TheExaminingQualityCultureproject:aimsandmethodology

2. Quality assurance as a component of quality culture

2.1.Theelementsofqualityculture

2.2.Qualityassuranceprocessesasunderstoodinthisproject

3. Mapping internal quality assurance processes: survey results

3.1.Qualityassurancestructures

3.2.Participationofstakeholders

3.3.Theuseofinformation

3.4.Qualityassuranceinteachingandlearning

3.5.ImplementationoftheESGs

4. Key trends and further reflection

4.1.Trendsandperceptions

4.2.Areasforfurtherdevelopment

5. Concluding remarks

Annex: Questionnaire

References

Table of contents

6

7

9

12

12

13

16

16

17

19

19

23

25

28

33

35

35

38

40

41

49


6

ThequalityofEuropeaneducation,andofhighereducationinparticular,hasbeenidentifiedasoneofthekeyfactorswhichwillallowEuropetosucceedinaglobalcompetition.In2003inBerlintheMinistersforhighereducationstatedthattheprimaryresponsibilityforqualityassurancelieswithhighereducationinstitutions.Further,variouspoliciesandactionlineshavebeendevelopedtoimprovequality;amongotherinitiatives,Europeandegreestructureshavebeenrevised,mobilityofstudentsandteachersisencouraged,andtransparencyandcomparabilityofqualificationsispromoted.

Inparallel,EUAhassupporteditsmembersinpromotinganinstitutionalqualityculturethatisfitforpurposeandthattakesaccountofthesignificantinstitutionaldiversitywhichexistsinEurope.Afteradecadeofworkinthisfield,andgiventheofficiallaunchoftheEuropeanHigherEducationAreain2010,wefeltthatthetimehadcometotakeamomenttoanalyseprogressmadeinthisrespect.

Theaimofthe“ExaminingQualityCultureinEuropeanHigherEducationInstitutions”projecthasbeen toask the institutionshow,and throughwhichactivities, theyare responding to thechallengeofassuringandenhancingthequalityof theirprovision.This report focusesontheactivitiesdevelopedbyuniversitiestoenhancetheirinternalquality,toimprovetheiraccountabilityandtherebyalsoimplementingtheEuropeanStandardsandGuidelines(ESGs)inpractice,andisbasedontheresultsofasurveycompletedby222institutionsacrossEurope.

Nevertheless,truehighqualityeducationcannotresultonlyfromformalqualityassuranceprocesses,but rather is a consequence of the emergence of a quality culture shared by all members of a highereducationcommunity.Hence,theworkofthisprojectwillcontinue.Theintentionistocomplementthisreport by another one which will address the complex relationship that exists between formal qualityprocessesandtheexistenceofanoverallinstitutional‘qualityculture’.Itwillalsopresentsomecasestudyexamplesofthisinteraction.

Wehopethatthispublicationwillbeofinteresttoallourmembersaswellastopolicymakers.Astheresultsdemonstrate,whileconsiderableprogresshasbeenmadethereremainsmuchtobedone.Inthatcontext,wehopethatthispublicationwillinvitereflectionsthatwillcontributetothiswork.

Foreword

Jean-Marc RappEUAPresident

7


Firstandforemost,onbehalfoftheprojectconsortium,wewouldliketoexpressourgratitudetoall thehighereducation institutionsthattookthetimetoanswerthequestionnaire.Thisreport isbasedon those repliesandwouldnothave seen the lightofdaywithout thecontributionofeveryonewithintheseinstitutions.Wearewellawarethatthetimespentrespondingtothesurveywastakenawayfromthedailytasksoftheinstitutionsandverymuchhopethattheresultsofthisprojectwillbenefitusallinthelongrun.Specialthanksgotothose14institutionsacrossEuropethattookpartinthetestingphaseofthequestionnaire,providinguswithinvaluableinputonthedesignofthequestionsanditstechnicalrealisation.

Secondly,wearegratefultothemembersoftheprojectSteeringCommittee(seelistbelow),chairedby Professor Henrik Toft Jensen, former President of Roskilde University in Denmark. Ever since the firstmeetingoftheSteeringCommitteewehavebeenimpressedbythesharedcommitmenttothisprojectandtopromotingaqualitycultureapproachwithinhighereducationinstitutions.Thiscommitmenthasbeendemonstratedbytheuntiringwillingnesstocommentondraftsofboththequestionnaireandthisreportaswellastocontributetotheanalysisoftheresults.

Furthermore, EUA would like to thank its consortium partners, the German Rectors’ Conference(HRK)andQAAScotlandforagreeingtoembarkonthischallengingadventureofmappingqualityassuranceprocessesandaddressingqualityculture.OurthanksalsogototheEuropeanCommission’sLifelongLearningProgrammeforco-fundingtheproject.

Finally, the authors wish to thank Barbara Michalk for acting as our sounding board during thewritingprocessandcontributingtothetext.Andlastbutbynomeansleast,thanksgotoJoanneByrne,whosehelphasbeeninvaluableinrespecttoallaspectsofthepracticalarrangementsrelatedtotheprojectandwhohasscrupulouslyprovidedfeedbackonvariousdraftversionsofthisreport.

TiaLoukkolaHeadofUnit

ThérèseZhangProjectOfficer

Steering CommitteeHenrik Toft Jensen,Chair,formerPresidentofRoskildeUniversity,DenmarkAndrea Blättler,representative,EuropeanStudents’UnionDavid Bottomley,AssistantHead,QAAScotland,UnitedKingdomKarl Dittrich,representative,EuropeanAssociationforQualityAssuranceinHigherEducation(ENQA)Tia Loukkola,HeadofUnit,EUABarbara Michalk,HeadofQualityManagementProject,GermanRectors’Conference(HRK),GermanyOliver Vettori,HeadofQualityManagementandProgrammeDelivery,ViennaUniversityofEconomicsandBusiness,Austria

Acknowledgements


8

9


1. Setting the stage1.HighqualityofprovisionhasbeenoneofthekeyaimsofthecurrentreformsinEuropeanhigher

education,andhasledtotheincreasingdemandforqualityassurance(QA).OfalltheworkcarriedoutinthisregardatEuropeanlevel,theEuropeanStandardsandGuidelinesforQualityAssuranceintheEuropeanHigherEducationArea(ESGs),adoptedin2005,areconsideredasacornerstone,reinforcingtheimportanceofinstitutionalautonomyandresponsibilityinQA.

2.When working on QA processes, higher education institutions (HEIs) are ideally expected todevelopinternalqualitycultureswhichtakeintoaccounttheirinstitutionalrealitiesandarerelatedtotheirorganisationalculture.

3.Theproject“ExaminingQualityCultureinHigherEducationInstitutions”(EQC)aimstoidentifyinstitutionalprocessesandstructuresthatsupportthedevelopmentofaninternalqualityculture.ThefirstphaseoftheprojectfocusedonmappingtheexistingQAprocessesthroughasurvey,whilethesecondphasewillprovideaqualitativeapproachandembracetheculturalandmoreinformalelementsofqualityculture.

4.Thereport,whichresultsfromthefirstphase,seekstoexaminehowhighereducationinstitutions(HEIs) respond in their activities to thedevelopments inQAatpolicy level. It is basedon thequantitativeresultsofasurveythatwasconductedduringspring2010.Atotalof222institutionsfrom36countriesacrossEuroperesponded.


5.Thestudybasesitsunderstandingof“qualityculture”onthedefinitionprovidedbyEUA’sQualityCultureproject (2006),which sees it as referring toanorganisational culture characterisedbyacultural/psychologicalelementontheonehand,andastructural/managerialelementontheotherhand.Itiscrucial,intheauthors’minds,todistinguishqualityculturefromqualityassuranceprocesses,whicharepartofthestructuralelement.

6.Thedefinitionofqualityassurancevariesfromonecountryandinstitutiontoanother.ThestudyusesQAinitsbroadestsense,includingallactivitiesrelatedtodefining,assuringandenhancingthequalityofanHEI,thusarguinginfavourofadoptinganall-encompassingapproachderivedfrominstitutions’ownstrategicgoals,fittingintotheirinternalqualityculture,whilealsofulfillingtheexternalrequirementsforQA.Theseactivitiesshouldinclude,butarenotlimitedto,activitiesmentionedbytheESGs.TheapproachadoptedbythesurveyisthereforenotlimitedtocheckingcompliancewiththeESGs,butalsoincludeselementsofinstitutionalstrategicmanagement.

Executive summary


10

3. Mapping internal quality assurance processes: survey resultsQuality assurance structures

7.Remarkable progress has been made in QA in recent years, and most of the responding HEIshavefundamentalpolicies,structuresandprocessesinplaceinthisregard–althoughinstitutionstend not to systematically identify or call all QA practices in place as such. A large variety oforganisationalstructuresexistwhenitcomestosupportingtheimplementationofQAprocesses.HEIswitha longerhistory inQAaremore likely tohavedevelopedsupport structures suchaspedagogicalinnovationandstaffdevelopment.

8.In terms of policy and associated procedures, most HEIs have a strategic document either atinstitutional(forthemajorityofcases)oratfacultylevel.

Participation of stakeholders

9.ThecrucialroleofinstitutionalleadershipindemonstratingcommitmenttoqualityhasbeentakenonboardbymostHEIs,whichhavetheirseniorleadershipinvolvedinonewayoranotherinQAprocesses.

10.WhereastheparticipationofstaffandstudentsisoneofthekeyprinciplesindevelopingbothaqualitycultureandQAprocesses,nearlyhalfoftherespondentsdonothaveacommitteeresponsibleforQA.CommitteesaremorelikelytobefoundinHEIshavingworkedforlongerinQA.HEIswithalongerhistoryinQAarealsomorelikelytogiveimportancetotheinfluenceofstudentsurveysaswelltheimportanceofafeedbackloopandinformingthestudentsaboutthefollow-upofQAactivitiestheyparticipatedin.Whiletheinvolvementofacademicstaffseemstobesystematicandcommoninallstages,fromcurriculumdesigntoinvolvementinformalQAprocesses,studentinvolvementisnotaswidespread.

11.InmostHEIs,externalstakeholders(employers,experts,alumni...)areinvolvedinQAprocessesinvariousways,butthelevelandthenatureoftheirparticipationvaries,fromsittingongovernancebodiestobeingconsultedassourcesofinformation–thislatterseemingtobethemorecommon.

The use of information

12.Practically all responding HEIs have an information system for monitoring their activities.Institutionstendtocollectinformationabouttheirprofileandwhattheyoffer,buttheinformationrelatedtoresourcesavailabletothestudents(suchaslibraryservices,computerfacilities...)ismorelimited.Moreover,theinformationcollectedisnotnecessarilytheonemadepublic.Usuallytheinformationmadepublicisthatonstudyprogrammes,althougheventhisdoesnotoftenincludeinformationongraduateemployment.

13.The link between collecting information and informing the staff or students involved in thisdatacollectionisnotobvious,assomeinformation(suchasteachers’performances)istypicallyconsideredasconfidentialoraccessibleonlyatleadershiplevel.Studentswhoprovidefeedbackthroughsurveysareinformedabouttheresultsandfollow-upactionsinabouthalfoftheHEIs,although a significantly higher percentage of institutions do actually conduct student surveys.Dataalsoshowthatinstitutionsthathaveprocessesinplacetoobligeateachertoimprovehis/herperformancegivemoreconsiderationtotheresultsofstudentsurveys.Theseinstitutions,again,arethosewithalongerhistoryinQA.

14.With regard to strategic management, in about two thirds of HEIs the institutional leadershipconductsanannualevaluation to reviewthegoals.However,onlya littleoverhalfof theHEIsreportedhavingformulatedkeyperformanceindicatorstomonitortheirprogress.

11


Quality assurance in teaching and learning

15.AbouttwothirdsofHEIshavedesignedtheirQAframeworkforteachingandlearningasinstitution-specific but following national frameworks and guidelines. Few HEIs chose to adopt externalmodels(CAF,ISO...)assuch.

16.Thecurriculumistypicallydesignedbyacommitteeoraworkinggroup.Afteraprogrammeisupandrunning,avarietyofprocessesformonitoringitexist.MostHEIsconductsomekindofinternalevaluationinadditiontoanexternalone,shouldtherebeone.

17.ThepercentageofHEIsthatreportedlyhavedevelopedlearningoutcomesishigherthan90%,buttheydonotallmakethempubliclyavailable.Lessthanhalfmeasurethestudentworkloadneededtoreachthedescribedlearningoutcomesthroughstudentsurveys.

18.However,where institutionshavedeveloped learningoutcomes, student assessment is directlyrelatedtothem.StudentassessmentcombinesavarietyofcharacteristicsacrossEurope.Assessmentmethodsandcriteriaareusuallymadetransparenttostudents.

19.HEIsofferlearningresources,buttheydonotallsystematicallymonitororevaluatethem.Studentsupportservicesaremore likelytobe inplaceandmonitoredin institutionshavingintroducedtheirQAsystembefore2000.

Implementation of the ESGs

20.ThelastpartofChapter3offersanoverviewontheimplementationoftheESGsinthelightofresultscollectedthroughthesurvey.

4. Key trends and further reflectionTrends and perceptions

21.The report argues that QA systems are largely in place, although their development in theircurrentformatisarecentphenomenon.Yet,developingaqualityculturetakestimeandeffort,asitiscloselyrelatedtovalues,beliefsandaculturalelementwhichcannotbechangedquickly.ParticipationofallstakeholdersintheimplementationofQAprocessesandstrivingforastrongerqualitycultureappearstobeessential,butstilldemandsattention.Moreover,HEIsseemtohavemoreinformationavailableontheinputandonwhatisoffered,thanontheoutput.Finally,HEIstendtobegoodatcollectinginformation,butpromotingabetterandmoreefficientuseofitmaybettercontributetostrategicplanningandfostercontinuousimprovement.

Areas for further development

22.SeveralkeyareasforfurtherdevelopmentininternalQAprocessesemergefromthisstudy.Amongthese, an all-encompassing approach to QA, the development of explicit feedback loops, theparticipationofallrelevantstakeholders,andtherelationbetweeninformationonstrategicgoalsandcommunicationstrategyshouldbeunderlined.Finally,thecomplexityoftheframework inwhichinternalQAprocessesoperateshouldnotbeunderestimated:otherdevelopmentsinhighereducation,externalregulations,financialconstraints,andpotentialreluctancefromtheinstitution’scommunityitselfaretobecarefullytakenintoaccountwhenfurtherdevelopingaqualityculture.


12

1.1. BackgroundTheriseofdemandforqualityassurance(QA)processes–bothinternalandexternal–hasusually

beenlinkedtothemassificationofhighereducation,totheincreaseofinvestmentanddoubtsconcerningthe possibility of maintaining quality in the resulting new circumstances, as well as to the belief in theimportanceofhighereducationinthenewknowledgesociety.

ThehighqualityofprovisionhasbeenoneofthekeyaimsoftheBolognaProcessandtheLisbonStrategyasameanstopromotetheattractivenessandcompetitivenessofEuropeanhighereducation.TheMinisterialmeetingswithintheBolognaProcesshaveshapedtheEuropeanqualityassuranceframework.In2003,theBerlinCommuniquéstatedthat“consistentwiththeprincipleofinstitutionalautonomy,theprimaryresponsibilityforqualityassuranceinhighereducationlieswitheachinstitutionitself”(BPMC2003).

TwoyearslatertheStandardsandGuidelinesforQualityAssuranceintheEuropeanHigherEducationArea(ESGs)wereadopted,basedonaproposalpreparedbytheE4Group(ENQA,ESU,EUAandEURASHE1).TheESGsyetagainreinforcedthecentralimportanceofinstitutionalautonomy,whichbringswithitheavyresponsibilitiesforHEIs(ENQA2005:11).In2007,theMinistersendorsedtheproposalofthesameE4GrouptosetupaEuropeanRegisterofQualityAssuranceAgencies(EQAR),thusconsolidatingtheframework.(EUA2010a:61-62)

Whiletheabove-mentionedEuropeanqualityassuranceframeworkwasbeingdeveloped,theworkonqualityassurancecontinuedatthegrass-rootslevel.HEIsareconstantlydevelopingandimplementingqualityassuranceprocessesandconsequently fosteringtheirqualityculture.Nevertheless,whereastherehavebeenvarious reportspreparedon theprogressmade in thefieldofexternalqualityassurance (forexample,ENQA2008)andotherreportsthathaveaimedatcoveringboth internalandexternalqualityprocesses(forexample,Rauhvargerset al.2009,EC2009;Westerheijdenet al.2010;ESU2009and2010),therehasnotbeenaEuropeanlevelreportspecificallyaimedatexamininghowhighereducationinstitutionshaverespondedtothedevelopmentsatpolicylevelintheirdailyactivities.

EUA’sTrendsseriestouchedbrieflyonthetopicofinternalQAaspartofthelargerBolognaProcessframework.TheTrends 2010resultsdemonstrateclearlythatinstitutionsfindqualityassurancereformstohavebeenamongthemostimportantdevelopmentsthathaveshapedtheirstrategyinrecentyears,since63%oftherespondentstoTrendsmentioneditamongthetopthreedevelopments.Furthermore,qualityassurancealsocontinuestobethesecondmostimportantdevelopmentexpectedtoimpactHEIsinthenextfiveyears,with21%havingmentioneditassuch.(EUA2010a:73)

IdeallyHEIsarenotmerelyworkingon theirqualityassuranceprocesses,butdeveloping internalqualityculturesadaptedto theirown institutional realities,which isamuchmorechallengingtask thanthat of simply settingupprocesses requiredby external parties. For quality culture is closely related toorganisationalcultureandfirmlybasedonsharedvalues,beliefs,expectationsandacommitmenttowardsquality,dimensionswhichmakeitadifficultconcepttomanage.

ItisinthiscontextthatEUAlaunchedthe“ExaminingQualityCultureinEuropeanHigherEducationInstitutions(EQC)”projectwithitspartners,theGermanRectors’Conference(HRK)andQAAScotland,withthegoalofexhibitingtheprocessesandstructuresthroughwhichthehighereducationinstitutionssupportthedevelopmentoftheirinternalqualityculturethusbothenhancingtheirqualitylevelsandrespondingtothedemandsofaccountability.

1. Setting the stage

1TheEuropeanAssociationforQualityAssuranceinHigherEducation(ENQA),theEuropeanStudents’Union(ESU),theEuropeanUniversityAssociation(EUA)andtheEuropeanAssociationofInstitutionsinHigherEducation(EURASHE).

13


1.2. The Examining Quality Culture project: aims and methodology

The project “Examining Quality Culture in European Higher Education Institutions” (EQC) waslaunchedinOctober2009andwilllastuntilautumn2011.Itbuildsuponthepartners’long-termworkondevelopingtheconceptofqualitycultureandexaminingitsnature,driversandobstacles(EUA2005andEUA2006inparticular).

Theaimoftheprojectistwofold:

1)IdentifyinternalqualityassuranceprocessesinplaceinHEIs,payingparticularinteresttohowtheinstitutionshaveimplementedthepartoftheESGsdedicatedtointernalQAwithinHEIs.

2)Discuss the dynamics between the development of institutional quality culture and qualityassuranceprocesseswhileidentifyingandpresentingcasepracticesinafinalreportinordertodisseminatethem.

Inthefirstphaseoftheprojectthefocushasbeenonmappingtheexistingqualityassuranceprocessesthroughasurveythatwaslaunchedtogatherquantitativeevidencetoreachthefirstaim.Chapter3ofthisreportisbasedontheanalysisofthedatagatheredviathesurvey,andChapter4includesfurtherreflections.

Thenextphaseoftheprojectwillfocusonthelatteraim,andwillcompriseofinterviewswhichwilltakeplaceduringthefirsthalfof2011.Intervieweeswillconsistofasampleofinstitutionrepresentativeswhohaveansweredthesurveyandindicatedtheirwillingnesstoparticipateinsuchaninterview.Theaimofthesecondphaseistocontributetointerpretingthequantitativedataprovidedduringthefirstphase.

This twofold approach was adopted so as to allow proper data collection, and to embrace aqualitativeapproachthatwasneededtocapturetheculturalandmoreinformalelements,whichcouldnotbedocumentedassuchbyaquantitativeapproachonly.

Keyquestions,towhichtheprojectaimstofindanswers,canbesummarisedasfollows:

•HowareEuropeanHEIsimplementingtheESGspart1inpractice?Whatkindofqualityassuranceprocessescanbeidentified?

•Atwhatstageistheimplementationofthequalityassuranceprocesses?WhileaEuropeanframeworkandprincipleshavebeendeveloped,howhavetheyinfluencedinstitutionalreality?Whatkindofmainchallengescanbeidentified?

•What is therelationshipbetweenformalqualityassuranceprocessesandan institutionalqualityculture? Does having one necessarily permit or preclude having the other? How do the twointeract?Howcananinstitutionensurethatthereissynergybetweenthemratherthandiscord?

•What kind of examples can be showcased to disseminate good practice or to demonstratechallengestoaqualityculture?

Thefirsttwoquestionsaboveareofprimaryinterestinthisreportandthesecondreportwilltakeacloserlookatthetwolatterpointsidentified.Therefore,itisworthunderliningthatthe main focus of this particular report is on the formal quality assurance processes and the progress HEIs have made in this regard in the light of the survey conducted.


14

The final project publication – which is to follow – will take the discussion one step furtherconsidering the relationshipbetweenqualitycultureand formalqualityassuranceprocessesaimingatabetterunderstandingoftheseconcepts(theirsynergies,similaritiesanddifferences)whilepresentingsomecaseexamplesandcitingobservationsmadeinthecourseoftheproject.

The survey

TheEQCsurveywasdesignedinordertocapturethedevelopmentsinqualityassuranceprocesseswithinEuropeanhighereducationinstitutions(HEIs)inaquantitativemanner.Thequestionnaire(seeannex)consistsoftwomainparts:onefocussingonquestionsdealingwiththedevelopmentoftheinstitutionalqualityassurancesystemorconcept,andthemainbodyofthequestionnairefocussingonquestionsonqualityassuranceinteachingandlearningwhileadaptingthelogicoftheESGs.

Priortothelaunchofthequestionnaire,theprojectpartnersinvited14HEIsacrossEuropetotestitandprovidefeedback.TheonlinesurveywaslaunchedatthebeginningofFebruary2010andadvertisedbythethreeprojectpartners.Aninvitationtoparticipate,alongwithreminders,wasaddressed,interalia,toallEUAmemberinstitutions.AnumberofpartnerorganisationsactiveinEuropeanhighereducationwerealsoaskedtoencouragetheirmemberstorespond.Theresponseswereanalysedintermsofgeneraltrendsandrespecting,atthisstage,theprincipleofconfidentiality:thereforenoinstitutionorindividualisidentifiedinthisreport.TheanalysisofresultsasstatedunderChapter3issolelybasedonresponsesreceivedfromthesurvey.

Atotalof222institutionsfrom36countriesansweredthequestionnairebythedeadlineof30April2010.Whereastherewasoneanswerfromeachofsevensmallercountries,itappearsthatnoneofthelargercountriesisoverrepresentedinthesample.Countriesfromwhichmorethan10answerswerereceivedareFinland,Germany,Ireland,Portugal,SpainandUnitedKingdom(SeeFigure2).

Thevastmajorityofrespondentswereuniversities(176),followedbyuniversitiesofappliedsciences(32),andotherhighereducationinstitutions(14)–thelatterbeingcomposedofdiscipline-basedinstitutionsanduniversitycolleges(seeFigure1).Mostoftherespondinginstitutions(195),includingalltypes,traintheirstudentsuptothedoctoralorthird-cycleequivalentlevel.

About59.5%(132)oftherespondinginstitutionscountmorethan10,000students,whereas35.1%(78)aremiddle-sized(between1,000and10,000students),and5.4%(12)small(lessthan1,000students).

72,78%

Figure 1: Ratio of types of institutions

14%

79%

6%

University of AppliedSciences, Polytechnic,Fachhochschule orequivalent

University

Other higher educationinstitution

15


Figure 2: Distribution of respondents per country

Countries with no answer

Countries with 10 or more answersCountries with 1 to 4 answers

Countries with 5 to 9 answers

2

104

5

1

11 8

8

2

26

21

4

4

8

1

99

1212

169

3

4

10

11

8

8

3

16

20

1

1

1


16

Thenameoftheproject–ExaminingQualityCulture–containsapromiseofexposingthetruestateandnatureofqualitycultureinEuropeanhighereducationinstitutions.Asdescribedinthepreviouschapter,thispublicationfocusesonthequantitativeresultsofthesurveyexaminingthequalityassuranceprocessesinplaceinhighereducationinstitutions.Beforescrutinisingtheseresultsitis,nevertheless,worthtouchinguponhowwehaveunderstoodbothqualitycultureandqualityassuranceprocessesintheframeworkofthisparticularprojectfortheresearchpurposes.Duringthecourseofthisproject,weexpecttoreachabetterunderstandingoftheseconceptsandthedynamicsbetweenthemandwillfurtherelaborateonthisinthefinalpublicationoftheproject.

2.1. The elements of quality cultureEUA’sQualityCultureproject’s (which ran in three rounds from2002 to2006)mainaimwas to

“increaseawarenessfortheneedtodevelopaninternalqualitycultureininstitutionsandtopromotetheintroductionofinternalqualitymanagementtoimprovequalitylevels”.(EUA2006:6-7)

As EQC continues the long-standing work of EUA in promoting quality culture, the followingdefinitionofqualityculturedevelopedbytheaforementionedprojectwaschosenasthestartingpointforourwork.Thus,inthecontextofthisproject:

[q]uality culture refers to an organisational culture that intends to enhance quality permanently and is characterised by two distinct elements: on the one hand, a cultural/psychological element of shared values, beliefs, expectations and commitment towards quality and, on the other hand, a structural/managerial element with defi ned processes that enhance quality and aim at coordinating individual efforts.(EUA2006:10)

The recentdiscussions in thefieldofquality andquality assurance inhighereducationarequiteunanimousinadvocatingpromotingqualityculture,somuchso,thatoftenqualitycultureisconsideredtobeasynonymfor“thedevelopmentof,andcompliancewith,processesofinternalqualityassurance”.(Harvey2009:1)

However,asdemonstratedbyFigure3,itiscrucialtodistinguishthesetwoconcepts:qualitycultureandqualityassurance.Whereasqualityassuranceprocessesare something tangibleandmanageablebyinstitutional decisions, the cultural aspect of quality culture – shared values, beliefs, expectations andcommitment–isfarmoredifficulttochange.(Ehlers2009)


“There needs to be a perceived value of quality assurance. Quality culture and quality assurance are not the same thing. You can have good QA in place but not necessarily a quality culture. The challenge is linking the outcomes of QA to the development of a quality culture that enhances the student experience.” - Respondent to the survey

“Much of the quality is dependent on the informal nature of staff/student relationships. The increasing calibration of quality indicators has led to a concern that this relationship will become formalised and thus less productive.” - Respondent to the survey

17


Thebasicassumptioninourprojecthasbeenthatqualitycultureandqualityassuranceprocessesareinterrelatedandthatqualityculturecanbeenforcedbystructuraldecisionswhichstimulatesharedvaluesandbeliefs.(Harvey&Stensaker2008:434)Furthermore,weacceptthatnoone(correct)qualitycultureexistsasacultureisalwayscloselylinkedtotheenvironmentandwithinoneHEItheremightevenbeseveralsub-culturesofquality.(Harvey&Stensaker2008;Harvey2009;Ehlers2009)

WhendescribinghowaHEIcouldgoaboutdevelopingitsqualityculture,Lanarèswrote:

There are at least two ways of seeing this. In some cases, the institution will introduce quality assurance. This will imply new values which will have to be integrated in the organisational culture. In other ones, the creation of quality assurance will start from the existing quality culture. Once finalised, quality assurance will in turn influence and modify the quality culture [...]. This second option may be preferable, considering that some continuity will facilitate change.(Lanarès2008:13)

ThismakesthechallengethatHEIsarefacing,andthusthetopicofourproject,evenmoreintriguing.

2.2. Quality assurance processes as understood in this project

Having established that quality assurance processes form one key component of quality culturealthoughtheydonotequalqualityculture,anddefinedthatthefocusofthisreportisonqualityassuranceprocesses,thequestionthatnaturallyfollowsis:whatdoweunderstandthesetobe?Indeed,itisclearthatthedefinitionvariesconsiderablyfromonecountryandinstitutiontoanother–nottomentionfromonepractitioner toanother.As theBolognaProcessStocktakingreport,whichaskedgovernmentsabout theprogressmadeinimplementingvariousBolognaProcessactionlines,observed:

It should be noted that the answers of some countries suggest that they think internal quality assurance within higher education institutions means only preparing self-assessment reports, without any reference to learning outcomes-based and improvement-oriented internal quality assurance systems. In addition, some HEIs have established a management system and they claim that it is a quality assurance system. However, some of these systems focus on measuring the performance of staff and/or units rather than on implementing ESG. This suggests there is a need to increase the focus on internal quality assurance within the EHEA.(Rauhvargerset al.2009:51)

Quality Culture

CommunicationParticipation

Trust

Formal qualityassurance processes

Tools and processes todefine, measure, evaluate,assure, and enhance quality

Quality commitmentCultural element

Individual level: personalcommitment to strive forquality

Collective level: individualattitudes and awarenessadd up to culture

Figure 3: Elements of quality culture (adapted from EUA 2006: 20 and EUA 2005: 18)


18

It iscorrect that, insomecontexts, internalqualityassuranceprocessesareseenastheprocessesthataimtoprepare the institutionor theprogramme foranexternalevaluation (i.e.preparing theself-evaluationprocess)orthemonitoringtaskassignedtoaspecificallyestablishedqualityunit. Ifthis isthewayqualityassuranceprocessesareregarded,itinvolvesrisks.WherearecentENQAsurveytoitsmemberagencies noted that three-quarters of respondents have recently changed or are about to change theirqualityassuranceapproach(ENQA2008:26),EUA’smostrecentTrends reportconcluded:

that the introduction of new national external evaluation procedures has caused some institutions to pay much less attention to their own internal accountability procedures, thus leading to a compliance culture. This seems to be particularly true when the external agency is perceived as being formalistic and bureaucratic.(EUA2010a:63)

Intheframeworkofthisprojectwehaveoptedto:

use quality assurance in the broad meaning of the term, including in practice all elements of a strong quality culture of a HEI. Internal QA in the context of this report should not be understood merely as a specific quality monitoring (such as process descriptions, data collection and analysis) or evaluation processes often carried out by a specific quality unit, but including all activities related to defining, assuring and enhancing the quality of an HEI. (EUA2009:13)

Furthermore, going back to our previous quote from the Stocktaking report, we agree withRauhvargerset al. thattheremaybeaneedtofocusorclarifywhat isunderstoodbyqualityassurance,but,inthelightofwhatwasdiscussedpreviouslyregardingcultureofcompliance,donotthinkthatqualityassuranceshouldbelimitedtocoveringonlytheactivitiesmentionedbytheESGs.

Wewouldevengoso faras toencourage institutionstoadoptanall-encompassingapproachtodevelopingtheirinternalqualityassuranceprocesses:anapproach,thus,thatwouldbetailor-madefortheinstitutionandderivedfromitsownstrategicgoalsandfitintothestate-of-artoftheinstitution’sinternalqualityculture,whilefulfillingtheexternalrequirementsintheprocess.

ItisinthiscontextthatwehaveintentionallychosennottolimitourselvesonlytomeasurescoveredbytheESGsasitwouldprovideaveryone-dimensionalviewonqualityassuranceprocesseswhichreallycan–andoftendo–takevariousformsandshapes.Also,itshouldbehighlightedthattheESGsonlydealwithteachingandlearningactivitywithinaninstitution,whereas,whenplanningtheiractivities,institutionstypicallyviewtheiractivitiesintheirentirety.Forexample,inaninstitutionwhichhasastrongresearchfocusandbasesteachingonresearch,canonecompletelyseparatethequalityassuranceofteachingandlearningfrom research? Furthermore, the strategic management and the quality of governance will inevitablyinfluencethequalityofteachingasOECD’sProgrammeonInstitutionalManagementinHigherEducation(IMHE)arguesintheirup-comingpublication(IMHE2010:2).

Also, EUA’s work over the years has stressed the importance of creating a link between qualityprocessesandinstitutionalstrategicplanning(EUA2006;EUA2010a).Aninstitution,havingclearlydefinedits mission and strategic goals and knowing what quality means in the light of its own goals, lays thegroundworkforawell-functioningqualityassurancesystem.Thenextstepinvolvessettingupprocessestoensurethatthisqualityisreachedandtomonitorprogressinthisregard(asaddressedbyEGSspart1)andfinally,beingabletoreactwhenallisnotwelloreven,ifallseemstobeworkingrelativelywell,beingableandwillingtoimprovecontinuously.Inthisregard,aninstitutionneedstodevelopafeedbackloopthatwouldstrengthenthelinkbetweentheresultsofmonitoringactivitiesandstrategicplanning.

Asaresult,weincludedinoursurveysomequestionsrelatedtotheinstitutionalqualityassuranceframework in general as well as questions related to quality assurance in research and to strategicmanagement.However,themainbodyofthesurveyremainsfocussedonqualityassuranceofteachingandlearningfollowingthelogicoftheESGsandaspirestomapthatstateofplayintheimplementationoftheESGstoprovidequantitativedataforthebasisofanypolicydiscussionsontheprogressmadeinthatregard.

19


3.1. Quality assurance structures

Data collected through the survey shows that most of the responding HEIs have fundamental quality assurance structures and processes inplaceandremarkableprogresshasbeenmadeinrecentyears,althoughanumberofchallengesremain.

Qualityassuranceprocessesmostcommonlycoverteachingandlearningactivities,whichisquiteunderstandable,asthecreationoftheEuropeanHigherEducationArea–andtheESGsasanintegralpartofit–hasfocussedonthispartofHEIs’mission.Thus,98.2%oftherespondentstooursurveyansweredthattheirqualityassuranceprocessescoverteachingandlearning,whilestudentsupportservices,whicharecloselyrelatedtoteachingandlearning,arecoveredinonly75.7%ofHEIs(seeTable1).Theotheractivitiesmentionedby6.7%(15)oftherespondentscoverareasasdiverseasartisticactivitiesandequalopportunitypolicies.

Table 1: Activities covered by institutional quality assurance processes

OnecommonphenomenonnotedwasthetendencynottorecogniseallQArelatedprocesseswithinaHEIassuch.Forexample,while79.3%oftherespondentsrepliedthattheirinstitutionalqualityassurancesprocessescoverresearchactivities,atotalof97.3%ofallrespondents–whenaskedwhetherornottheyhave specific processes in place with regard to QA in research – recognised that they do indeed haveindividualprocessesinplace.Themostcommonprocessesareinternalseminarswhereresearchprojectsandideasarediscussed(65.3%),preparingstatisticsonpublishedarticles(64.9%)andexternalpeerreviewofresearchprojectsinrelationtograntapplications(53.6%).Afurtherexampleofthisinabilitytodiscernthe

3. Mapping internal quality assurance processes: survey results

Which activities do your institutional quality assurance processes cover?Please choose all applicable options.

Teachingandlearning 98.2%

Research 79.3%

Servicetosociety 47.7%

Studentsupportservices 75.7%

Governanceandadministrationoftheinstitution 65.8%

Other 6.7%

“Quality culture must articulate Teaching, Research and Service to society. It should respect the diversity in specifi c domains and fi elds and rely on accurate objectives established according to the various institutional, political and cultural situations. Those objectives should be adapted to the broad context. Quality culture should aim at continuous improvement instead of ad hoc and instrumentalising evaluations.” - Respondent to the survey

“Some aspects of quality culture are consolidated in the university community, such as administrative services, but others are still “on paper” and not in “minds”. This is clear in all aspects of evaluating teaching and learning skills, and this is a big handicap we need to overcome to transfer from simply evaluating results to real effective decisions and political commitments.”

- Respondent to the survey


20

presenceofcertainQAprocesseswaspickedupthroughcross-checkinganswersregardingQAprocessesin services to society:whilstonly47.7%of the institutionsdeclaredcovering services to society in theirQAarchitecture,95.9%of the respondents,whenasked to specifywhichprocesses theyhave inplace,pinpointedoneorseveralprocessesrelatedtoservicestosociety.

AsEUA’sQualityCultureprojectidentifiedstrategicplanningascrucialinembeddingqualityculture(EUA2006:13)andthefirstguidelineforinternalqualityassurancementionedintheESGsstatesthatHEIs“shouldhaveapolicyandassociatedproceduresfortheassuranceofquality”,oursurveyaskedHEIsiftheyhadan institutional strategic plan or equivalent document.Anoverwhelmingmajority(92.8%)repliedthattheydohavesuchadocument.WhereassomeHEIsreportedthattheirstrategicplanshavebeendrawnupatfacultylevel,onlythreeinstitutionsadmittednothavingastrategicdocumentatall.Ourdatafurtherindicatesthatifaninstitutionhasastrategicplan,itusuallyalsohaseitheraseparateinstitutionalQApolicystatement(68.1%)oritsqualitystatementisincludedinthestrategicplan(25.5%).

FurthertoQApolicy,HEIshavealarge variety of organisational structuresinplacetosupporttheimplementationofQAprocessesbutnotypicalsolutiononhowtoarrangetheresponsibilitiesforQAwithinaHEIseemstoexist.Figure4demonstratesthedistributionofdifferentstructuresamongtherespondents.ItisnoteworthythatnearlyhalfoftherespondingHEIs(103)donothaveanycommittees(neitheratfacultynor institutional level)responsible forQAprocesses2. Inaddition,thereare80 institutionsthatansweredthatarectororvice-rectorisnotresponsibleforQAissues.Eveninthesecasestherewasnotypicalsolutiononhowtoorganisetheinstitutionallevelresponsibilitieswherethereisalackofinstitutionalleadership.Asexamplesoforganisationalfeatures,outofthese80institutions,andincombinationwithallsortsofotherfeatureswithnotypicalscheme,27institutionshaveacentralunitspecialisedinQA(amongotherthings),12haveapersoninchargeofQA(amongotherthings)withintherectorate,and13haveboth.

ItcanbenotedthatcentralisedunitsforQA,pedagogicaldevelopmentandstaffdevelopmentaremorelikelytobeinplaceinthoseuniversitiesthatworkedontheirQAsystembefore2000:respectively88.2%and55.8%ofinstitutionsthatintroducedtheirsystembeforeandinthe1990’shaveaunitresponsiblefor staffdevelopment,whereas thispercentagedropsandnever exceeds30% for institutions thathaveintroducedtheirsystemwithinthelastdecade.Theexistenceofaunitinchargeofpedagogicalinnovationfollowsthesametrend:respectively76.5and60.5%ofthosewhointroducedtheirsystembefore1990andinthe1990’shaveone,whilethepercentagedropsbelow50%forinstitutionsthatintroducedtheirsystemafter2000.

72,78%

Figure 4: Structures supporting the internal quality assurance processes

0 20 40 60 80 100 120 140 160

The rector or specially assigned vice-rector is in charge of QA issues.

There is a centralised QA unit, with specialised staff.

There is an institutional level quality committee or equivalent.

There is a unit responsible for pedagogical innovation (or equivalent) that offerssupport to the teachers in developing teaching methods.

There are contact persons or persons in charge of QA within their unit, who alsohave other responsibilities.

There are faculty level and/or department and/or programme level qualitycommittees or equivalent.

There is a unit responsible for staff development.

There is a person in charge of QA within the rectorate.

Other

There are QA units in each faculty with specialised staff.

64.0%

62.2%

53.6%

47.7%

45.0%

40.5%

38.3%

36.5%

9.9%

9%

Number of answers

2SeealsoChapter3.2.inthisregard.

21


Thus,HEIshavethefundamentalqualityassuranceprocessesandstructuresinplace,butourdataalsodemonstratesthatthe progress made in this regard is very recent.Alittleoverhalfoftherespondinginstitutions(52%)reportedthattheyintroducedtheirQAsystemin2005orlater,aconsiderableportionstillbeingintheplanningorintroductionphase(Figure5).

Thetimingofintroductiondoesnotappeartodependonthetypeorsizeofaninstitution,noraretheredistinctivedifferencesbetweencountries,withtheexceptionoftheUK.WhileinallothercountriesthereareHEIsthathavestartedworkingonQAprocessesonasystematicbasisatdifferenttimes,allUKinstitutionsreplyingtoourquestionnairealreadyhadtheirprocesses inplacebytheturnofthecentury(Figure6).

Figure 5: Introduction of a quality assurance system (or equivalent)

16%

36%

8%

19%

21%

In the 1990's

Between 2000 and 2005


Before 1990

We are currently designingand/or planning it.


22

Althoughthere isacleartrendthattheinstitutionsthat introducedaQAframeworkbefore1990are more inclined to have adopted an institutional approach, ranging from a policy statement to theinvolvementofinternalandexternalstakeholders,theseveryinstitutionsseemtograntlessimportancetomakinginformationandassessmentspublic.However,consideringthatavastmajorityoftheseinstitutionsisfromtheUK,thismaywelljustbeanationalfeatureandnofar-reachingconclusionscanbedrawnfromit.

In terms of how the institutional QA system was introduced, just over half (51.8%) of therespondentsbasedtheirconceptontherequirementsofthenationalQAagency,while40.5%saidthattheinstitutionalleadershipdecidedontheconceptandprovidedinstructions,trainingandsupporttotheunit to implement it. Inaddition,43.7%ofHEIsansweredthattheirQAconcept is theresultofvariousconsultationroundsamongtheacademicandadministrativestaffaswellasstudents,whileonethird(32%)usedpilotprojectsconductedbyselectedunitsintheintroductionphaseanddisseminatedgoodpracticesidentifiedthroughthem(Figure7).

Figure 6: Introduction of a quality assurance system (or equivalent) – Breakdown per country

No data

Before 2000


Currently designing/planning

23


3.2. Participation of Stakeholders

Institutional leadership

AsEUA’sQualityCultureproject’sonekeymessagewastounderlinethecrucialroleofinstitutionalleadership in demonstrating an institutional commitment to quality, thus laying a groundwork for theimplementationofqualityculture(EUA2006:20),itisworthwhiletakingacloserlookathowitsroleplaysinthelightofsurveyresults.

66.2%of the respondentsdefined the roleof senior leadership (rector, vice-rector) inbuildingaqualityculturewithintheirinstitutionastakingtheleadintheprocess.Inthosecaseswheretheleadershipwasnotleadingtheprocess, itwasusuallymentionedthatitmonitors,makesdecisionsorfacilitatestheprocessorsomecombinationofthese.Inuniversities,theleadershipismorelikelytoleadtheprocessandmakedecisionsthaninothertypesofinstitution:indeed,outoftheinstitutionswheretheleadershipdrivestheprocess,82.3%areuniversities–whichisslightlyhigherthantheratioofuniversitiesinthesampleofrespondentstothewholesurvey(79%).

Asmentionedabove(Figure7),in40.5%ofthecases,theinstitutionalleadershipdecidedontheQAconceptoftheHEI.WhenitcomestooperationalresponsibilityforQAprocesses,thepercentageofHEIswheretherectororspeciallyassignedvice-rectorisinchargeofQAissuesgrowsto64%(Figure4,Chapter3.1).Nevertheless,thedataindicatedthattheroleoftherector(orvice-rector)ismoredominantinthoseHEIsthathaveworkedontheirQAsystemformorethan10years.

Figure 7: How the internal quality assurance system (or equivalent) was introduced within the institution

64,0%

0 20 40 60 80 100 120

The concept is based on the requirement of the national QA agency whichdeveloped the standards and guidelines for this.

The concept is the result of various consultation rounds among the academicand administrative staff as well as students.

The institutional leadership decided on the concept, provided instructions,training and support to the units to implement it.

The concept was introduced through pilot projects conducted by some units.Good practices were disseminated based on these experiences.

The concept is the result of various consultation rounds among the academicstaff of the institution.

The concept is the result of various consultation rounds among the academicand administrative staff.

Other

51.8%

43.7%

40.5%

32.0%

14.4%

14.0%

13.1%

Number of answers

“The only way to achieve a functional quality culture is by convincing the members of the HEI that they have something to gain by analysing the qualitative processes of their day-to-day work.”

- Respondent to the survey

“The biggest challenge for quality culture implementation is to combine the top-down leadership and managerial approach with the bottom-up approach, while creating favourable learning environments for academic staff and students to be actively involved in quality culture implementation activities via their own initiatives and responsibilities.” - Respondent to the survey


24

Staff and students

Oneofthekeyprinciplesindevelopingbothqualityculture(EUA:2006,Harvey&Stensaker2008)and quality assurance processes (see various ministerial communiqués) is the participation of staff andstudents.AsindicatedbyFigure7(Chapter3.1),inalmost44%ofthecases,studentsandstafftookpartintheplanningoftheinstitutionalqualityassurancesystem.Inaddition,28.4%oftherespondentsinvolvedtheiracademicstaffintheconsultations(whichmakesatotalof72.1%involvingtheiracademicstaff),while14%involvedtheiradministrationstaff(totalof57.7%).

Settingupcommittee(s)forqualityassurancemightseemalogicalwayofensuringtheparticipationofstaffandstudents.However,aftercross-checkingtheresponsesonQAstructures(Figure4,Chapter3.1),itappears that theHEIswithcommitteesat institutional levelaremostly those thathavecommitteesatfaculty/departmentallevel.Thisleavesuswith46.4%oftherespondentswithnocommitteeresponsibleforqualityassurance.However,datadoesshowthatthelongertheHEIhasworkedonQA,themorelikelyitwillbetohaveinstitutionalorfacultylevelcommittees.

Figure8 illustrates theextent towhichvarious stakeholdersare involved in thequalityassuranceprocesses of the responding HEIs. This shows that the level of participation is relatively high, with theacademic staff being the most commonly involved. Furthermore, when HEIs reported that they have

Figure 8: The involvement of stakeholders in formal quality assurance processes

Through formal participationin consultation bodies

Through formal participationin governance bodies(with voting right)

They are not involved

By responding to the surveyson a regular basis (e.g.end ofeach course, academic year...)

By informally providinginformation on the issues atstake

Through formal involvementin self-evaluations or otherevaluation activities

Academic staff

Administrative staff

Leadership,institutional level

Leadership, faculty/department level

Students

Externalstakeholders

Alumni

0% 20% 40% 60% 80% 100%

81.5%74.8%

90.5%65.8%

53.6%

9.9%19.4%

25.7%41.4%

34.7%11.7%

41.9%41.0%

36.9%

21.2%9.0%

49.5%

0.5%

76.6%68.0%

74.3%63.1%

79.7%

77.5%64.9%

59.5%27.9%

76.1%

61.3%62.6%

58.6%68.5%

26.6%2.7%

85.1%66.7%66.2%

54.5%15.8%

25


formedworkinggroups toprepare curricula, academic staff is almost always (99.5%) included in theseworkinggroups,with37HEIsreportingthattheirworkinggroupsaresolelycomposedofacademicstaff.Itisnoteworthythatonlyinhalfofthecases(50.8%)arestudentsinvolved.

With regard to student participation in quality assurance, some previous studies (Rauhvargers et al. 2009:14; ESU200948-49)have shown that, althoughprogresshasbeenmade, there is still roomfor improvement,pointingout inparticular the fact that students arenotusually involved in follow-upmeasures.

Tosomeextent,ourdataconfirmthis.Thefeedbackloopbetweenstudentsurveysandthefollow-upactivitiesisdiscussedinfurtherdetailunderChapter3.3,butfromtheperspectiveofpromotingstudentparticipation,itcanbenotedthattheinfluenceofstudentsurveysandtheimportanceofthefeedbacklooparemuchmoredevelopedatinstitutionsthatintroducedtheirsystemearlyon,i.e.before2000.Indeed,94.1%oftheseHEIstakethesurveysintoaccountinthecurriculumdesign,discusstheminternally,andinformstudentsaboutthefollow-up.

Finally, with regard to the involvement of members of the institutional community, it is worthreferring to thefindings indicatedbyTable4 (Chapter3.3) thatconducting regular surveysamong themembersoftheinstitutionalcommunity(staffandstudents)toanalysetheirperceptionoftheinstitutionalstrategyand its implementationatgrass-roots leveldoesnotseemtobeacommonpractice (withonly27.9%reportingthattheydoso).Chapter3.3willfurtheranalysethefeedbackloopandcommunicationamongtheinstitutionalcommunity.

External stakeholders

External stakeholders (e.g. employers, experts) have been involved in QA processes from thebeginning.However,theirparticipationlevelvaries:around80%oftheHEIsincludethemindifferentwaysin their decisionmaking: throughgovernancebodies, consultationbodies or as sources of information,amongothers,whileroughly20%oftheHEIsdonotconsultorinvolvethematall.

External stakeholders are more likely to be involved in preparing a curriculum (for instance asmembersofworkinggroups)whentheultimatedecisiononthecurriculumismadebyabodyexternaltotheHEI.However,theresultsindicatethattheirroleisusuallythatofinformationprovidersratherthandecisionmakers.Furthermore,thealumniareseldominvolvedassuch(Figure8).

3.3. The use of informationThe purpose of this chapter is to discuss what kind of information HEIs collect and store about

themselves,howtheycommunicateonthebasisofthisinformation,andhowtheinformationfeedsintotheinternaldiscussionsanddecisionmakingprocesses.

Types and sources of information

93.2%of theresponding institutions reportedthat theyhaveacentralised informationsystem inplace.Inaddition,5.9%ofrespondents,whentheydonothaveacentralisedsystem,dohaveinformationsystemsthatexistat faculty level.Therefore,thenumberof institutionsthatdonotuseany informationsystemformonitoringtheiractivitiesisextremelylimited.


26

Mostcommonly,theseinformationsystemsincludestudentprogressionandsuccessrates(87.7%of respondents),aswell as theprofileof the studentpopulation (83.2%).The teacher-student ratiopereducationalunit also scoreshigh (65.5%),asdoes student satisfactionwith theirprogrammes (53.6%).However,this53.6%seemslowcomparedtothe71.6%ofrespondentswhoansweredthattheyconductstudentsurveysassessingtheteacher’sperformancesandcompetences.Table2givesanoverviewofthedifferentinformationincludedininstitutions’informationsystems.

Table 2: Information included in the information system(s) regarding study programmes

Whereas institutions collect information about their profile and what they offer, the informationbecomes scarcewhen it comes to resourcesavailable to the students. Indeed,whilemore than80%ofallrespondentsconfirmedthattheyofferlibraryservices,computerfacilities,laboratories,humansupport(tutors,counsellorsorotheradvisers)andotherkindsoflearningfacilities(seealsoFigure12,Chapter3.4.),only44.1%feedtheinformationontheavailablelearningresourcesand,wherepossible,theircosts,intotheirinformationsystem.

Interestingly,whattheinstitutionscollectmostoften(profileofstudentpopulation,studentprogressionandsuccess rates)doesnotnecessarilycorrespondwith the informationmadepublic (seeTable3).Thedatamosttypicallyfeaturedisinformationonstudyprogrammes,includinginformationonqualificationsgrantedbytheprogramme(86.9%)andonteaching,learningandassessmentprocedures(82%).Whereasqualificationsaredocumentedbymostinstitutions,theyarenotalwayspartoftheinformationonalumniemployment:only40.5%ofrespondentstrackgraduateemploymentandincludethisintheirinformationsystem,butTable3showsthat45.5%providesomekindofdetailonalumniemploymentintheinformationmadeavailableontheirstudyprogrammes.

Which of the following does the information system or systems include? Please choose all applicable options.

Studentprogressionandsuccessrates 87.7%

Teacher-studentratioperfaculty/department/instituteorintherespectivefaculty/department/institute 65.5%

Trackinggraduates’employment 40.5%

Students’satisfactionwiththeirprogrammes 53.6%

Profileofthestudentpopulation(e.g.,age,gender,educationbackground,socio-culturalbackground...) 83.2%

Availablelearningresourcesand,whenapplicable,theircosts 44.1%

Noneoftheabove 0.9%

Other(suchastheinstitution’sownperformanceindicators) 10.0%

27


Table 3: Information provided by the institution on its study programmes

Intermsofpublic informationonQA,institutionsmosttypicallymakepubliclyavailable(throughwebsites,publicationsorothersourcesofinformation)theresultsoftheexternalevaluations(61.3%),butaremorereluctanttomakepubliclyavailabletheresultsoftheirinternalevaluations(38.7%).Consideringthattheinternalevaluations–atleastideally–areimprovement-led,includingpointingoutshortcomingsandsuggestingremedies,thedesiretokeepthemasinternalworkingdocumentsisperhapsunderstandable.

Feedback loop and communication

HavingnotedthatHEIscollectdataontheirperformance,itshouldbeaddedthatthelinkbetweencollecting the information and informing the community involved in this data collection is not all thatobvious.Asanexampletoillustratethispoint,thegeneraltrendwithregardtoteachers’performancesistokeepthemconfidentialandavailableatinstitutionalorfacultyleadershiplevelonly:59%oftherespondentsreportedthatthisisthecaseintheirrespectiveHEIs.18.9%makethemavailableatinternallevel(forthoseinvolvedinQA,andfortheteachingcommunityingeneral),andonly6.3%(14institutions)makethempubliclyavailable.

Moreover,whileexploring further the feedback loop,wecansee that71.6%of respondentsusestudentsurveysasoneofthemeanstomonitorstudents’perceptionsoftheteachingtheyreceive.Amongthoseconductingstudentsurveys,92.4%taketheresultsintoaccountintheassessmentofteachingstaff.Butonly58.5%ofthemstatethatstudentswhoparticipatedinasurveyareinformedabouttheoutcomesandtheresultingactionstaken;andthepercentagedropsto6.3%whenitcomestomakingtheinformationonteachers’aptitudesandperformancepubliclyavailable.

In thecaseof22.3%of the respondents, the legal frameworkdoesnot foresee thepossibilityoftheremovalofanineffectiveteacher.ThisisoneoftheareaswherethestandardssetbytheESGsattheEuropeanlevelclashwithnationalregulations.Asourdata indicates,thefeedback loopclearlyfunctionsbetterwithintheinstitutionswithprocessesinplacetoobligeateacherimprovehis/herperformance.All61institutionsthathaveprocessesinplacetoremoveanineffectiveteacher,conductstudentsurveys,with49ofthemtakingtheresultsintoaccountintheassessmentofteachingstaff,and44oftheminformingthestudentsontheoutcomesandactiontakenasaresult.56oftheseinstitutionsusethestudentsurveysasonesourceofinformationwhendesigningand/orrevisingprogrammes.

The information on your institution’s study programmes include: Please choose all applicable options.

Numberofstudentscurrentlyinvolvedintheprogramme 76.1%

Numberofacademicstaffinvolvedintheprogramme 70.3%

Teacher-studentratiointherespectivefaculty/department/institute 44.6%

Informationontheintendedlearningoutcomesoftheprogramme 81.5%

Informationonqualificationsgrantedbytheprogramme 86.9%

Informationontheteaching,learningandassessmentproceduresusedwithintheprogramme 82.0%

Informationonthelearningopportunities(e.g.,traineeships,exchangeprogrammes,mobilitypossibilities,scholarships...)availabletothestudentsoftheprogramme

78.8%

Informationonalumniemployment 45.5%

Profileofthecurrentstudentpopulation 43.2%

Specificinformationtargetinginternationalstudents 64.0%

Accessibilityand/orpossibilitiesofferedtodisabledstudents 49.5%

Other 5.0%


28

TheresultsofthesurveyleadustobelievethattheQAofteachingstaffmaybecloselyconnectedtothetimeframeoftheintroductionofQA.Whilealmosttwothirds(63.4%)ofallinstitutionssurveyedhavespecifiedtheirownrequirementsforhiringteachingstaff,76.8%ofthosewhohadintroducedtheirQAsystembefore2000havedoneso.

Link to the strategic management

Further to thequestionsof involving thecommunity in the follow-upand the impactof studentsurveys in particular that was discussed in Chapter 3.2., the feedback loop between the results of QAprocessesandstrategicmanagement isoneofthekeysuccessfactors insustainingqualityculture.(EUA2006:13,18)

The most common way of ensuring the feedback loop to strategic management appears to betheconductingofannualevaluationsby the institutional leadership to review thegoals thathavebeenset (65.3%). In addition to the65.3%who chose this option, therewereotherswhodescribed similarprocesses,althoughunderliningthattheevaluationmadebytheleadershiphappensthroughdiscussionswithotherkeystakeholders(deans,headsofunitsetc.).However,thenatureofthisannualevaluationwasnotexaminedinfurtherdetailinthequestionnaireandthereforethedepthandeffectivenessofthisprocesscannotbeexploredhere.Annualevaluationsmaytakevariousshapesandfollowdifferentrationales–formalprocessesrequestedbyaMinistryorotherexternalbodies.

Table 4: Internal evaluation processes providing feedback to the strategic planning

Decisionson future strategicdirectionare ideallybasedon solidevidence,andQAprocessesaretypicallyconsideredtobeoneofthekeyinformationsources.Nevertheless,only55%oftherespondingHEIshaveformulatedkeyperformanceindicators(KPIs)tofollowtheirprogressbasedonthem(seeTable4).Itislogical,however,thatthosewhoarestillworkingontheprinciplesoftheirQAsystemarelesslikelytohaveKPIsinplace.

3.4. Quality assurance in teaching and learningWhenaskedaboutdesigningaqualityassuranceframeworkinteachingandlearninginparticular,

amajorityofHEIs(64.9%)indicatedthatitisinstitution-specificbutfollowsnationalQAframeworksandguidelines.Onequarter (25.7%)considered it tobe tailor-madeto the institution’sneedsanddoesnotapplyanyready-mademodel,whereaslessthan9.5%mentionedthatitappliesaready-mademodelsuchasISO,EFQM,andCAF.

Do you have an internal evaluation process that provides feedback to the strategic planning in place? Please choose all applicable options.

Theinstitutionalleadershipevaluatesannuallytheprogressmadeintermsofachievingthegoalssetbytheinstitution. 65.3%

Thefaculties(and/orrelevantunits)conductregularself-evaluationstoanalysetheircontributiontotheachievementofinstitutionalstrategicgoals. 55.9%

Theinstitutionconductsregularsurveysamongthemembersoftheinstitutionalcommunity(staffandstudents)toanalysetheirperceptionoftheinstitutionalstrategyanditsimplementationatgrass-rootslevel.

27.9%

Theinstitutionhasdefinedasetofkeyperformanceindicatorsandfollowsitsprogressbasedonthem. 55.0%

Theinstitutionalstrategyandtheachievementofthegoalssetinitarerevisitedwhenthedocumentisrevised(every3,5orNyears). 52.7%

Other 11.3%

29


Hence,qualityassuranceinteachingandlearningdoeshaveitsspecificitiesandfewHEIshavechosentoadoptexternalQAsystemmodelsas such.The fact that thenumberofHEIs following theirnationalQAframeworksisnothigherisaninterestingphenomenon,anditmayreflectthefactthatguidelinesforinstitutionallevelQAframeworkdonotexistineverycountry.IncountrieswithanexternalQAsystembasedoninstitutionallevel,amajority,ifnotallofHEIstendtoanswerthattheyfollownationalguidelines.Thisis,forexample,thecaseforFinland,France,IrelandandtheUK,wherenationalguidelinestargetingspecificallyinstitutionallevelQAarrangementdoexist.However,nationalguidelinesoninstitutionalQAprovisionsexistinsomecountrieswhereaccreditationisprogramme-based:forexample,inGermany,Spain,PolandandSweden,whereamajorityofrespondentsansweredthattheyfollownationalguidelinesinthisregard.

Thischapterpresentsselectedkeyresults regardingqualityassurance in teachingand learning inparticular,butitisworthnotingalsothatsomethemesintegrallyrelatedtotheteachingandlearningarealsodiscussedinotherchapters.

Curriculum design

Fromtheanswerstothequestionsrelatedtocurriculumdesign,wecandrawtheconclusionthatadelicatebalanceneedstobekeptduringthisprocess.Programmedesign–ingeneralconsideredbyacommittee(85.1%)-whoseobjectivesandexpectedlearningoutcomesarenecessarilyledatfacultylevel,ideallyhastocorrespondwiththestrategyoftheinstitutionasdefinedbytheHEIleadership.Mediationandcommunicationareanessentialpartofthediscussions.10.8%ofrespondentsalsoreportedthattheircurriculumisdesignedbytheMinistryorotherexternalbodyandthismakesthebalancingactallthemorechallenging.

Furthermore,thetraditions,internalstructures,proceduresaswellasthetypeofgovernancewithintheHEIleadtohighlyindividualisedprocessesinthisfield,whichmayvarydependingonthefacultieswithinoneinstitution.

0

50

100

150

200

Figure 9: Designing curriculum and programmes within the institution

Programme directoror equivalent person

prepares thecurriculum after which

staff members maycomment the draft.

Working group,committee or

equivalent preparesthe curriculum.

OtherThe curriculum isdesigned by theministry or otherexternal bodies.

Each staff memberproposes what

they find essentialfor the programme

and the curriculum iscreated on this basis.

36.0%

85.1%

28.4%

5.4%10.8%

Num

ber

of a

nsw

ers


30

Monitoring and improving an established programme

Onceaprogrammeisupandrunning,thefrequencyandmeansformonitoringitalsovaryfromoneinstitutiontoanother.Inaddition,mostinstitutionsseemtoconductavarietyofprocesses(seeFigure10)inavarietyofcombinations,leadingtotheconclusionthatthereisnotonetypicalprocessformonitoring.

Among the two thirds of respondents who answered that the evaluation of curriculum andprogrammes is part of an external accreditation process, only 12 HEIs (without significant geographicconcentration)answeredthattheydonothaveanyotherformofmonitoring.Thismeansthat, incaseswhensuchexternalprocessesarereported,mostofinstitutionsconduct,inaddition,somekindofinternalevaluationinordertomonitorandimprovetheircurriculumandprogrammes.

Learning outcomes and workload

Whereas95.6%ofinstitutionsstatedthattheyhavedevelopedexplicitlearningoutcomes–eitherforallprogrammes(67.9%)orforsomeprogrammes(27.7%)–theydonotallmakethempubliclyavailablethroughtheirwebsite,studyguidesorequivalent:71.7%havedoneso.Thenumberof institutionsthatinclude information on the intended learning outcomes in the information provided about their studyprogrammes is however higher (81.5%). Further studies and research may address this area further byexploring,forexample,howinstitutionsthatsaytheydevelopedthemmakeuseoflearningoutcomesintheirteachingpractices.

Whilst95.6%ofrespondentsdesignlearningoutcomes,only40.6%actuallymeasurethestudentworkloadthroughsurveysaddressedtostudentsinordertoreachthedescribedlearningoutcomes.ThiswouldbeanecessarystepinordertoimplementECTScorrectlysincethesebuildequallyuponlearningoutcomes and student workload as elements. In 44.8% of cases the teachers are solely responsible forindicatingthestudentworkloadandin4.3%noworkloadisindicatedatall.

Figure 10: Monitoring curriculum and programmes

The curriculum and programme contents, pedagogicalapproaches and intended learning outcomes are

evaluated as part of an external accreditationprocess or equivalent.

The curriculum and programme contents, pedagogicalapproaches and intended learning outcomes areevaluated regularly (every N years/semesters...).

The curriculum and programme contents are evaluatedcontinuously on an informal level (discussions between

staff members, staff and students...).

The curriculum and programme contents are evaluatedoccasionally (at the occasion of a self-evaluation

exercise, for an external evaluation body...).

Curriculum and programme design processes as such(that is, the effectiveness and comprehensiveness

of the processes) are evaluated regularly(every N years/semesters...).

Other

0 20 40 60 80 100 120 140 160

66.6%

57.2%

54.5%

36.5%

32.9%

6.8%

Number of answers

31


Student assessment

Figure11 summarises themain featuresof student assessmentprocedures.Whencross-checkingdata,itisclearthatthoseinstitutionswhoseassessmentisdirectlyrelatedtointendedlearningoutcomesarethesameonesthathavedevelopedexplicitlearningoutcomes,eitherinallorinsomeprogrammes.Thelinkbetweenhavinglearningoutcomesandtheirinfluenceinstudentassessmentproceduresisthereforeestablished.

Finally,onlytwoinstitutionsindicatedthattheyeitherhavenoneofthecharacteristicsmentioned,orthattheydonothaveassessmentprocedures.Inconclusion,althoughnoneofthesefeaturesscoredmorethan75%,mostinstitutionshaveacombinationofseveralcharacteristicsasmentionedintheESGs.

Withregardtoinformingstudentsabouttheassessmentprocedures,82.6%ofrespondentsmaketheirassessmentmethodsandcriteriapubliclyavailable,and83%relyontheteacherstoinformorremindstudentsaboutthisat thebeginningof thecourse.Thus,aclearmajorityof the institutionsseemtobetransparentaboutassessmentproceduresasrecommendedbytheEGSs.

Learning resources

AlmostallHEIsparticipatinginthesurveyofferbasiclearningresources,butwhenaskedspecificallyaboutmonitoringandevaluatingthem,itturnsoutthattheyoftendonotdoitsystematically(Figure12).However,interestinglyenough,thenumberofHEIsreportingthattheyregularlyimprovetheservicestheyofferishigherthanthosewhomonitororevaluatethem.Theoverviewseemstoendorsethefindingfromthe Trends series that regular evaluations of student learning services remain relatively rare while studyprogrammes,teachingstaffandresearchactivitiesareevaluatedmorefrequently.(EUA2010:86)

Figure 11: Characteristics of student assessment procedures

They have clear, pre-defined examinationor other assessment methods in place.

They are designed to measure the achievement of theintended learning outcomes and other

programme objectives.

They have clear regulations covering student absence,illness and other mitigating circumstances.

They have clear and publicly availablecriteria for marking/giving grades.

They ensure that assessments are conducted securely inaccordance with the institution's stated procedures.

The administration checks that the assessmentprocedures are followed.

Other

0 50 100 150 200

75.7%

69.8%

66.7%

65.3%

60.8%

48.2%

4.1%

Number of answers


32

Further, with regard to student support services there seems to be a trend that those HEIs whointroducedaQAsystemafter2000arelesslikelytohaveQAofstudentsupportservicesinplacethanthoseinstitutionsthathavehadaQAsystemsincebefore2000.

Figure 12: Institutions offering, monitoring, evaluating and improving their learning resources

Library

Computing facilities(including email account

and internet access)

Human support in the formof tutors, counsellors and

other advisers (in additionto teaching staff)

Laboratories

Learning facilities(language labs, musical

instruments, materialused for classes...)

0 50 100 150 200 250

93.2%

71.2%

69.4%

77.0%

91.3%

68.0%

63.9%

59.8%

55.3%

60.8%

75.8%

92.0%

91.7%

94.2%

66.3%

61.5%

67.3%

61.1%

55.8%

64.9%

Evaluated

Offered

Improved

Monitored

Number of answers

33


Corresponding ESG Summary findings based on the survey

1.1Policy and procedures for quality assurance:Institutionsshouldhaveapolicyandassociatedproceduresfortheassuranceofthequalityandstandardsoftheirprogrammesandawards.Theyshouldalsocommitthemselvesexplicitlytothedevelopmentofaculturewhichrecognisestheimportanceofquality,andqualityassurance,intheirwork.Toachievethis,institutionsshoulddevelopandimplementastrategyforthecontinuousenhancementofquality.Thestrategy,policyandproceduresshouldhaveaformalstatusandbepubliclyavailable.Theyshouldalsoincludearoleforstudentsandotherstakeholders.

Two-thirdsoftherespondentshadaseparateinstitutionalQApolicystatement(67.1%)andinaquarter(24.8%)ofthecasesthequalitystatementwasincludedinanotherinstitutionalpolicydocument.4.5%donothaveanyQApolicydocument.

98.2%oftherespondentsansweredthattheirqualityassuranceprocessescoverteachingandlearning.

1.2 Approval, monitoring and periodic review of programmes and awards:Institutionsshouldhaveformalmechanismsfortheapproval,periodicreviewandmonitoringoftheirprogrammesandawards.

95.5%oftherespondentshavedefinedexplicitlearningoutcomesforallorsomeofthestudyprogrammesand71.7%havemadethempubliclyavailable.

Inmostcases(85.1%)thecurriculumisdesignedbyaworkinggroupconsistingofvariousstakeholdersandultimatelyapprovedatinstitutionallevel(41%)orbyanexternalbody(30.6%).

Processesforreviewingandmonitoringprogrammesvarygreatlyandmostinstitutionsusecombinationsofvariousprocesses.

Involvementofstakeholdersisnotalwaystransparentorstructured.Studentsareinvolvedin40.5%ofHEIsinmeasuringstudentworkloadand,whenaworkinggrouppreparesthecurriculum,50.8%ofHEIsreportthatstudentsarepartofthegroup.

3.5. Implementation of the ESGsTheESGsprovideaEuropeanframeworkforacommonunderstandingofthegenericprinciplesof

qualityassuranceinteachingandlearningandassuchtheyareusuallyexpectedtoguidebothinternalandexternalqualityassurancearrangementsofHEIs.Thus,oneoftheaimsofthissurveyhasbeentogatherquantitativedataonstocktakingoftheimplementationofpart1oftheESGsinHEIs.Table5summarisesthekeyfindingswithregardtotheimplementationoftheESGs.

Table 5: Key findings corresponding to ESGs part 1


34

1.3Assessment of students:Studentsshouldbeassessedusingpublishedcriteria,regulationsandprocedureswhichareappliedconsistently.

75.7%ofinstitutionshaveclear,pre-definedexaminationorotherassessmentmethodsinplace,including,for66.7%,regulationscoveringstudentabsence,illnessorothermitigatingcircumstances.Mostinstitutionshaveamixofseveralfeaturesasmentionedintheguidelines.

82.4%ofinstitutionsmaketheassessmentmethodsandcriteriapubliclyavailablethroughtheirwebsite,studyguidesorequivalent.Inaboutthesamenumberofinstitutions,teachersinformthestudentsaboutthesemethodsandcriteriaatthebeginningofthecourse.

60.8%ofinstitutionsensurethatassessmentsareconductedsecurelyinaccordancewiththeinstitution’sstatedprocedures,and48.2%ofthemhavetheiradministrationcheckingthattheproceduresarefollowed.

1.4Quality assurance of teaching staff:Institutionsshouldhavewaysofsatisfyingthemselvesthatstaffinvolvedwiththeteachingofstudentsarequalifiedandcompetenttodoso.Theyshouldbeavailabletothoseundertakingexternalreviews,andcommenteduponinreports.

71.8%ofinstitutionsconductstudentsurveys,63.1%havespecifiedtheirownrequirementsforcompetenciesofpermanentteachingstaffwhenhiringthem.61.7%offeroptionalpedagogicaltrainingforteacherswhilst26.1%organisecompulsorytraining.

59%ofinstitutionskeeptheinformationonteachers’aptitudesandperformanceconfidentialandavailableonlyattheleadershiplevel(institutionand/orfacultyand/ordepartment).

Inthecaseof22.3%,thelegalframeworkdoesnotforeseethepossibilityofremovinganineffectiveteacher.

1.5Learning resources and student support:Institutionsshouldensurethattheresourcesavailableforthesupportofstudentlearningareadequateandappropriateforeachprogrammeoffered.

Learningresourcesarequitecommonlyoffered,themostcommonbeinglibrary(93.2%)andcomputerservices(90.1%).However,theirregularmonitoringandevaluationisnotquiteascommon.

1.6Information systems:Institutionsshouldensurethattheycollect,analyseanduserelevantinformationfortheeffectivemanagementoftheirprogrammesofstudyandotheractivities.

93.2%oftheinstitutionshavecentralisedinformationsystemsinplacethatincludeinformationontheirteachingmission.Mostcommonlythisinformationincludes:studentprogressionandsuccessrates(87.7%),profileofthestudentpopulation(83.2%)andteacher-studentratioperfaculty/department/institute(65.5%).

1.7Public information:Institutionsshouldregularlypublishup-to-date,impartialandobjectiveinformation,bothquantitativeandqualitative,abouttheprogrammesandawardstheyareoffering.

Themostcommonlypublishedinformationisonqualificationsgrantedbytheprogramme(86.9%),theteaching,learningandassessmentproceduresusedwithintheprogramme(82.0%)andtheintendedlearningoutcomesoftheprogramme(81.5%).Allinstitutionsoffer,withavarietyoffeatures,somesortofinformationontheirprogrammes.

35


4.1. Trends and perceptionsQuality assurance systems largely in place

DatafromtheEQCsurveyshowsthatqualityassurancedevelopments intheircurrentformatareveryrecentphenomena,andimpressiveprogresshasbeenmadeduringthelastdecade.Thisdevelopmentcoincideswith the European levelpolicydevelopments andwould indicate thatHEIshavemobilised torespondtotheQAframeworkputinplaceatthepolicylevel.In2003,theMinistersinhighereducationagreedthateachcountryshouldsetupanationalqualityassuranceframeworkandthatHEIsshouldhavetheirowninstitutionalQAsystemsandtwoyearslatertheESGswereadoptedprovidinggenericprinciplesforbothinternalandexternalQAprocesses.Consequently,since2005,HEIshavebeenincreasinglyworkingontheirQAsystems.

Hence, the Bologna Process and subsequent changes in the national frameworks have clearlypromotedagrowingawarenessandinsightintotheneedforQAinteachingandlearning.WhileMinisterialCommuniquéshavenotdefined indetailhowqualityassurance isunderstood inteachingand learning,the ESGs, adopted and published in 2005, have done so. In addition, various European level projects,organisations3aswellasevents4,havefurtherendorsedacommonperceptionoftheconcept.WhiletheESGsseemtohaveestablishedthis frameworkforteachingandlearning,wearetemptedtoconclude–basedontheresponsestoquestionsdealingwithotheractivitiesofaHEI–thatthedevelopmentofasharedunderstandingofqualityassuranceingeneralissomethingtobeworkedon.

ThefindingsdoindicatethatwhenitcomestotheactualprocessesandsupportstructuresinQA,agreatvarietyofcombinationsexists,whichendorsestheviewthattherearenoone-size-fits-allsolutionswhendecidingwhatan institutionalQAsystemshouldconsistof.AsvariedasstructuressupportingQAsystemsmaybe–suggestingthattheremaybeoverlapsandadditionalstructures,maybeeveninefficientdoublingsandcouplings–theyallareproofthatHEIsareworkingtowardstheirQAsystems.

Trends 2010notedthatrecentdevelopmentsininternalqualityprocessesinteachingandlearninghave not necessarily been linked to European QA developments, particularly to the ESGs. HEIs seemedto be mostly responding to the external QA requirements imposed on them, and these requirementsdidnot includethepartoftheESGsthatappliestoHEIs,norwasthe linkbetweenthemandEuropeanQA developments explicit. (EUA 2010: 63) The EQC survey responses indicate that a clear majority ofrespondents started implementing a quality assurance framework within their institution as from 2000.WhereasthisprobablyhashappenedasaconsequenceofthediscussionsanddecisionsatEuropeanlevelandnewlegislationstobeadoptedatnationallevels,itappears,indeed,thatmostinstitutionsdonotapplytheESGsasanintegratedwhole,buttendtoshowinterestinoneorseveralaspectsofthem.

4. Key trends and further reflection

3OrganisationssuchastheEuropeanAssociationforQualityAssuranceinHigherEducation(ENQA)atagencylevel,theEuropeanUniversityAssociation(EUA)andtheEuropeanAssociationofInstitutionsinHigherEducation(EURASHE)atinstitutionallevel,aswellasvariousdiscipline-basedassociationsatEuropeanlevel.

4SuchastheEuropeanQualityAssuranceForum,ayearlyeventgatheringtogethervariousstakeholdersinvolvedinqualityassuranceinhighereducation.

“Quality was introduced a long time ago even if it was not felt and stated as quality. The development and formalisation of quality processes is done step by step. It is diffi cult to measure the reality of the quality in a big institution.” - Respondent to the survey


36

Yet developing quality culture takes time and effort

AsreferredtoinChapter2,qualitycultureiscloselyrelatedtotheorganisationalcultureofrespectiveHEIs and, as such, always exists in the form of values, beliefs etc. Although based on the quantitativeresults related toqualityassuranceprocesses,no far-reachingconclusionscanbemadeon thestatusofqualitycultureintherespondingHEIs;numerousopenfieldcommentsindicatethattheultimategoalinmanoeuvringQAprocessesispursuinganimprovedqualityculture.

Hence, the responses demonstrate that institutions are building quality-related processes andstructures,butthatinsomecases,sometimesbecauseofthelegalframeworkencasingthem,institutionsmaynothaveyetachievedthekindofqualitycultureforwhichtheyarestriving.Evidently,thegoalofHEIsisagenuinequalityculturesupportedbywell-functioningQAprocesses.Nevertheless,thestateofplayatthemomentlooksasifHEIshavefoundawayofimplementingQAprocesses,butthatprocessesmaynothavenecessarilyquestioned,challengedorchangedHEIs’wayofcarryingouttheiractivities.Theeffectivenessandefficiencyoftheexistingqualityassuranceprocessesneedfurtherqualitativeexaminationthatasurveydoesnotallowtocapture.

Ensuring the participation of all stakeholders

Inthelightofourresults,itappearsthattheinstitutionalleadershipisoftenatleastformallyinvolvedintheQAprocessesandalsointhedevelopmentofqualitycultureinthemajorityoftherespondingHEIs.Nevertheless,onecouldarguethatthepercentageofrectorsorvice-rectorsinchargeofQAissuesshouldbehigherinordertodemonstrateatrueinstitutionalcommitmenttotheenhancementofquality.RelyingsolelyonorganisationalstructurestosustaincommitmentcancauseQAprocessestolosetouchwiththereality of academic work and thus, reduce the processes to bureaucratic exercises that take place in avacuumwithnofollow-upactivities.

Butatop-downapproachaloneisnotsufficient;thenotionofsharedvalues,beliefsandcommitmentsimpliestheparticipationofthewholecommunity.Furthermore,consideringthatoneoftheunderpinningprinciplesoftheEuropeanQAframeworkistheparticipationofstakeholders–bothinternal,suchasstudentsandstaff,andexternal–inthequalityassuranceprocesses,thissurveyclearlyshowsthatmoreworkstillremainstobedoneinthisrespect.

Although,theresponsesindicatethatacademicstaff–supportstafftoalesserextent–andstudentsarequiteoftenformallyinvolvedinQAprocesses(seeChapter3.2),theresults,whichatthisstagearearesultofaquantitativeanalysis,donotindicatehowactivetheinvolvementisandhowinfluentialitis.Inthiscontext,withregardtostudentparticipationweshalltherefore,refertotheconclusionsoftheEuropeanStudents’Union:

Overall student participation in QA has progressed since 2007. However, the analysis of the answers shows the serious gaps in terms of formal participation in decision-making processes and a rather un equal rate of participation in the different processes associated with QA across different countries. We can conclude that, in spite of students being accepted as a part of the follow up rather than technical processes, they still face reluctance towards their involvement in the decision-making process. This statement is valid on a case-by-case basis, and it is not currently possible to establish a general trend on the location or scale of the phenomenon.(ESU2009:49)

“Implementation takes a lot of time, especially when, traditionally, faculties are free to develop teaching and research. To implement a quality culture a number of people engaged in developing concepts and discussing the benefi ts and gains of QA are needed.” - Respondent to the survey

37


Furthermore,withregardtostudentparticipationinparticular,wehavetoconsiderthatstudentsgivetheirfeedbackon,forexample,coursesandmoduleswiththenextgenerationofstuden

examining quality culture: part 1 – quality assurance ... quality culture part i quality... ·...

Documents

Investigating Lighting Quality: Examining the Relationship ... · Investigating Lighting Quality: Examining the Relationship between Pedestrian Lighting environment and Perceived

EXAMINING THE RELATIONSHIP AMONG QUALITY ASSURANCE

Lab Exercise: Examining Water Quality: Most Probable ...faculty.sdmiramar.edu/dtrubovitz/micro/microhandouts/water.pdf · Lab Exercise: Examining Water Quality: Most Probable Number

Quality of Math Instruction Matters: Examining Validity

Examining Data Quality

Examining the Product Quality Attributes That Influences ...ijbssnet.com/journals/Vol_3_No_23_December_2012/23.pdf · Examining the Product Quality Attributes That Influences Customer

Quality Culture

Examining Transcendentalism Through Pop Culture By:

Examining Quality Culture Part II

the peter tomassi essay colt culture: examining

Examining Quality Inclusive Practices in Early Childhood

Article Culture change in elite sport performance teams: Examining

Examining the Region through Geography, History, Culture & Environment

ADVANCING QUALITY TEACHING FOR ALL SCHOOLS Examining …

Culture Or Quality

Examining Quality Culture: Part 1 – Quality Assurance Processes in

Re-Examining the Problem of Slavery in Western Culture

Jamba Juice: Examining New Business Strategies from Juice Cleanse Culture

Examining Paratextual Theory and its Applications in Digital Culture

Skeuomorphs and Spandrels: Examining the Interaction of Culture and Design

Examining the relationship between organizational culture ... · Examining the relationship between organizational culture and performance 261 utilizing CVF is a hybrid of four cultural

Developing; Quality Culture

FACULTY OF MEDICINE Quality Assurance Study Examining the

Quality Culture - MDICmdic.org/.../2016/10/Final.V-Cruz-Leaderships-Role-in-Quality-Culture.pdf · QUALITY CULTURE Why have a culture of Quality? Culture of Quality Product Quality

Does The Leader Fit The Place? Examining Leader-Culture

Examining the Quality of Speech and Language Therapy

Examining a Culture of Delay and Adjournments: Criminal

Toward Efficacy: Examining The Reported Impact of Quality

Quality CultureQuality Culture...Quality Culture “True Quality Culture – an environment in which employees not only follow quality guidelines but also consistently see others taking

Examining Culture and School Performance

Air Quality and Cement Production: Examining the

Hochschulentwicklung Theoretischer Input Thesen aus den … · 2018-10-26 · Loukkola, Tia/Zhang, Thérèse (2010). Examining Quality Culture Part I. Quality Assurance Processes

Examining the Effect of Organizational Culture on the

Data for Student Success Examining Student Work to Inform Instruction “It is about focusing on building a culture of quality data through professional

1 Summarize from: “Examining Trust in Information Technology Artifacts: The Effects of System Quality and Culture”, Journal of Management Information Systems,