NVAO’s external quality NVAO’s external quality assurance proceduresassurance procedures

Ann Van Neygen

La Rochelle11th of May 2012

2

‘Higher education’

‘Government’ ‘Society’

Enhancement

Accountability Information

Recognition

FundingSupport

Appreciatio

n

Value for €

Proof of useQuality

Access

(Consumer) protection

Support

• Stimulate quality culture institutional audit

• Commit professionals / Increase academic ownership

programme assessment

• Reward earned trustlimited programme assessment

• Stimulate HE to aim above thresholdaccredit as satisfactory, good, excellent

3

• Institutional audits • Initial accreditation

• (New) programmes that want to offer a recognised degree

• All bachelor and master’s programmes, including associate degrees

• Accreditation• programmes that already offer a recognised degree

• Publication of decisions (and officially recognised degrees)

4

Publication

Report

Official register

(Initial) accreditation• Concentrate on content

(& focus on performance) learning outcomes

Audit• Focus on policy & practice

re. internal QA system teaching and learning

5

Institution

Programme Report

6

1. What is the vision of the institution regarding the quality of the education it provides & of the development of a quality culture?

2. Does the institution have an appropriate policy to realise this vision?

3. How does the institution measure the degree to which this vision is realised?

4. How can the institution demonstrate that it systematically improves the quality of its programmes?

5. Does the institution have an effective organisation and decision-making structure regarding the quality of its programmes?

Positive / Negative / ConditionalPositive / Negative / Conditional

• Installed by NVAO

• At least 4 members, including 1 student

• The panel commands administrative (Board), educational and audit expertise, is acquainted with developments in the higher education sector at home and abroad, and is authoritative

• One of the members with Board expertise will act as chair

• The panel is independent (its members have had no ties with the institution over at least the past 5 years)

• Not part of panel: NVAO process co-ordinator and secretary (also independent of the institution)

7

• Conversation on management level

• Institutional profile: results of previous accreditation

• Critical reflection by HEI (max. 50 pages + appendices)

• Site visit• 1st visit (1-2 days): exploration of issues, first interviews and

feedback, and panel decision on topics for audit trails

• 2nd visit (2-3 days; 2 to 4 weeks after 1st visit): issues emerging from the meetings & documents studied during 1st visit; audit trails; at the end feedback to HEI on overall judgement and considerations

• Audit trails: panel follows the trail from the institutional level to the implementation of policy and/or the management of problems or vice versa

8

• Judgement on standards:• Meets, does not meet or partially meets the standard

• Overall judgement: • Based on its vision of the quality of the education

provided, the board of the institution has implemented an effective quality assurance system, which enables it to guarantee the quality of the programmes offered.

• Positive, negative or conditionally positive

9

• If positive judgement then NVAO approval valid for 6 years

• If conditionally positive then explicit statement of relevant conditions and positive judgement within 1 year:

• If conditions are met then NVAO approval for 6 years• If conditions are not met then approval expires

• If negative judgement then approval is withheld for at least 3 years:

• comprehensive programme assessments• additional assessments for those programmes which

have already had limited assessment

10

• For each standard:• Findings

• Considerations

• Judgment

• Improvement suggestions

• Summary for wider audience

• Report and decision is published on the site of NVAO

11

12

Extensive programme assessment

Limited programme assessment

Yes No

Is this institution recognised?

Recognition procedure

Has this institution undergone an institutional audit?

Yes No

++ decision -- decision

13

Is this programme on the register/ recognised?

(Lim./Compr.) initial accreditation

procedure

(Lim./Compr.) accreditation

procedure

-- decision ++ decision ++ decision -- decision

Extensive programme assessment

Limited programme assessment

Satisfactory - Good - Excellent

Deleted from register Included on register

Impr

ovem

ent

perio

d

14

Limited programme assessment

Extensive programme assessment

Assessment of•content of the programme•focus on performance

Detailed assessment of•content,•policy and•procedures

3 standards 16 standards

Themes

1. Intended learning outcomes 2. Programme and staff quality

3. Achieved learning outcomes & Learning assessment

1. Intended learning outcomes 2. Programme3. Staff4. Services5. Quality assurance system6. Achieved learning outcomes &

Learning assessment

Conclusion: unsatisfactory, satisfactory, good, excellent quality

• At least four members, one of whom is a student;• at least two authoritative subject-matter experts

• At least one with teaching experience at relevant level;

• aware of latest international developments in the discipline

• expertise in the professional field (where applicable)

• educational expertise

• assessment or audit expertise;

15

• The panel is independent• no ties with the institution offering the programme for

at least five years;

• an independent, external secretary trained and certified by NVAO;

• signed declarations of independence and codes of conduct;

• panel composition and declarations of independence will be published and made public;

• the parties involved in the assessment can report any matters that could affect the independence to NVAO.

16

17

Programme Module x Module y Module z ...

Intended LOs

Intended LOs

Intended LOs

Intended LOs

...

(C. &) LOs (C. &) LOs (C. &) LOs

Assessment Assessment Assessment ...

(ECTS) Course catalogue

Achieved?

Achieved?

Achieved?

Achieved?

…

• Who ?• New programmes: NVAO• Recognised programmes: evaluation organisation

• Information dossier (max. 50 pages + appendices)

• Site visit• Conversations with management, teachers, students, work

field • Overall judgment and considerations• Short feedback to HEI

18

• Judgement on standards:• New programme: Satisfactory, unsatisfactory

• Recognised programme: Satisfactory, good, excellent, unsatisfactory

• Overall judgement: • New programme: Satisfactory, unsatisfactory

• Recognised programma: Satisfactory, good, excellent, unsatisfactory

19

• New programmes:• If positive judgement then NVAO approval valid for 6

years

• If negative judgement: no approval

• Recognised programmes:• If positive judgement then NVAO approval valid for 6

years

• If negative judgement:• Limited approval• Conditions for improvement• If conditions are not met approval expires

20

• Generic (basic) qualityThe quality that in all reasonableness could be expected of a bachelor’s or master’s programme within higher education, and this from an international perspective.

• UnsatisfactoryThe programme does not provide generic quality.

• SatisfactoryThe programme provides generic quality.

• GoodThe programme is of notably higher quality than generic quality.

• ExcellentThe programme is of a quality very much above generic quality and fulfils an exemplary role for other relevant programmes.

21

• For each standard:• Findings

• Considerations

• Judgment

• Improvement suggestions

• Summary for wider audience

• Report and decision is published on the site of NVAO

22

23

Application

Allocation

Admissibility

Analysis

Proposed decision

Final decision

Publication

Letter Institutional details Programme details Expert panel report By managing director

First handler=policy advisor

Second handler Portfolio handler

= executive board member

Screening application Panel report

All standards judged Proof of payment Completeness

Template for Analysis Substantiation judgements

(Objective) findings (Subjective) considerations

Quality panel report ≠ quality programme

Decision NVAO Procedure followed Panel composition Essential programme details Substantiation decision Status and period of validity

NVAO website Netherlands: Centraal Register

Opleidingen Hoger Onderwijs (CROHO) Flanders: HE Register

24

25

• Open and inclusive approach• One framework for all assessments in a given type of

procedures• E.g. one assessment framework for all types of initial

accreditation

• Fit for purpose (~appropriate burden)

• Respect for institutional autonomy • Internal quality assurance system

• Assessment in line with institutional choices on the basis of the relevant framework

• Programme level• Institutional level

26?

• Generic, descriptive standards≠ Narrow, prescriptive and highly formulated

Broad acceptance

• Focus on what should be done (~fitness of purpose)

≠ How they should be achievedDo not dictate practice

• Focus on teaching & learning (~education)

• Incl. academic standards ≠ Research

• Incl. professional orientation ≠ Service to society

27

• Central documentation for External QA• Self-contained document – relatively brief

• Basic information• Institution / programme details

• Presents perspective shared by (internal) stakeholders• Addresses each standard in relevant framework

• rationale, practice, ambition

• Provide insight into strengths & weaknesses• Maximising strengths?

• Converting weaknesses into strengths?

• Mandatory annexes

28

• Direct window on what programmes aims to do & what a programme actually does• Less focus on policy, process & procedures

• But process & procedures are easier to assess

• Facilitates international transparency & comparability• E.g. International benchmarking

• Open to all types of teaching & learning• Technology Enhanced Learning / Distance learning

• Work-based learning / Assessment of prior learning

• Provides a common language for HE & stakeholders• Teaching & Learning

29