The European Peer Review Procedure

in Dutch VET(part 1)

Willem de RidderROC Aventus

Peer Review 2

What is a Peer Review?

• External evaluation– External group of experts, Peers– Invited by the institution

for a formative evaluation

• Advantages and benefits:– Quality assurance– Quality development– Enhancing accountability/governance– …

Peer Review 3

Peer Review 4

Desired conditions

• Commitment

• Object for evaluation

• Adequate documentation (self report of source grid)

• Relation with the Quality approach

• Tasks and responsabilities are clear– Peer Review Facilitator

Peer Review 5

Choosing quality areas

• Which quality areas show a good performance?

• Which quality areas (might) show problems?

• In which quality areas are there developments?

• Which quality area’s are most important to our stakeholders?

• + What is possible in 2 days?

Peer Review 6

Which CSF are important?

• Reliability– correct– complete

• Timely• Flexibility• Efficiency• Customer focus • …

Peer Review 7

Selection of the Peers

• No: colleagues from own institution• Yes: representatives of previous or following

education, companies, institutions etc.– Obligation: Training & procedure (forms)

– Thus» Objectivity » No discussion with the turkey about the

Christmas meat

• A relation with governance

Peer Review 8

Operationalisation

Step 1Indicator

Step 2Registration /observation

For example

Management shows commitment

remarks in minutes

mentioned importance in interviews

# minuten time Times on agenda

Look at agenda’s Observeinterview:- How do you …?- How important is it for you …?

Look in minutes

Peer Review 9

Operationalisation

• How?

– Decide what is the right indicator for the criterion

– Decide how to observe/measure this

– Decide which ‘scores’ are useful and which are not

Peer Review 10

We use a normative model

• An example …. The intake of students

• Each process has 4 cycles:1. Input (Which measures have been taken?)

2. Flexibility (Which measures have been taken?)

3. Learning capacity (Which measures have been taken?)

4. Output (Which measures have been taken?)

1 32 4

Peer Review 11

The methodology

• All choices must have underlying arguments

• The Review must be reproducible– Keep the attachments, the work documents and the

forms

Peer Review 12

During review

• Analyse the findings immediately after each activity/interview and discuss and record them a.s.a.p.

• Build up the review results by adapting the overview after each relevant event (use aflip over/flip chart!)

– Triangulation!• The opinion of students is very important for we are a

student centered organisation.

Peer Review 13

Peer Review 14

The Peer Team

• The Peer Coördinator (chair)• The Evaluation expert /Peer(secretary)

– They are involved in all reviews in the institute– The evaluation expert writes all the peer reports

• Two Peers– Asked for their expertise by the institution for their expertise in

the chosen quality areas

» Or– A transnational peer– An expert in gender mainstreaming

• All Peers must be able to work according the Peer Review Procedure (and work in tandems)

Peer Review 15

Types of Peer Reviews

• Problem-finding Review– What is the problem?– Why is this a problem?

• Diagnostic Review– A judgement about the correctness of the diagnosis of a

problem

• Design/blueprint Review– Will the design work? (problem-finding and diagnosis is

already done)

Peer Review 16

A useful Review for the organisation is ...

• ...relevant

• ...solid

• ...efficiently obtained

Peer Review 17

Which additional certainty can a Review provide?

• Additional:– Is there a management problem?– Is the diagnosis correct?– Is it possible to implement the designed

solution?– Is the designed solution efficient?– Is the designed solution effective?– Will there be adequate control during

implementation?

Peer Review 18

About giving advice to the institution

No• Procedure x is not followed

– ADVICE: follow procedure X!

Yes• Not following procedure X is caused by :

– Obscurity/lack of clarity in the procedure– lack of clarity in the division of tasks– Insufficient preparation– ….

Advice can only been given after investigating the problem and not just on professional judgement alone!

Peer Review 19

Disclosure van sources

Persons

Reality

Documents

Interview

Observation

Content analysis

Instrument

Peer Review 20

Analysis of documents

• Focussed reading:

• Relation to interview questions

• Reading assignments– quantitative: look for the relevant information– quantitative: look for the frequency of the

relevant information

Peer Review 21

Before the interview

• The introduction– Who we are,– Why we are here, who invited us,– The subject – The time of the interview – Which criteria for selection?– How the information will be used, – Why should the interviewee cooperate?

Peer Review 22

After the interview

• Summary

• Thanks

• What happens next?