How Student Assessment Increases the Learning Process at Catalan Universities

Elena CANO (Coord.)Nati CABRERA

Xavier GIMENOGeorgeta ION

AIDA

30th ANNUAL EAIR FORUM

24-27 August 2008

Copenhagen, Denmark

Index

Aim Objectives Backround Methodology Results Conclusion

Aim

To analyze the assessment practices currently in use, placing special emphasis on the ones considered to be excellent, in order to identify their strengths and to propose ways of disseminating specific actions that are likely to enhance our assessment activity in general..

Objectives

To define and characterize assessment practices of excellence.

To analyze and compare the views of teachers and students from public and private universities regarding the proposals for assessment processes;

To explore teachers' conceptions and planning processes that lead to the implementation of assessment activities; to investigate the variables that best explain what quality assessment practice should be.

To promote the use of teaching-learning processes with the most successful assessment systems, in terms of students' and teachers’ satisfaction with the learning achieved.

Background (I)

Educational assessment a recent discipline. evolved to evaluate institutions, programs, and professors.

Authors defined the objectives-driven model (Tyler) and the model of responsive evaluation (Stake), Parlett and Hamilton’s illuminative model, or Eisner’s artistic criticism model

Allal, Cardinet and Perrenoud (1979), Lafourcade (1985), Heywood (1989) and the Joint Committee (1988).

Background (II) The evaluation is an effect of the learning process,

but simultaneously it is a reason for it (Allen, 2000) and Brown, Glassner, 2003).

Assessment must constitute an opportunity for learning and must be used, and promoting these competencies in all the students- formative dimension (Hall, 2003; Kaftan, 2006).

Assessing competencies obliges us to use a diversity of instruments and to involve different agents in the process (McDonald et al., 2000; Stephen and Smith, 2003; Scallon, 2004; Gerard, 2005; Laurier, 2005; De Ketele; 2006).

Background (III)

The assessment has to be consistent with the rest of the elements of the formative design and integrated in it (Segers, 2001; Gijbels, 2005; Jennings, Rienzi, Lyda, 2006).

Assessment must be shared (Giles, Martin & Bryce (2004).

The assessment must heighten students’ awareness of their level of competencies (Boekaerts & Zeidner,

2000).

The 20`s characteristics of a good evaluation practice

PrecisePrecise

IndividualizedIndividualized

ContinuousContinuous

EthicalEthical

CollectiveCollective

PrecisePrecise

IndividualizedIndividualized

ContinuousContinuous

EthicalEthical

CollectiveCollective

ExplicitExplicit

FlexibleFlexible

DifferentDifferent

CorrectCorrect

ContextualizedContextualized

ExplicitExplicit

FlexibleFlexible

DifferentDifferent

CorrectCorrect

ContextualizedContextualized

Consistent

Proportionate

Accreditative*

Formative

Diagnostic

Consistent

Proportionate

Accreditative*

Formative

Diagnostic

ViableViableMeta-evaluativeMeta-evaluative

MotivatingMotivatingFriendlyFriendly

FastFast

ViableViableMeta-evaluativeMeta-evaluative

MotivatingMotivatingFriendlyFriendly

FastFast

Case studies UniversitUniversit

y of y of BarcelonaBarcelona

Autonomous Autonomous University of University of

BarcelonaBarcelona

Polytechnic Polytechnic University University

of Cataloniaof Catalonia

Open Open University University

of of CataloniaCatalonia

University University Rovira i Rovira i VirgiliVirgili

University University Pompeu Pompeu FabraFabra

Social Social Sciences Sciences

XX(Law)(Law)

Technologi-Technologi-cal Sciencescal Sciences

XXPharmacyPharmacy

XXEngineeringEngineering

XXMicro-Micro-BiologyBiology

Health Health SciencesSciences

XXPsychologyPsychology

Education Education Sciences Sciences

XXSchool School

TeachersTeachers

The extent to which they present features that the investigation group considers as determinants of good evaluation practices.

The teachers’ profile, based on aspects such as students’ satisfaction, leadership, membership of an innovation group, and the number of publications on the teaching/learning process and their impact factor.

A range of specialities, in order to be representative of different disciplines.

Use of innovating teaching/learning methodology such as problem-based learning, project-based learning, autonomous study and the assessment techniques used in the cooperative learning process.

Different types of formative experiences – e.g., face to face or e-learning.

Curriculum design.

Cases criteria

Data sources

Research Research Technique Technique

Information Information sources sources

InformationInformation

Interviews with Interviews with teachersteachers

Teachers Description of evaluation system

Explanation of the reasons for adopting and implementing the present evaluation methodology. Understanding the multidimensionality of the assessment process.

Questionnaires Questionnaires for students for students

Students Opinions of the characteristics of the evaluation system.

Students’ satisfaction with their evaluation system.

Documentary Documentary analysisanalysis

Curricular program

Assessment system design (exams, activities, practice handbook, portfolios, etc.)

Other

Correspondence between program components (objectives, contents, methodology, evaluation).

Correspondence between assessment tests and objectives, contents and methodology .e.g.

Teacher’s feed-back and contribution to students’ continuous assessment.

Results University

of Barcelona (UB)

Autonomous University of Barcelona (UAB)

Polytechnic University of Catalonia (UPC)

Open University of Catalonia (UOC)

University Rovira i Virgili (Tarragona) (URV)

University Pompeu Fabra (UPF)

Consistent Proportionate Accreditative* Formative Diagnostic Explicit Flexible Different Correct Contextualized Viable Meta-evaluative

Motivating Friendly Fast Precise Individualized Continuous Ethical Collective

Global results

A good evaluation has to be: formative and consistent diagnostic, flexible, and individualized correct

Results of each case study (I)

University of Barcelona (UB)University of Barcelona (UB) Students prepare a set of multi-choice questions,

and act as monitors of other pupils in the laboratory. Professors use diagnostic assessment (which is not

perceived as such by the pupils). “Surprise” tests are set during the course (which,

regardless of the grades obtained, helped to improve students’ final performance).

The teaching team constantly discussed their assessment techniques.

Results of each case study (II)

Open University of Catalonia (UOC)Open University of Catalonia (UOC)- Continuous assessment tests which students send

to their tutors during the course and which are returned both in a group (in the virtual classroom) and individually.

- The course chosen (Procedural Law) are its consistency with the objectives of the course and the fact that it is formative.

- The close relation between the course contents and the assessment activities, and the extent to which these activities correspond to the aims, especially the learning of certain procedures.

Results of each case study (III)

Polytechnic University of Catalonia (UPC)Polytechnic University of Catalonia (UPC)- Project work (in which a first version is presented and

discussed and improvements are then introduced in a second version).

- Continuous assessment, in which work is handed in (continuous assessment has been shown to increase both the number of students who sit the end-of course exams and the number who pass).

- Self-regulatory practices, in which each pupil writes an account of how a particular problem was resolved and how it should have been resolved, noting the mistakes to be avoided in the future.

- The teacher supports this kind of assessment in order to promote autonomous, reflective learning.

Results of each case study (IV) “Pompeu Fabra” University (UPF) Pompeu Fabra” University (UPF) - - The role of the assessment procedure in certifying

students’ knowledge. - Teacher and students stressed that the assessment

allowed them to know whether the basic concepts of the course had been mastered.

- The teacher considered that the procedures (of application, of reflection) and the transversal competencies are developed and assessed, but the students considered that the assessment was based more on the conceptual content.

- The varied and continuous nature of assessment is another emerging dimension that should be borne in mind.

Results of each case study (V)

Rovira I Virgili University (URV)Rovira I Virgili University (URV) Course based on portfolio work in which the pupils hand in

essays based on their learning and select the ones they consider best reflect their process of knowledge building.

We stress its consistent, formative and continuous nature. It is consistent with the teacher’s conceptions. It is also consistent with the teaching methodology, which

combines presentation sessions, the supervision of teamwork, oral presentations, autonomous work and guided practice for the case analysis.

The incorporation of self-assessment.

Results of each case study (VI)

Autonomous University of Barcelona (UAB)Autonomous University of Barcelona (UAB)- Portfolios- consistent with the dynamic of the course

and the classroom practices.- TThe process of negotiation of the assessment with

the classroom group and the individualized form.- Having an attitude that promotes initiative,

responsibility and systematic analysis of pre-conceived ideas are key competencies for student’s academic development and for success in the professional environment in the future.

Conclusion An appropriate evaluation design may help to bring the

formative results into line with the needs and expectations of prospective employers

Improving assessment improve educational performance.

Assessment should not be considered in isolation from the other elements of the curriculum.

Innovations in the field of assessment may help to open up new ways of approaching education and the teaching-learning process, and will encourage the use of new methodologies.