assessment and evaluationmvalcke/cv/6_ae_efe_assess.pdf · advance organizer • where is the...
TRANSCRIPT
![Page 2: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/2.jpg)
Structure • Consider evidence-base about
assessment and evaluation • Definition assessment and evaluation
– Activity 1: develop definitions
• Trends in adult A & E • Self assessment and peer assessment • Rubrics
– Activity 2: develop a rubric 2
![Page 3: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/3.jpg)
Advance organizer
• Where is the evidence about assessment and evaluation?
Hattie, 2009
![Page 4: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/4.jpg)
![Page 5: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/5.jpg)
5
![Page 6: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/6.jpg)
Assessment & Evaluation
Activity 1: define • What is assessment? • What is evaluation?
![Page 7: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/7.jpg)
Definition assessment & evaluation
• Three elements: – Measuring: get data from student about his/her
answer or behavior (test, exam, task) – Evaluating: what is the value of the “answer/
behavior”? Separately – Scoring: what score will be attributed to a
quality level in the answer or behavior?
![Page 8: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/8.jpg)
Assessment en Evaluation in adult education
• Characteristics of adult learners! – Voluntary learning – Informal – Self-directed – personal goals
• Motivational dimensions: autonomy, belonging and competence
8
![Page 9: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/9.jpg)
Trends in A & E of adults
• Major changes in the place, role, function, focus, approach, tools, … of A & E in adult education
• Trends in WHO is responsible for the evaluation and in the TOOLS being used
![Page 10: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/10.jpg)
Trends in A & E
• Trends share characteristics: – Focus on “behavior” – Focus on “authentic” behavior – Foicus on “complex” behavior – Explicit “criteria” – Explicit “standards” – Need for concrete feedback – Focus on “consequential validity”
Gielen, Dochy & Dierick (2003) Examples
![Page 11: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/11.jpg)
Rubrics
11 http://www.siue.edu/~lmillio/IT598/Resources/04_assessment/Rubrics%20and%20Adult%20Learners.pdf
![Page 12: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/12.jpg)
Self assessment
12 Boud, 2005
![Page 13: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/13.jpg)
Trends in A & E
Trends according to Fant et al. (1985, 2000): • Assessment centres • Self and Peer assessment • Portfolio assessment • Logbooks • Rubrics
![Page 14: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/14.jpg)
Individual learner Group learner
External institution
Teachers Expert teacher
Assessment system Institutional level
Who is responsible for A & E?
![Page 15: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/15.jpg)
Self- and peer assessment
• Learn about your own learning process. • Schmitz (1994): “assessment-as-learning”. • ~ self corrective feedback • See experiential learning cycle of Kolb. • Boekaerts (1991) self evaluation as a
competency. 15
![Page 16: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/16.jpg)
• Development of metacognitive knowledge and skills (see Brown, Bull & Pendlebury, 1998, p.181).
• Freeman & Lewis (1998, p.56-59): developing pro-active learners
16
![Page 17: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/17.jpg)
17 The Learning Cycle Model
![Page 18: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/18.jpg)
Learning to evaluate
• Develop checklists • Give criteria • Ask to look for quality indicators. • Analysis of examples good and less
good practices: develop a quality “nose”
18
![Page 19: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/19.jpg)
Learning to evaluate • Freeman & Lewis (1998, p.127) :
• Learner develops list of criteria. • Pairs of learners compare listed criteria. • Pairs develop a criterion checklist. • Individual application of checklist. • Use of checklist to evalute work of other learner. • Individual reworks his/her work. • Final result checkeed by teacher and result compared to learner
evaluation. • Pairs recheck their work on the base of teacher feedback.
19
![Page 20: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/20.jpg)
Learning to evaluate
• Peer evaluation is not the same as Peer grading
• Part of score could build on accuracy of self/peer evaluation and self-correction
• Example: peer evaliuation and feedback in writing
20
![Page 21: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/21.jpg)
21
Collaborative writing space
![Page 22: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/22.jpg)
22
![Page 23: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/23.jpg)
23
![Page 24: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/24.jpg)
Feedback
24
![Page 25: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/25.jpg)
25
![Page 26: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/26.jpg)
Feedback on feedback
26
![Page 27: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/27.jpg)
27
![Page 28: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/28.jpg)
Analysis example
• Authentic task • Collaborative setting • Clear criteria • Peer feedback • Re-writing • Feedback on feedback
28
Fits adult learner
![Page 29: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/29.jpg)
Trends in A & E
Trends (Fant et al., 1985, 2000): • Assessment centres • Self and Peer assessment • Portfolio assessment • Logbooks • Rubrics
![Page 30: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/30.jpg)
Rubrics
30
Quality indicators
Criteria
Quality levels
![Page 31: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/31.jpg)
Rubrics • Rubric: assessment tool for a qualitative
assessment of complex authentic activity. – A rubric builds on criteria that are enriched
with a scale that help to detremine mastery levels
– For each mastery level, standards/benchmarks can be defined.
– A rubric helps both the trainer and the trainee in view of what is expected at process/product level.
![Page 32: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/32.jpg)
32
![Page 33: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/33.jpg)
33
![Page 34: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/34.jpg)
34
![Page 35: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/35.jpg)
Critical thinking rubric
http://academic.pgcc.edu/~wpeirce/MCCCTR/Designingrubricsassessingthinking.html
![Page 36: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/36.jpg)
Activity 2
• Develop a rubric for assessing the quality of a conference contribution: – Define criteria – Define at least 4 quality levels with quality
indicators
36
Quality indicators
Criteria
![Page 37: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/37.jpg)
Research about rubrics • Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and
educational consequences. Educational Research Review, 2, 130–144.
– Reliable, especially if rubrics are analytic, topic-specific, and complemented with exemplars and/or rater training;
– Rubrics promote learning. Main reason: rubrics make expectations and criteria explicit, which also facilitates feedback and self-assessment.
![Page 38: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/38.jpg)
Conclusions
• Adult learning: assessment & evaluation special position
• Trends in assessment & evaluation align with adult learning – Self assessment and peer assessment – Rubrics
![Page 39: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/39.jpg)
Assessment and Evaluation
39
Martin Valcke [email protected]
http://users.ugent.be/~mvalcke/CV_2012/
![Page 40: Assessment and Evaluationmvalcke/CV/6_AE_EFE_ASSESS.pdf · Advance organizer • Where is the evidence about assessment and evaluation? Hattie, 2009 . 5 . Assessment & Evaluation](https://reader036.vdocuments.net/reader036/viewer/2022081405/5f0b51677e708231d42fec42/html5/thumbnails/40.jpg)
Informative websites • Overview tools, examples, theory, background, research: http://
school.discoveryeducation.com/schrockguide/assess.html • Critical thinking rubrics: http://academic.pgcc.edu/~wpeirce/MCCCTR/
Designingrubricsassessingthinking.html • Rubric generators: http://www.teach-nology.com/web_tools/rubrics/ • Intro on interesting rubric sites: http://web.njit.edu/~ronkowit/teaching/rubrics/
index.htm • Rubric APA research paper: http://web.njit.edu/~ronkowit/teaching/rubrics/
samples/rubric_apa_research.pdf • General intro and overview:
http://web.njit.edu/~ronkowit/teaching/rubrics/index.htm