claus brabrand (itu) dmlf Årsmøde 2007 – keynotesep 20, 2007 constructive alignment claus...

Claus Brabrand (ITU) DMLF Årsmøde 2007 – keynote Sep 20, 2007

Constructive Alignment

Claus Brabrand, Ph.D.((( brabrand@itu.dk )))

Associate Professor,IT-University of Copenhagen Denmark

DMLF'07 – Invited Talk

[ 2 ]Claus Brabrand (ITU) DMLF Årsmøde 2007 – keynote Sep 20, 2007

Outline

Warm up exercise: Post-It focus exercise...

Constructive Alignment: The Theory of Constructive Alignment

"From Theory to Practice": 'From content to competence'

Tips'n'Tricks: Tips'n'Tricks for teaching

Q'n'A: Questions (and possibly even answers...)

Warm up exercise

Post-It exercise:

1) Write down answer to: "what is good teaching?"2) Swap Post-Its...

Outline

Warm up exercise: "What is good teaching?"

The Theory of Constructive Alignment: FILM: "Teaching Teaching & Understanding Understanding"

Let's watch the short-film...

Teaching Teaching&

Understanding Understanding

Teaching Teaching&

Understanding Understanding(((((( ))))))

Available on DVD through Aarhus University Press:((( http://www.daimi.au.dk/~brabrand/short-film/ )))

Features Epilogue by John Biggs, DVD menu, and subtitles inEnglish, French, Spanish, Portuguese, Italian, German, and Danish

Inspired by: "Teaching for Quality Learning at University", John Biggs

Outline

From Content to Competence

(Concurrency 2004+2005) course aims: Given in terms of a 'content description':

Essentially:

This is a bad ideafor 2 reasons...!

The goal is...:

To understand: deadlock interference synchronization ...

Problem with 'content' as aim

Problem with 'content'as learning objectives?!

Objective:

analyze ...theorize ...

circa-explain ...circa-

describe ...

name solutions

recite conditons

Stud. C

Stud. A

Stud. B

analyze systems

explain causes

Censor

Teacher

BUT, even if it werepossible to agree, we know that the

exam will dictate thelearning anyways.

agreement

analyze systems

explain causes

tacit knowledge

from research-based tradition

t known by stud.)

Problem with 'understanding'

Problem with 'understanding'as learning objectives?!

The answer is simple:

concept of deadlock ?!

Objective:

It cannot be measured (!)

'Competence' as objectives !

'Competence' as learning objectives ! Evaluation = Have the student do something,

and then measure product and/or process

'SOLO' = Structure of the Observed Learning Outcome

Note': inherently operational (~ verbs)

Objective !

To learn to: analyze systems for... explain cause/effects... prove properties of... compare methods of... ...

Note:'understanding' is (of course) pre-requisitional (!)

[ Competence := knowledge + capacity to act upon it ]

Neighbour Discussion

Discuss with neighbour:"does this make sense ?!?"

(content competence)

'Competence' as objectives !

'Competence' as learning objectives ! Evaluation = Have the student do something,

and then measure product and/or process

'SOLO' = Structure of the Observed Learning Outcome

Note': inherently operational (~ verbs)

Objective !

To learn to: analyze systems for... explain cause/effects... prove properties of... compare methods of... ...

Note:'understanding' is (of course) pre-requisitional (!)

[ Competence := knowledge + capacity to act upon it ]

Advantages of 'SOLO'

SOLO 1 no understanding irrelevant information misses point ...

SOLO 2

SOLO 3

SOLO 4

SOLO 5

"pre-structural"

to identify to do procedure to recite ...

"uni-structural"

to classify to combine to enumerate ...

"multi-structural"

to relate to compare to analyze ...

"relational"

to generalize to hypothesize to theorize ...

"extended abstract"

Advantages of 'SOLO': Constructed for research-based (university) teaching

Converges on research (at SOLO 5) Strong correspondence to Piaget's learning stages

From cognitive psychology

SOLO (elaborated)

to theorize to generalize to hypothesize to predict to analyze to relate to compare to explain causes to describe to combine to classify to perform algorithm to do simple procedure to define to identify to recite

extendedabstract

relational

multi structural

SOLO 5

SOLO 4

SOLO 3

Graphic Illustration Legend

immediately relevant aspects – given! related or hypothetical – not given! irrellevant or inappropriate student responsex

SOLO 2

uni structural

SOLO (more verbs)

"extended abstract"

"relational"

"multi structural" &"uni structural"

to theorize to hypothesize to generalize to critize to predict

to apply theory (to 'near' problems) to reason about (reach conclusion) to explain (cause-effect) to explain (similarities-differences) to explain (strengths-weaknesses)

to describe to structure at collate to combine to classify to perform algorithm

to apply theory (to 'distant' problems) to put-into-perspective to reflect to judge to discuss

SOLO 2+3

SOLO 4

SOLO 5

to analyze to argue to relate to compare to integrate

to enumerate to paraphrase to do simple procedure to define to identify / name to recite

Note: the list is non-exhaustive

Concrete Example andConcrete Recommendations (4x)

Intended Learning Outcomes [Genetics 101]

After the course, the students are expected to be able to: locate genes on chromosomes do simple calculations : (e.g., recombination frequencies,

in-breeding coefficients, Hardy-Weinberg, evolutionary equilibria).

describe and perform connexion-analysis describe fundamental genetic concepts: (e.g., mutation

variation, in-breeding, natural selection). describe and analyze simple inheritancies analyze inheritance of multiple genes simultaneously

2) List sub-goals as 'bullets': Clearer than text

1) Use 'standard formulation':

put learning focus on student

(Note: competence formulation "to be able to")

3) Use 'Verb + Noun' formulation:

What the student is expected to

do with a given matter .V N

4) Avoid 'understanding-goals':

"To understand X", "Be familiar with Y", "Have a notion of Z", ...!

Post-It exercise

Write down 1-2 key competences (i.e., verbs)

(for your course)

Outline

5' Break

Please put the Post-Its on the wall

"What is good teaching?" Key competences(in your course)

Outline

Q'n'A: Questions & answers ...

Definition: “Good Teaching”

Definition:

Good news; we now know how to do this:

"Good teaching is getting most students to use the higher cognitive level

processes that the more academic students use spontaneously"

-- “Teaching for Quality Learning at University”, John Biggs, 2003

Alignment: Explicitly define: intended learning outcomes & Exam measure intended learning outcomes Teaching form intended learning outcomes

Implementation Process

Process (course specific):

1) Think carefully about: overall goal of course (what are the stud. to learn?)

2) Operationalize these goals: and express them as intended learning outcomes

3) Choose carefully the form(s) of examination ( intended learning outcomes)

4) Choose carefully the form(s) of teaching ( intended learning outcomes)

alignment

operationalization

A Case-Study on ImplementingAlignment

"Constructive Alignment for Teaching Model-Based Design for Concurrency" -- a case-study on implementing alignment

Claus BrabrandIn Proc. Workshop on Teaching Concurrency, June 2007, (aka., "TeaConc'07"), pp. 1--17((( Invited Paper for Invited Talk for TeaConc'07 )))

For more information about the implementation of alignment...:

Available on my homepage:((( http://www.itu.dk/people/brabrand/ )))

"The Learning Pyramid"Average

retention rate

Lecture

Reading

Audiovisual

Demonstration

Discussion group

Practice by doing

Teaching others

[Kilde: NTL Institute for Applied Behavioral Science, Bethel, Maine]

passive

activestudent

Tips'n'Tricks (activation)

Neighbour discussions:

Frequent breaks:

Post-It exercise:properties: anonymous (!) swap'able everyone will engageusages: focus: warm up / zoom in empathetic control shared knowledge pool

more questions (students dare ask them)

better questions (students had a chance to discuss)

1-2 min timeout [Phil Race]

Form variation:

lecturing blended with in-class activation exercises

Tips'n'Tricks (cont'd)

"Less-is-more":

Use many examples:(build on student pre-knowledge)

Explicit structure:

analyze compare relate

common deadlock, uncommon deadlock, A-synchronization, B-synchronization, hand-shake, multi-party synchronization, multi-party hand-shake, binary semaphores, generalized semaphores, blocking semaphores, recursive locks, ...

Emphasize depth over breadth (coverage)

NEWOLD

1. xxxxxxxxxx

2. yyyyyyyyyy

3. zzzzzzzzzz

4. wwwwwww

1. xxxxxxxxxx

2. yyyyyyyyyy

3. zzzzzzzzzz

4. wwwwwww

1. xxxxxxxxxx

2. yyyyyyyyyy

3. zzzzzzzzzz

4. wwwwwww

1. xxxxxxxxxx

2. yyyyyyyyyy

3. zzzzzzzzzz

4. wwwwwww

self evident to you [ teacher ] not to a learner [ student ] (esp. during learning process)

Student 'recap' at end:

after 1 dayafter 1 week

after 3 weeks

after 2 weeks

Outline

Questions...

"What is good teaching?"

R’’

The Short-Film

Cognition & structures

The SOLO Taxonomy

Association

new ~ old

analyzeexplain

'The Book'

John Biggs

"understanding"

content competence

Student activation

Student models

Susan & RobertTeacher models

levels 1 - 2 - 3

Intended learning outcomes (ILO)

Constructive AlignmentExam

SatisfactionExperiences

Pre vs. Post

Students at Uni

Model-based designfor Concurrency

My researchand teaching

'TLA'Teaching/Learning Activities

Tips'n'Tricks

Now, please: "3-minute recap"

Please spend 3' on thinking about and writing down the most important points from the talk – now!:

After 1 dayAfter 1 week

After 3 weeksAfter 2 weeks

Immediately

Thank You!

((( http://www.daimi.au.dk/~brabrand/short-film/ )))

Film homepage:

BONUS SLIDES

Background (~ this talk)

Concurrency 2004+2005: "Pre-alignment"

Learned about teaching/learning theories In particular: Constructive Alignment,

The SOLO Taxonomy, Constructivism, ...

Concurrency 2006+2007: "Post-alignment"

Disclaimer

not to exhibit a perfectly aligned course; but to show how the principles of alignment can be put to use (in particular, how it may serve as guidelines for the exam and TLA).

[ TLA := Teaching/Learning Activities ]

The point of this part is:

3) Choose carefully the form(s) of examination (~ intended learning outcomes)

4) Choose carefully the form(s) of teaching (~ intended learning outcomes)

alignment

Starting Point

Content description (Concurrency '04+'05):

What is the overall goal of the course...?(what are the students to learn)

Overall Course Philosophy

Model-Based Design for Concurrency:

Thoroughly Motivate(what can they do, if they 'bother' learning this?)

alignment

Intended Learning Outcomes

Model-based design

for Concurrency

Intended Learning Outcomes(based on The SOLO Taxonomy):

Note:explicitlyincludedas a non-goal

alignment

On Aligning the Exam (~ ILOs)

Pre-alignment (Concurrency 2004+2005): Group Project (50%) Individual Multiple-Choice Test (50%)

Post-alignment (Concurrency 2006+2007): Group Project (50%) Individual Multiple-Choice Test (50%)

However;BIG differences...!Coincidentally:

'Inherited' from pre-2004:

Because it seemed like agood idea to do a project

Carefully designed (~ILOs):

Project good for evaluating

model-based design process

Added in 2005:Politically motivated: exammust have individual part!

Carefully designed (~ILOs):

MC-test good for evaluating

analytical skills (~problem):

to analyze/compare models

Project (pre- vs. post-alignment)

2004 Project: "The Beer Factory":

2006 Project: "The Banana Republic":

Some student projects with no appearant model impl. relationship (at least, to me)!

No explicit learning objectives (only 'list of contents') No explicit project grading criteria

result

The Banana Republic

Project designed (~ ILO's): (a) Construct unsafe model (w/o controller); (b) Test model - observe that collisions with 'El Presidente' can occur; (c) Define safety property NO_CRASH; (d) Verify that collisions can occur; (e) Construct a controller (such that collisions can no longer occur); (f) Verify that collisions can no longer occur; (g) Define liveness property ('El Presidente' can eventually leave); (h) Implement model in Java.

Grading (of the report): construct models...

apply common solutions...

relate specmodel...

test model... define properties...

verify model wrt. properties...

implement model...

relate modelimpl...

All ILO's except:

analyze models

compare models

Betterevaluatedon MC-test

2004 MC-test:

2006 Project:

MC-test (pre- vs. post-alignment)

analyze models (and programs) wrt. behavior

(a bunch of seemingly reasonable questions):

Carefully designed (~ ILO's):

Bad Alignment

compare models (and program) wrt. behavior

Example: analyze models

Good Alignment

Example: compare models

Good Alignment

alignment

On Aligning the TLA (~ ILOs)

Pre-alignment (Concurrency 2004+2005): Lectures (2-3 hrs/week) 'Theoretical Exercise Classes' (2 hrs/week) 'Programming Lab' (2 hrs/week)

Post-alignment (Concurrency 2006+2007): Lectures (2-3 hrs/week) with activation exercises 'Theoretical Exercise Classes' (2h/w) apply common solutions

'Programming Lab' (2 hrs/week) hands-on training for project

Weekly hand-ins (every week) train for project (w/ feedback!)

MC-test sample questions (given early) train for MC-test

[ Idea due to colleague Thomas Hildebrandt at ITU ]

essentially teacher-centric "monologues"

TLA's (post-alignment)

Student-centric: 'Theoretical Exercise Classes' (2h/w) apply common solutions

'Programming Lab' (2 hrs/week) hands-on training for project

Weekly hand-ins (every week) train for project (w/ feedback!)

MC-test sample questions (given early) train for MC-test

Teacher-centric: Lectures (2-3 hrs/week) with activation exercises

{ apply common solutions }

{ construct, implement, test, verify, define, apply }

{ construct, implement, relate }

{ analyze, compare }

introduce fundamental concepts/problems/solutions (in terms of models & impl)

alignment??

Conclusions (pre vs. post)

Subjectively: Constructive Alignment (!!!):

To the point that I bothered making a film about it :)

Own behavior changed: From 'intuition' to conscious choices;

awareness of alternatives and of consequences of choices (~ student learning)

My students' behavior (from my perspective): More focusses on learning the objectives (esp. 'to relate')

Student background and prerequisites; The "Susan/Robert ratio"; Teacher's experience gain;

Disclaimer:(many factors involved that vary from-year-to-year)

... ...and many more

Objectively (I/III):(Questionnaire at end, 7-step scale) Student satisfaction:

"slightly more satisfied" ..or

"constructive alignment doesn't compromize student satisfaction"

Student proficiency:

More useful figures (~learning)! However: I only have

post-alignment data :( Thus: "inconclusive" :(

self-reported

Pre ('04-'05)Post ('06-'07)

Pre ('04+'05)Post ('06+'07)

Objectively (II/III):(Competences explicitly tested & trained)

Competences explicitly tested and trained for:

Conclusion: "Substantial SOLO-level increase" (~ good teaching) ! Much better projects (esp. 'modelimpl' relationship) !

Objectively (III/III):(Qualitative data from 2006 eval) Anonymous student in 2006 evaluation:

“This course has been awesome! It took me a while to be able to think in models, but I saw the light along the way.”

“Lectures have been great, the theoretical exercise classes have been rewarding and the feedback has been immense and insightful”

“I did not have a lot of time to do the exercises, but they seemed relevant from week to week.”

“The mini project was a good and solid exercise in analyzing a problem, making a model and implementing it. A very good exercise!”

Overall:

Teaching:

Exercises:

Project:

The Role of the Exam

Alignment: A theory of planning (over the course of a course) A theory of motivation (and incentive)

The exam as a...:"Necessary evil"

Motivational and learning-guidingpedagogical tool for the teacher(!)

applicationof alignment

"The exam does not come after, but before the course!"

Motivation Beyond the Exam

Motivational problem: “Why bother learn the course material?”:

Tell them why it is important to learn these things: How could these skills benefit them in their work/life/…

(focus on advantages)

[ Example… ]

Program worldModel world

ConcreteAbstract

P Mabstraction

concretization

Programs and Models

On Program EquivalenceProgram world

Model world

ConcreteAbstract

1. P ~ P’ ?2. abstract

3. M ~ M’ ?

4. relate

5. M ~ M’ !6. concretize7. P ~ P’ !

What discerns a really good programmer from one that is not so good is the capability of moving (consciously or unconsciously) between the concrete world of programs and the abstract world of models (via abstraction and concretization).

Specifically, such a programmer is capable of (consciously or unconsciously): - 1) abstracting programs into models - 2) reasoning about the models - 3) concretizing the insights back into the world of programs

On Property Satisfaction

What discerns a really good programmer from one that is not so good is the capability of moving (consciously or unconsciously) between the concrete world of programs and the abstract world of models (via abstraction and concretization).

Specifically, such a programmer is capable of (consciously or unconsciously): - 1) abstracting programs into models - 2) reasoning about the models - 3) concretizing the insights back into the world of programs

Notes on Skill Acquisition

From the world of psychoanalysis: Skill acquisition progresses according to the following

stages of learning: 1. Unconscious incompetence 2. Conscious incompetence 3. Conscious competence 4. Unconscious competence

5. Capacity for moving consciously between stages 3. and 4.: [ required by a teacher ]

A taxonomy / language for teaching impersonalizes teaching

Emotional detachment (aka. “dissociation”) The teacher is good/bad

identity: good/bad teacher The methods are good/bad

behavior: good/bad method knowledge: good/bad method

With dissociation: more capable of dealing with critique better to listen

to constructive advice (…just like with our research)

Impersonalization

identitymoralknowledgebehavior

“Neutological levels”

[model of the mind, “NLP”]

ethics

experience

reactions

convictions

capabilities

interaction

Based on John Biggs' Theories

2nd edition

(3rd edition expected this fall)

"Teaching for Quality Learning at University", John Biggs

Teacher’sintention

Student’sactivity

Exam’sassessment

e.g.- explain- relate- prove- apply

e.g.- memorize- describe

UNALIGNED COURSE

e.g.- memorize- describe

"Dealing with the test"

Teacher’sintention

Student’sactivity

Exam’sassessment

ALIGNED COURSE

Kommunikation (af fag-viden)

Speak-Listen:

Write-Read:

Personal

knowledge

Official

knowledge

Personal

knowledge

OfficializationOfficialization

Personalknowledge

Official

knowledge

Personal

knowledge

OfficializaOfficializa

Personlig vs. Officiel Viden

Personlig viden: "connaissance" (fransk) individuel uformel implicit subjektiv (kan være) ustabil (mht) specifikke situationer

Officiel viden: "savoir" (fransk) delt (med andre) formel eksplicit objektiv stabil (især NAT-videnskab) generel

To (modsat rettede) processer:

Personligviden

Officielviden

Officialization

Personalization

[ ~MBTI: "S" vs. "N" ]

Undervisningsformer

[kilde: Torben K. Jensen]

problem-orienteret

disciplin-orienteret

lærer-styret elev-styret

Forelæsning

Projekt-arbejde

Øvelser

Case-undervisning

Undervisningsforløb:Deduktion vs. Induktion

Deduktivt forløb: Induktivt forløb:

Introduktion

Øvelse

Opsamling

Introduktion

Øvelse

Analyse

claus brabrand (itu) dmlf Årsmøde 2007 – keynotesep 20, 2007 constructive alignment claus...

Documents

c ommunication via powerpoint claus brabrand it university...

microsoft research march 20, 2000 a programming language for...

claus brabrand (itu copenhagen) & jakob g. thomsen...

how to make sure your students learn what you want them to...

claus brabrand, ufpe, brazil aug 04, 2010data-flow analysis...

intraprocedural dataflow analysis for software product lines...

concurrency: deadlock ©magee/kramer claus brabrand...

n ov 12, 2008 c laus b rabrand p rogramming p aradigms l...

from intended learning outcomes to teaching/learning...

global software development 2013 claus brabrand (((...

“approximating context-free grammar ambiguity”november...

claus brabrand, ufpe, brazil aug 09, 2010data-flow analysis...

claus brabrand, itu, denmark feb 09, 2010black-box testing...

[ 1 ] claus brabrand, itu banana algebra march 28, 2009...

c laus b rabrand © c oncurrency (q3,’07) f eb 05, 2007 c...

itu seminar on teaching & learning claus brabrand (((...

j uly 18, 2007 ciaa'07, p rague "a nalyzing a mbiguity of c...

claus brabrand, itu, denmark mar 17, 2009graphs & graph...

c laus b rabrand © c oncurrency (q3,’06) j an 19, 2006 c...

the e i f f e l programming language claus brabrand (((...