June 2013

Learning Engineer/Project Manager

Mini OLI Authoring Training

Kim Larson

oli.cmu.edu

Learning Objectives

By the end of this session, you will be able to…

• Write a student centered observable or measurable

learning objective

• Identify component skills that make up that objective

• Write a learning activity, that includes hints and targeted

feedback.

oli.cmu.edu

The Course Design Triangle

Source: Eberly Center for Teaching ExcellenceCarnegie Mellon

Instructional Activities

(low stakes assessments)

Objectives

Assessments

(High stakes)

Tasks that provide

feedback on students’

knowledge and skills

Descriptions of what students

should be able to do at the

end of the course

Contexts and activities that foster

students’ active engagement in

learning

oli.cmu.edu

The Course Design Triangle

Source: Eberly Center for Teaching ExcellenceCarnegie Mellon

Instructional Activities

(low stakes assessments)

Objectives

Assessments

(High stakes)

Tasks that provide

feedback on students’

knowledge and skills

Descriptions of what students

should be able to do at the

end of the course

Contexts and activities that foster

students’ active engagement in

learning

oli.cmu.edu

Why Focus on Learning Objectives?

1. They communicate our intentions clearly to students and

to colleagues.

2. They provide a framework for selecting and organizing

course content.

3. They guide in decisions about assessment and

evaluation methods.

4. They provide a framework for selecting appropriate

teaching and learning activities.

5. They give students information for directing their

learning efforts and monitoring their own progress.

Based on A.H. Miller (1987), Course Design for University Lecturers. New York: Nichols Publishing.

Also see, C.I. Davidson & S. A. Ambrose (1994), The New Professor’s Handbook: A Guide to Teaching and Research in Engineering and Sciences. Bolton, MA:

Anker Publishing Company Inc.

oli.cmu.edu

Four key elements to good learning objectives:

1.student-centered

2.component skills/sub objectives

3.use action verbs

4.observable or measurable

oli.cmu.edu

Four key elements to good learning objectives:

Student-centered: Learning objectives should be student-centered; for example, stated as "Students should be able to _______.“

Component skills/sub objectives: They should break down the task and focus on specific cognitive processes. Many activities that faculty believe requires a single skill (for example, writing or problem solving) actually involve a synthesis of many component skills.

Use action verbs: Clear objectives should use action verbs that communicate to the students the kind of intellectual effort we expect of them. Furthermore, using action verbs reduces ambiguity in what it means to "understand."

Observable or measurable: Clear objectives should be observable or measurable. We should be able to easily check (that is, assess) whether students have mastered a skill (for example, asking students to "state" a given theorem, "solve" a textbook problem, or "identify" the appropriate principle).

oli.cmu.edu

Student Centered

Remember, the real goal of a course is not for us to teach a

body of material but rather for students to learn it.

Therefore, learning objectives are most effective when they

specify our goals from the students’ point of view.

oli.cmu.edu

Student Centered

The first key feature of a learning objective is that it expresses

what the student, not the teacher, will be doing. Look at the two

“objectives” below and ask yourself who would be accomplishing

each objective: the student or the teacher?

A. Apply two theories in social psychology to explain specific examples of

human behavior.

B. Introduce students to a range of theories in social psychology.

A describes what students will be able to do at the end of the course, it is a learning

objective.

B, however, indicates what the instructor wants to achieve and does not constitute a

learning objective.

oli.cmu.edu

Component skills/sub objectives

They should break down the task and focus on specific

cognitive processes.

Many activities that faculty believe requires a single skill (for

example, writing or problem solving) actually involve a

synthesis of many component skills.

oli.cmu.edu

Component skills/sub objectivesThe objective often requires more of a synthesized activity that requires students to be able to do a number of skills to show mastery of the objective.

For example, let's say you have a learning objective for an elementary algebra course like this one:

Simplifying, adding, subtracting, multiplying and dividing polynomials.

The component skills would be something like this:

Combine like termsUse order of operationUse distributive property

You would expect that in their prior knowledgethey would already be able to:

AddSubtractMultiplyDivide

oli.cmu.edu

Action verbs

Actions verbs help specify what you want students to be able to do.

By telling students what to do, we provide them with benchmarks by

which to assess their process--we inform them as to which tasks they

might use to monitor whether or not their performance of certain tasks is

up to par--metacognition.

As previously discussed, learning objectives both help students monitor

their own learning and also help students focus their energy

appropriately, and action verbs are also a big part of that contribute

significantly to both parts of that.

At the same time, action verbs enable instructors to focus on and clarify

the tasks we want to evaluate.

oli.cmu.edu

Observable or measurable

Observable or measurable

Look at the two objectives below. Which one is observable?

A. Understand Newton’s Second Law

B. Explain Newton’s Second Law

Notice that A is not observable, while B is. An instructor can’t

directly witness a student’s understanding. However, she could

observe and evaluate a student’s explanation.

Observable. Avoid using words like “understand”, “comprehend”, or “appreciate”.

These are cognitive processes that take place in a student’s head and are

invisible to you. Instead, ask yourself: What would students have to do that would

show me that they understand X, comprehend Y, or appreciate Z?

oli.cmu.edu

Different Kinds of Objectives

Bloom’s Revised Taxonomy identifies six levels of cognitive processes. http://www.celt.iastate.edu/teaching/RevisedBlooms1.html

Across these levels, knowledge is used in more sophisticated ways:

▫ Recall: remember, recognize, identify

▫ Understand: interpret, exemplify, classify, summarize, explain, compare

▫ Apply: execute, implement, use, carry out

▫ Analyze: differentiate, distinguish, organize, select

▫ Evaluate: check, critique, judge, monitor, test

▫ Create: generate, plan, produce, construct, hypothesize

oli.cmu.edu

NOTE: Watch Out for Verbs that are not Observable or MeasurableIn order for an objective to give maximum structure to instruction, it should be free of vague or ambiguous words or phrases. The following lists notoriously ambiguous words or phrases which should be avoided so that the intended outcome is concise and explicit.

WORDS TO AVOID:Believe, Hear, Realize, Capacity, Intelligence, Recognize, Comprehend, Know, See Conceptualize, Listen, Self-Actualize, Depth, Memorize, Think, Experience, Perceive, Understand, Feel

PHRASES TO AVOID:To Become: To Reduce: Appreciation for… Acquainted with… Anxiety Attitude of… Adjusted to… Immaturity Awareness of… Capable of… Insecurity Comprehension of… Cognizant of… Enjoyment of… Conscious of… Feeling for… Familiar with… Interest in… Interested in… Knowledge of… Knowledgeable about…. Understanding of… Self-Confident in.

oli.cmu.edu

Create an objective

Create a basic objective at your table, try to choose something that most people have domain expertise in. (It could be anything)

Then determine if there are component skills needed to for the student to show mastery in the objective.

For example you could use something like hand signals for driving:

OBJECTIVE:Describe the hand signals for driving when electronic signals are unavailable or not working in the US.

SUB SKILLIdentify the hand signal for making a right turn in the US

oli.cmu.edu

Let’s check your objective.

Checklist: Is the objective…?• Student centered (i.e., student should be able to…)

• Broken down into component skills (grain size)

• Phrased with an action verb

• Observable or Measurable

oli.cmu.edu

The Course Design Triangle

Eberly Center for Teaching ExcellenceCarnegie Mellon

Instructional Activities

Objectives

Assessments

Tasks that provide

feedback on students’

knowledge and skills

Descriptions of what students

should be able to do at the

end of the course

Contexts and activities that foster

students’ active engagement in

learning

oli.cmu.edu

What would happen if there was a learning objective with no activities?

All sorts of awful things▫ Fire and brimstone coming down from the skies

▫ Rivers and seas boiling

▫ 40 years of darkness

▫ Human sacrifice

▫ Dogs and cats living together

▫ Mass Hysteria!

- Ghostbusters

oli.cmu.edu

Learn By Doing

“Practice this material”

Scaffolded to give support “as needed” by providing tailored

hints and feedback.

Allow students to practice specific tasks and skills in a non-

threatening environment.

Learning Dashboard report student activity in an aggregate

format.

oli.cmu.edu

HintsThink about when a student is given task and they become

immediately stuck.

If the student asked you for help what would you say to

student to help them figure it out.

1st Hint is strategy for answering question:▫ General restatement of goal: your goal is X, try strategy Y (the

strategy)

2nd Hint is cognitive: ▫ Mid-level (compare angle A-B-C to X-Y-Z)

3rd Hint is bottom out, give them the answer:▫ Answer + explanation (the angle is 45 degrees)

oli.cmu.edu

Hint 1: Example

oli.cmu.edu

Hint 2: Example

oli.cmu.edu

Hint 3: Example

oli.cmu.edu

Feedback

Initial response, then short explanation.

▫ Not quite right. As you look at the histogram, notice that the

distribution is roughly symmetric.

▫ Correct. Since the distribution is symmetric, the mean and

median would be about the same.

▫ Incorrect. This data set is skewed and the Mean/SD are affected

by skew.

Appropriate distracters(answer choices):

▫ Reflect common misconceptions or errors,

oli.cmu.edu

Feedback: Example

oli.cmu.edu

Feedback: Example

oli.cmu.edu

Feedback: Example

oli.cmu.edu

Feedback: Example

oli.cmu.edu

Create a learning activity for the objective you wrote earlierThink about a question you would ask students to determine

if they could master the objective. Then try to write hints and

targeted feedback for that question.

Let’s go back to our example.

I will take the sub skill:

Identify the hand signal for making a right turn in the US

Q1. Which is the correct way to hand signal making a right

turn when your vehicle does not have signal lights or has

damaged signal lights?

oli.cmu.edu

Using distractors that address common misconceptions.Q1. Which is the correct way to hand signal making a right

turn when your vehicle does not have signal lights or has

damaged signal lights?

A. Put your left arm out of the left front window and point your

finger toward the right.

B. Open your right front window and reach over as far as you can

and put your hand out.

C. Extend your left upper-arm out to the left, horizontally and

angle your forearm vertically upward.

oli.cmu.edu

Hints and targeted feedback

Q1. Which is the correct way to hand signal making a right turn when your vehicle does not have signal lights or has damaged signal lights?Hints: 1. If you want to make a turn, think about what might be the safest and way to convey that to other drivers and pedestrians.2. You have to signal in a way that is visible to other drivers and pedestrian. 3. The correct answer is to extend your left upper arm out to the left, horizontally and angel your forearm vertically.

Feedback:A. Not quite right. While this may work it is not the universal signal in the US for

a right hand turn.B. Incorrect. Reaching across the inside of the car to signal a turn is not a safe

way to operate your vehicle, and is not the correct way to signal.C. Correct. This is the universal signal in the US to let other people know you

are taking a right hand turn.