CURRICULUM, INSTRUCTION & ASSESSMENT

SEPTEMBER 15, 2008

DR. DIANE KELLEY

Writing Measurable Goals and Objectives

Goals of this Presentation2

Appreciate the value of writing clear and measurable behavioral objectives.

Re-evaluate objectives you have written and work to enhance them.

Make additional efforts to enhance the congruency of: Objectives, Individual Professional Activities (PD, IPDPs)---Your Own Action Items, and/or School & Department Goals/Mission Statement.

Goals and Objectives

What are objectives?Objectives are NOT goals. Goals are broad, objectives

are specific.An objective is a “foreseen end that gives direction to

an activity” (John Dewey, 1977)

“An objective is an end toward which action is oriented, a condition or state of being to be reached. It reflects how the situation will be changed, improved or maintained. “ (Boyle)

An objective is a statement which specifies, in measurable terms, the changes we expect in our target audiences as a result of our programs.

3

Objectives4

Definition A statement in specific and measurable terms

that describes what the learner will know or be able to do.

Example Students will be able to construct a five

paragraph essay that follows the guidelines of the Florida Writes rubric within a 45-minute timed writing session in English class and explain the components in their essay that warrant the highest score.

Orienting Question

How do you tell a good objective from a bad one?

Elements of Strong Objectives

Target audience (who)

Criteria for coverage

Outcome (what)

Criteria for outcome

(measurable)

Personal actions to achieve

desired outcomes are SMMART (more later)

Parts of an Objective7

Measurable verb Construct Explain

Criteria 5 paragraphs Within 45-minutes

Conditions In their English class

EXAMPLE: Students will be able to construct a five paragraph essay that follows the guidelines of the Florida Writes rubric within a 45-minute timed writing session in English class and explain the components in their essay that warrant the highest score.

Sources of Objectives

Expressed---Needs Assessments, Surveys

Analysis of Data

Research Results

Policy, Legislation, Dictates

Taxonomies, Schemata, Frameworks

Strong & Weak Verbs for Objectives

Strong:to increase (what

by how much)to adopt (what

resource/practice, by what timeline)

to present or to demonstrate (what topics to whom, when)

Weak to promote to encourage to understand to become

aware to work with to support

Writing Useful Objectives

Use strong verbs

State only one purpose or aim per objective

Specify a single end-product or result

Specify a time frame for achieving results

May relate to taxonomies, schemata or frameworks

Depth of Knowledge

Adapted from the model used by Norman Webb, University of Wisconsin, to align standards with assessments

Used by the Council of Chief State School Officers (CCSSO) for assessment alignment in more than ten states

• Focuses on content standard in order to successfully complete an assessment/standard task

• Descriptive, not a taxonomy---Not the same as difficulty

Why “Depth of Knowledge”?

No Child Left Behind (NCLB) requires assessments to “measure the depth and breadth of the state academic content standards for a given grade level” (U.S. Department of Education, 2003, p. 12

Mechanism to ensure that the intent of the standard and the level of student demonstration required by that standard matches the assessment items (required under NCLB)

Provides cognitive processing ceiling (highest level students can be assessed) for item development

Webb’s Depth of Knowledge levels:

Recall and Reproduction: Level 1

Skills & Concepts: Level 2

Strategic Thinking: Level 3

Extended Thinking: Level 4

Bloom’s Taxonomy14

Levels of Cognition Knowledge (verbal recall) Comprehension Application Analysis Synthesis Evaluation

KnowledgeKnowledge

EvaluationEvaluation

Applying Bloom’s

Knowledge – List the items used by Goldilocks while she was in the Bears’ house.

Comprehension – Explain why Goldilocks liked Baby Bear’s chair the best.

Application – Demonstrate what Goldilocks would use if she came to your house.

Analysis – Compare this story to reality. What events could not really happen.

Synthesis – Propose how the story would be different if it were Goldilocks and the Three Fish.

Evaluation – Judge whether Goldilocks was good or bad. Defend your opinion.

Using the story Goldilocks and the Three Bears

Depth of Knowledge vs. Bloom’s Taxonomy

Level 1 (Recall) Knowledge

Comprehension

Level 2 (Skill/Concept) Application

Level 3 (Strategic Thinking)

Analysis

Level 4 (Extended Thinking)

Synthesis

Evaluation

Levels of Evidence by Claude Bennett (from lowest level 1 of evidence that program is effective to

highest level 7)

Level 1 InputsLevel 2 ActivitiesLevel 3 People InvolvementLevel 4 People ReactionsLevel 5 Knowledge, Attitudes, Skills,

Aspirations Change (KASA)Level 6 Behavior ChangeLevel 7 End Result (SEEC-Social,

Economic, Environmental, Civic)

Levels of Objectives

*Inputs*Activities*Participation*Reactions*Knowledge, Attitudes, Skills & Aspirations

(Behavioral intentions)*Behavioral or Practice Change*Social, Economic & Environmental

Conditions

Outcome (aka: Educational ) Objectives focus on:

Knowledge change (Level 5)Skill change (Level 5)Attitude change or Reactions (Level 5)Aspirations changes (Level 5)Short-term, mid-range and long-term

incremental behavior change (Level 6)May also include policy changes, decisions

made, etc.

Program Objectives focus on:

Resources or Inputs (Level 1)Activities (Level 2)Participation of target audience (Level 3)Meets micro needs

Examples of Measurable Objectives

Objective: 35 Young Men of Tomorrow members will practice

communication skills in relationships with parents or peers during the week of Sept. 15 as documented by their sponsors’ communication logs.

Objective: 100% of science teachers who attended my science

safety workshop will implement recommended safety practices all of the time after completing the program, as documented by district safety inspector.

Objective: Half of the teachers attending my math workshops will

implement at least one manipulative method for introducing a math concept this year as evidenced by lesson plans, observation, and follow-up on MLP.

General Format

To (increase/decrease) (what) by (% number) among (whom) by (when) as measured by (how do you know).

Note: Under certain extreme conditions, in the short-term, you may want to maintain (not increase/decrease) a problem at a certain level that has been steadily getting worse.

Measurable: A Technique

Who will change? Assigned audience/sWhat specific change will take place as a

result of your program? Specific targetWhen will the change take place? Timed How will you know the change has taken

place? Measurable. What standard or method will you use to know that the condition has changed.

Meaningful and Realistic are taken into account at every step of objectives development

Some Things To Remember About Writing Outcome Objectives

Objectives evolve from Goals and are tied to the assessment process.

Objectives are Future Focused. Remember, outcome objectives relate to outcomes

and not processes. For example, OCSD is planning a new program for

recently arrived ELLs. The objective is not "to provide information" but

rather "to reduce” the rate of retention among ELLs. A specific percentage increase would be needed, too.

Providing information is an activity (strategy) to enable the objective to be achieved.

Common Errors in Writing Goals and Objectives

Too specific to be meaningfulToo broad to be measurableToo many to be manageableToo few to be comprehensiveToo arbitrary to be relevantToo boilerplate or “cookie-

cutter” to be individualized

Comparison26

Goal Students will gain an appreciation and

understanding of the value of applying the Florida Writes rubric to their own writing.

Objective Students will be able to construct a five

paragraph essay that follows the guidelines of the Florida Writes rubric within a 45-minute timed writing session in their English class and explain the components in their essay that warrant the highest score.

Purposes of Objectives

Address the needs of your target public

Provide direction to your program or activity

Useful in selecting learning experiences and program methods

Communicate the expected or predicted results or changes to key stakeholders

Forms the basis for the evaluation if it is written in measurable terms

Levels of Objectives28

What is the level of this objective? Participants will be able to name the three

parts of a behavioral objective.

What is the level of this objective? Participants will be able to write a behavioral

objective that contains a measurable verb, condition, and criteria.

Specificity and Level29

Not So Specific: The learner will be able to: orally discuss the

elements of a storyMore Specific:

The learner will be able to: orally present a short story’s parts in a logical sequence (Setting, Characters, Conflict, Resolution), chronologically develop the elements, summarize the actions of the protagonist and antagonist, as well as highlight any details that may have been influenced by the author’s background or experiences.

Good Objectives are SMMART

Specific- tells what change is going to occur. Has a clear focus

Meaningful – relates to identified needsMeasurable- definite means to measure to

see if change occurred. Can observe it empirically

Assigned to a specific target audienceRealistic- can be accomplished Timed- specific time to be achieved by

Measurable and Observable

Can look back and tell whether you achieved your aim

Typically associated with quantifiable information, but can be qualitative

Ex: Expected level of improvement in math for an academic enrichment program as evidenced by pre and post program math test scores

Using the SMMART Process

When writing goals and objectives, keep them SMMART:

. Specific. Use specific rather than generalized language:

Clearly state the issue, the target group, the time and place of the program.

Poorly stated goal : The district’s Title I program will reduce the number of ELLs retained.

Clearer goal : As a strategy for reducing retention rates, ELLs will participate in an intensive supplementary instructional after-school program which will focus on supplementary instruction on grade level benchmarks using effective ESOL strategies.

Using the SMMART Process

Meaningful. Relates to identified needs. Example of a non-meaningful objective:

To support all of our schools in their efforts for success

Example of a more meaningful objective: To personally provide writing workshops in 10 of

our schools that are aligned with a need reflected in their disaggregated school data which will result in a 5% overall increase in Level 6 writing scores in those schools.

Using the SMMART Process

Measurable.Be clear in the objective about what will be changed and by howmuch. Setting this clearly at the startmakes it easier to evaluate:

Example of a non-measurable objective To raise awareness of the low literacy rate of parents of

ELLs. Example of a measurable objective

Within 6 weeks of running an evening family literacy program in each of the district’s 6 high schools, the attendance rate of participants will increase by 5% and survey results will indicate a 50% increase in the number of families that have implemented reading strategies at home as evidenced by reading logs.

Using The SMMART Process

Achievable. Be realistic about what the program can achieve in terms of the scale/scope of what is being done, the time and resources available:

Unrealistic: The percent of ELLs scoring at Level 1 will be reduced 50% at the end of the project period.

Realistic: When compared to the baseline, there will be a statistically significant reduction (5%) in the number and percent of ELLs scoring at Levels 1 & 2 in reading.

Using the SMMART Process

Relevant: Objectives need to relate to and be relevant to the goals. Remember objectives are the building blocks/steps toward meeting the goals:

Not Very relevant: 80% of the middle school students who received

supplementary services funded by this project will gain at least one level as measured by the FCAT in reading.

More relevant An analysis of end of the year reading scores for ELLs

will demonstrate that at least 80% of the middle school students will gain at least one level as measured by the FCAT in reading.

Using the SMMART Process

Time Specific. Be clear in the objectives about the timeframe in which the program/activities, as well as expected changes, will take place:

At the conclusion of the program, there will be a 7% (statistically significant) increase, when compared to baseline data, in the proportion of ELLs who have mastered listening, speaking, reading & writing in English.

What is a Goal?

A Goal is a general statement of a desired state toward which a program is directed.

see Rossi et al., 2004:98

Goals39

Definition A statement that describes in broad terms what

the learner will do. Example

Students will gain an appreciation and understanding of the value of applying the Florida Writes rubric to their own writing.

Goal Writing

Good goal writing takes into account the long-range vision and states it in short-range measures.

Good goal writing is essentially the same challenge regardless of the “form”.

Articulate measurable goals

Goals must be directly related to the student needs as identified in the present level data.

Goals are positive statements of student or teacher performance.

Goals must address reasonable, broader outcomes.

Goals provide the logical connection between student needs and service implications.

Sample Goal Objective Strategy

Your Goal is to ensure that middle school students get the adequate career information on a weekly basis.

Your SMMART Objective might be: By 5/31/09, 90% of middle schools in OCSD will have implemented career information units for their middle school students.

One of your Strategies might be: By 1/15/09, create & distribute packets to middle school principals educating them about the benefits of career information units for middle school students.

Examples—Make them better43

Adam will get better with adding and subtracting two digit numbers.

Change this to a SMMART Report

SMMART Report44

Adam will be able to add two digit numbers with re-grouping with 80% accuracy. He will also be able to subtract two digit numbers without re-grouping with 70% accuracy. We will continue to work on subtraction with and without borrowing. Please continue to practice the basic subtraction facts with Adam.

Some Things To Remember About Writing Objectives

Objectives May be Long-term (Impact):

Long-term but still realistic Based on expected achievements of program or actions

Deals with determinants of proficiency that can be measured (ex. level of understanding of math/science, speaking, reading & writing)

Deals with determinants of academic achievement that

can be measured (FCAT Levels) Do not happen immediately-measured after many months Dependent upon some action (strategies) personally

undertaken to improve student achievement SMMART!

References

Guion, Lisa A., Baugh, E. & Marcus, J. (2006). Writing “SMMART” objectives. EDIS. Florida Cooperative Extension Service, University of Florida. Publication # FY824

Impact Statements, Michigan Agricultural Experiment Station, available at http://www.maes.msu.edu/intranet/Report_impact.htm#Example2

Rockwell, K & Bennett, C (1995) Targeting Outcomes of Programs, available at http://citnews.unl.edu/TOP/english/index.html