developed & presented by edward j. caropreso, phd watson school of education university of north...

Developed & Presented byEdward J. Caropreso, PhDWatson School of EducationUniversity of North Carolina [email protected]

Director of AcademicsMarine Corps Combat Service Support SchoolsCamp Johnson, NC January, 2011

1. 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)2. Recognize/value the relationship between professional vocabulary and practice3. Recognize/value the use of professional vocabulary

2. 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation2. Recognize differences between objectives models3. Write objectives using a specific model4. Critique objectives using a specified evaluation system

3. 1. Recognize/value the role of appropriate information in decision-making2. Recognize/value the relationship between types of assessment & resulting information3. State the difference between objective & subjective (non-objective) assessment

4. 1. State the difference between traditional & alternative assessment models2. State the difference between models of alternative assessment3. State the purpose/s for rubrics in assessment4. Identify appropriate applications of rubrics (in specific assessment settings/contexts)

5. 1. Describe formative assessment strategies2. State appropriate applications for formative assessment/s

6. 1. Recognize/value the need for systematic planning for assessment2. Evaluate item types for given applications/purposes3. Construct items to meet specific Los in given learning domains4. Evaluate items/tests as to purpose/use of assessment information

7. 1. Describe validity; reliability2. Explain the relationship between high quality assessment and validity/reliability

8. 1. Recognize/value the role of instructors as student study supports2. Recognize the relationship between test item types & specific study strategies



3. 1. Recognize/value the role of appropriate information in decision-making2. Recognize/value the relationship between types of assessment & resulting

information3. State the difference between objective & subjective (non-objective) assessment


1. Introduction & Context for Training Knowledge-Practice Model Professional Vocabulary & Concepts

2. Context for Instruction Educational Goals & Objectives Models of Objectives Working with Objectives

3. Context for Assessment & Evaluation Role of Information for Decision-making

▪ Types of data related to types of decisions Assessment Model Types of Assessments

▪ Objective vs Subjective, Etc.4. Alternative Assessment: Performance Strategies Complementing Paper Assessment5. Formative Assessment: Information about Teaching & Learning to Improve Instruction6. Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment

items within local work context7. Assessment & Test Construction: Technical Features & “Quality” Tools

Validity Reliability

8. Supporting Student Learning through Assessment KSU Study Strategies; Test Preps Improving Performance; MIT Test Strategies





▪ Objective vs Subjective, Etc.

4. Alternative Assessment: Performance Strategies Complementing Paper Assessment

5. Formative Assessment: Information about Teaching & Learning to Improve Instruction

6. Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work context

7. Assessment & Test Construction: Technical Features & “Quality” Tools


8. Supporting Student Learning through Assessment KSU Study Strategies; Test Preps Improving Performance; MIT Test Strategies

Knowledge-Practice Model of Assessment & Evaluation

Professional Vocabulary for Concepts

Developing Professional Knowledge

Knowledge of Instructional Practice: Knowledge of Instructional Practice: Strategies & Activites (incl. Strategies & Activites (incl. assessment)assessment)

Knowledge of Learner Achievement Resulting from Instruction

Dynamic Reciprocity

Developing Professional Knowledge

Knowledge of Learner Achievement Resulting from Instruction

Dynamic &Reciprocal

Knowledge of Instructional Practices(Strategies/Activities/Assessments)

What’s the current status of your professional knowledge?

How have you developed your knowledge of instruction, assessment and evaluation?

What are your expectations for developing your professional knowledge & practice?

What are your professional development alternatives?

Resources & Databases New Horizons for Learning

http://www.newhorizons.org/strategies/assess/terminology.htm

Nat’l. Center for Research on Evaluation, Standards, & Student Testing (CRESST @ UCLA) http://www.cse.ucla.edu/products/glossary.html

Coalition of Essential Schools: Defining Assessment http://www.essentialschools.org/cs/resources/view/ces_re

s/124 System of Adult Basic Education Supports

http://www.sabes.org/assessment/glossary.htm Beyond Confusion: An Assessment Glossary, by Andrea

Leskes http://ctl.stanford.edu/Tomprof/postings/448.html

Continuous Assessment: Practical Guide for Teachers

Review the glossaries Identify the terms & concepts you know

Mark the ones you know & understand Compare your list to colleagues’/groups’

What’s the same; what’s different? Why are there differences?

What are the implications for professional practice given inconsistencies in the knowledge base?

Educational Goals & Objectives

Models of Objectives Working with Objectives

Educational Goals: “…those human activities which contribute

to the functioning of a society (including the functioning of an individual in society), and which can be acquired through learning.” (Gagne, Briggs, & Wagner, 1988, p. 39; cited in Nitko, 2001, p. 23)

Goals are stated in broad terms that give direction and purpose to planning overall educational activities.

Educational Goals (or global objectives) : Broad, complex learning outcomes that

require substantial time and instruction to accomplish

Typically encompassing many specific learning objectives which require learner success on most/all to reach the general goal (from Airasian, 2001)

Educational GoalsUT Examples of Goals vs ObjectivesMore Examples of Goals vs

Objectives Airasian’s 3-Level Model

Two Common Models: Mager & Gronlund

Mager TipsGagne & Briggs & ABCD Models

A Comparison of ModelsGreenberg’s 6 Keys to Successful Per

formance Objectives

Component Mager Gagné & Briggs ABCD

Performance(What will occur)

PerformancPerformance (Doing e (Doing

verb)verb)

Learned Learned capability, capability,

object, & actionobject, & action

BehaviorBehavior

Condition (Under what)

ConditionCondition SituationSituation ConditionCondition

Criteria (How well)

CriterionCriterion Tools/ConstraintsTools/Constraints DegreeDegree

Learner (By whom)

(Implied) (Implied) (Implied) (Implied) Audience Audience

Understanding the Context of Learning How do you make sense of learning? How do you identify/describe

expectations?Putting Bloom’s Taxonomy to WorkThree Types of LearningWhat Drives Statements of Learning:

Active Verbs

Identify & State 1-3 TLOs (goals; in your area) For each TLO (goal), using the course materials,

identify & state 1-2 knowledge & 1-2 performance objectives (ELOs) in at least two models we’ve reviewed

Compare your objectives with a colleague’s Using Greenberg’s Criteria to review your

objectives, how do they hold up? What’s similar to your colleague? What’s different?

Keep your objectives for later; we’ll use them again!

3. Context for Assessment & Evaluation Role of Information for Decision-

making▪ Types of data related to types of

decisions Assessment Model Types of Assessments▪ Objective vs Subjective, Etc.



3. 1. Recognize/value the role of appropriate information in decision-making2. Recognize/value the relationship between types of assessment & resulting information3. State the difference between objective & subjective (non- objective) assessment

Establishing vocabulary & conceptual framework

Assessment and Evaluation: What do they mean?

Relationship between assessment, evaluation and decision-making: The role of time

What about instructional outcomes? Instruction, assessment and evaluation Developing Criteria

Model of Assessment Identifying & applying appropriate

strategies to gather information for specific purposes &/or decisions

Determining appropriate assessment strategies

Appropriately applying strategies

Purposeful Decision Making

Assess Instructional Practices:Assess Instructional Practices:

Methods, Materials, Strategies & Methods, Materials, Strategies & ActivitiesActivities

Assess Learner Achievement Resulting from Instruction

Dynamic & Reciprocal

Types of Assessments by Purpose Bloom’s Taxonomy as Context Aim High

Objective Assessment Selected vs Constructed Response

Subjective (Non-Objective) Assessment Writing/Generating Responses Role & Control of Subjectivity in Assessment

Alternative Assessment Authentic Assessment Performance Assessment

Objective Assessment Objective assessments (usually multiple

choice, true false, matching, short answer) have correct answers.

These are good for testing recall of facts and can be automated.

Objective tests assume that there are true answers and assume that all students should learn the same things.

Subjective (Non-Objective) Assessment With subjective assessments, teacher's

judgment determines the grade These include essay tests. Essay tests take longer to answer and they

take longer to grade than objective questions and therefore only include a small number of questions, focusing on complex concepts.

Writing/Generating Responses Role & Control of Subjectivity: Criteria &/or

Rubrics





▪ Objective vs Subjective, Etc.

4. Alternative Assessment: Performance Strategies Complementing Paper Assessment

Alternative To what? Portfolio Assessment: An

exampleAuthentic

With respect to what?Performance

What types?

Alternative Possible examples? Portfolio Assessment: Possible

examples?Authentic

Possible examples?Performance

Possible examples?

3 Formal Models of Assessment: Holistic:

Applying all criteria as a single judgment Modified Holistic:

Adding levels of judgment Analytical:

Judgment by criteria; weighting factors; scaling Valid decision-making:

Matching the “tool” and the instructional outcome Examples of Rubrics: Rubric Primer

Information Skills Oral Presentation Creating Rubrics





▪ Objective vs Subjective, Etc.4. Alternative Assessment: Performance Strategies Complementing Paper Assessment

5. Formative Assessment: Information about Teaching & Learning to Improve Instruction

Formative Assessment Types of Formative Assessment

Objective assessments▪ Forms/versions of summative assessments that will be

implemented for final evaluation of students

Alternative assessments▪ Forms/versions of summative assessments that could be

implemented for final evaluation of students

Used to provide feedback on developing learning▪ Should not contribute to final grades

May or may not be “graded” assessments▪ Limited direct impact on final evaluation of student

Observation Strategies Using explicit formal tools Informal observations

Questioning Strategies Taba Tables

Self & Peer Assessment Strategies

Alternative assessments Used to provide feedback on developing learning

Examples: Formative Evaluation Form Summary Evaluation Form Observation Record Form Observation Guide Form Observation Assessment Form Open-ended Questions





▪ Objective vs Subjective, Etc.4. Alternative Assessment: Performance Strategies Complementing Paper Assessment5. Formative Assessment: Information about Teaching & Learning to Improve Instruction

6. Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work context






6. 1. Recognize/value the need for systematic planning for assessment

2. Evaluate item types for given applications/purposes3. Construct items to meet specific Los in given learning domains4. Evaluate items/tests as to purpose/use of assessment

information

Writing assessment plans Who’s responsible for developing plans? What’s the basis for developing plans?

Developing assessment tools & items Who develops summative assessments?▪ Who implements summative assessments?▪ Who interprets data from summative

assessments? Formative assessments Who develops formative assessments?▪ Who implements formative assessments?▪ Who interprets data from formative assessments?

Developing Objective Tests & Assessments

Planning AssessmentsTypes of Objective AssessmentsSample Table of Specifications Judging Assessments


Resources online @ Writing Test ItemsTesting PrimerMatching Items to Objectives


Test Construction Activity (online @ http://people.uncw.edu/caropresoe/)

Developing Subjective (Non-objective) Tests & Assessments See Test Primer materials (online @ http://people.uncw.edu/caropresoe/)

Developing Alternative Assessments See Test Primer materials(online @ http://people.uncw.edu/caropresoe/)





▪ Objective vs Subjective, Etc.4. Alternative Assessment: Performance Strategies Complementing Paper Assessment5. Formative Assessment: Information about Teaching & Learning to Improve Instruction6. Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items

within local work context

7. Assessment & Test Construction: Technical Features & “Quality” Tools






▪ Objective vs Subjective, Etc.4. Alternative Assessment: Performance Strategies Complementing Paper Assessment5. Formative Assessment: Information about Teaching & Learning to Improve Instruction6. Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work

context7. Assessment & Test Construction: Technical Features & “Quality” Tools

Validity Reliability Difficulty & Discrimination

8. Supporting Student Learning through Assessment▪ KSU Study Strategies; Test Preps ▪ Improving Performance; MIT Testing

Strategies

Improving Student Performance on Assessments

KSU Study Strategies Test Preps Improving Performance MIT Testing Strategies

The END!!!Hopefully, a beginning of improved professional practice!!!

Edward J. Caropreso, PhDWatson School of EducationUniversity of North Carolina Wilmington

Assessment Workshop: Part 1 (June, 2007)Revised

A context for instructional decision-making Establishing vocabulary and a conceptual framework

Performance assessment as a model Developing and applying criteria Exploring formal models of performance assessment:▪ Holistic; Modified Holistic; Analytical assessment

Planning: Instruction to assessment Expectations, outcomes, tasks & assessments Developing tasks and assessments▪ Review of current assignments, tasks, tools▪ Development: Revisions and new constructions

What about instructional outcomes? Intent, expectation and instruction Two models of instructional objectives▪ Gronlund: From the general to the specific▪ Mager: From the specific to the general

What’s the relationship between teaching and learning? Instruction and assessment?

Some web sites of interest http://www.ncrel.org/sdrs/areas/misc/glossary.htm http://ctl.stanford.edu/Tomprof/postings/448.html http://www.ericdigests.org/ http://www.ericdigests.org/pre-9213/terms.htm http://www.ericdigests.org/pre-9213/

Two models

Gronlund's General and Specific Objectives

Mager's Instructional Objectives

Gronlund's General and Specific Objectives

Gronlund’s claim: Performance-oriented objectives are effective with simple skills, but

Complex cognitive behaviors require another approach

Proposes that main objectives be expressed as general instructional outcomes

Each main objective should be elaborated in terms of specific objectives; linking general, abstract level with specific, concrete performances

Mager's Instructional Objectives

Must specify clearly what the learner should be able to do following instruction. ▪ Uses behavioral terms the actual, observable performance

of the student Specifies exactly what students must do to

demonstrate that they have reached the course goal

Provides teacher with specific guidelines for determining whether course goals have been reached; Uses specific, behavioral terms as opposed to general terms (general or vague terms include: student will understand, appreciate, know, etc.)

Some useful websites on learning outcomes http://web.bsu.edu/IRAA/AA/WB/cha

pter2.htm http://chiron.valdosta.edu/whuitt/col

/plan/plan.html http://www.harding.edu/USER/dlee/

WWW/objectives.doc http://www.e-learningguru.com/artic

les/art3_4.htm http://www.nbii.gov/datainfo/metada

ta/training/ttt/skills/pdf/objectives.pdf

Assessment and Evaluation: What do they mean?

Relationship between assessment, evaluation and decision-making: The role of time

What about instructional outcomes? Instruction, assessment and evaluation Developing Criteria

Using a T-chart: Instructional intent/expectation vs. Student performance (and its

assessment)The task and the assessment:

Not the same but necessarily relatedExpanding the chart beyond the “T”:

Identifying assessment criteriaAssessment and Evidence of

Learning

T-charts in education http://www.enchantedlearning.com/gra

phicorganizers/tchart/ http://www.myread.org/

organisation.htm http://www97.intel.com/en/

ProjectDesign/InstructionalStrategies/GraphicOrganizers/T_Charts.htm

T-charts in law http://carbon.cudenver.edu/~econry/

law2.doc

From criteria as evidence to a “tool” for assessment and evaluation: Checklists vs. Rating Scales

Whose tool is it anyway?: What teachers and students should

know to ensure learning occursEvaluation decisions and “grading”

student performance

Holistic: Applying all criteria as a single judgment

Modified Holistic: Adding levels of judgment

Analytical: Judgment by criteria; weighting factors; scaling http://www.teachervision.fen.com/teaching-methods/rubrics/

4524.html?detoured=1 Valid decision-making:

Matching the “tool” and the instructional outcome Examples of Rubrics

http://www.teachervision.fen.com/teaching-methods/rubrics/4524.html

http://www.tcet.unt.edu/START/instruct/general/rubrics.htm

Your expectations, outcomes, tasks & assessments

Developing tasks and assessments Review: Current assignments, tasks,

tools Development: ▪ Revisions and reconstructions▪ New constructions

Rubrics and other assessment tools http://www.teachervision.fen.com/teaching-methods/rubrics/

4521.html?detoured=1 http://www.tcet.unt.edu/START/instruct/general/rubrics.htm http://www.tcet.unt.edu/START/instruct/general/rubrics.htm http://www.curriculum.org/csc/library/profiles/10/html/

ELDBOP5.htm http://www.inmotionaame.org/education/

lesson.cfm;jsessionid=f8303421051155499527848?migration=&id=6_003LP&bhcp=1

http://school.discovery.com/schrockguide/assess.html http://www.ncsu.edu/midlink/ho.html http://www.uwstout.edu/soe/profdev/rubrics.shtml

Introduction & Major Categories http://www.officeport.com/edu/blooms.htm http://faculty.washington.edu/krumme/guides/bloom1.html

Applying Taxonomy (Cognitive Domain/Action Verbs) http://www.coun.uvic.ca/learn/program/hndouts/bloom.html http://chiron.valdosta.edu/whuitt/col/cogsys/bloom.html http://www.teachers.ash.org.au/researchskills/dalton.htm

Across the Domains http://www.nwlink.com/~donclark/hrd/bloom.html http://www.tedi.uq.edu.au/downloads/Bloom.pd

Psychomotor Domain http://www.olemiss.edu/depts/educ_school2/docs/stai_manual/

manual10.htm http://coe.sdsu.edu/eet/articles/BloomsLD/index.htm http://academic.udayton.edu/health/syllabi/health/lesson01b.htm

developed & presented by edward j. caropreso, phd watson school of education university of north...

Documents

types of assessment

high quality assessment

state appropriate applications

objectives models

specific los

specific model

use of professional

critique objectives