Creating Effective Outlines and Assessments

New Faculty OrientationAugust 21, 2012

Dr. Gosha Zywno, Faculty Associate, LTO3M National Teaching Fellow

Outlines at Ryerson must contain(see Policy 145 for more detail):• Course name, course number, semester and year• Prerequisites and exclusions• Professors name, contact information, office hours• Method of posting grades• Use of email instruction for faculty communication• Course calendar description• Course objectives and focus• Topics and schedule of topics• Type of teaching methods• Outside class activities, if any• Requirements for submission of work• Plagiarism detection tools used (Academic Integrity Office has a proper

statement)- deadlines for choosing not to be included in use of these tools• Field placement and technology requirements• Policy on use of electronic equipment in class• Policy on missed work

Good course outlines consider instructional design

• All components of the course should consider the learning objectives

• This includes the course content, assessments, activities, feedback to students

• Outlines in different fields are slightly different (you will have a chance to view some from different fields)

Good course outlines consider instructional design

The goals/objectives of the course are explicitly statedExample of learning objectives:

• To become familiar with the basic terms, concepts, principles, and techniques of research methods and statistics

• To appreciate the relations between the hypotheses being tested (question asked) and the most appropriate statistical test to use

• To become proficient in using SPSS software to conduct various forms of basic data analysis

• To feel equipped by term’s end to apply the knowledge and skills gained to better understanding the statistical methods and results in published literature and to conduct one’s own analyses

• To understand how to interpret the results from these analyses

AlignmentWhat are the desired learning outcomes?

“Writing Learning Outcomes,” British Columbia Institute of Technology Learning Resources Unit, https://helpdesk.bcit.ca/fsr/teach/courseprep/htoutcomes.pdf

Bloom’s Taxonomy• Benjamin Bloom (1956),

identified three domains of educational activities:

• Cognitive: mental skills (Knowledge)

• Affective: growth in feelings or emotional areas (Attitude)

• Psychomotor: manual or physical skills (Skills)

Think-Pair-Share Activity• Reflect individually on the topic

(THINK)• Turn to your neighbour (PAIR)

– Logistics: introduce yourself, decide Recorder-Reporter roles

• Recorder – your birth date is closest to today: take notes of your discussion

• Reporter – you may be called on to report to the large forum

– Take 5 minutes to discuss/record the topic

• Report back to the larger group (SHARE)– Some pairs may be called on to

report results

Total time: 15 minutes

Think-Pair-Share Activity• Logistics:

– Think (individual) – 90 sec– Pair (discuss) – 5 minutes– Share (large group) – 5 minutes

• Topic:– In a course you will be teaching,

identify one learning objective in each of the three domains (cognitive, affective, psychomotor)

Total time: 15 minutes

Decide at what cognitive level you want to accomplish your goals:

Different levels of cognitive learning

Bloom’s Taxonomy

New Version Old Version

http://www.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm http://www.officeport.com/edu/blooms.htm

Bloom’s Taxonomy

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure knowledge (common terms, routine info, definitions, descriptions, facts, principles, procedures), ask these kinds of questions: Define, Describe, Identify, Label, List, Match, Name, Outline, Reproduce, Select, State.

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

Example: ???

Bloom’s Taxonomy

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure knowledge (common terms, routine info, definitions, descriptions, facts, principles, procedures), ask these kinds of questions: Define, Describe, Identify, Label, List, Match, Name, Outline, Reproduce, Select, State.

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

List steps involved in titration

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure comprehension (understanding of facts and principles, interpretation of material), ask these kinds of questions: Convert, Defend, Distinguish, Estimate, Explain, Extend, Generalize, Give examples, Infer, Predict, Summarize.

Bloom’s Taxonomy

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

Example: ???

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure comprehension (understanding of facts and principles, interpretation of material), ask these kinds of questions: Convert, Defend, Distinguish, Estimate, Explain, Extend, Generalize, Give examples, Infer, Predict, Summarize.

Bloom’s Taxonomy

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

Summarize basic tenets of deconstructionism

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure application (solving problems, applying concepts and principles to new situations, using rules to perform calculations), ask these kinds of questions: Demonstrate, Modify, Operate, Prepare, Produce, Relate, Show, Solve, Use.

Bloom’s Taxonomy

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

Example: ???

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure application (solving problems, applying concepts and principles to new situations, using rules to perform calculations), ask these kinds of questions: Demonstrate, Modify, Operate, Prepare, Produce, Relate, Show, Solve, Use.

Bloom’s Taxonomy

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

Calculate the deflection of a beam under uniform loading

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure analysis (recognition of unstated assumptions or logical fallacies, ability to distinguish between facts and inferences, breakdown of problems to see structure, determine properties, see connections), ask these kinds of questions: Diagram, Differentiate, Distinguish, Illustrate, Infer, Point out, Relate, Select, Separate, Subdivide.

Bloom’s Taxonomy

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

Example: ???

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure analysis (recognition of unstated assumptions or logical fallacies, ability to distinguish between facts and inferences, breakdown of problems to see structure, determine properties, see connections), ask these kinds of questions: Diagram, Differentiate, Distinguish, Illustrate, Infer, Point out, Relate, Select, Separate, Subdivide.

Bloom’s Taxonomy

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

In the speech discussed, which statements are based on facts and which are based on assumptions?

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure synthesis (integrate learning from different areas or solve problems by creative thinking, design, form a new system or solution), ask these kinds of questions: Categorize, Combine, Compile, Devise, Design, Explain, Generate, Organize, Plan, Rearrange, Reconstruct, Revise, Tell.

Bloom’s Taxonomy

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

Example: ???

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure synthesis (integrate learning from different areas or solve problems by creative thinking, design, form a new system or solution), ask these kinds of questions: Categorize, Combine, Compile, Devise, Design, Explain, Generate, Organize, Plan, Rearrange, Reconstruct, Revise, Tell.

Bloom’s Taxonomy

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

How would you restructure the school day to reflect children's developmental needs?

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure evaluation (judging and assessing), ask these kinds of questions: Appraise, Compare, Conclude, Contrast, Criticize, Describe, Discriminate, Explain, Justify, Interpret, Support.

Bloom’s Taxonomy

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

Example: ???

reca ll

K now le dge

draw

C om prehens ion

ca lcu la te

A pplic a tion

categorize

A na lys is

design

S ynthes is

assess

E va lua tion

C ognitive D om ain

To measure evaluation (judging and assessing), ask these kinds of questions: Appraise, Compare, Conclude, Contrast, Criticize, Describe, Discriminate, Explain, Justify, Interpret, Support.

Bloom’s Taxonomy

Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 http://oed.berkeley.edu/bgd/quizzes.html

Why is Bach's Mass in B Minor acknowledged as a classic?

Bloom’s Taxonomy

Kelley, K. & McDonald, R. Information Literacy and Writing Assessment Project: Tutorial for Developing and Evaluating Assignments. University of Maryland University College.http://www.umuc.edu/library/libhow/informationliteracy_tutorial.cfm

The level of cognitive learning reached is dependent upon the type of activities assigned. Activities must be designed in order to achieve the desired level of cognitive learning.

Assessment Techniques Bloom’s Cognitive Level Reached

Exams & Quizzes:

• Multiple Choice•Fill in•Essay

Knowledge, Comprehension, Application

Homework Knowledge, Comprehension, Application

Problem Solving Knowledge, Comprehension, Application

Case Studies Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation

Journals Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation

Research Reports Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation

Portfolios Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation

http://www.fgcu.edu/onlinedesign/designDevd.html

AlignmentDesign assignments (and activities) to align with your desired course outcomes; some questions to ask:• What are the outcomes to be

assessed?• What are the capabilities/skills

(implicit or explicit) in the outcomes?• Does the specific assessment task

match the outcomes and skills?• Is the method efficient in terms of

student time and staff time?• What alternatives are there? What are

their advantages and disadvantages?• Are the marking schemes or criteria

appropriate?

Brown, G. (2001). Assessment: A Guide for Lecturers. The LTSN Generic Centre Assessment Series. http://www.heacademy.ac.uk/resources/detail/assessment/assessment_series

Effective Assessments - Summary• Align with Learning Objectives & Activities• Match appropriate level of Cognitive Learning• Have clear expectations • Have reasonable workloads• Engage students with material, are meaningful• Have fair and consistent grading - use rubrics

Effective Assessments - Summary• Align with Learning Objectives & Activities• Match appropriate level of Cognitive Learning• Have clear expectations • Have reasonable workloads• Engage students with material, are meaningful• Have fair and consistent grading - use rubrics• Encourage Academic Integrity• Support Information Literacy• Support EAL Students• Are Accessible (Universal Instructional Design)

Information LiteracyAn information literate student is able to:

• Determine the extent of information needed• Access the information effectively and efficiently• Evaluate information and its sources critically• Use information effectively to accomplish a

specific purpose• Understand the economic, legal and social issues

surrounding the use of information, and access and use information ethically and legally

Information LiteracyInformation literacy-based assessments:

• Encourage self-directed learning and critical thinking

• Favor analysis and interpretation over rote-learning

• Require research and evaluation on a variety of resources

• Provide concrete feedback on progress

Effective AssessmentsInformation Literacy Example

• Follow a research trend. • Select a present day issue or concern and then

search the past literature at five or ten year intervals.

• Discuss how theories or attitudes have endured or changed.

Touro College Libraries. http://www.tourolib.org/services/information-literacy/assignments

Avoid cheating and plagiarism:• Make assignments unique and personal• Vary assignments and rotate tests• Request rough drafts and description of research

methods, or assign work in steps• Be specific about grading criteria, consistent in

grading

For more information, visit: http://www.ryerson.ca/academicintegrity/

Make assignments as specific as possible:• Use more obscure material• Pose a more focused question• Require application rather than explanation• Consider a tight comparison• Make assignments unique and personal, change

oftenDiscouraging Plagiarism. Center for Innovative Teaching and Learning, Indiana University Bloomington. http://www.iub.edu/~citl/services/writing/plagiarism.php

Effective Assessments

Academic Integrity Example• Rather than: Explain the basic functions of the

vascular, skeletal, muscular and nervous systems• Try: A cat jumps off the end of a table onto the floor.

Describe how its vascular, muscular, skeletal and nervous systems contribute to this action.

Discouraging Plagiarism. Center for Innovative Teaching and Learning, Indiana University Bloomington. http://www.iub.edu/~citl/services/writing/plagiarism.php

• Break down assignments• Gives students time to ask questions• Encourage a simple, direct style• Emphasize ‘process’ rather than ‘product’• Incorporate low-stakes writing assignments• Provide models• Use explicit rubrics

http://www.ryerson.ca/studentservices/els/

Effective Assessments

EAL Learners Example• Use familiar, high frequency words, rather than low

frequency, academic or idiomatic language• Rather than: “Mitotic cell division is initiated in the

______.”• Try: “Mitotic cell division begins in the ______.”

Teemant, A. (2010). ESL Student Perspectives on University Classroom Testing Practices. Journal of the Scholarship of Teaching and Learning, 10(3), 89-105.

AccessibilityUniversal Instructional Design creates barrier free assignments

that benefit all students

• #1: Design instructions with the user in mind • Present information in small chunks separated by headers

• Put key tasks and dates in bold • Use a large, clear font

•#2: Reduce barriers to accessing the assignment• Provide access to instructions and supplemental material in multiple formats

Four Ways to Incorporate Universal Design for Learning into Your Assignment Instructions http://gwired.gwu.edu/dss/Newsletters/Fall05UDL/

Accessibility

• # 3 List assignment goals and evaluation criteria from the start

• With focus on objectives, it is easier to be flexible without losing academic rigour

• # 4 Provide Examples• Show different approaches to the assignment

• Get permission to share examples of previous students’ work

• Post samples in an accessible electronic format on Blackboard or class website

Four Ways to Incorporate Universal Design for Learning into Your Assignment Instructions http://gwired.gwu.edu/dss/Newsletters/Fall05UDL/

Universal Instructional Design creates barrier free assignments that benefit all students