David McConnellDepartment of Geology & Environmental

Science University of Akron

May 2008

SERC Woburn Workshop

Assessment of Student Learning in Geoscience Courses

Introductory Exercise 

Examine the six questions on page 2. Assume you are a student in classes where these questions would be appropriate.

Rank the questions from the easiest to most challenging based on the character of the question and nature of knowledge needed to answer it correctly.

Teaching and learning goals can be ordered using Bloom’s Taxonomy

Knowledge

Comprehension

Application

Analysis

Synthesis

Evaluation

memorization and

recall

understanding

using knowledge

taking apart

information

reorganizing

information

making judgements

Open-ended questions can be used for all categories.

Assessment of Learning Goals

Higher-order thinking

Content

Concepts

1. Knowledge

2. Comprehension

3. Application

4. Analysis

5. Synthesis

6. Evaluation

What is . . . ? Who, what, when, where, ...? Describe . . .

What would happen if . . . ? What does . . . illustrate about . . .? What is analogous to . . . ?

How could . . . be used to . . . ? What is another example of . . . ?

How does . . . affect . . . ? What are the differences (similarities) between . . . ? How does . . compare or contrast with . .?What is a solution for the problem of . . . ? How would you plan a new . . ? How does X relate to Y?

Why is . . . important? What is the best . . . , and why? Do you agree/disagree that . . . ?

Open-ended Question Stems

King, A., 1995, Teaching of Psychology, v. 22, p. 13-17.

Bloom’s Level Question Stems

Review the six questions on page 2.

Rank the questions using Bloom’s Taxonomy.

(See Summary of Bloom’s Taxonomy on page 4 if necessary.)

Introductory Exercise 

Knowledge

Comprehension

Application

Analysis

Synthesis

Evaluation

memorization and recall

understanding

using knowledge

taking apart information

reorganizing information

making judgements

Assessment of Learning Goals

Review examples of exercises used in an Earth Science course at UA and place each exercise into one of the six Bloom’s categories using the table on page 5.

Bloom’s Taxonomy Exercise Examples

Exercise Taxonomy Level

The Rock Cycle MC Questions K C Ap An S E

Atmospheric Pressure & Condensation K C Ap An S E

Life on Earth & Mars K C Ap An S E

Earthquake Warning System K C Ap An S E

Venn Diagram (Hurricanes vs. Tornadoes) K C Ap An S E

Reading Quiz: Coasts K C Ap An S E

Groundwater Rubric K C Ap An S E

Earth’s Layers Concept Map K C Ap An S E

Relative Time Diagram K C Ap An S E

Complete the scoring table on page 5

Assessment of SERC Woburn Site

Which Bloom’s Level is represented by this exercise?

Learning Module 4: Ground-Water Hydrology

1. The water-level data presented on the two maps represent information at discrete locations in the aquifer. Contouring these data requires interpolation between the known data points to produce a continuous surface known as a potentiometric surface. Using the guidelines given in the Chute reference, construct potentiometric surface maps for both sets of water-level data.

Assessment of SERC Woburn Site

Which Bloom’s Level is represented by this question?

Learning Module 2: Investigating Cancer and Clusters

After reviewing the module information on cancer and reviewing information specific to the Woburn cancer cluster, answer the following questions.

1. The Massachusetts Department of Health referred to the occurrence of cancer in the plaintiff's neighborhood as a "cluster". What is a cancer cluster and how does the medical community define a cluster?

Assessment of SERC Woburn Site

Learning Module 1: Timeline of Events

2. From the highlighted readings (Effective Expert Witness, Matson, 1999), the author provides suggestions and guidance on methods for being an effective expert witness. Refer to court testimony from days (insert specific references). Compare the testimony presented by the plaintiffs expert hydrogeologist, Dr. George Pinder and Grace's expert hydrogeologist Dr. John Guswa. Explain how each used or ignored Matsons advice regarding expert witness testimony.

Which Bloom’s Level is represented by this question?

Learning goals can be ordered using Bloom’s Taxonomy – but how do you assess student work?

Knowledge

Comprehension

Application

Analysis

Synthesis

Evaluation

Degrees of correctnes

s

Right/Wrong answers

Open-ended questions can be used for all categories.

Assessment of Learning Goals

Variation in form/conten

t of answers

More complex questions call for more sophisticated guides and responses

Writing Assessment with Rubrics

A rubric is a scoring protocol that:

• Specifies criteria and a measurement scale for different levels of proficiency.

• Establish criteria (e.g., presentation, reasoning) tied to exercise learning goals.

• Determine categories (e.g., needs improvement, satisfactory, good, exemplary) and scoring scale for components of answer.

• Analytic vs. Holistic

Sample Analytic Rubric

Level of Achievement General Presentation Reasoning, Argumentation

Exemplary (10 pts)

1. Provides a clear and thorough introduction and background

2. Addresses the question3. Presents arguments in a logical order4. Uses acceptable style and grammar (no errors)

1. Demonstrates an accurate and complete understanding of the question

2. Uses several arguments and backs arguments with examples, data that support the conclusion

Quality (8 pts)

1. Combination of above traits, but less consistently represented (1-2 errors)

2. Same as above but less thorough, still accurate

1. Uses only one argument and example that supports conclusion

Adequate (6 pts)

1. Does not address the question explicitly, though does so tangentially

2. States a somewhat relevant argument3. Presents some arguments in a logical order4. Uses adequate style and grammar (more than 2

errors)

1. Demonstrates minimal understanding of question, still accurate

2. Uses a small subset of possible ideas for support of the argument.

Needs improvement (4 pts)

1.Does not address the question2.States no relevant arguments3.Is not clearly or logically organized4.Fails to use acceptable style and grammar

1. Does not demonstrate understanding of the question, inaccurate

2. Does not provide evidence to support response to the question

No Answer (0 pts)

Rubric by Dr. Diane Ebert-May, Botany & Plant Pathology, Michigan State University.

Open-ended Questions and Rubrics

(CT)2: Critical Thinking for Civic ThinkingRead scenario and answer critical thinking and civic thinking questions with help of taxonomies.

Critical Thinking Question

Some residents have argued that pollution from the factory is causing cancer in their community. Identify and explain strengths and weaknesses of this conclusion.

Civic Thinking Question

As an affected citizen of this community, identify and explain a course, or courses, of action that you personally could take to address this situation effectively and responsibly.

SOLO Taxonomy (Holistic Rubric)

Biggs, J.B., and Collis, K.F., 1982, Evaluating the Quality of Learning: The SOLO taxonomy (Structure of Observed Learning Outcomes), Academic Press.

Level Level Characteristics

Prestructural No understanding demonstrated. Response does not address the question or restates the question.

Unistructural Limited understanding of topic. Response focuses on one conceptual item in a complex case.

Multistructural Understanding of several discrete components. Response is a collection of multiple items that are not related within the context of the exercise.

Relational Understanding of several components that are integrated conceptually. Response prioritizes information and is appropriate to the scale of the question.

Extended Abstract

Understanding demonstrated at a level extending beyond what has been dealt with in the question prompt. Response generalizes to situations beyond the scope of the question.

Answer Type

Points Answer Components Remarks Errors

Prestructural answer

0-10 

Response does not address exercise or restates exercise.  

  

Score reduced by 1 or more

points per error or for

incomplete or poorly written

answers.

Unistructural answer 

11-12 Only advantages or disadvantages described, key idea not discussed

Up to 3 additional points for your statements for/against idea and/or additional data.

13-14 One advantage and one disadvantage are partially defined, idea not discussed

15-16 One advantage and one disadvantage are well defined, idea not discussed

17 One advantage and one disadvantage are partially defined, idea is discussed

18 

One advantage and one disadvantage are well defined, idea is discussed

Multistructural answer 

19-20 Multiple advantages and disadvantages partially or well defined, article idea not discussed Up to 3 additional

points for your statements for/against idea and/or additional data.

21-22 Multiple advantages and disadvantages partially defined, article idea is discussed

23-24 

Multiple advantages and disadvantages well defined, key idea is discussed 

Relational or Extended Abstract answer

25-27 Multiple advantages and disadvantages well defined, key idea is discussed, and either answer presents case for/against ideas described or includes a description of new data needed.

Additional points may be awarded for examples or analogies beyond those described in article.

28-30 Multiple advantages and disadvantages well defined, key idea is discussed, and answer presents case for/against ideas described and answer includes a description of new data needed.

SOLO Taxonomy (Analytic Rubric)

(Level 0: I have never heard of Bloom’s Taxonomy.)

Level 1: I can identify how many levels there are in BT.

Level 2: I can place several levels of BT in correct order.

Level 3: I can describe the six categories of BT.

Level 4: I can explain how I might use BT in planning a lesson.

Comprehension Survey #1

Teaching Goal 1:

To understand the features of Bloom’s Taxonomy

Comprehension Survey #2

Teaching Goal 2:

To create exercises representing all categories of Bloom’s Taxonomy

Level 1: I can separate out low level and high level questions on the basis of BT.

Level 2: I can place simple multiple choice and essay questions using BT.

Level 3: I can analyze how to classify a series of exercises based on the categories of BT.

Level 4: I can make up questions representative of the six categories of BT.

1. Review existing Learning Modules to assess learning goals and appropriate level of questions.

2. Ensure exercises have multiple levels of learning and sufficient scaffolding of concepts.

3. Consider time and background knowledge necessary to complete exercises.

4. Write and post new questions that integrate multiple levels of Bloom’s Taxonomy.

5. Consider the components of a thorough answer and the development of related rubrics.

Assessment of Woburn Modules