Gaining A Geological Perspective Through Gaining A Geological Perspective Through Active Learning in the Large Lecture ClassroomActive Learning in the Large Lecture Classroom

Jessica Kapp, Randy Richardson, Allister Jessica Kapp, Randy Richardson, Allister Rees, Jon Patchett, Ross WaldripRees, Jon Patchett, Ross Waldrip

University of Arizona Department of University of Arizona Department of GeosciencesGeosciences

TraditionalTraditional Geological Geological Perspective Perspective

Tier 1 Gen Ed class for non-science majorsTier 1 Gen Ed class for non-science majors ~1200 students per year (~600 per semester)~1200 students per year (~600 per semester) ~74% freshman~74% freshman Four lectures per semester taught by 3 facultyFour lectures per semester taught by 3 faculty Lectures are ~150 students eachLectures are ~150 students each Optional study groupsOptional study groups Traditionally 8 GTAs and ~10 undergraduate Traditionally 8 GTAs and ~10 undergraduate

preceptors run study groupspreceptors run study groups Traditional lecture-style presentation with activities Traditional lecture-style presentation with activities

peppered throughout (varies based on faculty member)peppered throughout (varies based on faculty member) Instructional technology varies – animations, movies, Instructional technology varies – animations, movies,

demonstrationsdemonstrations

Goals of Geology RedesignGoals of Geology Redesign1. Develop consistency between lectures so that all 1. Develop consistency between lectures so that all

use an active learning approachuse an active learning approach

2. Reduce instructor face time and increase time spent 2. Reduce instructor face time and increase time spent peer learning (with other students or preceptors)peer learning (with other students or preceptors)

3. Reduce faculty time devoted to preparation and 3. Reduce faculty time devoted to preparation and implementation of the courseimplementation of the course

4. Reduce cost per student without compromising 4. Reduce cost per student without compromising learner outcomes and/or student perception of the learner outcomes and/or student perception of the classclass

5. Maintain or increase level of rigor of the course 5. Maintain or increase level of rigor of the course while increasing student enthusiasm/interestwhile increasing student enthusiasm/interest

No doubt we cut costsNo doubt we cut costsCost for teaching NATS 101

0

20000

40000

60000

80000

100000

120000

140000

160000

Faculty, TAs, Staff, Preceptors

Cost

per

sem

este

r ($)

TraditionalRedesigned

No doubt we cut costsNo doubt we cut costsHours Devoted to NATS 101

0

500

1000

1500

2000

2500

Faculty, TA, Staff, Preceptors

Hou

rs p

er s

emes

ter

TraditionalRedesigned

No doubt we cut costsNo doubt we cut costsCost per student per semester for teaching NATS 101

0

50

100

150

200

250

300

350

400

450

500

Cos

t ($)

per

stu

dent

per

sem

este

r

Traditional

Redesigned

Key features of redesigned Key features of redesigned coursecourse

Pre-class on-line reading quizzesPre-class on-line reading quizzes

In-class activities that promote In-class activities that promote student interaction and higher-level student interaction and higher-level learning (some graded, some not)learning (some graded, some not)

Mandatory break out sessions once Mandatory break out sessions once a week led by fewer GTAs and a week led by fewer GTAs and more preceptorsmore preceptors

Text – Text – The Good EarthThe Good Earth (McConnell (McConnell et al.)et al.)

Key features of redesigned Key features of redesigned coursecourse

All lectures prepared All lectures prepared ahead by J. Kapp, all ahead by J. Kapp, all instructors use same instructors use same lectureslectures

All lectures include built All lectures include built in activities that are in activities that are easy to gradeeasy to grade

No clickers – several No clickers – several studies are showing studies are showing they are not as effective they are not as effective at enhancing learning as at enhancing learning as they are at making it they are at making it easy to grade stuffeasy to grade stuff

Geosciences Concept Inventory Geosciences Concept Inventory 1.0 (Libarkin et al.) 1.0 (Libarkin et al.)

78 items, administered in three groups78 items, administered in three groups Only the 36 items most related to course Only the 36 items most related to course

were analyzedwere analyzed Quantitative measure of student learningQuantitative measure of student learning

Examples of GCI questions:Examples of GCI questions: If you could travel back in time to when the Earth first formed as a planet, what would If you could travel back in time to when the Earth first formed as a planet, what would

the Earth look like?the Earth look like? A. The Earth would be mostly covered with waterA. The Earth would be mostly covered with water B. The Earth would be mostly moltenB. The Earth would be mostly molten C. The Earth would be mostly covered with iceC. The Earth would be mostly covered with ice D. The Earth would be mostly rockyD. The Earth would be mostly rocky

If you could travel millions of years into the future, how big would the planet Earth be?If you could travel millions of years into the future, how big would the planet Earth be? A. Smaller than todayA. Smaller than today B. Larger than todayB. Larger than today C. Same size as todayC. Same size as today D. We have no way of knowingD. We have no way of knowing

Pilot semester learning Pilot semester learning outcomesoutcomes

Geosciences Concept InventoryGeosciences Concept Inventory GCI Traditional course: GCI Traditional course:

pre-test % correct = 28.67 (SD=11.45, n=96) pre-test % correct = 28.67 (SD=11.45, n=96) post-test % correct = 45.26 (SD=11.76, post-test % correct = 45.26 (SD=11.76,

n=84) n=84) GCI redesigned course: GCI redesigned course:

pre-test % correct = 38.62 (SD=8.42, n=144) pre-test % correct = 38.62 (SD=8.42, n=144) post-test % correct = 48.73 (SD=7.49, post-test % correct = 48.73 (SD=7.49,

n=132)n=132) full implementation post-test % correct = full implementation post-test % correct =

47.11 (SD = 9.01, n + 117)47.11 (SD = 9.01, n + 117)

-Gains from pre-test to post-test are statistically significant, but Gains from pre-test to post-test are statistically significant, but the difference in post-test GCI scores between the two groups is the difference in post-test GCI scores between the two groups is not. not. -Interpretation: Students’ knowledge levels, insofar as the GCI Interpretation: Students’ knowledge levels, insofar as the GCI can measure, were equivalent in both courses.can measure, were equivalent in both courses.

Other impacts of redesignOther impacts of redesign Analysis of common essay-style Analysis of common essay-style

exam questionsexam questions

Students’ written responses were Students’ written responses were analyzed in terms of three categories: analyzed in terms of three categories: (1) accuracy and completeness of the (1) accuracy and completeness of the response; (2) number of words; (3) response; (2) number of words; (3) numerical points assigned by gradersnumerical points assigned by graders

In all categories, the student-supplied In all categories, the student-supplied

responses from the two courses were responses from the two courses were indistinguishableindistinguishable

Common on-line quiz resultsCommon on-line quiz results Quiz on EarthquakesQuiz on Earthquakes Traditional course: Average = 75% (SD = 17)Traditional course: Average = 75% (SD = 17) Redesigned course: Average = 85% (No SD)Redesigned course: Average = 85% (No SD)

This seems significant BUT…This seems significant BUT…- We changed the wording of a few questionsWe changed the wording of a few questions- The lecture material changed a bit The lecture material changed a bit - These changes were made in response to These changes were made in response to

students’ poor performance on certain students’ poor performance on certain questionsquestions

Class SuccessClass Success Student success rates (C or higher in the course) Student success rates (C or higher in the course)

were 90% in the traditional course, 87.2% in the were 90% in the traditional course, 87.2% in the pilot semester, and 89.6% in the full redesign. The pilot semester, and 89.6% in the full redesign. The differences are statistically insignificant. differences are statistically insignificant.

Withdraw/drop rates were 3% in the traditional Withdraw/drop rates were 3% in the traditional course, 0.5% in the pilot semester, and 5.6% in the course, 0.5% in the pilot semester, and 5.6% in the full redesign. full redesign.

This last value is skewed, as in our full This last value is skewed, as in our full implementation semester our classes were implementation semester our classes were overenrolled without our consent or knowledge overenrolled without our consent or knowledge before the semester began, causing what before the semester began, causing what appeared to be an excessive drop rate. appeared to be an excessive drop rate.

Other impacts of redesignOther impacts of redesign Attitudes Toward Science – no statistical Attitudes Toward Science – no statistical

differencesdifferences Ratings on course evaluations unchangedRatings on course evaluations unchanged Lecture attendance virtually unchangedLecture attendance virtually unchanged Students positive about usefulness of Students positive about usefulness of

In class exercisesIn class exercises Break out sessionsBreak out sessions

Interview and focus group transcripts Interview and focus group transcripts reveal that students find the reformed reveal that students find the reformed course to be relevant to their lives and course to be relevant to their lives and educationally satisfying, both of which educationally satisfying, both of which are only infrequently observed among are only infrequently observed among traditional introductory science survey traditional introductory science survey courses courses

Issues and FindingsIssues and Findings Aside from J. Kapp, instructors vary.Aside from J. Kapp, instructors vary.

Instructors approach material differently.Instructors approach material differently. On-line course management system used in On-line course management system used in

traditional class had to be abandoned due to traditional class had to be abandoned due to glitches, lack of tools. Hard to compare results.glitches, lack of tools. Hard to compare results.

Grading still time consuming but students Grading still time consuming but students prefer the face time, and in-class assignments prefer the face time, and in-class assignments get them to class.get them to class.

Exams – still prefer some form of short answer Exams – still prefer some form of short answer (grading).(grading).

Issues and FindingsIssues and Findings PRECEPTORS!PRECEPTORS!

Great resource, students like themGreat resource, students like them Problem 1: High turn over rate. Very few Problem 1: High turn over rate. Very few

stay for several semestersstay for several semesters Problem 2: Lack of expertise in GeoscienceProblem 2: Lack of expertise in Geoscience Problem 3: Lack of confidence leading a Problem 3: Lack of confidence leading a

break out sessionbreak out session Rare (but real) problem: Some issues with Rare (but real) problem: Some issues with

boundaries, authority, etc.boundaries, authority, etc.

We attempted to address #3 with a training workshop for our preceptors. They enjoyed this very much and said they learned a lot about teaching techniques. However, it doesn’t change the fact that they are nervous about answering questions they don’t know the answer to (lack of expertise).

And most interesting?And most interesting? Interviews with students, TAs, preceptors, and Interviews with students, TAs, preceptors, and

faculty.faculty. More than 100 people were involved and 14 hours More than 100 people were involved and 14 hours

of transcripts were produced, in addition to the of transcripts were produced, in addition to the evaluators’ field notes. evaluators’ field notes.

All transcripts were inductively analyzed and All transcripts were inductively analyzed and reanalyzed until repeated themes emerged from reanalyzed until repeated themes emerged from the data across all transcripts. the data across all transcripts.

Four recurring themes were prominent across the Four recurring themes were prominent across the extensive qualitative data collected (listed by extensive qualitative data collected (listed by perceived importance to the success and value of perceived importance to the success and value of the supporting learning in the class) the supporting learning in the class)

1. Structured discussion 1. Structured discussion among students was among students was

meaningfulmeaningful-Students and instructors alike described the course as filled with repeated opportunities to absorb, process, and apply lecture information in guided and supportive settings. -Weekly discussion group sections provided time-on-task to “absorb” the ideas.-Discussion group activities focused on real life applications.-Most students understood discussion sessions were not intended to provide new information, but rather to emphasized the important aspects of lecture.-Several students agreed with the comment that, “I’m only doing well in this course because of the discussions.” -GTAs often felt that the cognitive level required of students in the discussion sections, and perhaps the course overall, was too low. -Undergraduate peer mentor instructors felt that the graduate teaching assistants, who were new experts in the field, didn’t have a reasonable opinion of what non-science majoring students needed to understand.

2. The interactive nature of the 2. The interactive nature of the course engaged students and course engaged students and

instructorsinstructors Students understood that the interactive, participatory nature of the course was Students understood that the interactive, participatory nature of the course was

substantively and crucially different than their other courses they were taking.substantively and crucially different than their other courses they were taking.

Students understood how the think-pair-share questions required students to Students understood how the think-pair-share questions required students to actively process information rather than engaging in passive, and occasionally actively process information rather than engaging in passive, and occasionally meaningless, “random note-taking” so often required in other classes.meaningless, “random note-taking” so often required in other classes.

““If you don’t pay attention you won’t do well, but you should because those are If you don’t pay attention you won’t do well, but you should because those are easy points to help your grade.”easy points to help your grade.”

Students also felt that the collaborative nature of the class, both in lectures and Students also felt that the collaborative nature of the class, both in lectures and during the required discussion sessions, helped them build a learning during the required discussion sessions, helped them build a learning community and that this sense of community led to increased interaction community and that this sense of community led to increased interaction between students in lecture.between students in lecture.

““You can’t do this kind of thing with 100 people who don’t know each other.”You can’t do this kind of thing with 100 people who don’t know each other.”

3. The non-adversarial nature 3. The non-adversarial nature of the course climate was of the course climate was

critical critical Applauded the course instructors on the emphasis and repeated Applauded the course instructors on the emphasis and repeated

importance of clear expectations. importance of clear expectations.

““There are no secrets in this class. They tell us over and over. If you There are no secrets in this class. They tell us over and over. If you don’t know what it takes to be successful, you aren’t paying attention.”don’t know what it takes to be successful, you aren’t paying attention.”

Students overwhelmingly wanted to comment enthusiastically on their Students overwhelmingly wanted to comment enthusiastically on their perceived instructors’ attitudes toward the students. The felt that perceived instructors’ attitudes toward the students. The felt that instructors cared about their learning, as evidenced in a number of waysinstructors cared about their learning, as evidenced in a number of ways instructors created a low-risk environmentinstructors created a low-risk environment professors demonstrated respect by learning everyone’s namesprofessors demonstrated respect by learning everyone’s names professors emailed students and greeted them by name on campusprofessors emailed students and greeted them by name on campus

4. The purposeful instructional 4. The purposeful instructional activities directly support activities directly support

metacognitionmetacognition Great value in seeing a variety of Great value in seeing a variety of

teaching modalities used in the class. teaching modalities used in the class. Students recognized that professors Students recognized that professors

had a reduced focus on vocabulary. had a reduced focus on vocabulary. Having shorter lecture periods Having shorter lecture periods

punctuated with purposeful punctuated with purposeful interactivity served to “reset” interactivity served to “reset” attention spans. attention spans.

Full ImplementationFull Implementation We continue to use consistent, pre-made We continue to use consistent, pre-made

lectures with built in activities lectures with built in activities Continuing effort to enhance lectures with Continuing effort to enhance lectures with

more multimedia material (you tube clips, more multimedia material (you tube clips, animations, video) animations, video)

Exam questions focus more on higher level Exam questions focus more on higher level learning (writing better multiple choice learning (writing better multiple choice questions)questions)

We continue to use an abundance of We continue to use an abundance of preceptors (29 this semester!)preceptors (29 this semester!)

We want to use more GTAs, as the We want to use more GTAs, as the preceptor/TA issue is paramountpreceptor/TA issue is paramount

Contact meContact meJessica Kapp Department of Jessica Kapp Department of

GeosciencesGeosciencesUniversity of ArizonaUniversity of Arizona

jkapp@email.arizona.edujkapp@email.arizona.edu(520) 626-5701(520) 626-5701