LEARNERS & LEARNING ENVIRONMENTS:Comparison of cognition level between

lecture-based classroom and experiential study abroad assessment

Abigail J. Lynch and William W. TaylorMichigan State University, Department of Fisheries and Wildlife, Center for Systems Integration and Sustainability

30 April 2013FAST Fellowship Symposium

Research need Research approach Preliminary results Implications

Fostering learning environments which result

in learning.

What is our objective as instructors?2

Evaluating effectiveness of learning environments to

ensure learning.

What is our objective as scholars?

Research need

Research approach

Preliminary results

Implications

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Study abroad: not available on campus

“My study abroad gave me tools and experiences that studying on campus could never provide. In fact, without studying abroad, I would never have known how valuable the experiences were.”

3

Research need

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Study abroad: experiential learning

“Most of the time they not only told us about their work and knowledge, but took us out into the field to show us too. This alone made this program a unique educational experience, because rather than just talking about what we were learning, we actually went to see where policy making or science and environmental restoration work takes place.”

4

Research need

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Study abroad: experience with impact

“This learning did not happen in a classroom and there was barely any lecture time…These people had devoted their lives to fish and water preservation, so at no point were they teaching us. They were actually physically showing us the things they were most passionate about. This is a type of learning that makes an impact on students.”

5

Research need

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Learning Environments

Lecture-based Experiential

Traditional Classroom Study Abroad

zaidlearn.blogspot.com

How can you compare?

Research need

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Comparing learning environments Foreign language competency

Segalowitz et al. (2004): U.S. classroom vs. study abroad in Spain Study abroad students: significantly greater

gains in oral proficiency, oral fluency but not grammar or pronunciation.

Cognitive thresholds for second language learning readiness (regardless of location)

7

Research need

Foreign language competency Segalowitz et al. (2004): U.S. classroom vs.

study abroad in Spain Freed et al. (2004): U.S. classroom vs.

summer immersion vs. study abroad in France Immersion students: significant gains in oral

proficiency Study abroad students: significant gains in speech

fluidity Classroom students: no statistically significant

gains Gains related more to hrs/week speaking and

writing French than anything else

Foreign language competency Segalowitz et al. (2004): U.S. classroom

vs. study abroad in Spain Freed et al. (2004): U.S. classroom vs.

summer immersion vs. study abroad Other subject areas?

Limited opportunities for paired design

Foreign language competency

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Paired comparison: FW 481Global issues in Fisheries and Wildlife

Classroom: Spring 2010, Spring 2012

Study abroad: Summer 2011, Summer 2013 Same learning objectives

Same essay assignments Similar number of students Similar demographics

8

Research approach

Classroom: Spring 2010, Spring 2012

Study abroad: Summer 2011, Summer 2013

Research need Research approach Preliminary results Implications

To compare cognition level between lecture-based students and experiential study-abroad students

Teaching-as-Research Objective

9

Research approach

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Cognition level

Bloom’s Taxonomy (1956): hierarchical classification of thinking skills

Most assessment is at lower levels of the taxonomy

Students remember more when they handle a topic at the higher levels of the taxonomy

Garavalia, Hummel, Wiley, & Huitt (1999)

10

Research approach

edweb.sdsu.edu 

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

“Blooming”

Scored essay assignments using Bloom’s Taxonomy (1956) Coded for highest competence at the

sentence, paragraph, and paper level 1: knowledge 2: comprehension 3: application 4: analysis 5: synthesis and evaluation

Standardized by number of sentences, paragraphs

Weighted by each level 0.5: sentence 0.3: paragraph 0.2: paper

11

Research approach

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

“Blooming” example12

Research approach

cognitive level sentences paragraphs

paper

knowledge 72 ⨯ 1 = 72

comprehension 38 ⨯ 2 = 76

application 22 ⨯ 3 = 66

analysis 15 ⨯ 4 = 60 4 ⨯ 4 = 16

synthesis and evaluation

8 ⨯ 5 = 40 3 ⨯ 5 = 15 1 ⨯ 5 = 5

sum 292 31 5

Scored essay assignments using Bloom’s Taxonomy (1956) Coded for highest competence

at the sentence, paragraph, and paper level

Standardized by number of sentences, paragraphs

Weighted by each level

full score

Research need Research approach Preliminary results Implications

Is there a difference between cognition level between:

• Classroom and study abroad students?

• Students the beginning and end of the course?

• Students with stated preference for active over lecture-based pedagogy?

Research Questions13

Research approach

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Learning environment effects Could learning environment affect

demonstrated cognition level? T-test for learning environment effects:

Is there a difference between demonstrated cognition level in the final essay between study abroad and classroom students?

14

Preliminary results

t = 4.0933df = 13.947

p-value = 0.001105

study abroad classroom

study abroad essay mean significantly higher cognition level than classroom essay mean

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Student effects

Could the course influence demonstrated cognition level? Self-selection? T-test for prior effects: Is there a

difference in the cognition level between the beginning and end of the study abroad course?

15

Preliminary results

t = -3.1145df = 7p-value = 0.01697

initial final

final essay mean significantly higher cognition level than initial essay mean

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Student effects

Could the course influence demonstrated cognition level? Self-selection?

Could pedagogical preference affect demonstrated cognition level? T-test for pedagogical effects: Is there a

difference in demonstrated cognition level between students who prefer active vs. lecture-based pedagogy?

16

Preliminary results

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Lecture vs. active learning preferences

Conducted a unit of the 2012 classroom course (three class periods) using different pedagogical styles Lecture Active learning

jig-saw and problem-based learning Distributed survey on pedagogical

preferences Likert scale Open ended questions

17

Preliminary results

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Lecture vs. active learning preferences

“Greater trust in [lecture] information because it comes from the TA/professor instead of classmates.”

“I like small group discussions because you hear multiple viewpoints and interpretations to better comprehend the material”

Lecture Active learning

18

Preliminary results

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Student effects

Could the course influence demonstrated cognition level? Self-selection?

Could pedagogical preference affect demonstrated cognition level? Test for pedagogical effects: Is there a

difference in demonstrated cognition level between students who prefer active vs. lecture-based pedagogy?

19

Preliminary results

t = 1.7805df = 7.999

p-value = 0.1129

active lecture

preference for active learning mean NOT significantly higher cognition level than preference for lecture mean

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Implications

Evidence that learner preference does not impact demonstrated cognition level Small sample size

Evidence that experiential study abroad learning environments can promote higher order thinking Final assignment mean cognition score:

Significantly higher than initial assignment Significantly higher than classroom final

assignment Need to compare initial assignments from both

learning environments to isolate effects

20

Implications

Research need Research approach Preliminary results ImplicationsResearch need Research approach Preliminary results Implications

Next steps

Add 2010 and 2013 students, 2012 initial essay

Add second reader to ensure reproducibility

Consider regression analyses? Consider publication?

Any recommendations? Journal of Natural Resources and Life Sciences

Education?

21

Implications

THANK YOU!lynchabi@msu.edu

Abigail J. Lynch and William W. TaylorMichigan State University, Department of Fisheries and Wildlife, Center for Systems Integration and Sustainability

30 April 2013FAST Fellowship Symposium

ACKNOWLEDGEMENTSFW 481 studentsFAST Fellowship Steering CommitteeFAST Fellows