PODCAST EFFECTIVENESS AS SCAFFOLDING SUPPORT FOR STUDENTS ENROLLED IN FIRST-SEMESTER

GENERAL CHEMISTRY LABORATORIES

M. Cynthia B. Powell

The Problem: Implementing inquiry-

based laboratories

• Students take an active role in planning experiments

• Inquiry-based labs can be difficult to implement if students do not have background knowledge and skills

Bruck, L. B.; Towns, M. H. Preparing Students to Benefit from Inquiry-Based Activities in the Chemistry Laboratory: Guidelines and Suggestions. J. Chem. Educ., 2009, 86, pp 820-822.

DIMENSIONS– Content– Methods– Sequence– Sociology Methods include

– Modeling– Coaching– Scaffolding and Fading– Articulation– Reflection– Exploration

Content includes

–Domain Knowledge

–Heuristic Strategies

–Control Strategies

–Learning Strategies

Sequence includes

–Global Skills before

Local Skills

–Increasing Complexity

–Increasing Diversity

Sociology includes

–Situated Learning

–Culture of Expert Practice

–Intrinsic Motivation

–Exploiting Cooperation

–Exploiting Competition

COGNITIVE APPRENTICESHIP

Collins, A., Brown, J.S. & Newman, S.E. Cognitive apprenticeship: Teaching the craft of reading, writing and mathematics. In Knowing, learning and instruction: Essays in honor of Robert Glaser, Resnick, L. B. Ed.; Lawrence Erlbaum Associates: Hillsdale, NJ, 1989.

Stewart, K.K.; Lagowski, J.J. Cognitive Apprenticeship and Graduate Chemistry Education. J. Chem. Educ., 2003, 80, 1362.

The Plan…• Resource files and podcasts that can

be accessed on the iPhones as needed to provide modeling, coaching and scaffolding during lab

• Select two treatment groups: podcast treatment group and a pre-lab lecture treatment group

• Gather quantitative and qualitative data to compare the effect of the treatments

• Pilot project to test data collection and data processing methods, TA training and podcasts

• ACU and UNT IRB approval!

Laboratory Resource FilesWritten laboratory materialsMSDS sheetsSyllabi, grading rubrics, and safety contractPodcasts in two categories: Chemistry calculations & concepts General laboratory techniques

The Fall 2009 sample: ACU General Chemistry Labs

• Six sections, lead by the same instructor • Saturation of mobile devices• 4 “podcast treatment” sections, n=81• 2 “lecture treatment” sections, n=51

• Students research teams of with assigned group roles

•Curriculum: 1. guided inquiry activity (Q) 2. experiment (E) 3. data processing (P)

Demographic Description Fall 2009 semester

Podcast treatment n = 81 Lecture treatment n = 51

ObservedMotivation

40.7% High40.7% Medium18.5% Low

39.2% High39.2% Medium21.6% Low

Classifications 75.6% Freshmen15.8% Sophomores 7.3% Juniors 1.2% Seniors

72.5% Freshmen15.7% Sophomores 7.8% Juniors 3.9% Seniors

Gender 43.2% Male56.8% Female

39.2% Male60.8% Female

Mean ACT 25.96 + 3.84 25.20 + 3.27

Mean GALT 9.32 + 2.00 9.00 + 2.16

No statistically significant differences

Summary of the research designDifference in curriculum?

Before experiment information presented in podcast vs. lecture format

Research Questions• Focus of Research Question 1:

Frequency of podcast usage

• Focus of Research Question 2: Number, types, and topics of scaffolding interactions between students research teams and instructors in two treatment groups

• Focus of Research Question 3: Performance differences between students in the two treatment groups

Research Question 1: Methodology

Research Question 1:

When relevant chemistry podcasts are available for on-demand access during a general chemistry laboratory taught with an inquiry-based curriculum, how frequently will student research teams access them?

– files in a secure file management system– pertinent podcasts loaded at the start of the lab– file usage log to track access events

Podcast Usage during the Lab Period

Experiment Week

Accesses podcasts Total access events

Mean access events per team

3 Vernier Gas Pressure Equipment 32 1.33

4 Collecting a Gas SampleUsing PipetsSimple Statistics

323963

1.331.632.63

5 FilteringMass DeterminationSimple Statistics

28118

1.170.460.33

8 Comparing Reactivity of MetalsUsing Acids Safely

5339

2.211.63

9 Using PipetsTitration Techniques

746

0.291.92

10 FilteringUsing Acids SafelySimple Statistics

4644

1.880.170.17

11 Planning an Experiment 59 2.46

Podcast Usage during the Lab Period

Accessed podcast Total access events Mean access events per team

Simple Statistics 75 3.13

Filtering 73 3.04

Planning an Experiment 59 2.46

Comparing Reactivity of Metals 53 2.21

Using Pipets 46 1.92

Titration Techniques 46 1.92

Using Acids Safely 43 1.79

Vernier Gas Pressure Equipment 32 1.33

Collecting a Gas Sample 32 1.33

Mass Determination 11 0.46

Research Question 2: MethodologyResearch Question 2: What differences are evident in the number, types, and topics of scaffolding interactions between student research teams and instructors in laboratory sections that have access to on-demand podcasts but no pre-laboratory lecture and those who have been instructed using a traditional pre-laboratory lecture?

Scaffolding Interaction Categorization Scheme (SICS)

Tier 1: Type of interaction Clarifying or Follow-up

Tier 2: Topic and Subtopic of interaction Q1 – numerical issueQ2 – ideological issueE3 – tools/equipmentE4 – investigative procedureE5 – data (quantitative or qualitative)E6 – safetyP7 – claims and evidencesP8 – prior knowledge or experiences

Tier 2: Topic of Interaction: Q, E, or P

Tier 2: Topic of Interaction: Q, E, or P

Tier 1: Type of Interaction: clarifying

or follow-up

Tier 1: Type of Interaction: clarifying

or follow-up

Tier 3: Subtopic of Interaction: 1-8

Tier 3: Subtopic of Interaction: 1-8

Inter-rater reliability for teaching assistants

Teaching assistants by treatment group

Number Type Topic(Q, E or P)

Subtopic(1-8)

Podcast treatment TAs mid-semester end-of-semester

1.00.96

.98

.96.98.96

.82

.89

Lecture treatment TAs mid-semester end-of-semester

.96

.90.96.90

.96

.95.50.60

all aligned data points ( all aligned data points + data points not aligned)

Ratio of clarifying interactions to follow-up interactions, 15:1

Clarifying interactions by experiment week

Clarifying interactions per team by treatment block

Treatment group Mean clarifying interactions by treatment block

Contrasting treatment

(Weeks 3, 5, 8, 9, 10, 11)

Equivalent treatment

(Weeks 6, 7 ,12)

Mixed treatment

(Week 4)

M SD M SD

Podcast treatment teams (n = 24)

2.942* .662 1.942 .485 3.950

Lecture treatment teams (n = 14)

4.478* .866 1.977 .605 4.210

* Welch’s t-test indicates these values are statistically significantly different at α = .05 level

Cohen’s d = ( Mt – Mc ) / Spooled

for contrasting treatment block = 2.18

Topic distribution of clarifying interactions

Treatment group

Mean clarifying interactions by treatment block

Contrasting treatment

Equivalent treatment Week 4

Q E P Q E P Q E P

Podcast treatment teams (n = 24) .709 1.94* .292 1.69 1.06 0 .50 3.08 .64

Lecture treatment teams (n = 14) .905 3.07* .502 1.71 1.14 0 .50 3.07 .50

* Welch’s t-test indicates these values are statistically significantly different at α = .05 level

Cohen’s d = ( Mt – Mc ) / Spooled

for E interactions in contrasting treatment block = 1.73

Research Question 3: MethodologyResearch Question 3: What do the student outcome measures of

laboratory report grade average, laboratory quiz average, laboratory final exam grade and laboratory course average indicate about performance differences between students who have access to the on-demand podcasts versus students who have received the same information in a traditional lecture format?

Graded laboratory assignments:– team laboratory reports– end-of-lab quizzes– individual student reflections– comprehensive final exam

Inter-rater reliability for assignment grading

Teaching assistants by treatment group

Percent difference between TA and faculty instructor grade assignment

Quizzes Laboratory reports with accompanying reflection

M SD M SD

Podcast treatment TAs –0.53% 2.76% –0.55% 1.73%

Lecture treatment TAs –0.83% 1.87% –0.55% 1.84%

Mean Values of Outcome Measures

Podcast treatment

n = 81

Lecture treatment

n = 51

Lab Reports 91.6 ± 6.74 90.61 ± 5.83

Quizzes 78.79 ± 11.49 75.84 ± 12.33

Lab Final Exam 72.38 ± 13.56 73.21 ± 11.43

Lab Course Grade 87.09 ± 7.91 85.92 ± 6.66

No category shows a statistically significantdifference at the α = .05 level

Demographics and interaction effects?

“Highly motivated”Podcast

treatmentn = 33

Lecture treatment

n = 20

Lab Reports 95.99 ± 2.74 91.80 ± 4.45

Quizzes 86.95 ± 6.56 79.44 ± 11.00

Lab Final Exam 83.24 ± 6.91 79.45 ± 10.28

Lab Course Grade 93.64* ± 3.13 88.72* ± 5.93

* ANOVA and Tukey Post-Hoc tests indicate that these values are statistically significantly different at α = .05 level

Conclusions

Research Question 1: *The mean number of access events per team was 2.80.

*Most frequently accessed podcasts covered calculations or concepts. *Natural “fading” observed in access patterns.

Research Question 2:*Fewer clarifying interactions with the instructors for the podcast treatment group during the contrasting treatment weeks, but not during equivalent treatment weeks. (“E” interactions)

Research Question 3:*Treatment groups performed comparably on outcome measures.*One observed demographic interaction effect: students with high observed motivation ratings in the podcast treatment group performed at a higher level than highly motivated students in the lecture treatment group.

Beyond the quantitative data: other uses for iPhones in the laboratory

Ready access to all posted resources

Online searching

Timers

Flashlights

Calculators

Cameras

Video Cameras

Continued research interests….

• Continued work on assessment for chemistry laboratory courses and training of teaching assistants

• Podcasts for upper division biochemistry laboratory with lab practicals to follow student progress

• Enduring resources for pre-service teachers in General Science course

AcknowledgementsDr. Diana Mason and the chemistry education

research group

Faculty and staff in Departments of Chemistry and Educational Psychology at UNT

Colleagues in Chemistry and Biochemistry Department and technical support staff at ACU

Mobile Learning Fellowship, ACU(2008-2009, 2009-2010)

R. B. Escue Scholarship, UNT