cirtl spring 2016 the college classroom meeting 8 - teaching-as-research

CIRTL – The College Classroom Meeting 8:

Teaching-as-Research

March 17, 2016

Unless otherwise noted, content is licensed under

a Creative Commons Attribution- 3.0 License.

Peter Newbury

Center for Engaged Teaching, UC San Diego

[email protected]

Tom Holme

Department of Chemistry, Iowa State University

[email protected]

collegeclassroom.ucsd.edu

Morning vs. Afternoon

Teaching as research - collegeclassroom.ucsd.edu 2

An instructor thinks the time-of-day affects student

learning. In the Fall, he teaches at 9:00 am. In the Winter,

he teaches the same course at 1:00 pm.

He uses the same final exam for both courses.

To test his hypothesis, he’ll compare the final grades of the

Fall and Winter classes.

Discussion procedure


1. You’ll be sent to Breakout Rooms 1 thru 6.

(Room 6: please return to the Main Room so we can

hear your conversation.) REMEMBER YOUR ROOM – YOU’LL GO BACK THERE

2. Open a browser window to your Room’s Google doc

3. The person whose first name comes first in the

alphabet will take notes in the Google doc.

4. The person whose first name comes second in the

alphabet will report back to the class.

(We’ll do this 4 or 5 times today and the roles of note-taker and reporter will change each time.)


Room Google doc

Main Room tinyurl.com/TARMainRoom

Room 1 tinyurl.com/TARRoom1





Discussion questions for Breakout Rooms:

• What concerns do you have about the experiment?

• How would you revise the experiment to address those concerns?

5

Scholarly

Teaching

Evaluate document observations summarize evaluation

Deliver deliver learning experiences * assess

learning outcomes * conduct observations

Design formulate big questions * identify learning

outcomes * plan assessment * design learning experiences

Adapted from Gabriele Wienhausen

Teaching as research - collegeclassroom.ucsd.edu

6

Scholarly

Teaching

Study identify key issues * analyze and synthesize

results * put results in context * solicit peer review * publish, present, disseminate






Scholarship

of

Teaching

and

Learning

(SoTL)



7

Scholarly

Teaching

Study identify key issues * analyze and synthesize

results * put results in context * solicit peer review * publish, present, disseminate






Scholarship

of

Teaching

and

Learning

(SoTL)



Teaching as Research (TAR)

Teaching as research

collegeclassroom.ucsd.edu 8

At the core of improving teaching and learning is the need

to accurately determine what students have learned as a result of

teaching practices. This is a research problem, to which instructors

can effectively apply their research skills and ways of knowing. In

so doing, instructors themselves become the agents for change in

teaching and learning.

Teaching-as-Research involves the deliberate, systematic,

and reflective use of research methods to develop and implement

teaching practices that advance the learning experiences and

outcomes of students and teachers.

CIRTL Network [1]

Categories of Educational Research[2]


Theoretical research

Action or practitioner research

Evaluative Experimental

'Cause and effect' research

Case study

Systematic review

Exploratory

Comparative

Grounded theory

Ethnography

The Effect of Interactive Instruction[3]


1 2

3 4

What is the value of course-specific learning goals?[4]


Research informs the

instructors about

how students are

responding

what’s working (or not)

what to use again (or

not)

Helps instructors become

better educators.


Active learning increases student performance

in science, engineering and mathematics[5]

established active learning

has positive impact on

learning

advances field of education

research: no need to

continue to replicate study

(in STEM)

Your education research posts


What can you study?


Students’ knowledge and skills

what are students able to do now that they couldn’t do

before taking the course?

are students thinking more like experts?

Students’ attitudes

what are they initially?

how did they change after you did X?

Depth of knowledge


A professor wants to find out the depth of her students’

knowledge. She classifies each question on the final exam

as basic, intermediate, or advanced.

After the exam, she’ll look at the students’ success rates on

the 3 levels of questions.

Depth of knowledge


A professor wants to find out the depth of her students’

knowledge. She classifies each question on the final exam

as basic, intermediate, or advanced.

After the exam, she’ll look at the students’ success rates on

the 3 levels of questions.

Return to your Breakout Room (Main, 1 – 5) and Google doc

• Pick a new note-taker and reporter

• Take notes on “Depth of Knowledge” worksheet in the same

Google doc

PowerPoint or Video?


A professor wants to determine which is better for

supporting a flipped class "flipped class" where students

prepare before coming to class: PowerPoint slides or video.

In a certain week, students whose last names begin A – L

read through PowerPoint slides before class; students

whose last names begin M – Z watch a video of the

instructor going over the PowerPoint slides. He’ll compare

their performance on the exam questions related to that

week’s material.

PowerPoint or Video?


A professor wants to determine which is better for

supporting a flipped class "flipped class" where students

prepare before coming to class: PowerPoint slides or video.

In a certain week, students whose last names begin A – L

read through PowerPoint slides before class; students

whose last names begin M – Z watch a video of the

instructor going over the PowerPoint slides. He’ll compare

their performance on the exam questions related to that

week’s material.



• Take notes on “PPT or video” worksheet in the same Google doc

Human Subjects Research[6]


Research: a systematic investigation, including research

development, testing, and evaluate, designed to develop or

contribute to generalizable knowledge

Human Subject: a living individual about whom an

investigator conducting research obtains

1. data through intervention or interaction with the

individual, or

2. identifiable private information

Ethical principles for research involving

human subjects (Belmont Report[7])


1. Respect for Persons

Individuals should be treated as autonomous agents and

people with diminished autonomy are entitled to

protection.

This mean the person involved must be capable or

making an informed decision on whether or not to

participate.




2. Beneficence

Persons are treated in an ethical manner not only by

respecting their decisions and protecting them from

harm, but also by making efforts to secure their well-

being. Actions must

do no harm

maximize possible benefits and minimize possible

harms




3. Justice

Who ought to receive the benefits of research and bear

its burdens?

An injustice occurs when some benefit to which a

person is entitled is denied without good reason or

when some burden is imposed unduly.

Teaching-as-Research Scenarios


With others in your group, try to determine

1. Does the experiment uphold the ethical principles of

respect for persons, beneficence, and justice?

2. If not, are there ways to revise the research question

and/or the experimental design so the research can

proceed?

3. Is it (interesting) research worth doing?


Room Google doc

Main Room tinyurl.com/TARMainRoom






Discussion questions for Breakout Rooms:

• Does the experiment uphold the ethical principles of respect for persons, beneficence, and justice?

• How would you revise the experiment to address those concerns?

Gender and Math


A professor knows there is a belief that women aren’t as

good at math as men. He wants to demonstrate women do

as well as, if not better than, men in math. On the final

exam, men are instructed to solve a certain problem,

women are instructed to solve a different, slightly harder

problem.

The professor will compare the men’s and women’s scores

on their problems.



• Take notes on matching worksheet in the same Google doc

Recognizing students’ diversity


An instructor recognizes she has a diverse class and wants

to figure out how to adjust her teaching to support each

student. She asks students to complete a background

survey giving their gender, age, race, ethnicity, sexual

orientation, first language, and whether or not they are the

first in their family to go to college. At the end of the

course, she’ll look for correlations with their final grades.



• Take notes on matching worksheet in the same Google doc

Humans are involved in TAR


Mission of the Institutional Review Board (IRB):

1. Protection of human subjects from physical harm.

2. Protection of your students’ privacy and success.

If you suspect your research could harm students or

their success, you need to talk to your IRB for

approval or an exemption.

Federal Common Rule 45 CFR

46.101(b) grants exemptions to…


(1) Research conducted in established or commonly accepted

educational settings, involving normal educational practices,

such as (i) research on regular and special education

instructional strategies, or (ii) research on the effectiveness of or

the comparison among instructional techniques, curricula, or

classroom management methods. [8]

Things a scholarly, reflective instructor

might and should do anyway.

Federal Common Rule 45 CFR

46.101(b) grants exemptions to…


(2) Research involving the use of educational tests (cognitive,

diagnostic, aptitude, achievement), survey procedures,

interview procedures or observation of public behavior, unless:

i. information obtained is recorded in such a manner that

human subjects can be identified, directly or through

identifiers linked to the subjects; and

ii. any disclosure of the human subjects' responses outside the

research could reasonably place the subjects at risk of

criminal or civil liability or be damaging to the subjects'

financial standing, employability, or reputation.[8]

Things a scholarly, reflective instructor

might and should do anyway.


You can use your skills

as a researcher

in your classroom, too.

Teaching - as - Research

References


1. Center for the Integration of Research, Teaching and Learning (2010) Teaching as Research.

www.cirtl.net/CoreIdeas/teaching_as_research

2. Lambert, M. (2012). A Beginner's Guide to Doing Your Education Research Project. Thousand Oaks, CA: SAGE

Publications Inc. via Tomorrow’s Professor cgi.stanford.edu/~dept-ctl/tomprof/posting.php?ID=1233

3. Prather, E.E, Rudolph, A.L., Brissenden, G., & Schlingman, W.M. (2009). A national study assessing the teaching and

learning of introductory astronomy. Part I. The effect of interactive instruction. Am. J. Phys. 77, 4, 320-330.

4. Simon, B., & Taylor, J. (2009). What is the Value of Course-Specific Learning Goals? J. College Science Teaching, 39, 2,

52-57.

5. Freeman, S., Eddy, S.L. McDonough, M., Smith, M., Okoroafor, N., Jordt,. H. & Wenderoth, M.P. (2014) Active

learning increases student performance in science, engineering, and mathematics. PNAS 111, 23, 8410–8415.

6. Collaborative Institutional Training Initiative, Module 2 “Students in Research” (CITI) citiprogram.org

7. Belmont Report (1979). The Belmont Report: Ethical principles and guidelines for the protection of human subjects of

research.

8. Basic HHS Policy for Protection of Human Research Subjects, U.S. Department of Health and Human Services,

Code of Federal Regulations

retrieved from www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html#46.101 3/16/ 2016.