Think Like a Researcher!

A Library/Faculty Collaboration to Improve Student Success

Heather Devrick Susan MikkelsenLecturer Instruction & Scholarly Communications LibrarianMerritt Writing Program UC Merced LibraryUniversity of California, Merced University of California, Mercedhdevrick@ucmerced.edu smikkelsen@ucmerced.edu

University of California Merced

The Collaboration Begins…

Summer 2013

Modeled after program at NMSU

UC Merced Library/Merritt Writing Program

Center for Research & Teaching Excellence Grant

Students in freshman composition courses struggle with academic research. Can curricular changes focused on the addition of information literacy lessons and activities help students be more successful researchers?

TRAIL

Program Name: TRAIL - Teaching Research And Information Literacy

Mantra: Think Like a Researcher!

Train-the Trainer model

TRAIL Goals

Improve integration of information literacy skills in Writing 10 (College Reading & Composition)

Teach research as a recursive process

Refocus the content of one-shot library instruction sessions

Raise the quality of argument papers (source selection, use of supporting evidence, multiple viewpoints)

Teach transferable research skills

Collaboration : Developing the Curriculum

Fall 2013

Faculty recruited

Writing assignments standardized

Information Literacy lessons and activities developed

Information Literacy course themes

Faculty-to-Faculty Collaboration

Assignments/Assignment Weight

Grading

Schedule

Readings

Course Themes

Course Text

Librarian Role

Meeting organizer and facilitator

Tutorial development

Information literacy assignment development

Input on research journal prompts

Collaboration : Implementing the Curriculum

Spring 2014

Piloted new curriculum in 6 sections of Writing 10

Embedded Librarian in Writing 10

Regular meetings and email check-ins

TRAIL sections have two library instruction sessions

Embedded Librarian Perspective

Students struggle to read the academic texts they are asked to use in their research assignments

Students struggle to distinguish between opinion and fact (evidence)

Students struggle to identify the problem that underlies their research topic or question

Students struggle to recognize bias in the articles they use for research as well as their own personal bias

Collaboration: Developing the Curriculum (round 2)

Summer 2014

Revised and added assignments to TRAIL curriculum based on what we learned in the pilot

Changed course text

Changed course theme

Collaboration : Implementing the Curriculum (round 2)

Fall 2014

9 Writing 10 sections implemented curriculum

Biweekly meetings with TRAIL faculty to debrief and prepare for upcoming lessons and activities

In-person library sessions restructured to focus on critical thinking and information literacy threshold concepts

TRAIL sections have one library instruction session

Information Literacy Assignments/Activities

Think Like A Researcher!

How to Read a Scholarly Article

Using Google for Academic Research

Reading for Keywords

Understanding the Knowledge/Information Cycle Tutorial

Guide on the Side Database Tutorials (Credo, Opposing Viewpoints, ASC)

Avoiding Research Bias

Developing Successful Research Topics/Questions

Research Reflection Journals

Information Literacy Course Readings

http://libguides.ucmerced.edu/think_like_a_researcher

Restructured One-Shot

How Do Databases Work? Fields, Tags, Keywords Match.com example

Sharing Research Questions Google Spreadsheet

Effective Search Strategies Students Searching in Front of Class

Assessment

ACRL Assessment in Action (2014)

Student Final Reflections – TRAIL only (Rubric)

Compare Writing 10 Final Papers (Rubric) TRAIL sections Traditional one-shot library instruction sections No library instruction sections

Post-semester Faculty Survey

4=Advanced 3=Developing 2=Emerging 1=Marginal No Score

Changes in Research Process

20% 31% 34% 13% 2%

Evaluating and Selecting Sources 23% 32% 33% 11% 1%

Overcoming Research Challenges

4% 11% 66% 15% 4%

Changes in Attitude and Confidence Level

17% 29% 33% 17% 4%

Transferability of Research Skills 17% 43% 26% 11% 3%

Understanding of “Think Like a Researcher”

23% 30% 33% 9% 5%

Final Reflection Rubric Scores

Final Reflections: Coded Response Analysis

88.5% indicate being more confident researchers 32.5% of these students indicate a BIG change in

confidence level

~80% anticipate using what they learned in future classes Upper Division Writing, CORE, Biology mentioned

most often

72% made source changes between their annotated bibliographies and final papers

Student Comments

“…now I read more effectively by reading the abstract, introduction, and the ending to determine if the article is suitable for reading fully and determining if it can be used as evidence.”

“Just because a source is academic does not always make it appropriate to use in a paper.”

“Not only do I feel more confident now but also I believe I will enjoy writing another research paper.”

Student Comments

“…I hated doing research because it was a slow, a really long and very involved process. However, by taking this class, I realize that research provides a firsthand opportunity to learn new information and also see and appreciate the diversity of the opinions which exist on my research topic.”

“Thinking like a researcher means thinking critically, seeing multiple sides and perspectives.”

Student Comments

“I believe learning more about the research process did help me in my other classes because the research process was more than just searching for sources. The most vital part of the process is actually the critical reading and analyzing which is something than can be used in any class.”

“…I believe thinking like a researcher means to question everything and tackle a problem from as many different angles as you can which is a skill we will always be able to use in future classes and even in life.”

Final Paper Rubric Scores

Compare papers from TRAIL sections with: Papers from sections that received traditional one-shot

library instruction Papers from sections that received no library instruction

Random selection

40 papers from each group

Scored by MWP faculty

Two scoring sessions

Rubric Criteria

Source Suitability

Argument & Evidence

Citation Style

Source Integration

Preliminary Results

Source Suitability

To what extent have students used suitable sources (credible, relevant) in their papers for evidence?

TRAIL students performed at more advanced levels than both other cohorts.

4 Advanced 3 Developing 2 Emerging 1 Marginal0

1

2

3

4

5

6

7

8

9

10

Source Suitability

TRAIL ScoreNon-TRAIL One Shot ScoreNon-TRAIL No Library Score

Preliminary Results

Argument & Evidence

Are students presenting arguments and counter arguments supported with evidence?

TRAIL students performed at more advanced levels than both other cohorts.

4 Advanced 3 Developing 2 Emerging 1 Marginal0

2

4

6

8

10

12

Argument & Evidence

TRAIL Score

Non-TRAIL One Shot Score

Non-TRAIL No Library Score

Preliminary Results

Citation Style

Do students cite source sources accurately? TRAIL students performed at more advanced levels than both other cohorts.

Only TRAIL students scored at the advanced level. 4 Advanced 3 Developing 2 Emerging 1 Marginal

0

2

4

6

8

10

12

Citation Style

TRAIL ScoreNon-TRAIL One Shot ScoreNon-TRAIL No Library Score

Preliminary Results

Source Integration

Do students successfully incorporate sources in their papers?

TRAIL students performed at lower levels than both other cohorts.

4 Advanced 3 Developing 2 Emerging 1 Marginal0

2

4

6

8

10

12

Source Integration

TRAIL ScoreNon-TRAIL One Shot ScoreNon-TRAIL No Library Score

Based on your own observations did your students

Yes, more so than previous students

No discernable differences

No, less so than previous students

…engage with research as an ongoing process? 5 0 0

…select suitable resources for their assignments? 4 1 0

…write strong research questions? 4 1 0

…incorporate evidence from multiple viewpoints? 3 2 0

…demonstrate persistence in information-finding? 4 1 0

Based on your experience, Would you promote this model of integrating research and writing skills to other faculty?

4 out of 5 TRAIL faculty responded Yes or Definitely

TRAIL Faculty Questionnaire Responses

TRAIL Faculty Questionnaire Responses

“By the end of the semester, the students in my TRAIL sections were more adept at working through the processes that researchers go through: examining assumptions, reviewing literature and analyzing it, finding gaps in the existing research, conducting primary research, etc.”

“…Throughout the semester I heard things like ‘Oh, you probably are using the wrong database or need new keywords’ or ‘dude you don't have to start over, just jump back into the research cycle’ or ‘I think you're being biased here’. This level of conversation about research is not typical.”

TRAIL Faculty Questionnaire Responses

Challenges with the TRAIL curriculum

“…not forgetting to emphasize other WRI 10 skills, like rhetorical analysis.”

“…The assignments have a lot of potential, and with some reworking, I think they could be very useful for students; however, the existing order of assignments, requirements, and parameters for the assignment were again not realistic models of the kind of research or process that researchers follow in their work.”

Faculty Perspectives

Quality of Research vs. Quality of Writing

Multiple Skills Integration

Scaffolding Research Assignments is Essential Topic Proposal Annotated Bibliography Final Argument Paper

Emphasize Research as Process

Flexibility and Creativity are Important

Assessment: Next Steps

AIA Assessment Short term

MWP score second batch of student papers (rubric) Compare final paper grades and course grades with

rubric scores Long term

Focus groups in upper division writing classes to determine which skills transfer

Institutionalizing the Curriculum

Potential to be more broadly adopted throughout Writing 10 Selected Activities: 1 hour Train-the-Trainer

Workshops Full TRAIL curriculum training: 2 day training +

semester-long support

Add an Information Literacy Lab to Writing 10 for 1 credit hour?

Seeking Undergraduate Council support

Questions?

http://libguides.ucmerced.edu/think_like_a_researcher