assessing information literacy from the ground up

ASSESSING INFORMATION LITERACY FROM THE GROUND UP – CREATING A CULTURE OF ASSESSMENT

Kristy Motz, Randall Schroeder & Mari Kermit-CanfieldFerris State University

FLITE ASSESSMENT TEAM

WHAT’S OUT THERE?

Starting Up – Graduate Level Assessments (CMS)

• Developed tools two years ago in Blackboard Vista product

• Ran assessments through several semesters

• Summer 2009: Refined questions and mapped them to Information Literacy Competency Standards for Higher Education

MMBA Pre- and Post-Instruction Assessments

• Mixed delivery or blended delivery class

• Two Saturdays on-site - remaining course online

• Class runs eight weeks = ½ semester

• Assessments run in class first and last day

• 8 questions pre-instruction (first day of class)

• 5 questions post-instruction (end of class)

ONLINE ASSESSMENT IN CMS

PRE-INSTRUCTION COURSE ASSESSMENT

POST-INSTRUCTION ASSESSMENT MMBA

MMBA CMS Issues• Surveys or ungraded quiz tools from

CMS don’t provide results.

• Instructor must allow point credit for results to remain in Gradebook.

• Students with first-day user ID/password problems can’t reach the assessment.

• Questions must be tweaked to fit CMS question software format.

FALL 2008 MMBA - FAGERMAN

SUMMER 2009 MMBA - HAMEL

ESPN 505 Special Education

• 19 students took Pre-Instruction Assessment

• 15 students took Post-Instruction Assessment

• Instructor added 2 point bonus for participation

• Instructor used CMS with on-campus course

ESPN 505 PRE AND POST INSTRUCTION ASSESSMENT

NEXT STEPS: REAL-TIME UNDERGRADUATE ASSESSMENTS

Vogel Library FLITE

• FLITE’s new department head: Feb 2008

• Assessment Team’s new focus on Information Literacy Instrument

• Announcement of new James Madison Information Literacy Testing

HOW TO BEGIN?VOGEL LIBRARY – WARTBURG COLLEGE

Vogel Library

• Seven years of pre-instruction and post-instruction information literacy testing

• Based on Information Literacy Competency Standards for Higher Education

• Designed question by question to highlight targeted standards

Vogel Library

• Developed by

• Librarians Randall Schroeder, Jill Gremmels, and Karen Lehman

• Dr. Judith Griffith, English Faculty Chair

• Led to Faculty Buy-In

Vogel Adaptations to FLITE• FSU Students

• FSU Focus and Mission

• Information Literacy a new Concept

• Implementation and Testing

• Faculty Buy-in: 3 volunteers worked with us

CHALLENGES

•Conflicts with Scantrons and Testing Center

•Need for definitive learning outcomes

•Testing on concepts not always covered in classes

WHAT ARE WE TESTING?16. A student who is deciding what topic to

research for an assigned paper approaches the reference librarian and says, “This summer was too hot. I wonder if it is because of global warming?”What might be the thesis statement of this student’s paper?

1. It was too hot this summer. 2. Global warming is causing the mean

temperature of the earth to rise, which could have disastrous effects on the ecosystems of the planet.

3. Did global warming cause abnormal heat around the planet?

4. We should buy hybrid vehicles because of global warming.

5. This paper will be about the effects of global warming.

SUCCESSES - SURVEY QUESTIONS Rank the activities on which you spent the most (4) to the least (1) time when you are on the computer.

Searching for specific information related to a school or personal task.

Communicating with individuals through Facebook/My Space, IM, or via e-mail.

  Downloading movies, music or graphics.

  Surfing the Internet for enjoyment or playing

games. 

WHAT DO WE KNOW ABOUT OUR STUDENTS?

WHERE DO YOU GET YOUR INFORMATION?

WHICH OF THESE HAVE YOU DONE WITHIN THE LAST YEAR?

HOW MANY PAPERS DID YOU WRITE?

Schroeder Words of Wisdom: Hard Lessons Learned

Give yourself permission to fail

Assess to learn things – Don’t just assess to assess

Be sure to have Faculty Buy-In

Starting Again

Involve new staff

Improve librarian buy-In

Imagine: What will happen when developers leave?

What are the Information Literacy Competency Standards?

http://ala.org/ala/mgrps/divs/acrl/standards/standards.pdf

FLITE INFORMATION LITERACY ASSESSMENT TEAM CREATES LEARNING OUTCOMES

In-depth study of the 5 competency standardsFocus on performance indicators

Curriculum mapping - Group discussion and consensus 7 performance indicators1-2 indicators from each standard

FLITE INFORMATION LITERACY ASSESSMENT TEAM CREATES LEARNING OUTCOMES

Group work: Individual questions written for each learning outcome (performance indicator) - 35 questions for assessments

Discussion and consensus: Team selected 34 questions for Question Bank

12 questions are survey questions asked for every assessment

22 questions can be tailored for class

Information Literacy – Round Two Fall 2009Question Bank built for use with

different classes

Individualized assessments built by individual library instructors from Question Bank

Assessments delivered electronically if possible

SURVEY MONKEY

Started in 1999, SurveyMonkey is an online survey tool that enables people of all experience levels to create their own surveys quickly and easily.

FLITE Library’s Assessment Team picked Survey Monkey because:

• Seven years of pre-instruction and post-instruction information literacy testing

• Based on Information Literacy Competency Standards for Higher Education

• Designed question by question to highlight targeted standards

FLITE LIBRARY’S ASSESSMENT TEAM PICKED SURVEY MONKEY BECAUSE…

History Team members have previous experience

with the program

Easy of use User needs no statistical background User needs no knowledge of statistical

software

Malleable Accessible from web; in class or at home Easily printable for a paper version

Success – Yes, You Can Too

We went beyond the “What if’s” and created a project.

We gave ourselves permission to fail.

We didn’t have to be perfect – we just had to start.

We created a Culture of Assessment: we refined what we did and moved on.

Next StepsPost-instruction assessment – Dec 2009

Refine questions based on data input

Expand to other faculty and departments

Adapt procedures for Distance Learning

Write new assessments for upper-level classes

Accomplished Major Goal

• Closed the loop

• Curriculum mapping: using assessment data to revise and improve instruction curriculum

• Keep assessing – TracDat

Culture of Assessment

Helps us inform faculty where we want them to go with us

Allows us to create customized instructions across the curriculum

Showcases FLITE – the Library is one of the early reporters of assessment data on campus

Just Start!

• Get Faculty Buy-in

• Give Yourself Permission to Fail

• Take the Plunge and Begin

Questions?

Kristen Motz motzk@ferris.edu

Randall Schroeder randallschroeder@ferris.edu

Mari Kermit-Canfield kermitt@ferris.edu