Preliminary Results from the Fall 2008 INTD 105 Assessment

Pretest vs. Posttest

About these results

• Preliminary• No analysis (yet) of wrong answers

Goal 1, Objective 1

• Students will be able to demonstrate an understanding of the organization of the library website

Goal 1, Objective 1

Students still unsure of the existence of or how to get to our Resources by Subject pages.

Pretest Posttest

Correctly found the Business RBS page

40% 46%

Found the English RBS page (multichoice)

48% 56%

Citing sources page

66% 75%

Goal I, Objective 2

• Students will be able to describe the purpose of:– IDS– SRC– Reference Desk– Circulation Desk– Citation Style Guides

Goal 1, Objective 2

Pretest Posttest

IDS 70% 95%STC 88% 93%Reference Desk 85% 93%Circulation Desk 86% 92%

Students can generally pick out the appropriate service from a list.

This does not imply that students are already aware that such a service exists.

We see big increases in the number of students who recognize IDS

Goal 2, Objective 1

• Students will be able to define GLOCAT and the types of material it contains

Goal 2, Objective 1

Pretest Posttest

What can you find in GLOCAT

11% 12%

What database to use to find Harry Potter

76% 92%

Students can figure out pretty easily that they need to use GLOCAT to find books, but they have major problems understanding that you can’t use it to find journal articles.

Goal 2, Objective 2

• Students will be able to find books by topic author or title

Goal 2, Objective 2

Pretest Posttest

Correct call number

47% 54%

Find a book about “running a small business out of your home”

44% 36%

In the pretest, 14% of students left the second question blank, as opposed to 4% in the posttest.

Most of the wrong answers were off topic. They were able to find a book, but it didn’t include one or more aspects of the topic.

Goal 2, Objective 3

• Students will be able to interpret results lists and item records

• 9 questions about this on the assessment

Goal 2, Objective 3Pretest Posttest

What info do you need to find the book in the library

58% 78%

Which item is on the lower level

43% 46%

Checked out? 89% 94%

ID the author? 99% 98%

ID subject? 89% 91%

Type of resource 96% 96%

Where to get full text?

96% 92%

In general, students can interpret our GLOCAT item records with ease from the beginning.

We see a big increase in the awareness that the call number will help them locate the book in Milne library.

They are not making the connection between the GLOCAT results screen and library floorplans.

Goal 2, Objective 4

• Students will be able to demonstrate an understanding of where books and journals are physically located in Milne library

Goal 2, Objective 4

Pretest Posttest

Given a results screen, which item is on the lower level?

43% 46%

Given 4 options, which is NOT upstairs?

59% 77%

I don’t think they are making it to our library floor plans. I think the increase in question 2 comes from students knowledge.

Goal 3, Objective 1

• Students will be able to select an appropriate periodical database for a given task

Goal 3, Objective 1

Pretest Posttest

Library recommended database for English

48% 56%

Which search engine is appropriate to find scholarly sources

90% 91%

Students are aware that Google, Yahoo!, etc. aren’t acceptable for scholarly research, but they don’t seem to know what is appropriate.

Open ended article question: In the pretest, 35% of students left the question blank, versus 16% in the posttest.

Students were not making the best choices in their choice of database, although they did slightly better in the posttest.

Resources Used to find an article

Pretest Percent

Posttest Percent

Multisearch 12% 30%

Proquest 14% 27%

JSTOR 7% 27%

[Blank] 35% 16%

WorldCat 1% 10%Health and Wellness Resource Center 5% 8%

Pubmed 3% 8%CINAHL Plus with full text 2% 8%Academic Search Premier 1% 6%

Google Scholar 3% 4%

Journal Finder 1% 4%

Ebsco 3% 3%

Resources Used to find an article

Pretest Percent

Posttest Percent

Advanced Multisearch 0% 3%

ScienceDirect 0% 2%

Medline 1% 1%

Google 1% 1%

Psycinfo 1% 1%

Scopus 0% 1%SAGE journals online 2% 0%

Search Engine 1% 0%medicine and health database 1% 0%www.findarticles.com 1% 0%

resources from A-Z 1% 0%

search 1% 0%

Wikipedia 1% 0%

Goal 3, Objective 2

• Students will be able to find appropriate articles on a given topic

• Open ended questions– One asked for an article about the portrayal of

Native Americans in film using Proquest– The other asked for an article about the

effectiveness of the HPV vaccine

Goal 3, Objective 2

Reason for result Pretest Percent Posttest Percent

Book Review 50% 77%Incomplete Citation 4% 0%

OK 5% 0%

Good 6% 4%

No Answer Given 24% 0%

Off Topic 11% 9%

Find an article about the portrayal of Native Americans in Film.

While the percent of students who found actual articles went down, the percent who found book reviews went way up. Almost all of the book reviews were on-topic.

Goal 3, Objective 2Reasons for result Pretest Percent Posttest Percent

Book 1% 2%

OK* 1% 26%Incomplete Citation 12% 11%

Not Scholarly 14% 12%

Scholarly 33% 25%

No Answer 35% 9%Government Report 0% 5%

Off topic 4% 10%

Find a scholarly article about the effectiveness of the new HPV vaccine.

Total number of students who found something Scholarly went up significantly, as did the number of students who even attempted the question.

*Difference in “OK” percent has more to do with my evaluation of whether a source was on topic or not. I need to go back and re-do this.

Goal 3, Objective 3

• Students will be able to determine the availability of full text articles online, in print at Milne Library, and through IDS

• Mostly a test of the Journal Finder feature

Goal 3, Objective 3

Pretest Posttest

Print Volume 58% 71%Online Volume 48% 48%IDS 49% 70%

It seems our Find Journals feature isn’t as straightforward as I originally thought.

A semester at college seems to significantly increase students’ ability to use the site, but the ability to figure out which online resource to use is still lacking.

Pretest (21%) and Posttest (26%) students all want to go to JSTOR to find the online volume, even though it isn’t listed anywhere in the screen shot.

Challenge Pretest Posttest NotesCitation 6% 5%Determining appropriateness of source 12.0% 7.4% Often in conjunction with finding materialEvaluation 4.0% 7.4% Sorting through search results, finding "relevant" materialFinding articles/books/sources 20.0% 22.1%

In the post-test, we are more likely to see students mention finding material "on a particular topic" or "on an obscure topic"

Finding help 2.0% 0.5% Online or in person

Finding print materials 2.0% 2.1%Sometimes finding print, sometimes finding things "in the library"

Getting full text 0.0% 1.1%Frustration at getting full text when the student thinks it is available

Information overload 3.0% 2.6%Internet searching 2.0% 0.5%

Interpretation 0.0% 4.7%Interpreting the material they find. Finding information in long articles or books

Item availability 0.0% 8.9%Frustration at the speed of IDS, lack of knowledge of how it works, inability to find things "here"

Laziness 0.0% 2.1%Mostly expressed as a desire to find things quickly, one student admitted laziness

No Answer 22.0% 8.4%Search strategy 5.0% 13.7% Mostly expressed as "keywords to use"Time 2.0% 1.1%Topic selection 0.0% 1.1%Using a database 1.0% 2.1% JSTOR and GLOCAT were mentioned specificallyWhere to start 8.0% 2.1%Which resource to use 11.0% 13.2%