· web viewrelevance feedback (trf) ... (8.4% of users), and generally (56% of users) ... then...

SAN JOSE STATE UNIVERSITY

San Jose, California

ASSIGNMENT 2: SUBJECT ANALYSIS & EVALUATION

A Paper Submitted in Partial Fulfillment

of the Requirements for

LIBR 202: Information Retrieval

Judith Homan

School of Library and Information Science

December 3, 2012

ASSIGNMENT 2: SUBJECT ANALYSIS 2

Judith Homan

Fall 2012

LIBR202 Tucker

Assignment 2: Subject Analysis & Evaluation

Contents Page

A3. Statement of Purpose, Data Structure, & Rules 3

A4. Postco Vocabulary List 6

A5. Database Records 8

A6. User Guide 28

B. Retrieval Analysis 30

C. Evaluation & Reflection 32


Statement of Purpose

The following database was created with the newly enrolled SLIS student in mind. The database

could be continued and more information added as new terminology is found by new student.

The idea of these specific words being chosen was based on the frequency of their use in relation

to other words. The terms can stand alone but often are found with other terms to create a new

concept. This database will help to see the relationship between words; this will be especially

useful for those who are new to the library world.


Database Structure

Textbase Structure

Textbase Information

Textbase: C:\Program Files\Inmagic\DBTextWorks\Assignment 2 A3 - HomanCreated: 11/25/2012 8:16:01 PMModified: 11/26/2012 10:10:18 PM

Field Summary: 1. DOC_NO: Automatic Number(next avail=16, increm=1), Term 2. AUTHOR: Text, Term & Word Validation: required 3. TITLE: Text, Word Validation: required 4. SOURCE: Text, Word Validation: required 5. ABSTRACT: Text, Word Validation: required 6. POSTCO: Text, Term Validation: required, valid-list

Log file enabled, showing 'DOC_NO'Leading articles: a an the Stop words: a an and by for from in of the to XML Match Fields: 1. DOC_NO

Textbase Defaults: Default indexing mode: SHARED IMMEDIATE Default sort order: <none>Textbase passwords: Master password = '' 0 Access passwords: No Silent password


Rules

DOC_NO - Document Number: When each record is entered it is given an automatic number. It

is required for ease of record keeping and search retrieval.

AUTHOR: This is text entered with authors last name first, and then their initials just as it is

written in APA formatting. The field is repeatable since more than one author can be placed in

this field. This field is also required. This field will give the ability to find number of article

written by a particular author.

TITLE: This field is where the title of the article is placed. This field is in the form of text and it

is a required field. The title field makes for easier search for full article.

SOURCE: This field is for the journal or periodical the record's article came from. The title will

make it easier to search for the article for full review. This field is required.

ABSTRACT: This field is where the article’s abstract is cut and pasted. This field is required; if

the article does not have one, please write a brief abstract of the article. This field allows a user

to get an overview of what the article is about.

POSTCO: This is where the terminology is placed from the article. Each article can have

multiple entries making it repeatable. This terminology must come from the validation list

provided in the database. This field is required.


POSTCO Vocabulary List

Access

Accessibility

Automation

Bibliographies

Boolean

Cataloging

Chunks

Classification

Communication

Control system

Controlled vocabulary

Database

Development

Document

Electronic data processing

End user

Evaluation

Folksonomies

Hypertext

Indexer


Indexing

Information Retrieval

Information Technology

Internet

Librarians

Library use

Metadata

Natural Language

Organization

Process

Public access

Query

Record

Reference

Relevance

Reproducibility

Researcher

Satisfaction

Searching

Tagging

User interface


Database Records

DOC_NO 1

AUTHOR Bates, M. J.

TITLE The invisible substrate of information science

SOURCE Journal of the American Society for Information Science

ABSTRACT The explicit, above-the-water-line paradigm of information science is

well known and widely discussed. Every disciplinary paradigm,

however, contains elements that are less conscious and explicit in the

thinking of its practitioners. The purpose of this article is to elucidate

key elements of the below-the-water-line portion of the information

science paradigm.

Particular emphasis is given to information science’s role as a meta-

science--conducting research and developing theory around the

documentary products of other disciplines and activities. The mental

activities of the professional practice of the field are seen to center

around representation and organization of information rather than

knowing information. It is argued that such representation engages

fundamentally different talents and skills from those required in other

professions and intellectual disciplines. Methodological approaches

and values of information science are also considered.

POSTCO Searching

Researcher

Reference

Record

Query

Process

Organization


Librarians



Indexer

Evaluation

Document

Development

Database

Cataloging

DOC_NO 2

AUTHOR Mat-Hassan, M.

Lavene, M.

TITLE Can navigational assistance improve search experience? A user study

SOURCE First Monday

ABSTRACT Providing navigational aids to assist users in finding information in

hypertext systems has been an ongoing research problem for well

over a decade. Despite this, the incorporation of navigational aids

into Web search tools has been slow. While search engines have

become very efficient in producing high quality rankings, support for

the navigational process is still far from satisfactory. To deal with this

shortcoming of search tools, we have developed a site specific search

and navigation engine that incorporates several recommended

navigational aids into its novel user interface, based on the concept

of a user trail. Herein, we report on a usability study whose aim was to

ascertain whether adding semi-automated navigational aids to a

search tool improves users' experience when "surfing" the Web. The

results we obtained from the study revealed that users of the


navigation engine performed better in solving the question set posed

than users of a conventional search engine. Moreover, users of the

navigation engine provided more accurate answers in less time and

with less clicks. Our results indicate that adding navigational aids to

search tools will enhance Web usability and take us a step further

POSTCO Access

Boolean

Document

Organization

Process

Query

Record

Reference

Relevance

Researcher

Satisfaction

Searching

User interface

DOC_NO 3

AUTHOR Swanson, D. R.

TITLE Historical Note: Information Retrieval and the Future of an Illusion


ABSTRACT More than thirty years ago there was good evidence to

suggest that information retrieval involved conceptual problems

of greater subtlety than is generally recognized. The

dramatic development and growth of online services since

then seems not to have been accompanied by much interest


in these conceptual problems, the limits they appear to impose,

or the potential for transcending such limits through

more creative use of the new services.

In this article, I offer a personal perspective on automatic

indexing and information retrieval, focusing not necessarily

on the mainstream of research but on those events and ideas

over a 34-year period that have led to the view stated above,

and that have influenced my perception of important directions

for future research.

Some experimental tests of information systems have

yielded good retrieval results and some very poor results. I

shall explain why I think that occurred, why I believe that

the poor results merit special attention, and why we should

reconsider a suggestion that Robert Fairthome put forward

in 1963 to develop postulates of impotence—statements of

what cannot be done. By understanding such limits we are

POSTCO Access

Chunks

Classification

Document

Communication

Development

Indexer


Natural Language

Process

Reference


Relevance

Researcher

DOC_NO 4

AUTHOR Harter, S. P.

Cheng, Y.

TITLE Colinked Descriptors: Improving Vocabulary Selection for End-User

searching


ABSTRACT This article introduces a new concept and technique for

information retrieval called collnked descriptors. Borrowed

from an analogous idea in bibliometrics—cocited references—

colinked descriptors provide a theory and method

for identifying search terms that, by hypothesis, will be superior

to those entered Initially by a searcher. The theory

suggests a means of moving automatically from two or

more initial search terms, to other terms that should be superior

in retrieval performance to the two original terms. A

research project designed to test this cotinked descriptor

hypottiesis is reported. The results suggest that the approach

is effective, although methodological problems in

testing the idea are reported. Algorithms to generate colinked

descriptors can be incorporated easily into system

interfaces, front-end or pre-search systems, or help software,

in any database that employs a thesaurus. The potential

use of colinked descriptors is a strong argument for

building richer and more complex thesauri that reflect as

POSTCO Access


Boolean

Database

Document

Evaluation

Searching


Internet

Librarians

Query

Natural Language

Process

Record

Reference

Relevance

Researcher

Indexing

Public access

DOC_NO 5

AUTHOR Spink, A.

TITLE Term relevance feedback and mediated database searching:

Implications for information retrieval practice and systems design

SOURCE Information processing & management

ABSTRACT Research into both the algorithmic and human approaches to

information retrieval is required to improve information retrieval

system design and database searching effectiveness. This study uses

the human approach to examine the sources and effectiveness of

search terms selected during mediated interactive information


retrieval. The study focuses on determining the retrieval effectiveness

of search terms identified by users and intermediaries from retrieved

items during term relevance feedback. Results show that terms

selected from particular database fields of retrieved items during term

relevance feedback (TRF) were more effective than search terms from

the intermediary, database thesauri or users' domain knowledge during

the interaction, but not as effective as terms from the users' written

question statements. Implications for the design and testing of

automatic relevance feedback techniques that place greater emphasis

on these sources and the practice of database searching are also

POSTCO Database

Development

Document


Natural Language

Process

Query

Record

Relevance

Researcher

End user

Indexing

DOC_NO 6

AUTHOR Simon, H. A.

TITLE Information-Processing Models of Cognition


ABSTRACT This article reviews recent progress in modeling human cognitive


processes. Particular attention is paid to the use of computer

programming languages as a formalism for modeling, and to computer

simulation of the behavior of the systems modeled. Theories of

human cognitive processes can be attempted at several levels: at the

level of neural processes, at the level of elementary information

processes (e.g., retrieval from memory, scanning down lists in

memory, comparing simple symbols, etc.), or at the level of higher

mental processes (e.g., problem solving, concept attainment). This

article will not deal at all with neural models; it focuses mainly upon

higher mental processes, but not without some attention to modeling

the elementary processes and especially to the relationships between

elementary and complex processes.

POSTCO Access

Accessibility

Chunks

Communication

Control system

Development

Evaluation

Natural Language

Organization

Process

Record

Record

Reference

Reproducibility

Researcher


DOC_NO 7

AUTHOR Najarian, S. E.

TITLE Organization factors in human memory: Implications for library

organization and access systems

SOURCE The Library Quarterly

ABSTRACT Examines psychological studies on memory and learning for what they

reveal about human categorizing processes and the organizing

principles and limitations of human memory. Findings suggest

considerations for the design of information systems that would take

conceptual organization of knowledge into account

POSTCO Access

Chunks

Classification


Database

Development

Indexer


Librarians

Library use

Natural Language

Organization

Process

Record

Reference

Searching

DOC_NO 8


AUTHOR Bodoff, D.

TITLE Relevance for browsing, relevance for searching.

SOURCE Journal of the American Society for Information Science and

Technology

ABSTRACT The concept of relevance has received a great deal of theoretical

attention. Separately, the relationship between focused search and

browsing has also received extensive theoretical attention. This article

aims to integrate these two literatures with a model and an empirical

study that relate relevance in focused searching to relevance in

browsing. Some factors affect both kinds of relevance in the same

direction; others affect them in different ways. In our empirical study,

we find that the latter factors dominate, so that there is actually a

negative correlation between the probability of a document's

relevance to a browsing user and its probability of relevance to a

focused searcher.

POSTCO Access

Database

Development

Document



Process

Query

Record

Reference

Relevance

Researcher


Searching

DOC_NO 9

AUTHOR Marchionini, G.

TITLE Interfaces for end-user information seeking


ABSTRACT Essential features of interfaces to support end-user information

seeking are discussed and illustrated. Examples of interfaces to

support the following basic information-seeking functions are

presented: problem definition, source selection, problem articulation,

examination of results, and information extraction, it is argued that

present interfaces focus on problem articulation and examination of

results functions, and research and development are needed to

support the problem definition and information extraction functions.

General recommendations for research on interfaces to support end-

user information seeking include: attention to multimedia information

sources, development of interfaces that integrate information-seeking

functions, support for collaborative information seeking, use of

multiple input/output devices in parallel, integration of advanced

information retrieval techniques in systems for end users, and

development of adaptable interfaces to meet individual difference and

POSTCO Electronic data processing

Access

Boolean

Classification

Communication

Database


Document

Natural Language

Organization

Query

Process

Satisfaction

User interface

End user

Researcher

Relevance

Public access

Reference

Record

Searching

Internet

DOC_NO 10

AUTHOR Mat-Hassan, M.

Leven, M.

TITLE Can Navigational Assistance Improve Search Experience? A User

Study

SOURCE First Monday

ABSTRACT Providing navigational aids to assist users in finding information in

hypertext systems has been an ongoing research problem for well

over a decade. Despite this, the incorporation of navigational aids

into Web search tools has been slow. While search engines have

become very efficient in producing high quality rankings, support for

the navigational process is still far from satisfactory. To deal with this


shortcoming of search tools, we have developed a site specific search

and navigation engine that incorporates several recommended

navigational aids into its novel user interface, based on the concept

of a user trail. Herein, we report on a usability study whose aim was to

ascertain whether adding semi-automated navigational aids to a

search tool improves users' experience when "surfing" the Web. The

results we obtained from the study revealed that users of the

navigation engine performed better in solving the question set posed

than users of a conventional search engine. Moreover, users of the

navigation engine provided more accurate answers in less time and

with less clicks. Our results indicate that adding navigational aids to

search tools will enhance Web usability and take us a step further

POSTCO Access

Boolean

Document


Organization

Process

Query

Record

Relevance

Researcher

Satisfaction

Searching

User interface

DOC_NO 11

AUTHOR Elings, M. W.


Waibel, G.

TITLE Metadata for All: Descriptive Standards and Metadata Sharing across

Libraries, Archives and Museums

SOURCE First Monday

ABSTRACT Integrating digital content from libraries, archives and museums

represents a persistent challenge. While the history of standards

development is rife with examples of cross-community

experimentation, in the end, libraries, archives and museums have

developed parallel descriptive strategies for cataloguing the materials

in their custody. Applying in particular data content standards by

material type, and not by community affiliation, could lead to greater

data interoperability within the cultural heritage community.

In making this argument, the article demystifies metadata by defining

and categorizing types of standards, provides a brief historical

overview of the rise of descriptive standards in museums, libraries

and archives, and considers the current tensions and ambitions in

making descriptive practice more economic [1].

POSTCO Indexing

End user

Searching

Relevance

Record

Process

Metadata

Internet

Development

Database



Cataloging

Automation

DOC_NO 12

AUTHOR Golder, S. A.

Huberman, B. A.

TITLE The Structure of Collaborative Tagging Systems

SOURCE arXiv.org.

ABSTRACT Collaborative tagging describes the process by which many users

add metadata in the form of keywords to shared

content. Recently, collaborative tagging has grown in popularity on

the web, on sites that allow users to tag

bookmarks, photographs and other content. In this paper we analyze

the structure of collaborative tagging systems as well as their

dynamical aspects. Specifically, we discovered regularities in user

activity, tag frequencies, kinds of tags used, bursts of popularity in

bookmarking and a remarkable stability in the relative proportions of

tags within a given url. We also present a dynamical model of

collaborative tagging that predicts these stable patterns and relates

them to imitation and shared knowledge.

POSTCO Classification

Document

Folksonomies

Organization

Process

Query

Reference


Researcher

Searching

Tagging

DOC_NO 13

AUTHOR Hildreth, C. R.

TITLE Accounting for users' inflated assessments of on-line catalogue

search performance and usefulness: an experimental study

SOURCE Information Research

ABSTRACT User-oriented approaches to information retrieval (IR) system

performance evaluation assign a major role to user satisfaction with

search results and overall system performance. Expressed

satisfaction with search results is often used as a measure of utility.

Many research studies indicate that users of on-line library catalogs

(OPACs) and other IR systems often express satisfaction with poor

search results. This phenomenon of "false positives," inflated

assessments of search results and system performance, has not been

adequately explained. Non-performance factors such as interface style

and ease of use may have an affect on a searcher's satisfaction with

search results. The research described in this report investigates this

phenomenon. This paper presents the findings of an experimental

study which compared users' search performance and assessments of

ease of use, system usefulness, and satisfaction with search results

after use of a Web OPAC or its conventional counterpart. The

primary questions addressed by this research center on the influence

of two experimental factors, OPAC search interface style and search

task level of difficulty, on the dependent variables: actual search

performance, perceptions of ease of use and system usefulness, and


assessments of satisfaction with search results. The study also

investigated associations between perceived ease of use, system

usefulness, and satisfaction with search results. Lastly, the study

looked for associations between the dependent variables and

personal characteristics. No association was found between

satisfaction with search results and actual search performance. Web

OPAC searchers outperformed Text OPAC searchers, but search task

level of difficulty is a major determinant of search success. A strong

positive correlation was found between perceptions of system ease of

POSTCO Indexing

End user

Hypertext

User interface

Searching

Satisfaction

Researcher

Relevance

Reference

Record

Query

Process

Librarians

Internet


Evaluation

Document

Database


DOC_NO 14

AUTHOR Rothenberg, J

TITLE Ensuring the Longevity of Digital Document

SOURCE Scientific American

ABSTRACT Digital documents are replacing paper in the most dramatic record-

keeping revolution since the

invention of printing. Is the current generation of these documents

doomed to be lost forever?

Note: this paper is an expanded version of the article “Ensuring the

Longevity of Digital Documents”

that appeared in the January 1995 edition of Scientific American (Vol.

272, Number 1, pp. 42-7).

As of the above date, this revision could be found at

http://www.clir.org/programs/otheractiv/ensuring.pdf

rev: 980327 Ensuring the Longevity of Digital Information—Jeff

Rothenberg Page 1 of 18

Ensuring the Longevity of Digital Information

POSTCO Access

Accessibility

Chunks

Database

Document

Evaluation


Internet

Organization

Process


Record

Reference

Reproducibility

DOC_NO 15

AUTHOR Jansen, B. J.

Spink, A.

Koshman, S.

TITLE Web searcher interaction with the Dogpile.com metasearch engine

SOURCE Journal of the American Society for Information Science and

Technology

ABSTRACT Metasearch engines are an intuitive method for improving

the performance of Web search by increasing coverage,

returning large numbers of results with a focus on relevance,

and presenting alternative views of information

needs. However, the use of metasearch engines in an

operational environment is not well understood. In this

study, we investigate the usage of Dogpile.com, a major

Web metasearch engine, with the aim of discovering how

Web searchers interact with metasearch engines. We

report results examining 2,465,145 interactions from

534,507 users of Dogpile.com on May 6, 2005 and compare

these results with findings from other Web searching

studies. We collect data on geographical location of

searchers, use of system feedback, content selection,

sessions, queries, and term usage. Findings show that

Dogpile.com searchers are mainly from the USA (84% of

searchers), use about 3 terms per query (mean 2.85),


implement system feedback moderately (8.4% of users),

and generally (56% of users) spend less than one

minute interacting with the Web search engine. Overall,

metasearchers seem to have higher degrees of interaction

than searchers on non-metasearch engines, but their

sessions are for a shorter period of time. These aspects of

metasearching may be what define the differences from

other forms of Web searching. We discuss the implications

of our findings in relation to metasearch for Web

POSTCO Indexing

Searching

Researcher

Relevance

Record

Query

Natural Language

Internet

Evaluation

Document

Database

Classification


Users Guide

In this database the idea is to be able to see what terminology is often linked together for

the sake of learning the art of indexing. As a new student to the SLIS program, it is a scary

endeavor to jump into a database search. This database will hopefully make it less scary to

search for terms and find articles that give an idea on what indexing can do for the seeker of

information. This database was also designed to allow indexers to continue to add information as

new articles are published.

To begin using this database, decide what information you are seeking. If you are seeking

a particular author begin searching by author’s last name in the author field. This field is required

for all the records in this database. There can be more than one author in the field, so you can

enter more than one name for this search. Beware of spelling; names must be spelled exact to

find a match.

The next two searchable fields are for article titles and journal titles. This field is a

required field that will have the complete title of the article and journal or periodical where it

was published. This will make it easier to view the entire article after the search reveals sought

after information. These fields are searchable by exact title or through certain words.

The abstract field will make it easier for the searcher to get a full understanding of what

the article is about. The abstract is an overview of the article. This will help with focusing on

certain topic by providing a general summary. This is searchable by words if that is a desired

field of interest.


The final field for searching is the Postco field. This field has a set list of words to search

from. This field can provide multiple terms for searches. If there is a set group of words that

you are seeking, then enter each word on an individual line. This will provide articles with same

terminology for search result.

The terminology selected for this field came from articles on different aspects of

information retrieval. The terms were some that were used frequently, but did not always have

the same idea if paired with other terms. These terms are stand alone words, but can also be

paired with certain term to help find relevant articles on specific topics you are seeking.

For example if you are looking for the POSTCO word Information retrieval in the journal

Technology Today you can create the query using the search information. When this database

was created it was designed to help gather key information that is essential to informational

retrieval and the different aspects. As the database grows the terminology will also grow and the

searching will continue to improve.


Retrieval Analysis

B1 – B3

Information need #1: I’m interested in research on database usage and user satisfaction.

Pre-determined relevant documents: 9, 13

Field Query Docs Retrieved Precision Recall Effectiveness

Title Database & satisfaction 0 0 0 0

Abstract (database & Satisfaction)

& user

1, 2, 4, 5, 7, 9,

10, 12,13 ,14

.02 .02 .0200

Post-Co Database & satisfaction 9, 13 1 1 1.000

Information need #2: I’m interested in research on information retrieval using natural language.

Pre-determined relevant documents: 5, 9

Field Query Docs Retrieved Precisio

n

Recall Effectiveness

Title Information

retrieval/natural language

0 0 0 0

Abstract Information

retrieval/natural language*

3, 4, 5, 13 .25 .25 .2500

Post-Co Information retrieval & 3, 4, 5, 7 .25 .25 .2500


natural language

Information need #3: I’m interested in research on controlled vocabulary found in chunks.

Pre-determined relevant documents: 7

Field Query Docs Retrieved Precision Recall Effectiveness

Title Controlled

vocabulary/chunks

0 0 0 0

Abstract Controlled

vocabulary/chunks*

0 0 0 0

Post-Co Controlled

vocabulary&chunks

3, 6, 7, 11, 14 0.2 0.2 0.200


Evaluation & Reflection

The evaluation of my database showed that this is not a strong database. Either I

performed searches wrong or I did not have enough choices or used the wrong terminology,

something because I was not very successful in the title or abstract queries. I did not pull up

information even if I tried different Boolean words.

When I performed queries in the Postco field, I did get several relevant hits. I was

pleased with the results in the Postco field. I however do see a need for my words in my

validation list. Maybe even a broader range of words. When I updated my validation list, I did

remove several words where I only got one hit out of 15 articles. I am thinking now these words

should have been left in the list.

I noticed that my abstract and title fields had the poorest number of hits, this made me

wonder how I could improve this results. Not many of my validation words were located into

the title field. I am think I should bring in some words from titles into my validation list. The

same goes for the abstract field. I began to wonder if I should also use abstract words in my

validation list.

This was my first real experience with a database of this nature. I have used Access by

Microsoft, but I grew frustrated with my unfamiliarity with the database. One area of frustration

was trying to find out how to put authors and postco fields on separate lines. I had to hunt in the

help section provided to find the answers. While I knew I could ask for help, but I was thinking

like many library patrons who will not ask for help. The new SLIS student who this was

designed for would be more apt to read through the help section, and then ask for help.


The use of the equations for precision and recall were an interesting way to put a value on

the accuracy of the queries. The ability to put a figure on a result instead of saying 2 out of 10

results was relevant. Even if math is not my strong point the results showed the weakness in the

database result.

I do not think my database is useful for retrieving titles and abstracts. I did a few queries

just using random words for the title and abstract fields. I was able to pull up more results in

these fields without using postco field. Somehow I need to build a strong postco field to ensure

that titles and abstracts result in more relevant retrievals.

I did find when I did a query using author field, I used author last name with initials. I

see that this can present a problem if an author shares same name and initials with another

author. I need to change this to use the entire name not just initials. This should result in better

query results.

I would have liked for someone else to have tested my database. I like being able to see

how others designed databases so I can get ideas to incorporate into mine. I also value the

feedback, no matter how painful, on their experience using the database.

I did look at validation list of my fellow classmates and many had the same or similar

words that I had. Again, looking back, I should not have removed seldom used validation words.

That is part of the learning process.

I really think the hardest part of this assignment for me was the statement of purpose,

rules and user guide. I would like to have the chance to look at examples other than the group

projects and group review. I looked online to find examples, but so many were rather confusing

to me and used terminology that seemed vague or just not useful. I am hoping to try things I


learned in this database program for use in Access or Open SQL so that I stay fresh in database

creation and usage. I was thinking this may be useful for creating a database for books I have

from 25 years in the youth ministry on different lessons style but similar subjects. This would

save me time of having to look through each book trying to find what I am looking for to create a

lesson plan. I see that this database could be useful for someone who needs to find an article

dealing with a weekly reading assignment on a specific topic or issue. Now to figure out how to

make sure it is a successful tool.