Relevance Feedback for the Modern Searcher: Elicitation Techniques RevisitedDiane KellySchool of Information and Library ScienceUniversity of North Carolina at Chapel Hill, USA

Once upon a time …

Explicit-O-Saurus

User-Model-O-Saurus

Implicit-O-Saurus

The Story of Explicit-O-Saurus • Review (some) past and present use of explicit feedback

techniques• Relevance feedback (short-term) (document and term)• User modeling (long-term)

• Present typical explanations for why these techniques are not used by searchers

• Argue that these explanations are no longer adequate

Schultz, C. K. (1968). H.P. Luhn: Pioneer of Information Science (p.32). London, UK: American Documentation Institute.

1950s: Luhn’s Selective Dissemination of Information

1960s: Salton, Rocchio, and Ide

Ide, E. (1967). User interaction with an automated information retrieval system. In G. Salton (Ed.) Information Storage and Retrieval: Scientific Report No. ISR-12.

Oddy, R. N. (1977). Information retrieval through man-machine dialogue. Journal of Documentation, 33(1), 1-14.

1970s: Oddy’s Thomas

1980s: User Modeling

Allen, R. B. (1990). User models: Theory, method, and practice. International Journal of Man-Machine Studies, 32, 511-543.

Rich, E. (1983). Users are individuals: Individualizing user models. International Journal of Human-Computer Studies, 51, 323-338.

“While the term ‘user model’ emphasizes the information about the person, it is obvious that a great deal of situational, task, or environmental information may be encoded in the model.”

Croft, W. B., & Thompson, R. H. (1987). I3R: A new approach to the design of document retrieval systems. Journal of the American Society for Information Science, 38, 389-404.

1980s: Intelligent IR

Maes, P. (1994). Agents that reduce work and information overload. CACM, 37(7), 30-40.

1990s: Agents

SYSTEM

USER

CAT-A-CONE: AN INTERACTIVEHearst, M. A. & Karadi, C. (1997). Cat-a-Cone: An Interactive Interface for Specifying Searches and Viewing Retrieval Results using a Large Category Hierarchy. Proc. of SIGIR ‘97.

1990S: And Some Visualization

Belkin, N. J., Cool, C., Kelly, D., Lin, S.-J., Park, S.Y., Perez-Carballo, J., & Sikora, C. (2001). Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval. Information Processing & Management 37(3), 404-434.

1990s: And more relevance feedback

2000s: Implicit Feedback

Click-through Dwell time Scrolling Queries

But explicit feedback is not really dead …

Date Viewed Online: October 12, 2010

http://www.movielens.org/rateMore and http://www.grouplens.org/

Dates Viewed Online: October 12 and December 06, 2010

Date Viewed Online: November 11, 2010

Date Viewed Online: November 11, 2010

Date Viewed Online: November 11, 2010

Date Viewed Online: December 06, 2010

http://hunch.com/

Date Viewed Online: October 12, 2010

What Caused the Supposed Demise of Explicit Feedback?

1. Users are unwilling to put forth the effort required to provide feedback

2. Users don’t have the additional cognitive resources to engage in explicit feedback

REASON 1: Users are unwilling to put forth the effort required to provide feedback.

http://www.pewinternet.org/http://www.visualeconomics.com/how-the-world-spends-its-time-online_2010-06-16/

REALLY?

REASON 2: Users don’t have the additional cognitive resources to engage in explicit feedback.

• Well, maybe back then …

• Better questions

• Better measures

A Future for Explicit Feedback

“While the term ‘user model’ emphasizes the information about the person, it is obvious that a great deal of situational, task, or environmental information may be encoded in the model.” (Allen, 1990)

• More creative

• More engaging

• More adaptive

Focused on information-seeking support not information search support.

A Future for Explicit Feedback