interactive informationretrieval 토인모_201202
DESCRIPTION
TRANSCRIPT
Interactive Information Retrieval
2012.2.18박 정 아
[email protected]://twitter.com/sunseed9
토요인지모임 2012 2월 발표
Contents
§ Interactive Information Retrieval§ Models of Information Seeking Behavior § Evaluation & Relevance § Search User Interfaces§ Towards User-centered Search
Information Technology
Background- 컴퓨터과학 (Computer Science) + 국어국문학
- 자연어처리 (Natural Language Processing)
- 인지과학 (Cognitive Science)
- 정보검색 (Information Retrieval)
Human Computer
Interacti
검색
Information Retrieval System
Interactive Information Retrieval
Human centered approach in Information Retrieval § Cognitive viewpoint in information retrieval
§ Traditional IR model concentrates on matching not user side & interaction
§ interactive IR§ more than simply developing interfaces for searching
§ Shneiderman, Byrd, & Croft, 1998
§ the strength of good research in IIR comes not only from a technical knowledge of interactive systems development but also from a knowledge of people’s search behavior and search context, including the environmental factors that influence behavior § Fidel & Pejtersen, 2004
§ HCIR (Human Computer Information Retrieval)§ Human-Computer Interaction + Information Retrieval§ HCI & IR § HCI and IR come from different traditions;
§ HCI, for example, places more emphasis on the published literature on usability § whereas IR emphasizes effectiveness.
Interactive information retrieval (IIR)
§ “ Interactive information retrieval (IIR) is, of its nature, cognitive. One important goal for IIR research is to study user interaction with a search system to learn about the user’s search intent and when they encounter relevant documents. Taking account of the user and her context has potential to improve understanding of the search process and the user’s intent. A search system that “knows” this information can improve its performance in retrieving documents that satisfy user’s needs. Awareness of demands imposed on user’s cognitive processing and levels of user’s knowledge can also contribute to improvements in system performance”
§ from "Inferring Cognitive States from Multimodal Measures in Information Science"
§ interactive IR is more than simply developing interfaces for searching (Shneiderman, Byrd, & Croft, 1998) and that the strength of good research in IIR comes not only from a technical knowledge of interactive systems development but also from a knowledge of people’s search behavior and search context, including the environmental factors that influence behavior (Fidel & Pejtersen, 2004).
Interactive Information Research problems
§ IIR research addresses three major problem areas § (1) understanding information seeking needs and behaviors;
§ (2) developing retrieval systems that respond to information needs and support information seeking behaviors and interactions;
§ (3) developing methods and measures to study and evaluate behaviors, interactions and systems.
§ issues§ information seeking behavior related information needs and query intent
§ Models of the Information Seeking Process
§ Design of Search User Interfaces§ Presentation of Search Results§ includes document surrogates, properties of results listings, summaries (snippets)
as used in search results
§ How people evaluate IR systems / Search Quality
Models of information seeking behavior
Models of the Information Seeking Behavior
§ Bates’ berrypicking – acts in searching§ Dervin's sense-making theory – gap, bridge§ Ellis’s Information Seeking Process§ Kuhlthau’s information search process§ Ingwersen's cognitive model§ Wilson's information-seeking behaviour model§ Saracevic's model of stratified interaction
Models of the Information Seeking (1)
§ Bates' Berrypicking - “dynamic” not “static”
Models of the Information Seeking (2)
§ Dervin's sense-making theory
§ A problem-solving mode§ The solution of the problem, the resolution of the discrepancy, the
advance from uncertainty to certainty
Figure 3: Dervin's 'sense-making'
questioning that can reveal the nature of a
problematic situation, the extent to which
information serves to bridge the gap of uncertainty,
confusion, or whatever, and the nature of the
outcomes from the use of information.
Models of the Information Seeking (3)
§ Ellis’s Information Seeking Process
§ Kuhlthau’s information search process
Figure 5: A process model based on Ellis's 'characteristics'
Figure 2.2: Kuhlthau's model of the search process
Models of the Information Seeking (4)
§ Ingwersen's cognitive model§ 'traditional' model
§ represents IR as a two prong set (system and user) of elements and processes converging on comparison or matching...';
§ Ingwersen's cognitive model [27], (cognition)
§ concentrates on identifying processes of cognition which may occur in all the information processing elements involved.
Models of the Information Seeking (5)
§ Saracevic's model of stratified interaction § Saracevic then goes on to propose what he calls a 'stratified interaction model' developed within an overall
framework of an 'acquisition-cognition-application' model of information use.
Figure 9: Saracevic's model of stratified
The levels or strata posited by Saracevic
are simplified (in his words) to three:
1. surface, or the level of interaction
between the user and the system
interface;
2. cognition, or the level of interaction
with the texts or their representation
3. situation, or the context that
provides the initial problem at hand.
Evaluation & Relevance
Evaluation of interactive information retrieval systems with users
§ THE EVALUATION OF SEARCH USER INTERFACES
§ Measure§ search interfaces are usually evaluated in terms of three main aspects of usability:
effectiveness, efficiency, and satisfaction, which are defined by ISO 9241-11, 1998 as:
§ Effectiveness§ Accuracy and completeness with which users achieve specified
goals.
§ Efficiency§ Resources expended in relation to the accuracy and completeness
with which users achieve goals.
§ Satisfaction§ Freedom from discomfort, and positive attitudes towards the use of
the product.
Evaluation of interactive information retrieval systems with users
§ STANDARD INFORMATION RETRIEVAL EVALUATION§ Text REtrieval Conference (TREC), run by the U.S. National Institute of
Standards (NIST) for more than 15 years (Voorhees and Harman, 2000)
§ The most common evaluation measures used for assessing ranking algorithms
§ Precision, Recall, F-measure, Mean Average Precision (MAP).
§ The TREC evaluation method has been enormously valuable for comparison of competing ranking algorithms.
§ discounted cumulative gain (DCG) (Järvelin and Kekäläinen, 2000, Kekäläinen, 2005)
USABILITY TESTING - the results of an eye-tracking study
Arrows indicate dominant directions of eye movement; “hotter” colors indicate more frequent eye fixations, and X's indicate locations of clicks. From Search Engine Results: 2010, by Enquiro Research
The Dynamics of relevance judgment and its order effect§ dynamics of relevance
§ change-in-meaning hypothesis => direct impression
§ subject's relevance judgments would vary as a consequence of the order of presentation
§ first received information affected participants' impression of the following information
§ learning effect (Harter, 1992)
§ fatigue effects ( clancy & wachsler 1971)§ information item이 어느정도로 증가하면 subject는 너무 fatigue해서 carefully하게 반응 어려움.
§ the dynamics of relevance judgment
§ (a) the changing external task situation and demand that modify a user’s information need (Cuadra & Katter, 1967; Park, 1993); (b) the changing cognitive state of the user as a result of encountering relevant documents (Harter, 1992; Xu, 2007b); (c) the different modes of document presentation § such as showing only the title, the abstract, bibliographic information, or the whole
document content (Ingwersen & Järvelin, 2005).
§ belief-adjustment model§ Hogarth & Einhorn 1992
Summary styles compared in an experiment by Aula, 2004.
Brand Awareness and the Evaluation of Search Results
검색 엔진 브랜드가 검색 결과 품질에 미치는 영향 (The Effect of Brand Awarenes On the Evaluation of Search Engine Results)
• 연구 내용 요약◦ 사람들에게 같은 검색 결과가 제시되어도 검색 엔진이 무엇이냐에 따라 검색 품질 평가가 달라진다는 것을 재미
있는 실험을 통해 증명함으로써, 검색 엔진 브랜드가 검색 품질 평가의 한 요소임을 밝힌 연구
• 실험 방법 및 결과◦ 사용한 쿼리
■ camping mexico, laser removal, manufactured home, techo music■ 150만개 가량의 e-commerce 검색 로그로 부터 분야별로 4개의 쿼리 선정
◦ 검색 결과■ 선정된 4개의 검색 쿼리를 구글 검색에 던져서 검색 결과 저장
◦ 검색 로고■ 각각 Google, MSN, Yahoo의 로고들을 캡쳐하여 사용하고, AI2RS라는 새로운 검색 브랜드 로고 추가로
생성하여 사용• 실험 대상
◦ 18~25세 사이의 미국 대학생 32명( 남자 24명, 여자 8명)• 실험 결과
■ 결과적으로 이름 없는 AI2RS 는 평균적으로 10% 떨어지는 평가를 받았다.■ Yahoo는 4개 쿼리 모두 평균 이상의 평가를 받으며 높은 브랜드 인지도를 나타내었다.
• 결론◦ 검색 성능(품질) 평가에서 검색 엔진 브랜드의 영향을 살펴본 결과, 동일 한 검색 결과라도 검색 엔진 브랜드에
대한 인식에 따라 검색 품질 평가에 상당한 영향을 미칠 수 있다는 것을 알 수 있다.
적합성 (Relevance)
• 시스템 중심 적합성 vs 사용자 중심 적합성
• 시스템 중심 적합성– 벡터 스페이스 적합성(vector space relevance), 확률 적합성
(probabilistic relevance), 불린 적합성(Boolean relevance)• Borlund, 2003
시스템 중심 사용자 중심
논리적 (Cooper, 1971) 심리적 (Wilson, 1973)
주제적 (Cooper, 1971; Park, 1994) 상황적 (Wilson, 1973; Harter, 1993)
객관적 (Swanson, 1986; Howard, 1994) 주관적 (Swanson, 1986; Howard, 1994)
적합성 분류 (출처. Maglaughlin & Sonnenwald, 2002)
사용자 중심 적합성
• 사용자 중심 적합성 연구 활발
• 특히 90년대 연구 집중
– Froehlich, 1994; Green, 1995; Harter, 1992; Janes, 1994; Mizzaro, 1998; Park,
1994; Saracevic, 1996; Schamber, Eisenberg, & Nilan, 1990
• 사용자 중심 적합성에 대한 정의
– “적합성이란 다차원의 인지적 개념으로써 사용자의 정보 인식과 정
보 이용자의 정보 요구 상황에 상당 부분 의존한다” (Borlund, 2003)
– 상황 적합성(Schamber, 1990), 심리적 적합성(Harter, 1992), 과제
기반 적합성 (Cosijn, 2000; Reid 1999) 등
Saracevic의 적합성 분류(1996)
• 시스템 중심 적합성
– 시스템 적합성
• 검색어와 문서간의 유사도
• 사용자 중심 적합성
– 주제 적합성
• 검색어와 문서의 주제
– 인지 적합성
• 문서가 정보 이용자의 지식 상태와 인지적인 정보 요구에 얼마나 잘 부합하는지
– 상황 적합성• 문서가 상황이나 현재 문제에 얼마나 잘 적합한지
Saracevic’s stratified model of IR interaction
Tefko Saracevic 28
INTE
RACT
ION
& RE
LEVA
NCE
STRA
TA
Adap
tatio
n
Adap
tatio
n
Use
of in
form
atio
n
COM
PUTE
R
USER
Contentinf. objects; representations...
Processingsoftware; algorithms …
Engineeringhardware; connections...
Surface level
INTERFACE
Querycharacteristics …
Affectiveintent; motivation ...
Cognitiveknowledge; structure...
Context
Situationaltasks; work context...
social, cultural
사용자 적합성 판단 기준 연구들출처 도메인 참가자 판단 기준
개수
Schamber, 1991 날씨정보검색 직장인 30명 10
Park, 19913 석사논문연구주제 검색 대학원생 11명 22
Barry, 1994 온라인 정보 검색 학생 18명 23
Spink et al. 1998 연구 정보 검색 교수와 학생 11명 27
Bateman, 1998 논문검색 대학원생 35명 40
Wang & Sorel, 1998 연구 프로젝트 검색 대학원생 25명 11
Tang & Solomon, 1998 기말 논문 검색 대학원생 1명 10
Hirsh, 1999 스포츠 검색 초등학생 10명 14
Maglaughlin & Sonnenwald, 2002
논문검색 대학원생 12명 29
Choi & Rasumuseen, 2002
이미지 검색 학생 38명 9
Savolainen & Kari, 2006 웹 검색 학생 9명 18
My Dissertation
정보 검색에서의 사용자 중심 적합성 판단 모형 개발 및 평가
• 개요 (Overview)– 한국 통합 검색 환경에서의 사용자 적합성 판단에 관한 연구
– 정보 검색 과제 별 사용자가 적합성을 판단하는 기준과 적합성 유형과의 관계
• 연구1. 사용자 적합성 판단 기준에 관한 탐색적 연구 – 한국 통합 검색 환경
– 반구조(semi-structured) 인터뷰
• 연구2. 사용자 적합성 판단 모형 개발 – 정보 검색 과제별 적합성 판단 기준과 적합성 유형의 관계
– Xu & Chen 적합성 판단 기준 정량적 연구 기반– 반제어(semi – controlled) 설문
– “적합성 유형”별, “정보 검색 과제”별
• 연구 3. 정보 검색 과제별 동적 검색 랭킹 모델 구현 및 검증 – 사용자 적합성 판단 기준 랭킹 요소로 정보 검색 시스템 반영
– 사용자 평가 비교 실험• 정적 검색 랭킹 모델 vs 동적 검색 랭킹 모델
정보 검색 과제
Navarro-Prieto,
Scaife, & Rogers (1999)
Kelly et al.
(2002)
Limberg, L.
(1999)
White, Jose,
& Ruthven (2003)
Freund (2008) This research
Fact finding
Fact finding
(fact question)
Ex)“How long does it take to get a passport?
Fact-finding fact search
e x ) a n a m e d person’ s current email address
Fact finding
사실 검색
Exploratory
Procedural
(task question)
Ex) “How do I get a passport?”
Understanding a topic
b a c k g r o u n d search
ex) dust allergies
Learning 문제 해결 검색
Exploratory
Procedural
(task question)
Ex) “How do I get a passport?”
Understanding a topic
b a c k g r o u n d search
ex) dust allergies How to
문제 해결 검색
Exploratory
Procedural
(task question)
Ex) “How do I get a passport?”
Understanding a topic
b a c k g r o u n d search
ex) dust allergies
Solve a problem
문제 해결 검색
Exploratory
Procedural
(task question)
Ex) “How do I get a passport?”
assessing an
i s s u e a n d r e a c h i n g a decision
decision search
ex) find Rome’ s best museum f o r impress ion i s t art
Make a decision 의사 결정 검색
• 정보 검색 과제에 따른 사용자 행동 또는 반응 기존 연구
Search User Interfaces
Search User Interfaces
Presentation of Search Results (Ranked List)
Search results listings from Infoseek in 1997 (left) and Google in 2007 (right), courtesy Jan Pedersen.
Designing Search for Humans - Provide Memory Aids
Suggest the Search Action in or near the Query Form
www.yelp.com, www.powerset.com
Memory AidsProvide Access to Recent Actions
PubMed
amazon.com Dumais et al., Stuff I’ve Seen, SIGIR 2003
38
Memory Aids; Anchoring Aids
Dynamic Query Suggestions
http://www.daum.net
http://google.com
39
Suggest Next Steps: Query suggestions
Show suggestions after the query has been issued.
http://bing.com
http://yahoo.com
40
Suggest Next Steps: Query suggestions
http://nextbio.com
PubMed
41
Putting It All Together: Faceted Navigation
§ Suggests next steps§ Helps with Vocabulary Problem and Anchoring
Problem§ Promotes Flow
§ Show users structure as a starting point, rather than requiring them to generate queries
§ Organize results into a recognizable structure
§ Eliminates empty results sets
42
A New Development: Faceted Breadcrumbs
Nudelman, http://www.boxesandarrows.com/view/faceted-finding-with
Towards User-centered Search
Search is not easy
Frowns, Sighs, and Advanced Queries -- How does search behavior change as search becomes more difficult?
Instant Search
43
http://goo.gl/p0Alw
Interface in Mobile (1)
청각적(voice) 정보를 이용한 검색 시각적(visual) 정보를 이용한 검색
Interface in Mobile (2)초성검색 통합웹 컷오프
모바일 화면의 제약으로 인한 컴팩트한 검색 결과 제공
'Natural' Search User Interfaces
§ 'Natural' Search User Interfaces | November 2011 | Communications of the ACM§ 사람들은 자연스러운(natural) 인터페이스 - 타이핑 입력보다 말하기를, 텍스트 읽기 보다 동영상 보기 - 를 원하고, 혼자가 아닌 함께(social)를 원한다.§ "Users will speak rather than type, watch video rather than read,
and use technology socially rather than alone"
§ 따라서 검색 인터페이스는 natural 과 social 을 지원하는 방향으로 발전해 가야한다.
§ Siri / Social Search
Siri, artificially intelligent voice search assistant
소셜 검색 (What is Social search?)
Trapit, Discovery Engine
[Curation]
• 인간의 요소, 인간만이 - 패턴을 인식하는 인간 고유의 능력 @ <큐레이션, 인간을 지향하다>
"프로그래머와 큐레이터로서 인간의 역할이 사라지는 일은 일어나지 않을 거에요. 컴퓨터가 절대로 따라올 수 없는 부분이 있으니까요. 그게 바로 인간의 요소, 인간만이 떠맡을 수 있는 부분이죠"
Beyond Search
“Contextual discovery will take data gathered from people’s browsing data and location profiles and use it to serve up interesting and relevant results – without the search”
