Techniques forDeep Query

Understanding“Beware of the man who knows the answer before he

understands the question”

Guided By:

Dr. Dhaval Patel,Assistant Professor,

Department Of CSE,IIT Roorkee.

Presented By:

Abhay Prakash, En. No. – 10211002, CSI, V Year, IIT Roorkee.

(Source: Google)

Purpose:

To understand what exactly the user is searching for – his precise intent

To correct mistakes and guide user to formulate a precise intended query

Introduction: Query Understanding

Query Refinement

Why only this phrase in Bold?(Source: Google)Query Suggestion

Emerging Variety of Queries

Natural Language Queries instead of Keyword Represented Queries

“who is the best classical singer in India” instead of “best classical singer India”

Use of NL Queries increasing (Makoto et.al in [1])

Local Search Queries

“Where can I eat cheesecake right now?”

Context Dependent Queries (Interactive Question Answering

(Source: Bing – location set as US)

Background: How results are generated

High Level Architecture of Search Mechanism (Source: Self Made)

INDEX(Knowledge Base)

Document Understanding

(What and how to Index)

Query Understanding

(which index parameters to be used)

User Query

Results Ranking

Entities: Hotel ABC, cheesecakeLocation: Civil LinesQuality: 0.8Time: 8:15 PM

“where can I eat cheesecake right now?”

Data(Text documents,

User Reviews, Blogs, Tweets, Linkedin …)

Review: Hotel ABC, Civil Lines:I ate cheesecake, which was really awesome. (4/5 star)

[Time: 8:15 PM]

Intent: Hotel SearchSearch for: cheesecakeLocation: Civil linesTime: 8:20 PM

Mechanism in Basic Search (Source: Self Made)

Background: QU & Adv. In Search (Weotta in [3])

1. Basic Search

Direct text match based retrieval of documents

Restrict search space using facet values provided by user

Current day example: Online shopping sites

Example of Facets (Source: Flipkart.com)

Background: QU & Adv. In Search (Weotta in [3])

2. Advanced Search

Ranking of result documents based on:

TF-IDF to identify more relevant documents

Website authority and popularity

Keyword weighting

Not Considered:

Context, NLP for semantic understanding

Location of query, time of query

Example: Google as was in its early stage

Background: QU & Adv. In Search (Weotta in [3])

3. Deep Search

What difference does it bring?

Requirements:

Semantic Understanding of Query

Knowledge of Context, previous tasks

User Understanding and Personalization

Architecture: Query Understanding Module

Query

Query Suggestion

Query Correction

Query Expansion

Query Classification

Semantic Tagging

3. Query Intent Detection2. Query Refinement

QUERY UNDERSTASNDING MODULE

ANSWER GENERATION

MODULE

1. Query Suggestion

Result

Components of Query Understanding Module (Source: Self Made)

Architecture: Query Understanding Module

Query

Query Suggestion

Query Correction

Query Expansion

Query Classification

Semantic Tagging

Example of purpose of each Component (Source: Self Made)

michal jrdan

michael jordan

i) michael jordan berkleyii) michael jordan NBA

i) michael jordan berkleyii) michael l. jordan berkley

i) michael jordan berkley: academicii) michael l. Jordan Berkley: academic

i) [michael jordan: PersonName][berkley: Location]: academic

ii) [michael l. jordan: PersonName][berkley: Location]: academic

Reformulates the ill-formed (mistaken) search queries

ex. Macine learning Machine Learning

Refinements:

Spelling error, Two words Merged together, One word separated

Phrase segmentation (machine + learning machine learning)

Acronym Expansion (CSE Computer Science & Engineering)

Refinement may be mutually dependent

“lectures on machne learn”

learn is a correct term, but should have been learning

Hence, different terms need to be addressed simultaneously

Query Correction

Problem Modeled by Jiafeng et.al in [10] as

Original Query () Corrected Query ()

Get y(complete sequence) which has maximum probability of occurrence, given the sequence x.

Simple Technique

Assume terms independent take with max

Prime Disadvantage:

Reality deviates a lot from assumption

Ex. “Lectures on machine learning”

Independent Corrections

Query Correction

What is CRF?

Probabilistic graphical model, models conditional distributionof unobserved state sequences

Trained on given observation sequence

Trained for getting

Why use CRF? Conditioned on?

Sequence of words matters (learning machine?)

conditioned on other s as well, along with

Corrections are mutually dependent (e.g. machine learning)

Disadvantage:

Will require very large amount of data, candidates’ domain open

Query Correction Using Conventional CRF

Conventional CRF

Restricting space of y for the given x

conditioned on operation also

, such that required to get from

is operation like deletion, insertion of characters, etc.

Learning and Prediction

Dataset of

Features

, where the prob. calculated using corpus

Whether --{0|1}

Basic CRF-QR Model

Query Correction Basic CRF-QR Model (Jiafeng et. al in [10])

What is new?

Handles scenario with more than one refinements

Machine learm learn learning

Sequence of (sequence of operation)

i.e. multiple operations on each word

Intermediate results:

Extended CRF-QR

Query Correction Extended CRF-QR Model (Jiafeng et. al in [10])

Query Suggestion

Purpose:

Suggest similar queries

Query auto-completion

Requirements

Context consideration [7]

Identifying Interleaved Tasks [9]

Personalized suggestion [2]

Suggestions on “iit r..”

Query Suggestion Mechanism (Source: [7])

Query – mapped Concept

Concept Suffix tree from log

Suggestion time: Transition on tree with each query’s concept

Suggest top queries of that state

Query Suggestion

Context aware Query Suggestion (Huanhuan et.al in [7])

Concept Suffix Tree

Concept Discovery

Queries clustered using set of clicked URLs

Feature vector

Each identified cluster is taken as a Concept

Concept Suffix Tree

Vertex: state after transition through a sequence of concepts (of queries)

Transition in a session

C2C3C1: transition Beginning C1 C3 C2

Click-Through Bipartite

Query Suggestion

Context aware Query Suggestion (Huanhuan et.al in [7])

Why task identification Important?

Considering Off-Task query in context adversely affect quality of recommendation

30% sessions contained multiple tasks (Zhen et.al in [8])

5% sessions have Interleaved tasks (Zhen et.al in [8])

Identify similar previous queries as On-Task

consider only On-Task queries as context

Query Suggestion

Task aware Query Suggestion (Allan et.al in [9])

Effect of On-Task and Off-Taskqueries

Query Suggestion

Task aware Query Suggestion (Allan et.al in [9])

Measures to evaluate similarity between two queries

Lexical Score: captures similarity at word level directly. Average of:

Jaccard Coefficient between trigrams from the two queries: how many common trigrams?

(1 - Levenshtein Edit Distance), which shows closeness at word level

Semantic Score: maximum of the following two

: cosine similarity of vector of tf-idf score of Wikipedia documents w.r.t the two queries.

: similar to above on Wiktionary entries

Final Similarity() = . Lexical Score + (1-) . Semantic Score

If Similarity(, Reference_q) greater than threshold is On-Task Query

Query Suggestion

Personalization in Query Suggestion (Milad et.al in [2])

On character hit of ‘i’

“Instagram” more popular for female below 25

“Imdb” more popular for male in 25-44.

Candidate queries generated by prior general method

Personalization by re-ranking candidate queries

Features for feedback earlier global rank Original position

Original score

Short History Features 3-Gram similarity with just previous query

Avg. 3-gram similarity with all previous queries in the session

Query Suggestion

Personalization in Query Suggestion (Source: [2])

Long History Features

No. of times candidate query issued in past

Avg. 3-gram similarity with all previous queries in the past

Demographic Features

Candidate query frequency over queries by same age group

Candidate query likelihood -- same age group

Candidate query frequency -- same gender group

Candidate query likelihood -- same gender group

Candidate query frequency -- same region group

Candidate query likelihood -- same region group

Query Expansion

Sending more words (should generate similar result) to tackle term-miss

Ex. “Tutorial lecture on ABC” “Video Lecture on ABC”

Expansion Tasks:

Adding synonyms of words

Morphological words by stemming

Naïve Approach

Exhaustive lookup in thesaurus

Time taking

Still miss terms of similar intent (terms even semantically far)

Query Expansion

Path Constrained Random Walk (Jianfeng et.al in [11])

Exploiting search logs for identifying terms having similar end result

Search log data of <Query, Document> clicks

Graph Representation

Node Q: seed queryNodes Q’: queries in search logNodes D: documentsNodes W: words that occur in queries and documents

Word nodes are the candidate expansion terms

Edges have scoring function

Represents probability of transition from start node to end node

Search Log as Graph

Query Expansion

Path Constrained Random Walk (Jiafeng et.al in [11])

Probability of using w as an expansion word?

Product of probabilities in Paths starting at node Q and ending at w

Top probable words picked, obtained from random walk

Search Log as Graph

Query Classification

Classifying given query in a predefined Intent Class

Ex. michael Jordan berkley: academic

Precise intent by sequence of nodes from root to leaf

More challenging than document classification

Short length

Keyword representation, makes more ambiguous

Ex. query “brazil germany”

Older basic techniques

Considering single query statistical techniques like 2-gram/3-gram inference

Example Taxonomy (Source: [6])

Query Classification

Context aware Query Classification (Huanhuan et.al in [6])

Resolving ambiguity using context

Previous Queries sports, then “Michael Jordan” sports (Basketball Player)

Previous Queries academic, then “Michael Jordan” academic (ML professor)

Use of CRF (because training and prediction on sequence)

Local Features

Query Terms: Each supports a target category

Pseudo Feedback:

with concept , submitted to an external web directory

How many of top M results have concept?

Implicit Feedback:

Instead of Top M results – only the clicked documents taken

Query Classification

Context aware Query Classification (Huanhuan et.al in [6])

Contextual Features

Direct association between adjacent labels

Number of occurrences of adjacent labels

Higher weight higher probability of transit from to

Taxonomy-based association between adjacent labels

Given pair of adjacent labels at level n

n-1 features of taxonomy-based association between considered

e.g. Computer/Software related to Computer/Hardware, matching at (n-1)th level Computer

Semantic Tagging

Identifies the semantic concepts of a word or phrase [michael jordan: PersonName] [berkley: Location]: academic

Useful only if phrases in documents also tagged

Shallow Parsing Methods

Part of Speech Tags: e.g. Clubbing consecutive nouns for Named Entity Recognition

Disadvantage: Sentence Level Long Segments can’t be identified

Semantic Tagging

Hierarchical Parsing Structures

Trained a semi-Markov CRF on segments

Features

Syntactic Features

Parse tree of sentence

Plot

Semantic Tagging

Semantic Dependency Features

leverage the information about dependencies among different segments

Ex. “show me a funny movie starring Johnny and featuring Carribbean Pirates”

‘Featuring’ takes arguments – “funny movie” and “Carribbean Pirates”

long distance semantic dependency between the object “movie” and attribute <Plot>

Conclusion & Future Work

End-to-End Discussion of Query Understanding Module Tasks

Semantic Understanding of queries for intent detection has lot of scope

Use of NL (grammatically correct) queries rising

Understanding at the structure level

User community detection for its application in Query Suggestion

Based on search behavior

Community/Topic specific temporal trending of search query

References

[1] Makoto P. Kato, Takehiro Yamamoto, Hiroaki Ohshima and Katsumi Tanaka, "Cognitive Search Intents Hidden Behind Queries: A User Study on Query Formulations," in WWW Companion, Seoul, Korea, 2014.

[2] Milad Shokouhi, "Learning to Personalize Query Auto-Completion," in SIGIR, Dublin, Ireland, 2013.

[3] Weotta, "Deep Search," 10 6 2014. [Online]. Available: http://streamhacker.com/2014/06/10/deepsearch/. [Accessed 6 8 2014].

[4] W. Bruce Croft, Michael Bendersky, Hang Li and Gu Xu, "Query Understanding and Representation," SIGIR Forum, vol. 44, no. 2, pp. 48-53, 2010.

[5] Jingjing Liu, Panupong Pasupat, Yining Wang, Scott Cyphers and Jim Glass, "Query Understanding Enhanced by Hierarchical Parsing Structures," in ASRU, 2013.

[6] Huanhuan Cao, Derek Hao Hu, Dou Shen and Daxin Jiang, "Context-Aware Query Classification," in SIGIR, Boston, Massachusetts, USA, 2009.

References (Continued…)

[7] Huanhuan Cao, Daxin Jiang, Jian Pei, Qi He, Zhen Liao, Enhong Chen, Hang Li, "Context-Aware Query Suggestion by Mining Click-Through and Session Data," in KDD, Las Vegas, Nevada, USA, 2008.

[8] Zhen Liao, Yang Song, Li-wei He and Yalou Huang, "Evaluating the Effectiveness of Search Task Trails," in WWW, Lyon, France, 2012.

[9] Allan, Henry Feild and James, "Task-Aware Query Recommendation," in SIGIR, Dublin, Ireland, 2013.

[10] Jiafeng Guo, Gu Xu, Hang Li and Xueqi Cheng, "A Unified and Discriminative Model for Query Refinement,“ in SIGIR, Singapore, 2008.

[11] Jianfeng Gao, Gu Xu and Jinxi Xu, "Query Expansion Using Path-Constrained Random Walks," in SIGIR, Dublin, Ireland, 2013.