snow workshop, 8th april 2014 real-time topic detection with bursty ngrams: rgu participation in...

SNOW Workshop, 8th April 2014

Real-time topic detection with bursty ngrams:

RGU participation in SNOW 2014 challengeCarlos Martin and Ayse Goker (Robert Gordon University)

Outline

• Architecture diagram• Results• Future work

#2

Architecture diagram

#3

Crawler EntitiesExtractor

Solr

Tweets(English)

Tweets

Tweets (with Entities)

Architecture diagram

#4

Crawler EntitiesExtractor BNgram

Keyword Extractor

TopicAggregator

Solr

Topics Combiner

Query Builder

Topic Labeller

Tweets(English)

Tweets

Tweets (with Entities)

Ranked topics

Topics (+ keywords, entities, hashtags

and urls)

Merged topics Topics (+ label)Topics (+ tweets)

Entities Extractor

• Extract entities per tweet using Stanford NER (http://nlp.stanford.edu/software/CRF-NER.shtml).

• 3 class model Identifies Person, Location and Organization.

• Efficient enough for a real-time system.

#5

Architecture diagram

#6

Crawler EntitiesExtractor BNgram

Solr

Tweets(English)

Tweets

Tweets (with Entities)

Ranked topics

BNgram approach

• Detection of bursty ngrams based on df-idf score Bursty entities, hashtags and urls are also included in the approach. Re ngrams, 2- and 3-grams are considered (no unigrams anymore).

• Variant of tf-idf Penalization of frequent terms in previous timeslots.

• Terms containing hashtags, entities, urls are boosted.• Two previous timeslots (s=2) were considered in our

experiments.

#7

BNgram approach

• “Partial” membership clustering approach is an interesting alternative as one term could belong to different clusters (For example, entity “Obama” for the stories “Obama wins in Ohio” and “Obama wins in Illinois”).

• Apriori clustering algorithm has been used in the experiments of SNOW challenge

• Explore maximal associations between terms based on the number of shared tweets.

#8

BNgram approach

• Output: Clusters of trending terms with tweets from the last timeslot associated to them.

• A tweet should contain a minimum number of cluster terms to be included.

• Clusters are ranked by their bursty scores (maximum df-idf value of topic terms)

#9

Architecture diagram

#10

Crawler EntitiesExtractor BNgram

Keyword Extractor

TopicAggregator

Solr

Tweets(English)

Tweets

Tweets (with Entities)

Ranked topics

Topics (+ keywords, entities, hashtags

and urls)

Keyword Extractor and Topic Aggregator modules• Topic Aggregator module:

– Aggregate entities, hashtags and urls per topic (coming from topic tweets of the corresponding timeslot) keeping their frequencies.

– Keep those ones whose frequency is higher than a threshold.• Keyword Extractor module:

– Extract main keywords (including ngrams) per topic (not extracted from Topic Aggregator) using bursty terms from the clusters.

– Removal of urls, hashtags, user mentions, entities and acronyms.

– Overlaps are also removed.– Keep df-idf scores as their weights.

#11

Architecture diagram

#12

Crawler EntitiesExtractor BNgram

Keyword Extractor

TopicAggregator

Solr

Topics Combiner

Tweets(English)

Tweets

Tweets (with Entities)

Ranked topics

Topics (+ keywords, entities, hashtags

and urls)

Merged topics

Topic Combiner module

• Topic Combiner module:– Merge similar topics from the same timeslot.– Based on the co-occurrence of keywords (unigrams),

entities, hashtags and urls from the compared topics.– According to preliminary results, Apriori algorithm makes

this module more accurate as one term could belong to different topics.

#13

Architecture diagram

#14

Crawler EntitiesExtractor BNgram

Keyword Extractor

TopicAggregator

Solr

Topics Combiner

Query Builder

Tweets(English)

Tweets

Tweets (with Entities)

Ranked topics

Topics (+ keywords, entities, hashtags

and urls)

Merged topics Topics (+ tweets)

Query Builder module

• Creation of final queries to retrieve all the related tweets to the topic (Solr queries) and also filtering by time (simulating real-time scenario).

• 3 types of queries:– Keywords– Entities and Hashtags– Urls

• If keywords and entities in topic, keywords closer to the entities are the selected ones.

• Image population: If tweets contains links to images (metadata), they are added to the topic.

#15

Query Builder module

• Replies are also considered. Be careful with spam replies

• Replies are not text-query dependent. More diversity?.

• Sentiment analysis, extraction of relevant keywords.

#16

Query Builder module

• Diverse tweets are computed based on cosine similarity.

• This approach could be more or less strict depending on the selected threshold.

#17

Architecture diagram

#18

Crawler EntitiesExtractor BNgram

Keyword Extractor

TopicAggregator

Solr

Topics Combiner

Query Builder

Topic Labeller

Tweets(English)

Tweets

Tweets (with Entities)

Ranked topics

Topics (+ keywords, entities, hashtags

and urls)

Merged topics Topics (+ label)Topics (+ tweets)

Topic Labeller module

• BuzzFeed editor-in-chief Ben Smith: “Headlines sure look a lot like tweets these days.” (http://perryhewitt.com/5-lessons-buzzfeed-harvard/)

• For each topic tweet, a score is computed based on the following formula.

where α = 0.8. The tweet with the highest score is selected as the Topic label after cleaning it.

#19

Topic Labeller module

• Example of tweets after cleaning them

• Granularity is still an issue Some topic labels are too general or specific.

#20

Results - Examples of topics

#21

Future work

• Improve Topic Combiner module – use of similarity measures.

• Further research on the use of replies and diverse tweets per Topic.

• Improve Topic Labeller module – granularity issue.• Modifications in QueryBuilder module – use of term

weights (Solr).

#22

Thank you!

E-mail address: c.j.martin-dancausa@rgu.ac.ukTwitter account: @martincarloscit