Exploring the news | Always multi-source, multimodal and personalized

News Rover

The News Today

The News Today

The News Today

The News Today

The News Today

The News Today

Trends and StatisticsNews consumption is moving online

Advantages:• Access on any device• Available any time, any

place• Find related content

Where did you get your news yesterday?

Source: PEW RESEARCH CENTER 2012 News Consumption Survey

sportspoliticstechsports economicspolitics

Heterogeneous Large-scale Often Isolated

NewsRover Live Recording System

100 TV channels recorded continuouslyLinked to Online News and Twitter topics

Per day:• 110 hours video recorded• 1,380 video stories indexed• 460 Google News topics crawled• 550 Twitter topics crawled

• Total size:– 28,000 hours video

recorded– 464,000 video stories– 24,500 News topics– 80,000 Twitter topics

Navigating the News with StructureW

hen?

Who

?W

hat?

Whe

re?

‘Same-sex Marriage’ ‘France/Mali’ conflict

‘Debt Limit’

Topical Organization

‘Cong. Budget Office’ ‘Consumer Debt’

Coverage Trendline

Event Geo-visualization

Name Extraction & Normalization

Multimodal Topic Extraction & Linking

Who Said What? in VideoFe

atur

e M

odes

Audiotrack:

Segmentation:

Speech Segmentation

Visual Speaker Detection

Speaker Gender Classification

Face Tracking & Clustering

Segmented Audiotrack:

Text

Visu

al

Aud

io

Anchor Detection

Name Extraction from Aligned CC-ASR

Name Extraction from OCR

Name Gender Classification

Speaker Diarization

Edit

Dist

ance

Al

ignm

ent

Male GMM

Female GMM

MFC

C

Clas

sifier

Gend

er

New york times -> search

Shih-Fu Chang, 11/2011

Read, Tag and Search