GTM-UVigo System for MultimodalPerson Discovery in Broadcast TV Taskat MediaEval 2016

Paula Lopez-OteroLaura Docio-FernandezCarmen Garcia-Mateo

Main contributions

I A different point of view of the Person Discovery task

I Video i-vectors

Proposed strategy

Proposed strategy

Proposed strategy

Proposed strategy

Proposed strategy

Proposed strategy

Video i-vectors

Video i-vectors

Video i-vectors

Video i-vectors

Issues to be solvedFacial feature extraction

I Face tracking → baseline approachI Face detection and feature extraction → Bob toolkit

I Features are extracted for the “best” face in the shot

Issues to be solvedFacial feature extraction

I Face tracking → baseline approachI Face detection and feature extraction → Bob toolkit

I Features are extracted for the “best” face in the shot

Issues to be solvedFacial feature extraction

I Face tracking → baseline approachI Face detection and feature extraction → Bob toolkit

I Features are extracted for the “best” face in the shot

Issues to be solvedSpeech-based search

I Shots are assumed to have speech of one speaker → Speakersegmentation

Issues to be solvedSpeech-based search

I Shots are assumed to have speech of one speaker → Speakersegmentation

Issues to be solvedWritten name detection

amie de la famille sarraute

al hopital cochin hotel dieu

carte de fra mauro

femme de frederic dard

ma fille vend son corps

domicile de marie et remy

message offert par france 2

comme la riviere au delta

anzos pourquoi bernard tapie aurai

rome : un pape travailleur

Errors at the baseline name de-tection stage!

I Natural languageprocessing

I Named entity detection

Issues to be solvedWritten name detection

amie de la famille sarraute

al hopital cochin hotel dieu

carte de fra mauro

femme de frederic dard

ma fille vend son corps

domicile de marie et remy

message offert par france 2

comme la riviere au delta

anzos pourquoi bernard tapie aurai

rome : un pape travailleur

Errors at the baseline name de-tection stage!

I Natural languageprocessing

I Named entity detection

Future challenges

Fully multimodal Person Discovery

I Audiovisual i-vectors

I Early fusion

Future challenges

Fully multimodal Person Discovery

I Audiovisual i-vectors

I Early fusion

GTM-UVigo System for MultimodalPerson Discovery in Broadcast TV Taskat MediaEval 2016