introduction (motivation) · gps information etc.) visual + metadata. 4 benchmark datasets:...
TRANSCRIPT
1
Introduction(Motivation)
In2015,atotalof728millionsofpublicpictureswereuploadedtoFlickr
Suchlargeamountof user-generateddatamakesmultimediaindexingandretrievalamorechallengingtask
However,italsoopensnewopportunitiesfordevelopmentofnovelandmoreefficienttools
2
Introduction(Motivation)User-generated multimedia contents depictindividual experiences or collective activities
WhatisanEvent?
Arealworldhappening toWho?,What?,When?andWhere?
Aneventisplannedbypeopleattendedbypeopleandrelatedmediaarealsocapturedbypeople
Personalexperiences
Collectiveactivities
3
EventDetectioninImages:State-of-the-art
VisualInformation
Metadata(tags,GPSinformation
etc.)
Visual+Metadata
4
BenchmarkDatasets:State-of-the-art
Currentdatasetsfor
eventdetectioninimages
lownumberofimages(e.g.,EIMM[1],Cultural
eventrecognitiondatabase[3])
limitedvarietyofevents/eventclasses(e.g.,EiMM [2]andSED2013
database[2])
Unbalancedeventclasses(e.g., EiMM [1]andSED2013[2])
1. R.Mattivi etal..Exploitationoftimeconstraintsfor(sub-)eventrecognition.InProceedingsofthe2011jointACMworkshoponModelingandrepresentingevents,pages7(12).ACM,2011..
2. T.Reuteretal..Socialeventdetectionatmediaeval2013:Challenges,datasets,andevaluation.InMediaEval Workshop,2013..3. S.Escalera etal..ChaLearn LookingatPeople2015:ApparentAgeandCulturalEventRecognitionDatasetsandResults,ICCV2015
5
USED:AlargeScaleSocialEventDetectionDatasetAlargecollectionofimages
Covers14differenteventsclasses
AbalanceddatasetEqualnumberofimagesineachclass(35,000)
Event-classesinUSEDDataset
6
USED:AlargeScaleSocialEventDetectionDataset
DiversityincontentsIndoorVs.outdoorGrouppicturesVs.SingleportraitImagesofkey-momentsinaneventMulti-culturalOutliersandborderlinecasesaremanuallyremoved
Somesampleimagesfromweddingclass
7
USED:AlargeScaleSocialEventDetectionDataset
USED490,000 Eventrelated
imagesdepictinga widevarietyof
events
8
Comparisonswithstate-of-the-artdatasets
ExistingdatasetsforEventDetectionCulturalEventDetectionDatasetEiMMSED
DatasetName #Event-classes Total Images Minimagesinaclass
Max.images inaclass
EiMM 8 (socialevents) 13219 795 2253
SED 7 82213 342 71556
CulturalEvents 50 11776 180-200(Avg.) 180-200(Avg.)
USED 14 490000 35000 35000
Comparisons ofUSEDwithotherDatasets
9
ExperimentalValidationofUSED
DISCOVERINGEVENTSFROMSINGLEPICTURESUSINGACONVOLUTIONALNEURALNETWORK
10
Validation/ExperimentalSetup
Fine-tuningCNN
Classification
Pre-training
ParametersofaCNN(Alexnet)pre-trainedonImageNet dataset
[NIPS2012]
Fine-tunedonnewlycollecteddatasets
Reduced overalllearningrateIncreasedlearningrateof
newlayerMomentum=.9
WeightDecay=.0005
11
PreliminaryResultsDataset
USED
Event Type Accuracy EventType Accuracy
Concert 74.20% Conference 75.70%
Graduation 66.43% Exhibition 58.54%
Meeting 78.70% Fashion 65.43%
MountainTrip 67.00% Protest 74.58%
Picnic 54.42% Sports 72.24%
Sea-holiday 74.24% Theater 51.90%
Ski-holiday 48.00%
Wedding 51.00%
ResultsonUSEDdataset
DataAssemblageTrainingset=20,000imagesperclassValidationset=7000perclassTestset=7000imagesperclass
12
ComparisonsofaCNNtrainedonUSEDwithBaselineApproaches
ComparisonwithRosani etal.,[IEEETMM2015]
EiMMDataset SEDDatasetOurApproach 71.54 59.42BaselineApproach 38.8 31.15
0
10
20
30
40
50
60
70
80Ac
curacy(%
)
A.Rosani,G.Baoto,F.G.B.DeNatale,“EventMask:agame-basedframeworkforEvent-saliencyidentificationinImages”,IEEETransactionsonMultimedia2015
13
USED:ALarge-scaleSocialEventDetectionDataset
490,000 Event-relatedimages, 14differentevent-classes,35,000imagesper
class
ENJOY USED!