selecting user-generated videos for augmented reality ...hto/resources/bigmm2016.pdf · 1 selecting...
TRANSCRIPT
1
SelectingUser-GeneratedVideosforAugmentedRealityApplications
IEEEBigMM 2016
HienTo,Hyerim Park,Seon HoKim,CyrusShahabi.
IntegratedMediaSystemsCenterUniversityofSouthernCalifornia
2
Motivationü Popularityofaugmented-reality(AR)
ü Scarcity ofARcontent
ü Lackofusercontextscausesimpreciseregistration
DifficulttocreatecontentinARbrowsersAdoptmultimediacontentfromsocialmedia
ARbrowsers,e.g.,Layar,Wikitude,andJunaio
Needcontentsourcewithrichcontextinformation
3
Motivation(Cont…)Mobilecamerasareeverywhere!
AnywaytousemobilevideosasARcontent?
4
Motivation(Cont…)
• Currentmostimportantkeywordsinvideomarketareasfollows:(byKleiner PerkinsCaufield &Byers (KPCB), 2015)–User-generated– Tagged– Curated– Indexed&Searchable
5
Challenges
Knownissuesofmobilevideos
1.OutofControl 2.HardtoSearch 3.HardlySystematic
HardtosearchforinterestingvideosforARcontent
6
RelatedWorkü Content-basedretrieval• Extensivecomputationrequired,stillinaccurate
ü Spatialmetadata(cameralocation,orientation)• Geo-taggingrequired,sensorsonmobiledevice
𝑁𝑑
pR𝜃
p:cameralocationd:cameradirection𝜃:viewableangleR:viewabledistancet:timestamp
FieldOfView(FOV)
MediaQ:mobilemediamanagementsystemusingspatialmetadatafromsensors(http://mediaq.usc.edu)
7
Geo-taggedMobileVideos
üSelectingvideosforacertainareaisnowfast!
8
ProblemStatementüSimplegeospatialrelevanceisnotenough.Forexample,selecting“interesting”videosforARapplicationsfrommanyatalocation?
üIdentifyinteresting(orsignificant)videostoARusersasasequenceofFOVsthatfollowaparticularcamerashootingpattern usedinfilmingsuchastracking,zooming,panning,andarchingscenes
9
ModelingSignificantVideoSegmentsSingle Multiple
SingleZoom Pan
MultipleTrack Arch
PositionDirection
1d2d
3d
4d5d
1d2d 3d 4d
5dθ
1d
2d3d
4d
5d1d
2d3d
4d 5d
3θ4θ 5θ
0
< 𝑡'(), 𝑑'+, >
< 𝑡'(), 𝑑'+, , 𝑟'+, > < 𝑡'(), 𝑑'(), 𝑟'+, >
< 𝑡'(), 𝑑'+, >
10
Statistics ValueNumberofvideoswithgeo-metadata 2,397Averagelengthper videowithcontent(sec) 72.14Averagecameramovingspeed(km/h) 4.5Averagecamerarotationspeed(degrees/sec) 10Numberofusers 289Numberofvideos byeachuser 8.29NumberofFOVs 208,978NumberofFOVpersecond 1.03NumberofFOVpervideo 74.16
[ACMMMSys 2016DatasetPaper,USCMediaQMobileVideoData]
ExperimentUsingRealDatasets
11
SignificantVideoSegments
Numberofscenesfound Searchingtime(onevideo)
0
500
1000
1500
2000
Track Arch Zoom Pan
Num
bero
fscenes
051015202530
Track Arch Zoom Pan
Runtime(m
s)
𝑡'() = 15, 𝑑'+, = 15, 𝑑'() = 120
Trackingisthemostpopularwayofcapturingmobilevideos
Overallsearchtimeisfast
12
ZoomingScene
TrackingScene
PanningScene
ArchingScene
13
DetectingHotSpots
Hotspotscoveredbymanyvideoframes
0
200
400
600
800
1000
100 90 80 70 60 50 40 30 20 10Totalnum
berofhotspots
MinimumFOVsperhotspot
14
ConclusionandFutureWork
üProposedmodelsandalgorithmstoidentifyinterestingvideosegmentsandhotspots
üExperimentsonrealgeo-taggedvideodatashowthatthealgorithmsarefastandabletoidentifyinterestingscenes
üWillconsiderusermobilityandrankingofvideosegments