![Page 1: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/1.jpg)
CAP6412AdvancedComputerVision
http://www.cs.ucf.edu/~bgong/CAP6412.html
Boqing GongJan26,2016
![Page 2: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/2.jpg)
Today
• Administrivia• Abiggerpictureandsomecommonquestions• Objectdetectionproposals,bySamer
![Page 3: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/3.jpg)
Pastdue(12pmtoday)
• Assignment2:Reviewthefollowingpaper
{Major}[DetectionProposals]J.Hosang,R.Benenson,P.Dollár,andB.Schiele.Whatmakesforeffectivedetectionproposals?PAMI2015.
Templateforpaperreview:http://www.cs.ucf.edu/~bgong/CAP6412/Review.docx
![Page 4: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/4.jpg)
Anassignmentwithnoduedates
• See“PaperPresentation”onUCFwebcourse
• Sharingyourslides• Refertotheoriginalssourcesofimages,figures,etc.inyourslides• ConvertthemtoaPDFfile• UploadthePDFfileto“PaperPresentation”afteryourpresentation
![Page 5: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/5.jpg)
ScheduleupdateWeek2 CNNvisualization&objectrecognition
Week3 CNN&objectlocalization
Week4 CNN&transferlearning
Week5 CNN &segmentation,super-resolution
Week6 CNN&videos(opticalflow,pose)
Week7 Imagecaptioning&attentionmodel
Week8 Visualquestionanswering
Week9 Attentionmodel,aligningbookswithmovies
Week10--16 Video:tracking,action,surveillanceHuman-centered CV3DCVLow-levelCV,etc.
![Page 6: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/6.jpg)
Nextweek:Imagecaptioning&attentionmodel
Tuesday(02/02)
Harish RaviPrakash
Karpathy, Andrej, and Li Fei-Fei. “Deep visual-semantic alignments forgenerating image descriptions.” arXiv preprint arXiv:1412.2306(2014).
& Secondary papersThursday(02/04)
Karan Daei-Mojdehi
Xu, Kelvin, Jimmy Ba, Ryan Kiros, Aaron Courville, RuslanSalakhutdinov, Richard Zemel, and Yoshua Bengio. “Show, attend andtell: Neural image caption generation with visual attention.” arXivpreprint arXiv:1502.03044 (2015).
& Secondary papers
![Page 7: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/7.jpg)
Beginningnextclass
• Makegoodpresentations--- #3courseobjective- Title,authors(fullname),authors’institutes,yournameandemail- Motivationoftheresearch(1—2slides)- Problemstatement(1—2slides)- Maincontributionsofthepaper- Approachoutline(1slide)- Detailsoftheproposedapproach- Experiments- Relatedwork(1—3slides)- Conclusion:take-homemessage(1—2slides)- Strengths&weaknessesofthepaper(1—2slides)- Overallrating&why(howyouweighthestrengthsandweaknesses)(1slide)- Futuredirections(1—3slides)
![Page 8: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/8.jpg)
Beginningnextclass
• Makegoodpresentations--- #3courseobjective- Title,authors(fullname),authors’institutes,yournameandemail- Motivationoftheresearch(1—2slides)- Problemstatement(1—2slides)- Maincontributionsofthepaper- Approachoutline(1slide)- Detailsoftheproposedapproach- Experiments- Relatedwork(1—3slides)- Conclusion:take-homemessage(1—2slides)- Strengths&weaknessesofthepaper(1—2slides)- Overallrating&why(howyouweighthestrengthsandweaknesses)(1slide)- Futuredirections(1—3slides)
40minsonlyLeavemetimetocover:• Underexploitedpointsinslides/discussion• Techniquedetails• Morerelatedworkandreadingreferences• Myowncomments
![Page 9: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/9.jpg)
Today
• Administrivia• Abiggerpictureandsomecommonquestions• Objectdetectionproposals,bySamer
![Page 10: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/10.jpg)
Whywereadthesepapers: Apersonalizedandbiasedperspective
![Page 11: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/11.jpg)
Whywereadthesepapers: Apersonalizedandbiasedperspective
Time Event RelatedPapers Read?
01/2012 NegativeCVPRreviews
[LeNet]YannLeCun,L.Bottou,Y.Bengio,andP.Haffner.Gradient-based learningapplied todocument recognition.ProceedingsoftheIEEE,november 1998.
Yes
![Page 12: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/12.jpg)
Whywereadthesepapers: Apersonalizedandbiasedperspective
Time Event RelatedPapers Read?
01/2012 NegativeCVPRreviews
[LeNet]YannLeCun,L.Bottou,Y.Bengio,andP.Haffner.Gradient-based learningapplied todocument recognition.ProceedingsoftheIEEE,november 1998.
Yes
10/2012 AlexNet winsILSVRC2012
[AlexNet] Krizhevsky,Alex,IlyaSutskever,andGeoffreyE.Hinton. “Imagenet classificationwithdeepconvolutionalneuralnetworks.”InNIPS,2012.
Yes
![Page 13: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/13.jpg)
Whywereadthesepapers: Apersonalizedandbiasedperspective
Time Event RelatedPapers Read?
01/2012 NegativeCVPRreviews
[LeNet]YannLeCun,L.Bottou,Y.Bengio,andP.Haffner.Gradient-based learningapplied todocument recognition.ProceedingsoftheIEEE,november 1998.
Yes
10/2012 AlexNet winsILSVRC2012
[AlexNet] Krizhevsky,Alex,IlyaSutskever,andGeoffreyE.Hinton. “Imagenet classificationwithdeepconvolutionalneuralnetworks.”InNIPS,2012.
Yes
11/2013 Visualize&understandCNNs
[Visualization] Zeiler,MatthewD.,andRobFergus.“Visualizingandunderstanding convolutionalnetworks.”InECCV,2014.
Yes
![Page 14: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/14.jpg)
Whywereadthesepapers: Apersonalizedandbiasedperspective
Time Event RelatedPapers Read?
01/2012 NegativeCVPRreviews
[LeNet]YannLeCun,L.Bottou,Y.Bengio,andP.Haffner.Gradient-based learningapplied todocument recognition.ProceedingsoftheIEEE,november 1998.
Yes
10/2012 AlexNet winsILSVRC2012
[AlexNet] Krizhevsky,Alex,IlyaSutskever,andGeoffreyE.Hinton. “Imagenet classificationwithdeepconvolutionalneuralnetworks.”InNIPS,2012.
Yes
11/2013 Visualize&understandCNNs
[Visualization] Zeiler,MatthewD.,andRobFergus.“Visualizingandunderstanding convolutionalnetworks.”InECCV,2014.
Yes
2014 CNN winsonobjectdetection
Girshick,Ross,JeffDonahue, TrevorDarrell,andJagannathMalik."Richfeaturehierarchiesforaccurateobjectdetectionandsemanticsegmentation."InCVPR, 2014.
ThisThursday
![Page 15: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/15.jpg)
Basicnetworkstructures--- whereisCNN?
• Feed-forwardnetworks • Recurrentneuralnetworks
Imagecredit:http://mesin-belajar.blogspot.com/2016/01/a-brief-history-of-neural-nets-and-deep_84.html
![Page 16: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/16.jpg)
CNN:aspecialformoffeed-forwardnetworks
• Seewhiteboard
![Page 17: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/17.jpg)
Detour:WeightsharinginCNN
Convolutionlayer
Neuronsofthesamefeaturemapsharethesameweights(thefilter)
Significantlyreduced#parameters
Imagecredit:deeplearning.net/tutorial/lenet.html
![Page 18: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/18.jpg)
Detour:SparseconnectioninCNN
TheLeNet [LeCun etal.’1998]
Sparseconnectionsvs. FullconnectionSmaller#parameters,betterlearningefficiency
![Page 19: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/19.jpg)
Today
• Administrivia• Abiggerpictureandsomecommonquestions• Objectdetectionproposals,bySamer
![Page 20: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/20.jpg)
Whatmakesforeffectivedetectionproposals?
JanHosang1,RodrigoBeneson1,PiotrDollar2,andBernt Schiele1
1MaxPlanckInstituteforInformatics2FacebookAIResearch(FAIR)
Presentedby:Samer Iskander
![Page 21: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/21.jpg)
Motivation• Highperformingobjectdetectorsarebasedonobjectproposals,inordertoavoidexhaustiveslidingwindowsearchacrosstheimage.
![Page 22: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/22.jpg)
• Asaresultofthat,anin-depthanalysisofdifferentmethodsisrequired,inordertostudytheirimpactondetectionperformance.
![Page 23: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/23.jpg)
ProblemStatement
• Althoughthewidespreaduseofdetectionproposals,itisnecessarytostudytheperformancemetricstrade-offswhenemployingthem.
![Page 24: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/24.jpg)
MainContributions• Asystematicoverviewofdetectionproposalmethodsisprovided.
• Thenotionofproposalrepeatabilityisintroduced.• Objectrecallmetricisstudiedondifferentdatasets.• Theinfluenceofdifferentproposalmethodswhenappliedonselectedobjectsdetectionalgorithms(DPM,R-CNNandFastR-CNN).
• Anovelmetric,theaveragerecall(AR),whichrewardsbothproposallocalizationandrecallperformancemetricsandeffectsthedetectionperformanceisproposed.
![Page 25: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/25.jpg)
ApproachOutline1.DetectionProposalMethods1.1BaselineProposalMethod
2.EvaluationMetricsforObjectProposals3.ProposalRepeatability4.ProposalRecall5.UsingTheDetectionProposals5.1DetectorResponsesAroundObjects5.2LM-LLDA,R-CNNandFastR-CNNdetection
performance5.3Predictingdetectionperformance
![Page 26: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/26.jpg)
1.DetectionProposalMethods
DetailsofTheProposedApproach
![Page 27: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/27.jpg)
DetectionProposalMethods
Grouping ProposalMethods
• Theyattempttogeneratesegments(maybeoverlapped) thatarelikely tocorrespond toobjects
WindowScoringMethods
• Theyscoreeachcandidatewindowaccordingtohowlikelyitistocontainanobject.
• Itisfaster.• Ifnotgeneratesdensely
windows, lowlocalizationaccuracy
![Page 28: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/28.jpg)
1.1BaselineProposalMethodA.Uniform:Togenerateproposals,itisnecessarytouniformlysampletheboundingboxcenterposition(x,y),squarerootareaandlogaspectratio.
ThePASCALVOC2007trainingsetisusedtoestimatetheseparameters.
![Page 29: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/29.jpg)
B.Gaussian:Togenerateproposals,itisnecessarytomultivariateGaussiandistributiontheboundingboxcenterposition(x,y),squarerootareaandlogaspectratio.
![Page 30: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/30.jpg)
C.SlidingWindow:Equallydistributedwindowsinspacearegenerated.BING(Binarized NormedGradientsforObjectness Estimationat300fps)uses29specificsizes,thismethodspreadthissizeshomogeneouslyinsidetheimage.
![Page 31: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/31.jpg)
D.Superpixels:Superpixels aregeneratedfromEfficientGraph-BasedImageSegmentation.
![Page 32: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/32.jpg)
2.EvaluationMetricsforObjectProposals
![Page 33: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/33.jpg)
1.IntersectionOverUnion(IOU):• Themetricsusedforevaluatingobjectproposalsarealltypicallyfunctionsofintersectionoverunion(IOU)betweengeneratedproposalsandground-truthannotations.
• Fortwoboxes/regionsbi andbj ,IOUisdefinedas:
𝐼𝑂𝑈 𝑏%, 𝑏' =𝑎𝑟𝑒𝑎 𝑏% ∩ 𝑏'𝑎𝑟𝑒𝑎 𝑏% ∪ 𝑏'
![Page 34: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/34.jpg)
2.Recall@IOUThresholdt:• Foreachground-truthinstance,checkwhetherthebestproposalfromlistLhasIOU>t.
• Ifso,thisground-truthinstanceisconsidereddetectedorrecalled.
• Thenaveragerecallismeasuredoveralltheground-truthinstances.
𝑟𝑒𝑐𝑎𝑙𝑙@𝑡 =1|𝐺| 5 𝐼 max
9:∈<𝐼𝑂𝑈 𝑔%, 𝑙% > 𝑡
?:∈@
I[.]isanindicatorfunctionforlogicalprepositionintheargument
![Page 35: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/35.jpg)
• Objectproposalsareevaluatedusingthismetricintwoways:1.Plottingrecallvs.tbyfixing#proposalsinL.
2.Plottingrecallvs.#proposalsbyfixingt.
![Page 36: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/36.jpg)
3.AverageBestOverlap(ABO):Thismetriceliminatestheneedforthethreshold.Calculatetheoverlapbetweeneachground-truthannotationgiϵGandthebestobjecthypothesisinL.
𝐴𝐵𝑂 =1|𝐺| 5 max
9:∈<𝐼𝑂𝑈 𝑔%, 𝑙%
?:∈@
4.AverageRecall(AR):
𝐴𝐵𝑂 = D|@|∑ max
9:∈<(𝐼𝑂𝑈 𝑔%, 𝑙%?:∈@ -0.5,0)
Averagerecall(forIOUbetween0.5:1)vs.#proposals
![Page 37: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/37.jpg)
5.VolumeUnderSurface(VUS):Itplotsrecallasafunctionofbothtand#proposalsandcomputesthevolumeunderthesurface.
![Page 38: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/38.jpg)
3.ProposalRepeatability
![Page 39: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/39.jpg)
1.ForeachimageinthePASCALVOC2007testset,severalperturbedversionsaregenerated(blur,rotation,scale,illumination,JPEGcompression,and“saltandpepper”noise).
![Page 40: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/40.jpg)
2.Foreachpairofreferenceandperturbedimages,detectionproposalsarecomputedwithagivenmethod(generating1000windowsperimage).3.Theproposalsareprojectedbackfromtheperturbedintothereferenceimageandthenmatchedtotheproposalsinthereferenceimage.4.Then,plotrecallvs.IOUt(0:1),andrepeatabilityistheareaunderthecurve.5.MethodsthatproposewindowsatsimilarlocationsathighIoU—andthusonsimilarimagecontent—aremorerepeatable,sincetheareaunderthecurveislarger.6.Largewindowsaremorelikelytomatchthansmalleronessincethesameperturbationwillhavealargerrelativeeffectonsmallerwindows.
![Page 41: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/41.jpg)
![Page 42: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/42.jpg)
• Scale:AllmethodsexceptBingshowadrasticdropwithsmallscalechanges,butsufferonlyminordegradationforlargerchanges.Bingismorerobusttosmallscalechanges;however,itismoresensitivetolargerchangesduetoitsuseofacoarsesetofboxsizeswhilesearchingforcandidates.
![Page 43: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/43.jpg)
• JPEGCompression:Smallcompressionhasalargeeffectandmoreaggressivecompressionshowsmonotonicdegradation.Despiteusinggradientinformation,Bingismostrobusttothesekindofchanges.
![Page 44: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/44.jpg)
• Rotation:Allproposalmethodsareaffectedbyimagerotation.Therepeatabilitylossisduetomatchingrotatedboundingboxes.
![Page 45: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/45.jpg)
• Illumination:Methodsbasedonsuperpixels areheavilyaffected.Bingismorerobust,likelyduetouseofgradientinformationwhichisknowntobefairlyrobusttoilluminationchanges.
![Page 46: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/46.jpg)
• Blur:Therepeatabilityresultsagainexhibitasimilartrendalthoughthedropisstronger(incomparisonwithothereffects)forasmall.
![Page 47: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/47.jpg)
• Saltandpeppernoise:Significantdegradationinrepeatabilityforthemajorityofthemethodsoccurswhenmerelytenpixelsaremodified.
![Page 48: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/48.jpg)
4.ProposalRecall
• Ifrepeatabilityisaconcern,theproposalmethodshouldbeselectedwithcare.
• Forobjectdetection,anotheraspectofinterestisrecall.
![Page 49: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/49.jpg)
Dataset Description
1. PASCAL Itincludes20objectcategoriesthatarepresentedinnearly5000unconstrained images.
2.ImageNet InlargerImageNet2013,thereare200categoriesinover20,000images.
Differenttypesofobjectsareincluded thatarenotinPASCAL.
ImageNet andPASCALhavethesamenumber ofobjects/imageandsizeofobjects.
3.MSCOCO MicrosoftCommonobjectsinContext(MSCOCO)hasmoreobjects/image,smallerobjects,butfewerobjectclasses(80objectcategories).
![Page 50: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/50.jpg)
Overall,themethodsfallintotwogroups:1.WelllocalizedmethodsthatgraduallyloserecallastheIoU thresholdincreases.2.Methodsthatonlyprovidecoarseboundingboxlocations,sotheirrecalldropsrapidly.
![Page 51: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/51.jpg)
![Page 52: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/52.jpg)
![Page 53: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/53.jpg)
![Page 54: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/54.jpg)
5.UsingTheDetectionProposals
• Thisisananalysisofdetectionproposalstobeusedwithobjectdetection.
• Themain2goals:1. Measuringtheperformanceofproposal
methodsforobjectdetection.2. Theeffectofobjectproposalsmetriconfinal
detectionperformance.
![Page 55: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/55.jpg)
5.1DetectorResponsesAroundObjects
• Itisnecessarytochecktheimportanceandrelationshipbetweenwelllocalizedproposals(highIOU)andobjectdetection(recall).
![Page 56: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/56.jpg)
5.2LM-LLDA,R-CNNandFastR-CNNdetectionperformance
1. ApplyLM-LLDAmodelstogeneratedensedetectionsusingthestandardslidingwindow.
2. Applydifferentobjectproposalstofilterthesedetectionsattesttime.
*Thesestepsareusedtoevaluatetheeffectofproposalsondetectionquality.
![Page 57: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/57.jpg)
• Usingonly1000proposals,thedetectionqualityisreduced.
• But,methodswithhighaveragerecall(AR)alsohavehighmeanaverageprecision(mAP),andviceversa.
![Page 58: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/58.jpg)
• Fromtablebelow:(1)clearlyhurtperformance(bicycle,boat,bottle,car,chair,horse,mbike,person),reducingtherecallandprecisionbecauseofbadlocalization.(2)improveperformance(cat,table,dog),(3)donotshowsignificantchange(allremainingclasses).
![Page 59: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/59.jpg)
• FastR-CNNafterre-trainingforeachmethod.• Intherightmostcolumn,FastR-CNNtrainedwith1000SelectiveSearch proposalsandappliedattesttimewithagivenproposalmethod,versusFastR-CNNtrainedforthetesttimeproposalmethod.
![Page 60: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/60.jpg)
5.3Predictingdetectionperformance
![Page 61: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/61.jpg)
RelatedWork:
FasterR-CNN:TowardsReal-TimeObjectDetectionwithRegionProposalNetworks
Shaoqing Ren1,Kaiming He2,RossGirshick,andJianSun3
1UniversityofScienceandTechnologyofChina2MicrosoftResearch
3FacebookAIResearch
![Page 62: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/62.jpg)
• Thisobjectdetectionsystemiscomposedoftwomodules.Thefirstmoduleisadeepfullyconvolutionalnetworkthatproposesregions,andthesecondmoduleistheFastR-CNNdetectorthatusestheproposedregions.
• TheRPNmoduletellstheFastR-CNNmodulewheretolook.
• ARegionProposalNetwork(RPN)takesanimage(ofanysize)asinputandoutputsasetofrectangularobjectproposals,eachwithanobjectness score.
![Page 63: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/63.jpg)
![Page 64: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/64.jpg)
• Forregionproposalsgeneration,slideasmallnetworkovertheconvolutionalfeaturemapoutputbythelastsharedconvolutionallayer.
• Thissmallnetworktakesasinputannxn spatialwindowoftheinputconvolutionalfeaturemap.
• Eachslidingwindowismappedtoalower-dimensionalfeature(256-dforZFand512-dforVGG,withReLU following).
• Thisfeatureisfedintotwosiblingfullyconnectedlayers—abox-regressionlayer(reg)andabox-classificationlayer(cls).
![Page 65: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/65.jpg)
![Page 66: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/66.jpg)
Conclusion• Thispaperrevisitsthemajorityofexistingdetectionproposalmethods,proposednewevaluationmetrics,andperformedanextensiveanddirectcomparisonofexistingmethods.
• Therepeatabilityofallproposalmethodsislimited:smallchangestoanimagecauseanoticeablechangeinthesetofproducedproposals.
• Forobjectdetection,improvingproposallocalizationaccuracy(improvedIoU)isasimportantasimprovingrecall.
• Tosimultaneouslymeasurebothproposalrecallandlocalizationaccuracy,averagerecall(AR)summarizesthedistributionofrecallacrossarangeofoverlapthresholds.
![Page 67: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/67.jpg)
Strengths
• Thispaperprovidesanewmetric,AverageRecall(AC),thatrelatesbetweenaccuracy(recall)andgoodlocalization(IOU).
• Itdemonstratesdifferentevaluationprotocoltocomparebetweenproposalmethods(repeatability,recallandusingproposalmethodsforobjectdetection).
![Page 68: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/68.jpg)
Weaknesses
• Thispaperdependsonlyon12proposalmethods,becausetheirimplementationsareavailable.
• Thebaselineproposalmethodsarenotalgorithms(uniform,Gaussian,slidingwindowandsuperpixels).
![Page 69: CAP 6412 Advanced Computer Vision - UCF CRCV · Week 4 CNN& transfer learning Week 5 CNN& segmentation, super-resolution Week 6 CNN & videos (optical flow, pose) Week 7 Image captioning](https://reader033.vdocuments.net/reader033/viewer/2022042213/5eb662bdc57dec6f316eb178/html5/thumbnails/69.jpg)
OverallRating• MyRatingScale(0-5):1ThenewperformancemetricwhichisAverageRecall(AC)isjustanAverageBestOverlap(ABO)withinrange0.5:1
Comparisonistakenplacebetween12proposalmethodsonly.