week 8: web-assisted object detection alejandro torroella & amir r. zamir
TRANSCRIPT
![Page 1: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/1.jpg)
WEEK 8:
WEB-ASSISTED OBJECT
DETECTION
A L E J A N D R O TO R R O E L L A &
AM
I R R . ZA M
I R
![Page 2: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/2.jpg)
GEOMETRY METHOD PROCEDURE
For each query image we manually set orientation, angle of view, range of view, and location of camera.
: Camera location
: Object locations
: Field of vision
![Page 3: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/3.jpg)
GEOMETRY METHOD PROCEDUREUsing the obtained FOV, select only the objects that are within the
FOV
Calculate the degrees from the left limit of the FOV and store in a vector specific to the object’s class These vectors will be
our “true” layout of objects.
.
.
.
![Page 4: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/4.jpg)
GEOMETRY METHOD PROCEDUREWe then run our DPM detectors for the classes in question on the
query image.
Below are results for Street Lights (green) and Traffic Signals (red).
![Page 5: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/5.jpg)
GEOMETRY METHOD PROCEDUREWe sift through the detections that completely disagree with the
“true” GIS layout.
![Page 6: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/6.jpg)
GEOMETRY METHOD PROCEDUREWe then sift through the detections again by size of bounding boxes
(too large or too small)
![Page 7: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/7.jpg)
GEOMETRY METHOD PROCEDUREUsing the sifted bounding boxes we generate all possible
combinations (no repeats, order doesn’t matter) of possible layouts.
For each class: Out of the N detections, choose k of them for the possible layout. Where
For each layout combination we calculate the “cost” of it compared to the obtained “true” GIS layout and keep track of the combination that returned the minimum
Two cost functions we’ve tested:
Absolute value:
Standard deviation:
![Page 8: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/8.jpg)
GEOMETRY METHOD PROCEDUREOnce we’ve traversed through all the possible combinations, we
display the detections that resulted in the minimum of the cost function.
𝑆𝑇𝐷
𝐴𝐵𝑆
![Page 9: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/9.jpg)
GEOMETRY METHOD RESULTS: BEFORE
𝑆𝑇𝐷
Traffic SignalsTrash CansTraffic Signs
![Page 10: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/10.jpg)
GEOMETRY METHOD RESULTS: AFTER GIS SIFT
𝑆𝑇𝐷
Traffic SignalsTrash CansTraffic Signs
![Page 11: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/11.jpg)
GEOMETRY METHOD RESULTS: AFTER SIZE SIFT
𝑆𝑇𝐷
Traffic SignalsTrash CansTraffic Signs
![Page 12: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/12.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (ABS)
𝑆𝑇𝐷
Traffic SignalsTrash CansTraffic Signs
![Page 13: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/13.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (STD)
𝑆𝑇𝐷
Traffic SignalsTrash CansTraffic Signs
![Page 14: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/14.jpg)
GEOMETRY METHOD RESULTS: BEFORE
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 15: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/15.jpg)
GEOMETRY METHOD RESULTS: AFTER GIS SIFT
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 16: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/16.jpg)
GEOMETRY METHOD RESULTS: AFTER SIZE SIFT
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 17: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/17.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (ABS)
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 18: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/18.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (STD)
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 19: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/19.jpg)
GEOMETRY METHOD RESULTS: BEFORE
𝑆𝑇𝐷
Street LightsTraffic Signs
![Page 20: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/20.jpg)
GEOMETRY METHOD RESULTS: AFTER GIS SIFT
𝑆𝑇𝐷
Street LightsTraffic Signs
![Page 21: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/21.jpg)
GEOMETRY METHOD RESULTS: AFTER SIZE SIFT
𝑆𝑇𝐷
Street LightsTraffic Signs
![Page 22: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/22.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (ABS)
𝑆𝑇𝐷
Street LightsTraffic Signs
![Page 23: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/23.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (STD)
𝑆𝑇𝐷
Street LightsTraffic Signs
![Page 24: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/24.jpg)
GEOMETRY METHOD RESULTS: BEFORE
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 25: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/25.jpg)
GEOMETRY METHOD RESULTS: AFTER GIS SIFT
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 26: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/26.jpg)
GEOMETRY METHOD RESULTS: AFTER SIZE SIFT
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 27: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/27.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (ABS)
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 28: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/28.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (STD)
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 29: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/29.jpg)
GEOMETRY METHOD RESULTS: BEFORE
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 30: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/30.jpg)
GEOMETRY METHOD RESULTS: AFTER GIS SIFT
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 31: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/31.jpg)
GEOMETRY METHOD RESULTS: AFTER SIZE SIFT
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 32: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/32.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (ABS)
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 33: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/33.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (STD)
𝑆𝑇𝐷
Traffic SignalsStreet Lights
![Page 34: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/34.jpg)
GEOMETRY METHOD RESULTS: BEFORE
𝑆𝑇𝐷
Fire HydrantsStreet Lights
![Page 35: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/35.jpg)
GEOMETRY METHOD RESULTS: AFTER GIS SIFT
𝑆𝑇𝐷
Fire HydrantsStreet Lights
![Page 36: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/36.jpg)
GEOMETRY METHOD RESULTS: AFTER SIZE SIFT
𝑆𝑇𝐷
Fire HydrantsStreet Lights
![Page 37: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/37.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (ABS)
𝑆𝑇𝐷
Fire HydrantsStreet Lights
![Page 38: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/38.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (STD)
𝑆𝑇𝐷
Fire HydrantsStreet Lights
![Page 39: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/39.jpg)
GEOMETRY METHOD RESULTS: BEFORE
𝑆𝑇𝐷
Street LightsTraffic Signs
![Page 40: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/40.jpg)
GEOMETRY METHOD RESULTS: AFTER GIS SIFT
𝑆𝑇𝐷
Street LightsTraffic Signs
![Page 41: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/41.jpg)
GEOMETRY METHOD RESULTS: AFTER SIZE SIFT
𝑆𝑇𝐷
Street LightsTraffic Signs
![Page 42: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/42.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (ABS)
𝑆𝑇𝐷
Street LightsTraffic Signs
![Page 43: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/43.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (STD)
𝑆𝑇𝐷
Street LightsTraffic Signs
![Page 44: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/44.jpg)
GEOMETRY METHOD RESULTS: BEFORE
𝑆𝑇𝐷
Traffic SignalsTraffic Signs
![Page 45: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/45.jpg)
GEOMETRY METHOD RESULTS: AFTER GIS SIFT
𝑆𝑇𝐷
Traffic SignalsTraffic Signs
![Page 46: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/46.jpg)
GEOMETRY METHOD RESULTS: AFTER SIZE SIFT
𝑆𝑇𝐷
Traffic SignalsTraffic Signs
![Page 47: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/47.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (ABS)
𝑆𝑇𝐷
Traffic SignalsTraffic Signs
![Page 48: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/48.jpg)
GEOMETRY METHOD RESULTS: AFTER FUSION (STD)
𝑆𝑇𝐷
Traffic SignalsTraffic Signs
![Page 49: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/49.jpg)
GEOMETRY METHOD CONCLUSIONS
• Using the Standard deviation cost function resulted in better results compared to the Absolute value function on average.• A more advanced cost function would probably result in
even better results• Results look promising considering we haven’t implemented a
robust sensor model for creating the “true” GIS layout
![Page 50: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/50.jpg)
GOALS FOR NEXT WEEK
• Implement a robust sensor model• Look into more advanced cost functions• Instead of crudely sifting through the bounding boxes by size
using a threshold based on the size of the image, use the distances of the objects from the camera to estimate how large the bounding box should be.
![Page 51: WEEK 8: WEB-ASSISTED OBJECT DETECTION ALEJANDRO TORROELLA & AMIR R. ZAMIR](https://reader033.vdocuments.net/reader033/viewer/2022051316/56649e9a5503460f94b9ceb8/html5/thumbnails/51.jpg)
THANK YOUFIN.