![Page 1: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/1.jpg)
Robust Object Tracking by Hierarchical Association ofDetection ResponsesPresent by fakewen
![Page 2: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/2.jpg)
outline•Introduction•Hierarchical Association of Detection
Responses▫Low-Level Association▫Middle-Level Association▫High-Level Association
•Experimental Results•conclusion
![Page 3: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/3.jpg)
•a detection-based three-level hierarchical association approach to robustly track multiple objects in crowded environments from a single camera.
![Page 4: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/4.jpg)
•low level, reliable tracklets.•middle level, these tracklets are further
associated to form longer tracklets based on more complex affinity measures.
•high level, entries, exits and scene occluders are estimated using the already computed tracklets, which are used to refine the final trajectories.
![Page 5: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/5.jpg)
![Page 6: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/6.jpg)
![Page 7: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/7.jpg)
outline•Introduction•Hierarchical Association of Detection
Responses▫Low-Level Association▫Middle-Level Association▫High-Level Association
•Experimental Results•conclusion
![Page 8: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/8.jpg)
notation• detection response• position • size• occurrence frame index• color histogram
![Page 9: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/9.jpg)
notation• object
trajectory/tracklet• object trajectory/tracklet
set• association results of the
low level, the middle level and the high level respectively
![Page 10: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/10.jpg)
outline•Introduction•Hierarchical Association of Detection
Responses▫Low-Level Association▫Middle-Level Association▫High-Level Association
•Experimental Results•conclusion
![Page 11: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/11.jpg)
Low-Level Association• : set of all detection responses•
![Page 12: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/12.jpg)
outline•Introduction•Hierarchical Association of Detection
Responses▫Low-Level Association▫Middle-Level Association▫High-Level Association
•Experimental Results•conclusion
![Page 13: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/13.jpg)
Middle-Level Association•The middle level association is an
iterative process: each round takes the tracklets generated in the previous round as the input and does further association
![Page 14: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/14.jpg)
•First round• input• tracklet association
▫lk is the number of tracklets in Sk.• corresponding
trajectory of Sk
• tracklet association set.
![Page 15: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/15.jpg)
MAP problem
![Page 16: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/16.jpg)
Hungarian Algorithm
![Page 17: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/17.jpg)
Hungarian Algorithm(1)•Arrange your information in a matrix
with the "people" on the left and the "activity" along the top, with the "cost" for each pair in the middle.
![Page 18: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/18.jpg)
Hungarian Algorithm(2)•Ensure that the matrix is square by
the addition of dummy rows/columns if necessary. Conventionally, each element in the dummy row/column is the same as the largest number in the matrix.
![Page 19: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/19.jpg)
Hungarian Algorithm(3)•Reduce the rows by subtracting the
minimum value of each row from that row.
![Page 20: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/20.jpg)
Hungarian Algorithm(4)•Reduce the columns by subtracting
the minimum value of each column from that column.
![Page 21: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/21.jpg)
Hungarian Algorithm(5)•Cover the zero elements with the
minimum number of lines it is possible to cover them with. (If the number of lines is equal to the number of rows then go to step 9)
![Page 22: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/22.jpg)
Hungarian Algorithm(6)•Add the minimum uncovered element
to every covered element. If an element is covered twice, add the minimum element to it twice.
![Page 23: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/23.jpg)
Hungarian Algorithm(7)•Subtract the minimum element from
every element in the matrix.
![Page 24: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/24.jpg)
Hungarian Algorithm(8)•Cover the zero elements again. If the
number of lines covering the zero elements is not equal to the number of rows, return to step 6.
![Page 25: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/25.jpg)
Hungarian Algorithm(9)•Select a matching by choosing a set of
zeros so that each row or column has only one selected.
![Page 26: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/26.jpg)
Hungarian Algorithm(10)•Apply the matching to the original
matrix, disregarding dummy rows. This shows who should do which activity, and adding the costs will give the total minimum cost.
![Page 27: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/27.jpg)
![Page 28: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/28.jpg)
![Page 29: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/29.jpg)
![Page 30: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/30.jpg)
Implementation Details
•for each input tracklet, a Kalman Filter is used to refine the positions and sizes of its detection responses and estimate their velocities.
• refined color histogram by a RANSAC method
![Page 31: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/31.jpg)
•appearance affinity
•motion affinity
• frame gap between the tail of and the head of
![Page 32: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/32.jpg)
•temporal affinity
• number of frames in which the tracked object is occluded by other objects
• number of frames in which the tracked object is visible but missed by the detector
![Page 33: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/33.jpg)
![Page 34: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/34.jpg)
•Initialization and termination probabilities of each tracklet
•In the following rounds, tracklets with longer frame gaps are associated by progressively increasing .
![Page 35: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/35.jpg)
outline•Introduction•Hierarchical Association of Detection
Responses▫Low-Level Association▫Middle-Level Association▫High-Level Association
•Experimental Results•conclusion
![Page 36: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/36.jpg)
High-Level Association•During the middle-level association, all
tracklets have the same initialization/termination probabilities as there is no prior knowledge about entries and exits at that stage.
![Page 37: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/37.jpg)
High-Level Association•At the high level, an entry map and an
exit map are inferred from T M, which are used to specify the initialization/termination of each tracklet in the scene.
•a scene occluder map is also inferred from T M to revise the link probabilities.
![Page 38: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/38.jpg)
•The three maps, as hidden variables, constitute a scene structure model in the high-level association.
•the scene structure model is estimated in the ground plane coordinates for better accuracy.
•solve this coupled scene-estimation tracklet-association problem by an EM-like algorithm.
![Page 39: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/39.jpg)
E-step•Bayesian inference
•indicator function for entries, exits or scene occluders (q = {en; ex; oc}) at position x on the ground plane.
![Page 40: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/40.jpg)
![Page 41: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/41.jpg)
•The complete version of a tracklet Tk that includes missed detections, , is obtained by filling the gaps between inconsecutive detection responses with interpolated ones.
![Page 42: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/42.jpg)
entry/exit map
• estimated position and velocity at the head/tail of by the Kalman Filter
• short time span for predicting the positions of the entry and the exit.
![Page 43: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/43.jpg)
entry/exit map
•where is the position of response .
![Page 44: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/44.jpg)
occluder map
• subset of the complete tracklet
![Page 45: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/45.jpg)
![Page 46: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/46.jpg)
![Page 47: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/47.jpg)
M-step•In the M-step, the tracklets in T M are
further associated to form even longer ones.
•based on the scene structure model obtained from the E-step, the initialization and termination probabilities
![Page 48: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/48.jpg)
M-step
• the frame number of missed detection part between the head (or tail) of Tk to the nearest entry (or exit):
![Page 49: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/49.jpg)
•This is used to reivse the temporal affinity in Equ.13 by considering occlusions by scene
•occluders when counting the occluded frame number .
![Page 50: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/50.jpg)
![Page 51: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/51.jpg)
outline•Introduction•Hierarchical Association of Detection
Responses▫Low-Level Association▫Middle-Level Association▫High-Level Association
•Experimental Results•conclusion
![Page 52: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/52.jpg)
Evaluation Metric•(1) MOTA: Multiple Object Tracking
Accuracy, calculated from the number of false alarms, missed detections, and identity switches;
•(2) FGTIM: Fraction of Ground Truth Instances Missed; and
•(3) FAPF: False Alarm Per Frame.
![Page 53: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/53.jpg)
![Page 54: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/54.jpg)
![Page 55: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/55.jpg)
![Page 56: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/56.jpg)
![Page 57: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/57.jpg)
outline•Introduction•Hierarchical Association of Detection
Responses▫Low-Level Association▫Middle-Level Association▫High-Level Association
•Experimental Results•conclusion
![Page 58: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/58.jpg)
Conclusion•we present a robust hierarchical
association framework for the multiple object tracking problem
•effectively associating tracklets with inaccurate detection responses and long-time occlusions.
![Page 59: Robust Object Tracking by Hierarchical Association of Detection Responses](https://reader035.vdocuments.net/reader035/viewer/2022062315/5681634e550346895dd3eab7/html5/thumbnails/59.jpg)
The end•Thank you!