mining and understanding events in crowd...
TRANSCRIPT
Mining and Understanding
Events in Crowd Scenes
Weiyao Lin (林巍峣)
Shanghai Jiao Tong University
Oct. 13. 2016
Background
Issues in crowd scene understanding:
• Semantic Region Segmentation Crowd Activity Recognition
• Unsupervised Recurrent Motion Pattern Mining
• Cross-view Motion Pattern Matching
Background Related works
• Trajectory-based methods
• Trajectory (or tracklet) extraction in crowed scenes
become difficult and unreliable
• Low-level feature based methods
• Have limitations in achieving precise motion flow
patterns under scenes with complex motions
Our Approach
• Parsing crowd scenes based on coherent motion regions
Framework
• Coarse-to-Fine Thermal Diffusion Better motion field:
Thermal Energy Field (TEF)
• Two-Step Clustering Finding reliable semantic regions
• Cluster and merge Finding recurrent motion patterns
Coherent Motion Region Detection Coarse-to-fine Thermal Diffusion Process
Thermal diffusion equation: The final diffused thermal
energy for P after l sec:
The final individual thermal energy from Q to P:
Coherent Motion Region Detection
Framework
• Coarse-to-Fine Thermal Diffusion Better motion field:
Thermal Energy Field (TEF)
• Two-Step Clustering Finding reliable semantic regions
• Cluster and merge Finding recurrent motion patterns
Finding Semantic Regions
Two step Clustering
• Step 1: Cluster coherent motions
• Similarity between coherent motions
where
Finding Semantic Regions
Two step Clustering
• Step 2: Cluster to find semantic regions
Framework
• Coarse-to-Fine Thermal Diffusion Better motion field:
Thermal Energy Field (TEF)
• Two-Step Clustering Finding reliable semantic regions
• Cluster and merge Finding recurrent motion patterns
Finding recurrent motion patterns Cluster and Merge Process
• Step 1: Iterative frame-level clustering
Finding recurrent motion patterns
Similarity between frames
Joint similarity for matched
coherent regions
Joint similarity for unmatched
coherent regions
Finding recurrent motion patterns Cluster and Merge Process
• Step 2: Iterative frame-level clustering
• Step 3: Flow curve extraction
Cross-view Motion Pattern Matching
Define matching cliques
Find best matching by optimizing clique cost
Experimental Results
(a): Ground Truth, (b): Our approach, (c): CVPR’07, (d): IEEE Trans. SMC’ 12, (e):
ECCV’12, (f): CVPR’13, (g): ECCV’10, (h) CVIU’13
Experimental Results
Semantic region detection
Experimental Results
Recognizing crowd activities
Dense
Trajectory
Experimental Results
Mining Recurrent Motion Patterns
Experimental Results
Matching cross-scene motion patterns
References
[1] Weiyao Lin*, Y. Mi, W. Wang, et al. "A diffusion and clustering-based
approach for finding coherent motions and understanding crowd scenes,"
IEEE Trans. Image Processing, vol. 25, no. 4, pp. 1674-1687, 2016.
[2] W. Wang, Weiyao Lin* et al., "Finding coherent motions and semantic
regions in crowd scenes: a diffusion and clustering approach," ECCV, 2014.
[3] Weiyao Lin*, Y. Mi et al., "Finding coherent motions and understanding
crowd scenes: a diffusion and clustering-based approach," CVPR Scene
UNderstanding workshop, 2015.
[4] L. Liu, Weiyao Lin*, et al., "Traffic flow matching with clique and triplet
cues," MMSP, 2015.
For more information, please visit my personal
webpage:
• http://wylin2.drivehq.com/
Thanks!