estimating human body configurations using shape context matching
DESCRIPTION
Estimating Human Body Configurations using Shape Context Matching. Greg Mori and Jitendra Malik. Problem. Approach: Exemplar-based Matching. Set of stored exemplars with hand-labeled keypoints Obtain sample points Deformable matching to exemplars: - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/1.jpg)
Computer Vision GroupUniversity of California Berkeley
Estimating Human Body Configurations using Shape Context Matching
Greg Mori and Jitendra Malik
![Page 2: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/2.jpg)
Computer Vision GroupUniversity of California Berkeley
Problem
![Page 3: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/3.jpg)
Computer Vision GroupUniversity of California Berkeley
Approach: Exemplar-based Matching
• Set of stored exemplars with hand-labeled keypoints
• Obtain sample points
• Deformable matching to exemplars:– Shape context matching to get correspondences– Kinematic chain deformation model
• Estimate 3D body configuration
![Page 4: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/4.jpg)
Computer Vision GroupUniversity of California Berkeley
Comparing Pointsets
![Page 5: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/5.jpg)
Computer Vision GroupUniversity of California Berkeley
Shape ContextCount the number of points inside each bin, e.g.:
Count = 4
Count = 10
...
Compact representation of distribution of points relative to each point
![Page 6: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/6.jpg)
Computer Vision GroupUniversity of California Berkeley
Shape Context
![Page 7: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/7.jpg)
Computer Vision GroupUniversity of California Berkeley
Comparing Shape ContextsCompute matching costs using Chi Squared distance:
Recover correspondences by solving linear assignment problem with costs Cij
[Jonker & Volgenant 1987]
![Page 8: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/8.jpg)
Computer Vision GroupUniversity of California Berkeley
Deformable Matching
• Kinematic chain-based deformation model
• Use iterations of correspondence and deformation
• Keypoints on exemplars are deformed to locations on query image
![Page 9: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/9.jpg)
Computer Vision GroupUniversity of California Berkeley
![Page 10: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/10.jpg)
Computer Vision GroupUniversity of California Berkeley
Problem
![Page 11: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/11.jpg)
Computer Vision GroupUniversity of California Berkeley
Estimate 3D Body Configuration [Taylor ’00]
• Known:– Relative lengths of body segments– (x,y) Image locations of keypoints– “closer endpoint” labels for each segment– Scaled orthographic camera model
• Solve for 3D locations of keypoints up to some scale factor– Scale factor can be estimated automatically
![Page 12: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/12.jpg)
Computer Vision GroupUniversity of California Berkeley
Solving for Foreshortening
)()()( 2121212222 ZZYYXXl
2221
221
2 /))()(( suuuuldZ
![Page 13: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/13.jpg)
Computer Vision GroupUniversity of California Berkeley
Choosing Scale
))()(( 221
221 vvuu
ls
![Page 14: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/14.jpg)
Computer Vision GroupUniversity of California Berkeley
Results
![Page 15: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/15.jpg)
Computer Vision GroupUniversity of California Berkeley
![Page 16: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/16.jpg)
Computer Vision GroupUniversity of California Berkeley
Multiple Exemplars
• Parts-based approach– Use a combination of keypoints/whole limbs from different
exemplars– Reduces the number of exemplars needed
• Compute a matching cost for each limb from every exemplar
• Compute pairwise “consistency” costs for neighbouring limbs
• Use dynamic programming to find best K configurations
![Page 17: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/17.jpg)
Computer Vision GroupUniversity of California Berkeley
Parts-based Approach
![Page 18: Estimating Human Body Configurations using Shape Context Matching](https://reader035.vdocuments.net/reader035/viewer/2022062316/56816825550346895dddb5ef/html5/thumbnails/18.jpg)
Computer Vision GroupUniversity of California Berkeley
Tracking by Repeated Finding