outline facial attributes analysis animated pose templates(apt) for modeling and detecting human...
TRANSCRIPT
![Page 1: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/1.jpg)
APPLICATIONS OF DEEP MODEL
![Page 2: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/2.jpg)
Outline
Facial Attributes Analysis Animated Pose Templates(APT) for Modeling
and Detecting Human Actions Unsupervised Structure Learning of Stochastic
And-Or Grammars
![Page 3: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/3.jpg)
Outline
Facial Attributes Analysis Animated Pose Templates for Modeling and
Detecting Human Actions Unsupervised Structure Learning of Stochastic
And-Or Grammars
![Page 4: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/4.jpg)
A Deep Sum-Product Architecture for Robust Facial Attributes Analysis
Motivation:An attribute can be estimated from small
regionOccluded region can be inferred with
respect to othersAttributes may indicate the absence or
presence of others
![Page 5: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/5.jpg)
Algorithm
Use discriminative binary decision tree(DDT) for each attribute.Each node of tree contains a detector(locate
the region) and a classifier(determine the presence or absence of an attribute)
DDT
![Page 6: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/6.jpg)
Sum-product Tree(SPT)
Model joint probabilityThe value of the root equals the joint
probability of the variables. All the children of a product node are
sums, all the children of a sum node are products or terminals.
Sum node with its children has weights.
![Page 7: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/7.jpg)
Sum-product Tree(SPT)
With SPT, we can efficiently infer the value of an unobserved variable using MPE inference.
When = 1 and is unobserved!We use MPE can find that the most probable explanation of is 0 when = 1.
![Page 8: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/8.jpg)
Algorithm Transform DDT to a sum-product tree(SPT)
to explorer interdependencies of regions. Be able to handle occlusions even train data has
no occlusions
separator
cluster
Sum node
Product node
![Page 9: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/9.jpg)
Algorithm Organize all the SPTs into a sum-product
network(SPN) to learn correlations of different attributes.(Learned by EM)
means 3 different type of sum weights
![Page 10: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/10.jpg)
Inference
Run region detector with sliding window Locate a region Apply a region classifier
![Page 11: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/11.jpg)
Learning
1) Train DDT for each attribute 2) transform DDT to SPT 3) build SPN
E-step: infer unobserved dataM-step: renormalize parametersPrune edges with zero weights
![Page 12: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/12.jpg)
Outline
Facial Attributes Analysis Animated Pose Templates for Modeling and
Detecting Human Actions Unsupervised Structure Learning of Stochastic
And-Or Grammars
![Page 13: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/13.jpg)
Formulation
Short-term action snippets( 2~5 frames )Moving pose templates
Long-term transitions between the pose templatesAPTs
Contextual objects
![Page 14: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/14.jpg)
Short-term action snippets
Moving pose templates for each pose =
Shape template(HOG) + Motion template(HOF) Human geometry, appearance, motion jointly
![Page 15: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/15.jpg)
![Page 16: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/16.jpg)
Moving Pose Template(MPT) MPT
appearance(HOG), deformation and motion(variation of HOF).
![Page 17: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/17.jpg)
Long-term actions
Animated pose templateA sequence of moving pose templates
![Page 18: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/18.jpg)
Animated Pose Templates HMM model
Transition Probability for the MPT labelsTracking probability for the movement of
parts between frames
![Page 19: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/19.jpg)
Animated Pose Templates(APT)
![Page 20: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/20.jpg)
Animated Pose Templates with Contextual Object
Contextual ObjectsWeak objects( e.g. cigarette and ground )
○ Too small or too diverse○ Using body parts
Strong objects( e.g. cup )○ Distinguishable○ Using HOG
Treat these objects in the same way as the body parts.
![Page 21: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/21.jpg)
![Page 22: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/22.jpg)
Inference
![Page 23: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/23.jpg)
Learning
Semi-supervised Structure SVMAnnotated key framesCluster them into pose templates by EMFor unannotated frames and model parameters
○ Learn model using labeled data by LSVM○ Accept high score frames as labeled frames
![Page 24: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/24.jpg)
Outline
Facial Attributes Analysis Animated Pose Templates for Modeling and
Detecting Human Actions Unsupervised Structure Learning of Stochastic
And-Or Grammars
![Page 25: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/25.jpg)
Unsupervised Structure Learning Problem Definition
G is grammar X is the training data
![Page 26: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/26.jpg)
Algorithm Framework
Introduce new intermediate nonterminal nodes to increase its posterior probability.
![Page 27: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/27.jpg)
And-Or Fragments
And-fragmentsFailed when training data is scarce.
Or-fragmentsDecrease posterior probability.
And-Or fragmentsAnd-rules and Or-rules are learned in a
more unified manner.
![Page 28: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/28.jpg)
Likelihood Gain
= likelihood changes * context matrix changes
Prior Gain = size of grammar increase + reductions of configurations
Posterior Gain = Likelihood Gain * Prior Gain
![Page 29: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/29.jpg)
![Page 30: Outline Facial Attributes Analysis Animated Pose Templates(APT) for Modeling and Detecting Human Actions Unsupervised Structure Learning of Stochastic](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649cf85503460f949c9095/html5/thumbnails/30.jpg)