semantizing complex 3d scenes using constrained attribute...
TRANSCRIPT
![Page 1: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/1.jpg)
Semantizing Complex 3D Scenesusing Constrained Attribute Grammars
A. Boulch, S. Houllier,R. Marlet and O. Tournaire
slight variant of the materialpresented at SGP 2013
![Page 2: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/2.jpg)
Motivation
Semantizing complex objects in 3D scenes
Bare geometry
Semantized geometry
2
![Page 3: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/3.jpg)
Motivation
Semantizing complex objects in 3D scenes
Bare geometry
Semantized geometry
(rendered)
2
![Page 4: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/4.jpg)
Motivation
Semantizing complex objects in 3D scenes
⌅ Building industry⇤
for the renovation market
⌅ point cloud ! building model
⇤for architects
⌅ building sketch ! rendering⌅ Game industry
⇤for graphic designers
⌅ basic level design ! rendering⌅ Object mining in shape databases
⇤for semantic queries
⌅ 3D object ! semantic labeling
Semantized geometry
2
![Page 5: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/5.jpg)
Motivation
Semantizing complex objects in 3D scenes
⌅ Building industry⇤
for the renovation market
⌅ point cloud ! building model⇤
for architects
⌅ building sketch ! rendering
⌅ Game industry⇤
for graphic designers
⌅ basic level design ! rendering⌅ Object mining in shape databases
⇤for semantic queries
⌅ 3D object ! semantic labeling
Semantized geometry
(rendered)
2
![Page 6: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/6.jpg)
Motivation
Semantizing complex objects in 3D scenes
⌅ Building industry⇤
for the renovation market
⌅ point cloud ! building model⇤
for architects
⌅ building sketch ! rendering⌅ Game industry
⇤for graphic designers
⌅ basic level design ! rendering
⌅ Object mining in shape databases⇤
for semantic queries
⌅ 3D object ! semantic labeling
Semantized geometry
(rendered)
2
![Page 7: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/7.jpg)
Motivation
Semantizing complex objects in 3D scenes
⌅ Building industry⇤
for the renovation market
⌅ point cloud ! building model⇤
for architects
⌅ building sketch ! rendering⌅ Game industry
⇤for graphic designers
⌅ basic level design ! rendering⌅ Object mining in shape databases
⇤for semantic queries
⌅ 3D object ! semantic labelingSemantized geometry
2
![Page 8: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/8.jpg)
Outline
Constrained attribute grammars
Scene interpretation
Bottom-up parsing
Experiments
3
![Page 9: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/9.jpg)
Outline
Constrained attribute grammars
Scene interpretation
Bottom-up parsing
Experiments
4
![Page 10: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/10.jpg)
Grammars to express hierarchical decomposition
Sentence
Subject Verb Complement
NounAdjective
The super computer rules Discovery One
Building
Facade Roof Floor
Window WallArticle
4
![Page 11: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/11.jpg)
Complex, non-hierarchic relations between components
Sentence
Subject Verb Complement
NounAdjective
The super computer rules Discovery One
Building
Facade Roof Floor
Window Wall
agreement (singular)
adjArticle
5
![Page 12: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/12.jpg)
Constrained attribute grammars
G = (N,T ,P , S)
⌅ N: nonterminals ($ complex forms) e.g., window, wall
⌅ T : terminals ($ geometric primitives) e.g., polygon, cylinder
⌅ P : production rules ($ hierarchical decomposition and constraints)⌅ S : start symbols ($ root shapes) e.g., building
6
![Page 13: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/13.jpg)
Basic rules
Decomposition of a complex object y of type Y into its constituentsxi of type Xi :
Y y �! X1
x1
, . . . ,Xn xn
Example:
step s �! riser r , tread t
Rule application:⌅ top-down view:
y decomposes into x1
, . . . , xn⌅ bottom-up view:
given some x1
, . . . , xn, create a new object y
riser
tread
7
![Page 14: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/14.jpg)
Basic rules
Decomposition of a complex object y of type Y into its constituentsxi of type Xi :
Y y �! X1
x1
, . . . ,Xn xn
Example:
step s �! riser r , tread t
Rule application:⌅ top-down view:
y decomposes into x1
, . . . , xn⌅ bottom-up view:
given some x1
, . . . , xn, create a new object y
riser
tread
tread ?
riser ?
7
![Page 15: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/15.jpg)
Constraints
Conditional rule application (conjonction of predicates):
Y y �! X1
x1
,X2
x2
, . . . h cstr1
(x1
), cstr2
(x1
, x2
), . . . i
Example:
riser r �! polygon p h vertical(p) itread t �! polygon p h horizontal(p) istep s �! riser r , tread t
h edgeAdj(r , t), above(t, r) i
riser
tread
8
![Page 16: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/16.jpg)
Attributes
Features attached to each grammar element:⌅ at creation time (primitives)⌅ at rule application (synthetized attributes)
Examples:⌅ length, width⌅ bounding box⌅ ... r.widthr.length
9
![Page 17: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/17.jpg)
Predicates
Predicates on grammar elements:⌅ adj, edgeAdj⌅ orthogonal, parallel⌅ vertical, horizontal⌅ ...Predicates on attributes:⌅ �, >, , ...⌅ =, 6=⌅ ...
t.length
r.length
Example: r .length == t.length
10
![Page 18: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/18.jpg)
Collections of similar elements
Grouping elements via recursion (Y as set of X s):Y y �! X x
Y y �! X x ,Y y2
Grouping elements via specific collection operators:Y y �! coll(X ) xs
Useful operators: maximal collections⌅ maxconn: maximal set of connected components⌅ maxseq: maximal sequence⌅ ...Example:
stairway sw �! maxseq(step, adjEdge) ss
11
![Page 19: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/19.jpg)
Collections of similar elements
Grouping elements via recursion (Y as set of X s):Y y �! X x
Y y �! X x ,Y y2
Grouping elements via specific collection operators:Y y �! coll(X ) xs
Useful operators: maximal collections⌅ maxconn: maximal set of connected components⌅ maxseq: maximal sequence⌅ ... [see complete stairway grammar]
Example:stairway sw �! maxseq(step, adjEdge) ss
11
![Page 20: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/20.jpg)
Outline
Constrained attribute grammars
Scene interpretation
Bottom-up parsing
Experiments
12
![Page 21: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/21.jpg)
Scene interpretation: parse tree
Tree representation of a grammatical analysis of the scene:⌅ leaves: terminals representing primitives⌅ non-leaf nodes: instanciations of grammar rules
12
![Page 22: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/22.jpg)
Scene interpretations: parse forest
Set of parse treeswith systematic sharing (DAG)
Compact representationof all possible interpretations
13
![Page 23: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/23.jpg)
Ambiguity and the exclusivity constraint
Example (assuming no height ordering constraint):
step s �! riser r , tread t h edgeAdj(r , t) i step 2
step 1
Exclusivity constraint:
at most 1 occurrenceof a grammar elementper interpretation
Stairway
Step 2Step 1
Riser 2 Tread 1 Riser 1
14
![Page 24: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/24.jpg)
Outline
Constrained attribute grammars
Scene interpretation
Bottom-up parsing
Experiments
15
![Page 25: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/25.jpg)
Parse forest computation
Bottom-up parsing:construction of the parse forestfrom leaves (terminals) to roots (start symbols)
⌅ create one terminal for each primitives⌅ iteratively create new grammar elements from existing ones
⇤e.g., given grammar rule step s �! riser r , tread t
given existing instances riser r23, tread t18create new instance step s5
⌅ merge identical trees on the fly⌅ stop iterating when no rule applies
15
![Page 26: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/26.jpg)
Rule application order
⌅ Simple rules: use any order⌅ Maximal operators: wait for all subelements to ensure maximality
⇤reverse topological sort of nonterminal dependency graph
maxseq
16
![Page 27: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/27.jpg)
Rule application order
⌅ Simple rules: use any order⌅ Maximal operators: wait for all subelements to ensure maximality
⇤reverse topological sort of nonterminal dependency graph
maxseq
16
![Page 28: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/28.jpg)
Rule application order
⌅ Simple rules: use any order⌅ Maximal operators: wait for all subelements to ensure maximality
⇤reverse topological sort of nonterminal dependency graph
maxseq
16
![Page 29: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/29.jpg)
Mastering the combinatorial explosion
Usual drawback of bottom-up analysis: combinatorial explosion⌅ all trees⌅ all sets, all sequences, ...⌅ all combinations
Our solution:⌅ tree sharing: construction of a parse forest (exp. ! lin.)⌅ maximal operators, with efficient implementation (> exp. ! polyn.)⌅ constraint propagation: predicate ordering ) early pruning
17
![Page 30: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/30.jpg)
Maximal operators
maxseq
vs
allseq
18
![Page 31: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/31.jpg)
Maximal operators
maxseq
vs
max(allseq)
18
![Page 32: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/32.jpg)
Constraint propagation as predicate orderingA simple 2D example
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
19
![Page 33: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/33.jpg)
Constraint propagation as predicate orderingExample of poor ordering
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
1. orthogonality
2. adjacency3. verticality
Complexity:O(#seg2)
constraints to test
20
![Page 34: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/34.jpg)
Constraint propagation as predicate orderingExample of poor ordering
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
1. orthogonality
2. adjacency3. verticality
Complexity:O(#seg2)
satisfied constraints
20
![Page 35: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/35.jpg)
Constraint propagation as predicate orderingExample of poor ordering
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
1. orthogonality2. adjacency
3. verticality
Complexity:O(#seg2 +
#seg ⇥maxDeg)
= O(#seg2)
constraints to test
20
![Page 36: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/36.jpg)
Constraint propagation as predicate orderingExample of poor ordering
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
1. orthogonality2. adjacency
3. verticality
Complexity:O(#seg2 +
#seg ⇥maxDeg)
= O(#seg2)
satisfied constraints
20
![Page 37: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/37.jpg)
Constraint propagation as predicate orderingExample of poor ordering
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
1. orthogonality2. adjacency3. verticality
Complexity:O(#seg2 +
#seg ⇥maxDeg +#seg)
= O(#seg2)constraints to test
20
![Page 38: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/38.jpg)
Constraint propagation as predicate orderingExample of poor ordering
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
1. orthogonality2. adjacency3. verticality
Complexity:O(#seg2 +
#seg ⇥maxDeg +#seg)
= O(#seg2)satisfied constraints
20
![Page 39: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/39.jpg)
Constraint propagation as predicate orderingExample of good ordering
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
1. verticality
2. adjacency3. orthogonality
Complexity :O(#seg)
constraints to test
21
![Page 40: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/40.jpg)
Constraint propagation as predicate orderingExample of good ordering
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
1. verticality
2. adjacency3. orthogonality
Complexity :O(#seg)
satisfied constraints
21
![Page 41: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/41.jpg)
Constraint propagation as predicate orderingExample of good ordering
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
1. verticality2. adjacency
3. orthogonality
Complexity :O(#seg +
#seg ⇥maxDeg)
= O(#seg ⇥maxDeg)
constraints to test
21
![Page 42: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/42.jpg)
Constraint propagation as predicate orderingExample of good ordering
pair p �! seg s1
, seg s2
, h orthogonal(s1
, s2
), adj(s1
, s2
), vertical(s1
) i
1. verticality2. adjacency3. orthogonality
Complexity :O(#seg +
#seg ⇥maxDeg +#seg ⇥maxDeg)
= O(#seg ⇥maxDeg) ⌧ O(#seg2)satisfied constraints
21
![Page 43: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/43.jpg)
Constraint propagation as predicate ordering
1. Unary predicates
2. Invertible predicates that are partially instantiated3. General predicates
= constraints implying only 1 element: complexity O(#element)
Example:riser r �! polygon p h vertical(p) i
22
![Page 44: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/44.jpg)
Constraint propagation as predicate ordering
1. Unary predicates2. Invertible predicates that are partially instantiated
3. General predicates
= constraints with small cardinality when some arguments are fixed
Example:
step s �! riser r , tread t h edgeAdj(r , t) i
22
![Page 45: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/45.jpg)
Constraint propagation as predicate ordering
1. Unary predicates2. Invertible predicates that are partially instantiated3. General predicates
= remaining relations
Example:
step s �! riser r , tread t h orthogonal(r , t) i
22
![Page 46: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/46.jpg)
Outline
Constrained attribute grammars
Scene interpretation
Bottom-up parsing
Experiments
23
![Page 47: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/47.jpg)
Semantization pipeline
Semantizedmodel
Polygonextraction Parsing
Grammar
CAD Model(triangle soup)
Point cloud Primitivedetecion
23
![Page 48: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/48.jpg)
CAD modelsPreprocessing
⌅ Region growing over trianglesfor polygon creation
⌅ Computation of exact andapproximate adjacency graphs
24
![Page 49: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/49.jpg)
CAD modelsDetection of stairs (more examples in supplem. material)
25
![Page 50: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/50.jpg)
CAD modelsDetection of walls, roofs and openings (more examples in supplem. material)
26
![Page 51: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/51.jpg)
Real data: photogrammetry⌅ Preprocessing (point cloud)
⇤clustering using RANSAC
(or region growing)
⇤polygons bounded
by alpha shapes
⌅ Problems:⇤
missing primitives
⇤false primitives
⇤wrong adjacencies
⌅ Solution:⇤
use of a relaxed grammar
⌅ looser bounds⌅ 1-2 missing items OK
⇤22 openings out of 31
27
![Page 52: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/52.jpg)
Quasi-real data: simulated LIDAR
⌅ Planes by region growing in depth image⌅ Polygons as oriented bounding rectangles⌅ Adjacency based on pixels in depth image
28
![Page 53: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/53.jpg)
Size and parsing time (CAD models)
# of # of Parsing time (s)Name triangles polygons stairs openingsLcG 48332 9705 5 15LcA 111979 26585 14 42LcC 385541 111732 33 306LcD 313012 75257 25 111LcF 286996 84347 39 322
29
![Page 54: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/54.jpg)
Precision and recall (%, CAD models)
# of # of Stairs OpeningsName stairs steps Prec. Rec. # Prec. Rec.LcG 3 45 100 93 83 100 90LcA 6 84 100 100 62 98 83LcC 30 210 100 100 196 100 98LcD 5 61 93 100 74 100 93LcF 7 98 100 50 99 100 96
30
![Page 55: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/55.jpg)
Future work
⌅ Principled way to deal with partial or missing primitives⌅ Exploitation of occlusion/visibility information⌅ Scoring of interpretations: pick best tree(s)
31
![Page 56: Semantizing Complex 3D Scenes using Constrained Attribute ...imagine.enpc.fr/~marletr/publi/SGP-2013-Boulch-et-al_slides.pdfpresented at SGP 2013. Motivation Semantizing complex objects](https://reader033.vdocuments.net/reader033/viewer/2022050323/5f7d3e4573ab7419f51d3843/html5/thumbnails/56.jpg)
Conclusion
Constrained attribute grammars:⌅ appropriate to semantize complex objects⌅ high-level specification language
⇤being expert is enough, computer scientist not required
⌅ efficient even on large models
This work:⌅ well-delimited first step: perfect data⌅ extensions required for incomplete/noisy data
On the web⌅ http://imagine.enpc.fr/⌅ sites.google.com/site/boulchalexandre/
32