interpreting line drawings of smooth shapes

Algorithm 1: Label contour orientation and inflate Algorithm 2: Reconstruct surface normals from patches Prior work focused on precise, “blocks world” shapes: Approach: Example-based 1. Find keypoints in drawing, connect with graph 2. Select set of examples at each keypoint 3. Find most consistent global configuration 4. (Optional) Fit surface to solution Training Set: Random blobby shapes Drawing Normal map Goal: 3D shape interpretation from line drawings of blobby, “organic” shapes 4. Inflate surface and compare with human perception 1. Keypoints are line pixels 2. Look up candidates from training data 3. Average guesses over graph Graph: segmented curves Consistent contour orientation Orientation guesses 1a. Place keypoints at image corners at varying scales 2a. Find patch candidates based on appearance Test image Shape candidates Lines Normals Context 2b. Rate compatibility of neighboring patches Graph connectivity 3. Find best global solution with inference on Markov Random Field from keypoint graph Test image / Original shape MAP solution, average of all patches MAP solution, fine-scale only 4. Fit surface to fine-scale patches Success if: Output matches human shape perception (not original 3D shape) PROS: matches human perception on some shapes CONS: brittle; hard to extend beyond occluding contours PROS: flexible; generalizes to any kind of line CONS: ??? (too early to say) 0° 90° Disagreement Original shape Occluding contours Inflated surface Comparison with humans 1b. Connect keypoints based on image proximity Keypoints (drawings are computer-generated from 3D models) GOOD OK BAD BAD OK BAD BAD BAD GOOD A B Patch A Patch B test drawing recon. drawing recon. normals original normals Key: [Malik 1987] [Ulupinar and Nevatia 1993] [Wang et al. 2009] 100 blobs x 20 views per blob = 2000 training pairs drawn with occluding and suggestive contours [DeCarlo et al. 2003] Blurred Drawing Corner Strength Graph connectivity SIFT descriptor [Lowe 1999] SIFT descriptor [Lowe 1999] Corner strength as defined by [Harris and Stephens 1988] 1a.i. Add extra points to cover all line pixels: All keypoints [DeCarlo et al. 2003] [Cole et al. 2008] Interpreting Line Drawings of Smooth Shapes Forrester Cole, William T. Freeman, Frédo Durand, Edward H. Adelson Massachusetts Institute of Technology

Upload: others

Post on 16-Nov-2021

2 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: Interpreting Line Drawings of Smooth Shapes

Algorithm 1: Label contour orientation and in�ate

Algorithm 2: Reconstruct surface normals from patchesPrior work focused on precise, “blocks world” shapes:

Approach: Example-based1. Find keypoints in drawing, connect with graph2. Select set of examples at each keypoint3. Find most consistent global con�guration4. (Optional) Fit surface to solution

Training Set: Random blobby shapes

Drawing

Normal map

Goal: 3D shape interpretation from line drawings of blobby, “organic” shapes

4. In�ate surface and compare with human perception

1. Keypoints are line pixels2. Look up candidates from training data

3. Average guesses over graph

Graph:segmentedcurves

Consistent contourorientation

Orientationguesses

1a. Place keypoints at image corners at varying scales

2a. Find patch candidates based on appearance