why is photorealism the aim? people paint! what is npr? npr issues nonphotorealistic rendering

• Why is photorealism the aim? People paint!

• What is NPR?

• NPR issues

NonPhotorealistic Rendering

Why photorealism? people paint!

Why photorealism ?

• Physics provides starting point for models

• Photographs provide a foil to test against

But people paint !

• Artwork shows off important regions better

• Photography is just one depictive style amongst many

What is NPR?

• NPR is a sub-branch of Computer Graphics that studies making images that do not look photographic.

• NPR is said to have started around 1990.

• Several sub-divisions exist– paint-boxes– rendering 3D models– rendering from photographs and/or video

Paint boxes

from the simple… …to the complex

Rendering 3D models

Breslav et al, SIGGRAPH 2007

Rendering from photographs and/or video

NPR issues

• NPR depends on perception and interpretation– “drawing implies seeing”– models are much more complex– interaction is common

• Mark making– What kind of mark?– Where to mark?

• No single foil to test against– there is no “correct” drawing

NPR depends on Perception and Interpretation

No single foil to test against

Mark making: what and where?

What: Emulate real media

pencil, oil paint, chalk, …..

physical models possible

can test for quality of a given mark

What: Produce new media

Can the computer be used to make marks not made before?

Where: Should marks be made locally, globally, or both?

How to decide where to place marks?

Where: How can marks be made stable in animations?

• The problems

• Some solutions

NPR from models

The problems

Consider a pencil drawing, marks are used to depict:

• object boundaries and other contours

• shadows and shading

• texture

Apparent ridges

Judd et al, SIGGRAPH 2007

For apparent ridges we need:

• differential geometry of surfaces

• projection

The curvature at a surface point S(x,y) – how curved is the surface?

Use the Hessian, H, which is a 2x2 matrix of 2nd order partials of S.

The EVD of H = UKUT

principle directions, U = [u1 u2], principle curvature, K = [k1 k2]

Ridges and valleys where k1 is an extremum in direction u1.

Project the local surface geometry (tangents, normals ‘n all),

Look for extremum in this observed geometry.

Shadows and shading

Shading lines – cross hatches etc

Can be based in “image space”………or in “object space”

Again, curvature can be used, this time to direct pencil lines.

Other methods use solid texture maps, that adapt to local lighting and geometry

Marks make textures too…

With NPR, even the marks can be animated (look at the sea)

• Where is this used?• Why is this difficult?• What has been done so far?

NPR from images

Where is NPR from images used?

• Photo-booths

• Film special effects

• Adverts

is related to

• Image Based Rendering

What has been done to date?

• 1990 Paint by Numbers

• 1994 Video Impressionism

• 2001 Image Analogies

• 2002 Gaze Directed Rendering

• 2003 Video Tooning

• 2007 Simpler shapes

Why is NPR from images difficult?

• Painting using STROKES– WHAT is a stroke?– WHERE to place them?

• Strokes are not enough

1990: Paul Haeberli “Paint by numbers”

User interaction,

place “strokes” over a photograph.

Stroke colour from pixel

Stoke direction from image gradient

Stoke look from a texture map

a STROKE represents the action of a brush/pencil etc on paper/canvas etc.

There is a lot of work to model strokes;

• Paint brushes

• Palette knives

• Pencil

• Copper plate

We’ll consider shaped dots

We’ll place place and shape them at edges

Where to place a stroke (here a dot)

Artists draw lines at object boundaries

Edge detectors respond to object boundaries

To get edges, typically convolve with Gaussian derivatives

f(x,y)|grad(f*g)|

Shaping a dot

Edges are found using “local” information,

but strokes are best placed using GLOBAL information

Edges do not correspond well to lines drawn by humans

distinguishing between different kinds of “edges” allows differential emphasis

Let’s use the “edge” mapas a kind of target.

Let’s place stokes near-randomly on one painting, but make MANY paintings.

Keep those that match the target best.

Make new painting by spawning strokes from the best fits.

Continue until happy

Detail is kept where needed

Lost where not needed

The situation is worse still when the painting moves…

But NPR is more than strokes