looking through the glass · 2019-06-08 · looking through the glass annalisa crannell contents...

Looking Through the Glass

Annalisa Crannell

Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2A Brief History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3A New Mathematical Object: The Point of Projective Geometry . . . . . . . . . . . . . . . . . . . . . . . . 5

Ideal Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Vanishing Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Where Was the Camera? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8A Consequence of Viewing Distances: Illusion, Distortion, and Anamorphism . . . . . . . . . . . . 10

Dolly Zoom. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Anamorphic Art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Impossible Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Going Backward from Pictures to 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Homogeneous Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Multiple View Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

The Ames Room. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Reconstructing Objects from Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Cross-References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

The description in section “Going Backward from Pictures to 3D” of the three steps forreconstructing three-dimensional objects from a collection of photographs was influenced by atalk by Joe Kileel at the Algebraic Vision session at the SIAM conference on Applied AlgebraicGeometry, July 31 2017 in Atlanta GA. Joe had just finished his PhD from UC Berkeley and washeaded for Princeton.

A. Crannell (�)Franklin & Marshall College, Lancaster, PA, USAe-mail: [email protected]

© Springer Nature Switzerland AG 2019B. Sriraman (ed.), Handbook of the Mathematics of the Arts and Sciences,https://doi.org/10.1007/978-3-319-70658-0_41-1

1

http://crossmark.crossref.org/dialog/?doi=10.1007/978-3-319-70658-0_41-1&domain=pdfmailto:[email protected]://doi.org/10.1007/978-3-319-70658-0_41-1

2 A. Crannell

Abstract

Projective geometry allows us, as its name suggests, to project a three-dimensional world onto a two-dimensional canvas. A perspective projectionoften includes objects called vanishing points, which are the images of projectiveideal points; the geometry of these points frequently allows us to either createimages or to reconstruct scenes from existing images. We give a particularexample of using a pair of vanishing points to locate the position of the artistCanaletto as he painted the Clock Tower in the Piazza San Marco. However,because mappings from three-dimensional space to a two-dimensional plane arenot invertible, we can also use perspective and projective techniques to createand analyze illusions (e.g., anamorphic art, impossible figures, the dolly zoom,and the Ames room). Moving beyond constructive (e.g., ruler and compass)projective geometry into analytical projective geometry via homogeneouscoordinates allows us to create and analyze digital perspective images. Theubiquity of digital images in the present day allows us to ask whether we canuse two (or many) images of the same object to reconstruct that object in partor in entirety. Such a question leads us into the emerging field of multiple viewgeometry, straddling projective geometry, algebraic geometry, and computervision.

KeywordsLinear perspective · Multiple view geometry · Projective geometry ·Anamorphism

Introduction

This chapter is about perspective art, and in particular about the role that projectivegeometry plays in perspective art. Most people are aware that perspective techniquesbegan to flourish during the Renaissance, and as a result drawings and paintings ofthat era became demonstrably more “realistic” or “lifelike” than art in previous eras.Now we are living through a similar Renaissance, especially in the technologicalrealm (which includes our animated movies, video games, medical imaging, andmore). The mathematics that transformed our world several centuries ago stillflourishes around us; it continues to have relevance and power in the way we lookat the world today.

The word perspective comes from the Medieval Latin roots per (“through”) andspecere (“look at” – the same root that gives us “spectacles”). So the title of thischapter is a deliberate pun: like the book written in 1871 by the mathematicianCharles Dodgson under his pen name, Lewis Carroll (1871), perspective art literallyintends us to look through a window to see the objects it portrays lying on theother side. And as Carroll’s book suggests, sometimes the view that we get bylooking through the glass will give us glimpses of the world that are surprising –even wonderful – feats of illusion and magic.

Looking Through the Glass 3

A Brief History

There is a lore that projective geometry has been a subject intimately connectedwith, and arising from, the development of perspective art. That lore is not entirelyin accordance with historical fact. (For a much more comprehensive description ofthe history of perspective art than this chapter can provide, see Andersen’s excellentvolume Andersen 2006.)

The formal introduction of linear perspective is generally credited to FilippoBrunelleschi, an Italian designer, architect, and engineer who lived 1377–1446(See also � “Renaissance Architecture”). His perspective demonstrations reliedextensively on geometry but also on physical apparatuses – he interposed mirrorsbetween his canvas and the pictured scenes to validate the accuracy of his images.Brunelleschi’s work had an almost immediate influence on Leon Battista Alberti(1404–1472), an Italian polymath (architect, priest, artist, and author). In 1435,Alberti published Della pittura, his seminal work on perspective, whose influencereached far and wide.

For two centuries, perspective art remained largely in the arena whereBrunelleschi and Alberti had placed it: as an exercise in Euclidean geometry andengineering. The German mathematician and astronomer Johannes Kepler (1571–1630) may have been the first person to introduce the projective notion of “pointsat infinity.” However, Kepler’s motivation arose not from perspective art, but ratherfrom developing a unified theory of conics (e.g., “closing up” the parabola).

In the early-to-mid 1600s, Girard Desargues (1591–1661) published a seriesof short works, some in perspective art (notably, Desargues 1987) and others inprojective geometry. Like Brunelleschi and Alberti, Desargues was a mathematicianand engineer. The theorem that bears his name to this day appears in a work ofhomage by his contemporary, Abraham Bosse (1648). Desargues’s theorem statesthat “A pair of triangles perspective from a point is also perspective from a line.”This theorem does indeed have perspective art interpretations: see Fig. 1, whichdepicts a lamp casting a shadow. In this figure, the corresponding vertices of twotriangles are collinear from the bulb of the lamp (a point), and the correspondingedges of the triangles are coincident with the line where the glass meets the ground.But it is not clear that the theorem was directly motivated by a similar situation;in Bosse’s manuscript, the formulation of Desargues’s theorem is separated fromhis description of Desargues’s work in perspective; the diagram and proof are bothhighly abstract.

Desargues’s work seems to have been lost or neglected in the period thatfollows, possibly because the algebraic approach to geometry put forward by hiscontemporary, Rene Descartes, proved more versatile. A century later, for example,the artist Canaletto (whom we will return to in section “Where Was the Camera?”)was creating his paintings with the camera obscura rather than with geometry.Across the channel, the English mathematician Brook Taylor (of Taylor’s seriesfame) would publish his highly celebrated “New Principles of Linear Perspective:or the Art of Designing on a Plane the Representations of all sorts of Objects, in amore General and Simple Method than has been done before” Taylor (1719). But in

http://link.springer.com/``Renaissance Architecture''

4 A. Crannell

Fig. 1 A perspective interpretation of Desargues’s theorem

spite of the promise of the first word of this title, the book contained very little thatwas “new”; it relied almost exclusively on Euclidean geometry (moreover, it wasoften described as far from “simple” to read).

Two centuries after Desargues introduced projective geometry, another Frenchengineer and mathematician – Jean-Victor Poncelet (1788–1867) – resurrected it.Famously, Poncelet wrote much of what would become his “Traité des propriétésprojectives des figures” during a two-year imprisonment; he had been capturedduring Napoleon’s campaign against the Russian Empire. Poncelet’s geometry wasaxiomatic and theoretical, and was not explicitly motivated by, nor applied to,perspective art.

The centuries that followed have seen projective geometry take a variety offorms. Perhaps farthest from perspective art is the subfield of finite projectivegeometry, with points and lines abstracted (as in the Fano plane, Fig. 2).

But fittingly, given the coincident geometric contributions of Desargues andDescartes, it is in the realm of analytical projective geometry where we see recent,exciting applications to perspective images, as well as to reconstructing the objectsthat make those images. In the sections that follow, we build from perspectiveapplications of “traditional” (ruler and compass) projective geometry toward theseanalytical applications.


Fig. 2 The Fano planecontains seven points, eachincident with three “lines”,and seven “lines”, eachincident with three points

A New Mathematical Object: The Point of Projective Geometry

Traditional perspective art assumes that there is an artist looking with one eyethrough a window or canvas at the world. We call the location of the viewer’s eyethe center of the projection and denote it by the point O; we’ll denote the pictureplane by the greek letter ρ, and the image of a real-world point X on the canvas ρwe’ll denote by the symbol X′.

There are other physical setups that give us similar projections on planes. Forexample, a camera might have a lens or pin-hole that projects objects in the realworld onto a sheet of film or a set of pixels; again, we call the lens the center O ofthe projection, with the film lying in a plane ρ and the object and its image similarlydenoted by X and X′, respectively. Or we might have a light source casting a shadowon the ground; the light source in this case would play the role of the center O; theground becomes the image plane ρ, and the object and its shadow are X and X′.

What all these situations have in common is that the points O, X, and X′ arecollinear and that X′ is the intersection of the line through O and X with the planeρ. (In shorthand mathematical notation, we write X′ = (OX) · ρ.)

This simple notion runs into difficulties, however, if the point X lies in an“awkward” place: if the line OX is parallel to the plane ρ, then the intersection(OX) · ρ is empty (at least in the usual realm of Euclidean geometry). Fortunatelyfor artists, this situation does not seem to arise often; if an artist wanted to draw herfeet (which presumably are directly below her eye), she would tilt the picture planerather than leaving the canvas vertical. A much more frequent artistic conundrumis that sometimes the image X′ appears to exist even though the object X does not:this situation arises in the case of the well-known vanishing point. The vanishingpoint where the two railroad tracks appear to meet together on the horizon plays anextremely important role in a perspective picture, even though there is no such pointin the real world.

6 A. Crannell

Ideal Points

To counteract both of the above difficulties with single solution, mathematiciansexpanded the notion of Euclidean space to a larger space; if we use analyticproperties such as coordinates in this space, we call it “projective space” (P 3(R)),or if we use purely geometric properties, we call it “Extended Euclidean space”(E3). This larger space includes not only all the familiar points in R3, but also anadditional set of points called ideal points (or sometimes points at infinity). In thespaces P 3(R) and E3, we must alter our conception of parallel lines; in particular,lines in R3 that are parallel meet in E3 at an ideal point. We will delve further intoP 3(R) in section “Homogeneous Coordinates”; until then, this text will only needthe geometric properties of E3.

Ideal points are created by what we call a formal definition, meaning that thedefinition itself “forms” the object. This kind of definition is different than one thatmerely identifies an existing object: we could define

√2 to be “the positive real

number x with the property that x2 = 2.” The definition of √2 is not a formaldefinition, because such a number already exists in R. But the definition of idealpoints creates something new, in the same way that defining the imaginary numberi to be “a number z with the property that z2 = −1” creates something that does notexist in R, leading to the formation of the complex plane C. In the same way, thespace E3 is larger than and has different properties from R3.

In particular, in E3, every line and plane intersect in a point (unless the line isa subset of the plane, in which case their intersection is a line). This means that ifthe center O is not a subset of the image plane ρ and if O �= X, the image pointX′ = (OX) · ρ is always well defined.

Similarly, two lines in E3 are coplanar if and only if they intersect in exactly onepoint. In this sense, as we noted above, “parallel” lines are coplanar and intersect inan ideal point. Artistically speaking, the existence of ideal points as the intersectionof parallel lines allows us to say that if X′ is a vanishing point in our picture, thenthe object X that it portrays exists and is a point “at infinity.”

Because vanishing points play such a crucial role in understanding perspectivepictures, it is worth looking at these objects more carefully.

Vanishing Points

In the same way we say someone is a “parent” when that person is the parent ofsome child or group of children, a “vanishing point” is always a vanishing pointof some line or collection of lines. An examination of Fig. 3 shows that the line �appears to vanish when the artist at point O is looking parallel to �; it follows that apoint V ∈ ρ is the vanishing point for the line � if and only if

OV ‖ �.


r

�

O

A B C DE

F

A′

B′

C′

D′

E ′

F ′

V

Fig. 3 Points on the line � project to the plane ρ from the center O. Points A and B project to A′and B ′ like a camera with O like a lens; points C and D project to C′ and D′ like shadows with Olike a light source; points E and F project to E′ and F ′ like drawing on a window, with O like theartist’s eye. The line OV is parallel to �; we say V is the vanishing point of �

As we noted above, the vanishing point V is the image of the ideal point (the pointat infinity) on �.

It follows that if several lines �1, �2, �3, . . . are parallel to one another, then theline OV is parallel to all of them if and only if OV is parallel to any one of them,so the lines �1, �2, �3, etc. all have the same vanishing point. If the lines �1, �2, �3,etc. are parallel to one another but not parallel to the picture plane, it follows that Vis a real (rather than ideal) point, so their images �′1, �′2, �′3, etc. are not parallel butrather all intersect at that point V (giving us, e.g., the drawing of the railroad tracksthat converge at a point in the horizon). If the lines are parallel to one another andalso parallel to the picture plane, then OV is likewise parallel to the picture plane,implying V is an ideal point, and so the images �′1, �′2, �′3, etc. will all be parallel toeach other (as well as to the original lines).

Note that this definition of vanishing point implies something significant aboutinterpreting a piece of art. If we know something about a set of lines (say, we caninfer that the lines in the road were running perpendicularly to the canvas), and wecan locate the vanishing points of those lines on the canvas, then this means weknow something about the location O of the artist, and this location is somethingwe explore further in the next section.

8 A. Crannell

Where Was the Camera?

In the previous section, we claimed that the location of vanishing points helpsus determine the location of the artist or camera that made the picture. In thissection, we explore the implications of this claim. Determining the location of anartist or of a camera is the source of a good amount of mathematical inquiry (see,e.g., references Byers and Henle 2004, Crannell 2006, Futamura and Lehr 2017,Robin 1978, and Tripp 1987). Moreover, the methods for solving this questionlead to multiple applications, as we will see in the sections that follow (See also� “Geometries of Light and Shadows from Piero della Francesca to James Turrell”).

Here we give one simple example of using geometry to locate the originalposition of an artist: a standard, back-of-the envelope calculation that uses twovanishing points to determine the original viewing location. Figure 4 shows apainting from circa 1730 of the Clock Tower in the Piazza San Marco, a notedtourist attraction. The artist, Giovanni Antonio Canal (better known as “Canaletto”),was noted for his realistic city scapes; he often used a camera obscura to projectimages onto canvas where he would capture them in paint. As such, his works giveus excellent examples of perspective projections.

In Fig. 4 we can see that images of vertical lines in the Piazza have vertical imageson Canaletto’s canvas. Likewise, the horizontal lines in the front face of the clocktower building also have horizontal images. This tells us that Canaletto’s canvas was

Fig. 4 “The Clock Tower in the Piazza San Marco”, Canaletto (circa 1730)

http://link.springer.com/``Geometries of Light and Shadows from Piero della Francesca to James Turrell''


Fig. 5 The vanishing point V of those lines that are perpendicular to the canvas shows thatCanaletto painted this canvas from a second-story location. The vanishing point D the diagonalline of a vertical rectangle lies directly above V

set up parallel to that face of the building. We can deduce that a third set of linesdepicted in the picture run perpendicularly to the canvas. Figure 5 shows that theselines have images that converge at a point V in the second floor of the building, nearthe main doorway and below the clock in the painting.

Because this third set of real-world lines are perpendicular to the canvas, itfollows that Canaletto was perpendicularly across from the point V depicted in thepicture – in other words, he was not standing on the ground, but was stationed onthe second floor of another building.

But we can be even more specific. The picture also contains clues that help usdeduce his horizontal distance from the clock tower. On the right side of the plazais a building with semicircular arches. Around one of these arches, we can draw theimage of a rectangle that is twice as long as it is high. We draw the image of thediagonal line through this rectangle, which has vanishing point D directly above V(see Figs. 5 and 6). Because the slope of the real-world line is 1/2, the geometry ofsimilar triangles allows us to deduce that the viewing distance (the distance fromCanaletto to the canvas) is twice the length of the segment V D. Assuming the clocktower to be approximately 70 ft tall (based on its height relative to the people in thepicture), we get that the height of the clock tower appears to be 35% the length of‖V D‖, so Canaletto was approximately 200 ft from the clock tower.

10 A. Crannell

O

d

V

D

C

rface of clock tow

er

Fig. 6 A side view showing the location O of the artist and the picture plane ρ. The point V ∈ ρis both the image of a point C on the second floor of the clock tower and also the vanishing pointfor lines perpendicular to the picture plane ρ. The line d has vanishing point D ∈ ρ, so OD ‖ d;therefore, the slope of OD is 1/2

In conclusion, a few standard assumptions about Canaletto’s world (buildingswere constructed with right angles, the arches were semicircles, and people wereapproximately the same height they are today) allow us to reconstruct the locationof that artist as he painted this picture 300 years ago.

A Consequence of Viewing Distances: Illusion, Distortion, andAnamorphism

Understanding where the artist stood is more than a historical exercise; it also has thepower to affect how we view photographs and the apparent distortion within them.Almost every person has had the experience of seeing a breathtaking vista and tryingto capture it on camera, only later to lament that the photograph didn’t do justice tothe power of the original view. Often, the problem is not with the mechanics of thephotograph or the photographer, but with the small size of the image coupled withthe too-far distance of the person looking at the photograph. If the photograph werelarger, or if its viewer were closer, the sense of awe for the vista might return.

Figure 7 gives an example of why the size of a photograph, a movie screen, or areproduction of a perspective painting matters. Good perspective artists often placetheir vanishing points far off the picture because doing so “reduces distortion.” InFig. 7, we have instead sized the drawing in such a way that the vanishing pointsare readily apparent on the page (like consolidating a magnificent scenic view intoa photo that is only as wide as a phone or a laptop). Notice that the word “LIFE”appears to be highly distorted. In particular, the bottom corner of the “L” has an


×

Fig. 7 “LIFE” in two-point perspective, with vanishing points indicated on the horizon. The nearbottom corner of the “L” has an angle of 48◦, but if you look at the picture with one eye from veryclose to the × on the horizon line, the angle appears to be “correct”; that is, it appears to be 90◦

angle in the drawing of 48◦, even though this vertex is supposed to represent a right-angled corner. We could have made the corner appear more like a right angle byplacing the vanishing points further apart. But surprisingly, we can also make thecorner appear more like a right angle by moving ourselves closer to the drawing. Ifa viewer moves uncomfortably close to this picture (in particular, if a person lookswith one eye from a location very close to the × on the horizon), the angles in theword appear to be correct, 90◦ angles.

Figure 8 explains why moving our eye close to the image helps the picture appearmore realistic. The viewer at O1 is far from the image – just as most readers of thischapter will view “LIFE” in Fig. 7 from a comfortable distance. The lines of sightto the two vanishing points for the viewer at O1 form an acute angle θ . Recall thatwhen an artist draws a scene through a window, the vanishing points in the pictureplane will lie on those lines of sight that are parallel to the lines she is drawing inthe “real world.” Therefore, for the viewer at O1, the drawing appears to depict anobject that is likewise formed by the acute angle θ .

On the other hand, the viewer at O2 is closer to the picture plane, at a place wherethe lines of sight from O2 to the vanishing points are perpendicular. Therefore,

12 A. Crannell

×

r

O2

O1

q

q

Fig. 8 A top view showing two viewers looking at the picture plane ρ. The lines of sight from theviewers to the two vanishing points are parallel to the lines of the objects they appear to see in the“real world”; hence the viewer at O1 sees a diamond-shaped object, while the viewer at O2 sees arectangle

for this viewer, the drawing appears to depict an object in the real world formedby lines that are likewise perpendicular to one another. In other words, if thedrawing is supposed to depict an object with right angles, the closer viewer seesan “undistorted” picture, whereas the further viewer sees a distorted image.

The reason our photographs don’t capture what we remember seeing is notbecause the camera messed up; it is because we view the small photographs fromtoo far away. Enlarging the photos or moving closer to the photos will restore theillusion of depth.

Dolly Zoom

Cinematographers make effective use of altered viewing distances to create adistinctive mood. Figure 9 shows one of the most effective and common of these:a movie camera technique called the dolly zoom. (The dolly zoom has many othernames – including the Hitchcock zoom, because it first appeared in that director’sfilm Vertigo when it was pioneered by cameraman Irmin Roberts.) In this zoom, thecamera is placed along a track and pulled backward while simultaneously zoomingin on the figure in the foreground.

The effect of this technique appears in Fig. 10. When the camera is close, eventhough the house and tree are large objects, they are far from the camera, so thenearby person seems relatively large compared to the background objects. But asthe camera zooms in on the person’s face while simultaneously drawing backward,


Fig. 9 Side views showing the camera up close, and then drawn back while zoomed in. Note thatthe image of the house in the distant background grows larger relative to the image of the person

Fig. 10 In the first figure, the camera is close, so the nearby person seems relatively largecompared to the background objects; in the second figure, the camera zooms in on the person’sface while simultaneously drawing back, so that the background objects seem to swell ominously

the background objects seem to swell in size. The effect is to make the world appearto loom large, giving that short scene a disturbingly ominous feeling.

If the camera pulls back slowly (as in a diner scene in Goodfellas), thepsychological effect is one of creeping unease. The audience is aware of somethingbeing not quite right, but can’t quite place the source of trouble. Often, however, thecamera zooms back quickly: in The Lord of the Rings: Fellowship of the Rings, asFrodo stands on a road, the accompanying dolly zooms last a fraction of a second,evoking a feeling of terror. It’s no surprise, then, that Michael Jackson’s Thrillervideo ends with a similar, speedy zoom! These sudden zooms are technicallydifficult and costly, but clearly they are worth the expense and effort to the directors

14 A. Crannell

of these films. See Boing Boing (2015), for example, for a video clip purporting tobe “23 of the best dolly zooms in cinematic history.”

Of course, the effect can be reversed (even with a virtual camera); at the end ofFiona’s battle with Robin Hood’s men in the animated movie Shrek, there is a split-second reverse dolly zoom, giving the sudden impression that the battle is over andall is right with the world.

Anamorphic Art

The word “LIFE” in Fig. 7 looks moderately distorted because of the unusuallyclose viewing distance, but the word is still recognizable because the viewing target(at the “×”) is centered on the horizon. That is, if we hold this picture in front ofus, we’ll be centered on the viewing target; the distortion comes solely from thedistance between our eye and that target.

Perspective techniques allow artists to create even more significant illusions bylocating the viewing target close to an edge of the canvas – or even off the edge of thecanvas. One of the most famous examples of this technique, called anamorphism,appears in a 1533 painting by the German-born artist Hans Holbein the Younger.The Ambassadors (Fig. 11) appears to show a wealthy landowner and a Bishopsurrounded by objects both secular and religious. Toward the bottom of the paintingis an odd gray-and-black smear; this smear is in fact meant to be viewed from theextreme right edge of the painting. A viewer standing at this extreme angle wouldnot be able to see the men and their possessions clearly, but would clearly be able tosee a skull hidden in plain view within the painting (Fig. 12).

Anamorphic art is hardly confined to the sixteenth century; it abounds todayin curated museum shows, in public spheres (e.g., in the New York subwaysystem), and in art-gone-viral (just perform an Internet search for the sidewalk chalkartist, Julian Beever, sidewalk art (Beever 2019)). See � “Anamorphosis: BetweenPerspective and Catopritics” for a fuller treatment of the topic. Anamorphism has itspractical aspects, too: turn arrows painted on roadways look highly distorted whenseen from directly above but appear correct to the drivers approaching along theroad. There are parking garages that paint anamorphic exit signs, which make senseto the cars needing to leave the building but appear to be a jumble otherwise.

Impossible Figures

The above examples show how perspective art can “hide” or distort the imageof a real-world, three-dimensional object within a two-dimensional canvas. Butperspective art can also make unreal objects appear to exist. One famous exampleof such an example is the eponymous Penrose triangle (Fig. 13), popularized in the1950s by the father-son team of Lionel and Sir Roger Penrose, a psychologist andmathematician.

http://link.springer.com/``Anamorphosis: Between Perspective and Catopritics''


Fig. 11 The Ambassadors by Hans Holbein the younger (Holbein, 1533)

This triangle is one of the simplest and most iconic examples of what we call“impossible figures.” Locally, at each corner of the object, this appears to be theimage of a solid three-dimensional object made of flat surfaces with linear edges.But the object as a whole contradicts the local analysis. For example, as we travelaround the object counter-clockwise, each subsequent corner appears to be closer tothe viewer than the previous one – an impossibility in a closed loop!

Many artists include “impossible figures” in their work, including Swedish artistOscar Reutersvard – who is credited with the 1930s discovery of the triangle thatwould later bear the Penrose name – and M.C. Escher, whose Waterfall, Ascendingand Descending, Belvedere (among many others) have captivated and perplexedgenerations of curious viewers.

For this reason, it’s especially interesting that artists have created three-dimensional statues depicting impossible figures – see, for example, Fig. 14.

16 A. Crannell

Fig. 12 Viewed from theextreme right and close to thecanvas, the smear on TheAmbassadors appears to be askull

Fig. 13 A Penrose triangle isan “impossible figure”

These statues, even more than their two-dimensional counterparts, require a strictalignment with a particular viewing position for the illusion to be effective.

The observation that the same object (such as the Penrose Triangle sculptureabove) can have very different appearances when viewed from two differentlocations is one of the reasons that reconstruction of three-dimensional objects fromtwo-dimensional photographs is such a challenging one. This challenge is the focusof the next section.

Going Backward from Pictures to 3D

In the centuries that saw Desargues, Canaletto, and Poncelet, the task of drawingaccurate images and maps was a significant technological challenge. But in today’s


Fig. 14 Two views of a Penrose Triangle sculpture at the Deutsches Technikmuseum, Berlin,February 2008 (Deutsches-Technikmuseum, 2008)

Fig. 15 A reconstruction of the Colosseum from photographs uploaded to Flickr (Agarwal et al.,2011)

world – where cameras are built into cell phones – accurate images surround us. Theubiquity of digital images has allowed us to attempt new technological challengesof our day: to recreate a three-dimensional world from a collection of photographs.

See for example Fig. 15, the lead figure from a highly cited paper entitled“Building Rome in a Day” by Agarwal et al. (2011). In this rendering of theColosseum, each triangle in the picture is the location of one of more than 2000cameras that had uploaded photos to Flickr.

The authors describe their work in this way:

Entering the search term “Rome” on Flickr returns more than two million photographs. Thiscollection represents an increasingly complete photographic record of the city, capturingevery popular site, facade, interior, fountain, sculpture, painting, cafe, and so forth. Italso offers us an unprecedented opportunity to richly capture, explore and study the threedimensional shape of the city.

We are all familiar with computer games that allow us to move through a virtual3D world, and also with online sites (such as Google Maps) that allow us to virtually“move” through city streets while seated at our computers. These newly familiarexperiences rely on already knowing the structure of space. Virtual gaming worlds

18 A. Crannell

have a three-dimensional structure already encoded into the software; Google mapstakes images from satellite or roving, calibrated cameras with GPS coordinatesencoded into the image.

What makes the work of Agarwal (etc.) a geometrical challenge is the almostcomplete absence of a priori geographic or spatial information. Piecing the worldback together from a collection of random of photographs is like fitting together ajigsaw puzzle with a million pieces, some of which are missing and many of whichare redundant. (Almost no one takes pictures of the dumpster behind the grocerystore; millions of people take photographs of a famous statue.)

Reconstructing three-dimensional objects from a collection of photographsrequires going through these three steps:

1. identifying feature points or lines that match across images;2. doing a reconstruction from pairs or possibly triplets of images; and3. piecing together and refine these many reconstructions using optimization.

The first step requires careful use of a cluster of computers, one of which isdesignated as the “master node” that distributes images to individual computers(nodes) in a balanced manner. The nodes each toil away at pre-processing imagesby verifying they are readable and extracting available camera information (if any isattached). The process of matching images is not entirely random; in the same waymost people begin solving a jigsaw puzzle by looking for edge pieces, the matchingalgorithm uses a library of SIFT (Scale Invariant Feature Transform) features.

Likewise, the third (final) step uses intense use of computational algorithms,outside the scope of this chapter.

Step two is where projective geometry comes in; this step requires “undoing”the kind of perspective map that Desargues and Caneletto mastered long ago. Thisstep is the basis for the field of multiple view geometry, an increasingly fertile areaof research for theoretical and applied mathematicians alike. Indeed, the authorwas introduced to this subject at an energetic week-long gathering of universityprofessors and Google engineers at a conference on Algebraic Vision hosted by theAmerican Institute of Mathematics in summer 2016.

To describe the locations of real-world points and their photographic imagesin a way that is amenable to computer algorithms, we will need to understandhomogeneous coordinates for space; that is the subject of the next section.

Homogeneous Coordinates

To motivate the use of homogeneous coordinates (as contrasted with Cartesiancoordinates), we return to the notion of an observer positioned at the origin 0 =(0, 0, 0) ∈ R3, gazing at the world through a picture plane. To this observer, everypoint along a given line of sight will map to the same point on the picture plane. In


particular, points (x, y, z) and (λx, λy, λz) have the same image whenever λ �= 0. Ifthe picture plane is z = 1, for example, then both of these points map to

(x

z,y

z, 1

).

In this vein, we form P 2(R), the projective plane, as equivalence classes of pointsin R3\{0}. A point in P 2(R) can be written in homogeneous coordinates in the form[x : y : z]T for (x, y, z) ∈ R3 \ {0}; we say

⎡⎣xy

z

⎤⎦ =

⎡⎣λxλy

λz

⎤⎦

whenever λ �= 0. Just as points in P 2(R) correspond to real lines through the origin;lines in P 2(R) correspond to real planes through the origin. Said another way,projective points [x1 : y1 : z1]T , [x2 : y2 : z2]T , and [x3 : y3 : z3]T are collinearin P 2(R) precisely when real points (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3) arecoplanar in R3.

The projective plane P 2(R) and the Extended Euclidean plane E2 (see section“A New Mathematical Object: The Point of Projective Geometry”) have a naturalcorrespondence. If we think of E2 as the extension of the particular plane z = 1, thenwe can identify the projective point [a : b : c]T with the ordinary point ( a

c, b

c, 1)

whenever c �= 0; projective points of the form [a : b : 0]T correspond to ideal pointsin E2. This makes some intuitive sense, as these correspond to the observer’s linesof sight that are parallel to the picture plane, and so “intersect” the plane z = 1 “atinfinity”.

We define P 3(R) analagously: projective points take the form

[x : y : z : w]T = [λx : λy : λz : λw]T ∈ P 3(R)

for (x, y, z,w) ∈ R4 \ {0} and λ �= 0. As before, we can find a naturalcorrespondence between P 3(R) and E3 (say, via the identification using w = 1).These homogeneous coordinates underlie much of the field of analytical projectivegeometry.

To understand how the use of homogeneous coordinates helps us understandcamera projections, consider the case of an observer standing at [0 : 0 : 0 : 1]T ,looking through a planar window located at z = d, w = 0, which we think ofas an embedding of P 2(R) ⊂ P 3(R). To such an observer, the point [x : y : z : w]Twould have an image on the window located at

[dx

z: dy

z: d

]T= [dx : dy : z]T .

That is, we can compute the transformation P 3(R) → P 2(R) above via the matrixmultiplication

20 A. Crannell

Fig. 16 Using a spreadsheet to draw the perspective image of a cube with viewing distance 4 andviewing target (2, 7)

P

⎡⎢⎢⎣

x

y

z

w

⎤⎥⎥⎦ =

⎛⎝ d 0 0 00 d 0 0

0 0 1 0

⎞⎠

⎡⎢⎢⎣

x

y

z

w

⎤⎥⎥⎦ =

⎡⎣dxdy

z

⎤⎦ .

The computation above shows why algebraic geometers define a camera to be a3×4 matrix. Moving the viewer, shifting the film, rotating the image plane, or usinga camera with non-square pixels has the effect of changing the entries of the cameramatrix P . (See (Hartley and Zisserman, 2003, Chapter 6) for a fuller description.)

Figure 16 demonstrates putting this into practice in a rather simple spreadsheet.In this sheet, we draw the one-point perspective image of a cube; the viewingdistance is 4 and the viewing target is (2, 7).

Multiple View Geometry

How do we recover information about a three-dimensional world from two-dimensional images?

Suppose we have two images of the same real-world object. Usually, one ofthe first steps in reconstruction of the 3D scene is to determine what is called thefundamental mapping taking points in the first image α to a certain set of lines inthe second image β. The description below explains how and why this mappingemerges.


We say points xα ∈ α and xβ ∈ β are corresponding points if they are images viathe appropriate maps of a common point X ∈ P 3(R). That is, X projects onto xα ∈α from the point Oα , and X projects onto xβ ∈ β from the point Oβ . Then the fivepoints X, xα , Oα , xβ , and Oβ are necessarily coplanar. Note that the line (OαOβ)– called the epipolar line – lies in every such plane constructed from correspondingpoints. Of particular interest along this line are the epipolar points eα = α · (OαOβ)and eβ = β · (OαOβ). We can think of eα as the image in α of the camera at Oβ ,and eβ as the image in β of the camera at Oα .

The point xα might correspond to several different points in the plane β. Forexample, the camera at Oα might appear to show a tree growing out of a person’shead: the point xα could come from both the person’s hat and the trunk of thetree. The images of the hat and trunk in another photograph β might not coincidewith each other, but because of the coplanar relationship described in the previousparagraph and illustrated in Fig. 17, they must be collinear with the epipolar pointeβ . Accordingly, a pair of photographs of the same scene, taken from two differentcamera locations, describe a function from points xα in α to lines (eβxβ ) in β. Thisfunction is called the fundamental mapping.

Because xα and xβ can be thought of as points in P 2(R), we can representthe fundamental mapping with a 3 × 3 matrix F , called the fundamental matrix.In general, we can determine F from 7 pairs of corresponding points in generalposition (the matrix is a rank-2 matrix and therefore has 7 degrees of freedom). Foreach of these corresponding pairs of points, the mapping satisfies Fxα = (eβxβ);that is to say,

Oa Ob

X

b

a

ea

xaxb

eb

Fig. 17 The point X and its images xα and xβ lie in a plane with the line containing the centers(Oα and Oβ ) and the epipoles (eα and eβ )

22 A. Crannell

xTβ Fxα = 0.

In the previous section, we described a camera as a 3 × 4 matrix. If we have twoimages α and β, then the fundamental matrix allows us to describe a relationshipbetween the two cameras Pα and Pβ which created the two images. Why is this?For any point X ∈ P 3(R), we have

(XT P Tβ )F (PαX) = (xTβ )Fxα = (xTβ )(eβxβ) = 0.

Therefore, it follows that P Tβ FPα is a skew-symmetric matrix. This fact is a foot-in-the-door for developing reconstruction algorithms.

How, then, do we use the fundamental matrix to reconstruct the real-world scene?The answer is not simple, as the figure of the Ames room below shows.

The Ames Room

The Ames room, designed by perceptual psychologist Adelbert Ames, Jr., is anillusion room. Viewers who peer into the room from a peephole in the wall seemto see objects that grow and shrink as the objects move from one side to the other.The illusion works because from the correct vantage point, the room appears to be a“normal,” rectangular room. But in fact, the walls, ceiling, and floors are trapezoids,with the short edges close to the vantage point and the long edges far from thevantage point. The illusion that the room is rectangular, and not trapezoidal, can behard to overcome, even when viewers have been inside the room or see people theyknow walking through it, appearing to shrink or grow as they walk (Fig. 18).

Said another way, the Ames room is projectively equivalent to a normal room;there is a collineation P 3(R) → P 3(R) (a function that takes points to points andlines to lines) that maps the Ames room onto a normal, rectangular room. For thisreason, the methods described above can determine the relationship between twocameras – and thereby the reconstruction of the three-dimensional scene – only upto projective equivalence. The fundamental mapping by itself can help us distinguishbetween an Ames room and an A-frame house, but it can’t tell an Ames room from aregular rectangular room. We can’t extract distance or angle measurements of real-world objects without a priori information about the scene or the cameras.

Reconstructing Objects from Images

Knowing real-world information vastly increases the ease with which we canreconstruct objects from images. A “calibrated camera” makes the reconstructionprocess much simpler. For instance, many modern digital cameras come availablewith GPS information encoded into the image. For even more accuracy, many 3Dscanners use a known camera that is a fixed distance from a turntable rotating at


Fig. 18 Ames room: “Room constructed to make a person appear large or small depending onperspective, in the city of Rio de Janeiro, Brazil.” (Courtesy of Andrevruas) (Andrevruas, 2011)

known angles. Knowing the focal length of the camera allows us to account forphenomena such as the dolly zoom; knowing the viewing target allows us to accountfor anamorphic effects (see section “A Consequence of Viewing Distances: Illusion,Distortion, and Anamorphism”).

Real-world information is useful as well. Note that in analyzing Canaletto’spainting in section “Where Was the Camera?,” we used standard observations aboutreal-world parallel lines, and also about real-world perpendicular lines, to gaininformation about Canaletto’s viewing position. In general (meaning, if the sceneis not an Ames-room-like scene), this kind of assumption means that reconstructingscenes with architectural features is simpler than, say, reconstructing landscapes. Wecan see the importance of knowing such geometric information for understandingdrawings like “LIFE” (Fig. 7) or the Penrose sculpture (Fig. 13).

In the analysis of Canaletto’s painting, we also used information about propor-tions (by assuming the arch was a semicircle) and about actual size (e.g., the heightsof the people pictured). This kind of detective work is another part of reconstruction;without it, we can’t distinguish between photographs of, for example, a single-family home and a doll’s house.

24 A. Crannell

In practice, the task of reconstruction is further complicated by “noise” and error:points are infinitesimal, but pixels are discrete and finite. So optimization and erroranalysis also enter into reconstruction algorithms.

Nonetheless, at the heart of any reconstruction lies the language of homogeneouscoordinates and analytical projective geometry.

Conclusion

The long and storied history of projective geometry weaves itself through thelast half-millennium of mathematics; it is a subject that has been discovered andrediscovered by mathematicians searching for answers beyond Euclidean geometry.Its reemergence under Poncelet points to the aesthetic elegance of its axiomaticstructure; the subject has also led to deeper understandings of conics (e.g., under theinfluence of Steiner) and of topology (e.g., under Möbius).

But its utility in perspective drawings and photographs is where the subjectof projective geometry becomes most applied and touches our lived experiencesmost directly. With the passing of time, this tool is becoming even more relevantand powerful than when Desargues first introduced it. We live in a world that isincreasingly visual, a world in which technology creates, reproduces, and altersimages constantly; analytic projective geometry is the machinery that allows us tocreate, explain, and analyze these digitized images.

Beyond the technical aspect of analyzing digital images, constructive projectivegeometry gives us all a way to see our surroundings and the objects in them: tobetter understand how to look at paintings or our vacation photographs, to createor to dispel illusions, and to interpret the way we look at our wonderful, three-dimensional world.

Cross-References

�Anamorphosis: Between Perspective and Catopritics�Geometries of Light and Shadows from Piero della Francesca to James Turrell�Renaissance Architecture

References

Agarwal S, Furukawa Y, Snavely N, Simon I, Curless B, Seitz SM, Szeliski R (2011) BuildingRome in a day. Commun ACM 54(10):105–112. With a Technical Perspective by Prof. CarloTomasi

Andersen K (2006) The geometry of art: the history of the mathematical theory of perspective fromAlberti to Monge. Springer, New York

Andrevruas (2011) Português: Casa construída de forma a fazer a pessoa parecer grande oupequena dependendo da perspectiva, na cidade do Rio de Janeiro, 24 Jan 2011. https://commons.wikimedia.org/wiki/File:Casaperspectiva.jpg, from Wikimedia Commons

http://link.springer.com/Anamorphosis: Between Perspective and Catopriticshttp://link.springer.com/Geometries of Light and Shadows from Piero della Francesca to James Turrellhttp://link.springer.com/Renaissance Architecturehttps://commons.wikimedia.org/wiki/File:Casaperspectiva.jpghttps://commons.wikimedia.org/wiki/File:Casaperspectiva.jpg


Beever J (2019) Julian Beever’s website. http://www.julianbeever.net/Boing Boing (2015) Watch 23 of the best dolly zooms in cinematic history, 26 Jan 2015. https://

boingboing.net/2015/01/26/watch-23-of-the-best-dolly-zoo.htmlBosse A (1648) Manière universelle de Mr. Desargues, pour pratiquer la perspective par petit-pied,

comme le Geometral, ParisByers K, Henle J (2004) Where the camera was. Math Mag 77:4:251–259Canaletto GA (circa 1730) The Clock Tower in the Piazza San Marco. https://commons.wikimedia.

org, oil on canvas, 69.22 × 86.36 cm, current location at the Nelson-Atkins Museum of ArtCarroll L (1871) Through the looking-glass. Macmillan & Co, LondonCrannell A (2006) Where the camera was, take two. Math Mag 79:4:306–308Desargues G (1987) Exemple de l’une des manieres universelles du s.g.d.l. touchant la pratique

de la perspective sans emploier aucun tiers point, de distance ny d’autre nature, qui soit horsdu champ de l’ouvrage. In: The geometrical work of Girard Desargues. Springer, New York,p 1636

Deutsches-Technikmuseum (2008) Penrose triangle sculpture. https://commons.wikimedia.org/w/index.php?curid=3597501, images from Wikimedia Commons

Futamura F, Lehr R (2017) A new perspective on finding the viewpoint. Math Mag 90(4):267–277Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, 2nd edn. Cambridge

University Press, New YorkHolbein H (1533) The Ambassadors. https://commons.wikimedia.org, oil on oak, 209.5 cm×

207 cmRobin AC (1978) Photomeasurement. Math Gaz 62:77–85Taylor B (1719) New principles of linear perspective: or the art of designing on a plane the

representations of all sorts of objects, in a more general and simple method than has been donebefore, London

Tripp C (1987) Where is the camera? The use of a theorem in projective geometry to find from aphotograph the location of a camera. Math Gaz 71:8–14

http://www.julianbeever.net/https://boingboing.net/2015/01/26/watch-23-of-the-best-dolly-zoo.htmlhttps://boingboing.net/2015/01/26/watch-23-of-the-best-dolly-zoo.htmlhttps://commons.wikimedia.orghttps://commons.wikimedia.orghttps://commons.wikimedia.org/w/index.php?curid=3597501https://commons.wikimedia.org/w/index.php?curid=3597501https://commons.wikimedia.org

Looking Through the GlassContentsIntroductionA Brief HistoryA New Mathematical Object: The Point of Projective GeometryIdeal PointsVanishing Points

Where Was the Camera?A Consequence of Viewing Distances: Illusion, Distortion, and AnamorphismDolly ZoomAnamorphic ArtImpossible Figures

Going Backward from Pictures to 3DHomogeneous CoordinatesMultiple View GeometryThe Ames RoomReconstructing Objects from Images

ConclusionCross-ReferencesReferences

looking through the glass · 2019-06-08 · looking through the glass annalisa crannell contents...

Documents