Visual Intelligence Visual Intelligence Chapters 6 & 7Chapters 6 & 7

When the World Stopped When the World Stopped MovingMoving

Tiffany S. WilliamsTiffany S. Williams

44thth Year Drama and Year Drama and Psych major.Psych major.

Believes she may Believes she may be be quadrichromatic.quadrichromatic.

Borna RangchiBorna Rangchi

44thth Year Psych Year Psych MajorMajor

Believes Tiffany Believes Tiffany may be may be quadrichromatic.quadrichromatic.

How could you possibly How could you possibly see an object but not its see an object but not its

motion?motion?

This Question would have This Question would have puzzled John Locke!puzzled John Locke!

Asserted that body Asserted that body and motion were and motion were inseparable and inseparable and that body and its that body and its color are color are separable. separable.

Motion is primary Motion is primary and inseparable.and inseparable.

The Case of L.M.The Case of L.M.

Had “Had “AKINETOPSIAAKINETOPSIA” – brain ” – brain damage resulting in the inability to damage resulting in the inability to perceive moving objects, despite perceive moving objects, despite stationary objects remaining more or stationary objects remaining more or less visible.less visible.

Loss of motion is permanent. Loss of motion is permanent. Akinetopsia would have been a hard Akinetopsia would have been a hard

concept for Locke to comprehend. concept for Locke to comprehend.

Motion is Constructed!Motion is Constructed! Our visual intelligence constructs Our visual intelligence constructs

motion.motion. Logic = “If you construct objects, then you must Logic = “If you construct objects, then you must

construct their motions. If what moves is your construct their motions. If what moves is your construction then how it moves must be as well. The construction then how it moves must be as well. The what and how of motion are, of necessity, intimately what and how of motion are, of necessity, intimately linked. linked.

How can we experience the loss of How can we experience the loss of motion without brain damage?motion without brain damage?

TMS (TRANSCRANIAL MAGNETIC TMS (TRANSCRANIAL MAGNETIC STIMULATION)STIMULATION) – process where anyone – process where anyone can experience a brief loss of motion can experience a brief loss of motion through the use of magnetic fields that through the use of magnetic fields that impair normal electric function in V5. impair normal electric function in V5.

Motion is ConstructedMotion is Constructed

Need Some Proof Do Need Some Proof Do You?You?

Phi Motion (Sigmund Phi Motion (Sigmund Exner; 1875)Exner; 1875) Two dots are flashed, Two dots are flashed,

one after the other.one after the other. With the correct With the correct

interstimulus interval interstimulus interval (time and space (time and space between dots), two between dots), two objects are transformed objects are transformed into one moving object.into one moving object.

If ISI is between a half If ISI is between a half of a second and a tenth of a second and a tenth of a second, the illusion of a second, the illusion of motion is created by of motion is created by your visual intelligence. your visual intelligence. The precise range of The precise range of ISIs depends on the ISIs depends on the distance between dots.distance between dots.

Flashed: 200ms

Blank: 400ms

Flash: 200ms

Blank: 50ms

Flash: 10ms

Blank: 0ms

Implications of “Phi” Implications of “Phi” MotionMotion

Objects and motions are constructed Objects and motions are constructed interdependently.interdependently.

The construction of motion depends The construction of motion depends on the construction of objects and vice on the construction of objects and vice versa.versa.

ex. If your visual intelligenceex. If your visual intelligence opts to opts to construct one object, construct one object, it will move across the it will move across the screen rather than two separate screen rather than two separate blinking blinking objects.objects.

Color Phi MotionColor Phi Motion Color Phi Motion (Max Color Phi Motion (Max

Wertheimer; 1912)Wertheimer; 1912) Same as Exner’s Same as Exner’s

experiment but with experiment but with different color dots.different color dots.

If ISI is appropriate, you If ISI is appropriate, you should see the first half should see the first half of the motion represent of the motion represent the color of the first dot, the color of the first dot, and the second half of and the second half of the motion represent the the motion represent the color of the second dot.color of the second dot.

What about the What about the sequence of events? sequence of events?

Motion Over Curved Motion Over Curved PathsPaths

With the right ISI the dot should With the right ISI the dot should move over the arc.move over the arc.

Curved Motions Curved Motions ContinuedContinued

Curved motions can be constructed when Curved motions can be constructed when no curve is flashed.no curve is flashed.

• A single dot is constructed moving left to right in a curved path above of below the box.

• No fancy loops or twists

• From this we can infer that motions are constructed on minimal (simple) paths and motions.Rule 29: Create the simplest possible motions.

Two Dots at a TimeTwo Dots at a Time

Long ISI = Group Long ISI = Group MotionMotion Group Motion: Two dots move Group Motion: Two dots move

together rigidly from left to right.together rigidly from left to right. Long Range MotionLong Range Motion

Short ISI = Element Short ISI = Element MotionMotion Dot in the middle stays put, and the Dot in the middle stays put, and the

dot on the left jumps around it from dot on the left jumps around it from left to right.left to right.

Short Range MotionShort Range Motion

Group Motion

Element Motion

More Group/Element More Group/Element MotionMotion

Multilayered constructionMultilayered construction

Group Motion Element Motion

Two Dots at OnceTwo Dots at Once

Two different motions Two different motions are constructed.are constructed. Dots move up & down or Dots move up & down or

side to side.side to side. Dots DO NOT merge. Dots DO NOT merge.

Either they BOTH move up Either they BOTH move up & down or they BOTH move & down or they BOTH move side to side.side to side.

Rule 30Rule 30

Rule 30: Rule 30: When making motion, construct as few objects as possible, and conserve them as much as possible.

“These motions make two dots merge into one, and leave one dot unmatched. You prefer to conserve objects as much as possible when you construct motion. You don’t want to make two objects magically turn into one, and have another object magically appear, when instead you could just make two objects total, and conserve them over the two frames.”

-(Hoffman pg. 148)

Coherence of MotionCoherence of Motion

Do you notice that Do you notice that motions of the dots motions of the dots are the same?are the same?

Even when one Even when one switches from switches from horizontal to horizontal to vertical and vice vertical and vice versa, the rest versa, the rest switch in switch in concordance.concordance.

……..which leads us to……..which leads us to……

RULE RULE 31!!!!31!!!!

Rule 31Rule 31

Rule 31: Construct motion to be Rule 31: Construct motion to be as uniform over space as as uniform over space as possible.possible.

RotationsRotations

Clockwise or Counterclockwise?Clockwise or Counterclockwise?

Rotation with “Inertia”Rotation with “Inertia”

Clockwise or Counterclockwise?Clockwise or Counterclockwise?

Circular SquareCircular Square

Given the correct ISI, motion from Given the correct ISI, motion from four points can be interpreted in a four points can be interpreted in a circular fashion.circular fashion.

Look off to the side for the full experience.

Constructing motion Constructing motion depends on the Contextdepends on the Context

Karl Dunker (1929) – illustrated a Karl Dunker (1929) – illustrated a “Cycloid.”“Cycloid.”

See light as bouncing like a ball and not rotating on a See light as bouncing like a ball and not rotating on a wheel. wheel.

Dunker was ambitious and decided to add yet another light, this time with one at the hub and one to the wheel…

2 or more Dots2 or more Dots We will create two types of motion:We will create two types of motion:

1.1. Global Motion – all the dots togetherGlobal Motion – all the dots together2.2. Relative Motion– the motion of each dot relative to Relative Motion– the motion of each dot relative to

the global motion. the global motion.

How do we divide motion into global How do we divide motion into global and/or relative?and/or relative?

James Cutting & Dennis Proffitt (1982) – suggested James Cutting & Dennis Proffitt (1982) – suggested it is to minimize both common and relative it is to minimize both common and relative motions. motions.

Doesn’t predict what you see in Dunker, Rubin, Doesn’t predict what you see in Dunker, Rubin, and Richards displays. “Because you minimize and Richards displays. “Because you minimize common motion by making it uniform translation, common motion by making it uniform translation, but you don’t minimize both common and relative but you don’t minimize both common and relative motions together. If you did, you’d see both dots motions together. If you did, you’d see both dots rotate about a point halfway between them.” rotate about a point halfway between them.”

What’s up with those What’s up with those barbershop poles?barbershop poles?

J.P. Gilford (1929) – J.P. Gilford (1929) – noticed that stripes on noticed that stripes on barber shop poles barber shop poles appear to move appear to move vertically although it is vertically although it is turning on a horizontal turning on a horizontal axis. axis.

This paradoxical motion This paradoxical motion is fundamental in is fundamental in understanding the understanding the APERTURE APERTURE PROBLEM.PROBLEM.

The Aperture ProblemThe Aperture Problem

Rule 32: Construct the smoothest Velocity Rule 32: Construct the smoothest Velocity FieldField

““Velocity Field” – the path of motionVelocity Field” – the path of motion The smoothest path is where the velocity of the motion The smoothest path is where the velocity of the motion

changes the least.changes the least.

So in the Case of the Barber shop pole…So in the Case of the Barber shop pole… The smoothest velocity field has the stripes moving straight The smoothest velocity field has the stripes moving straight

up. up.

The motion you construct at one point depends The motion you construct at one point depends on the motion you construct at nearby points.on the motion you construct at nearby points.

Smoothness is a relation between motions at Smoothness is a relation between motions at different points.different points.

Exploring the Interaction Exploring the Interaction between the Construction of between the Construction of

Objects and MotionObjects and Motion

Ted Adelson & Tony Movshon – A “grating.” Ted Adelson & Tony Movshon – A “grating.”

““motion orthogonal to its lines” – the motion orthogonal to its lines” – the motion direction you always make for motion direction you always make for that particular grating.that particular grating.

Do u see two independent gratings (combo of 1st and 2nd grating) or one unit?

You Coordinate your creations of motions and

objects by intelligently using size, contrastcontrast, depth, and color.

What will happen if the What will happen if the gratings differ?gratings differ?

Differ in size=no Differ in size=no integrationintegration

The greater the The greater the difference the more difference the more likely to see 2 likely to see 2 independent gratingsindependent gratings

Differ in contrast = Differ in contrast = no integrationno integration

The same is true with differences of color and depth.

The POWERFUL Principle The POWERFUL Principle of Rigid Motion!!!of Rigid Motion!!!

Points in space move rigidly if all Points in space move rigidly if all distances between them remain distances between them remain constant during the motion.constant during the motion.

3D Cylinder3D Cylinder Dots appearing at slightly different positions from Dots appearing at slightly different positions from

one frame to the next over a span of 12 frames. They one frame to the next over a span of 12 frames. They appear to take 3D form…different from just a 2D appear to take 3D form…different from just a 2D movie. movie.

Rule 33: If possible, and if other rules permit, Rule 33: If possible, and if other rules permit, interpret image motions as projections of rigid interpret image motions as projections of rigid motions in three dimensions.motions in three dimensions.

Um…ok, so how can you do Um…ok, so how can you do this in practice?this in practice?

Ullman’s Rigidity Theorem: “Suppose you are Ullman’s Rigidity Theorem: “Suppose you are given three frames, each containing at least given three frames, each containing at least four points. If the points are placed at random four points. If the points are placed at random in each frame, then the probability is zero that in each frame, then the probability is zero that they have a rigid interpretation in three they have a rigid interpretation in three dimensions. If the points do have a rigid dimensions. If the points do have a rigid interpretation, then they almost surely have interpretation, then they almost surely have exactly two interpretations (which are mirror-exactly two interpretations (which are mirror-symmetric).”symmetric).”

It means you don’t need that much info to decide It means you don’t need that much info to decide whether or not to create a rigid 3D object and its motion. whether or not to create a rigid 3D object and its motion. All you need is as little as 3 frames each having 4 points! All you need is as little as 3 frames each having 4 points!

Uh…so what does that mean?

This is advancing the field of computer vision systems. At the moment they are good at constructing 3D objects, but not recognizing them.

Limitations of Rigidity Limitations of Rigidity PrinciplePrinciple

Biological MotionBiological Motion – motion of the – motion of the human bodyhuman body

Eg. Black Light Eg. Black Light Theater of Prague.Theater of Prague.

Coming to our own Coming to our own Irvine Barclay Irvine Barclay Theatre March 19Theatre March 19thth and 20and 20thth

UCI’sUCI’s Beall Center Beall Center for Art and for Art and Technology Technology explores the idea of explores the idea of biological motion.biological motion.

Johansson’s moviesJohansson’s movies Placed lights on joints of Placed lights on joints of

actors.actors. Issue for Rigidity Issue for Rigidity

Principle: You can’t have Principle: You can’t have groups of 4 points that groups of 4 points that move together rigidly.move together rigidly.

Knee and ankle points Knee and ankle points move rigidly, but hip and move rigidly, but hip and ankle do not.ankle do not.

You can still construct 3D You can still construct 3D objects w/o having to objects w/o having to create groups of 4 rigid create groups of 4 rigid pointspoints

So, you mean there’s more So, you mean there’s more than just the Rigidity than just the Rigidity

Principle? Yup!Principle? Yup! Rule 34: If possible, and Rule 34: If possible, and

if other rules permit, if other rules permit, interpret image motions interpret image motions as projections of 3D as projections of 3D motions that are rigid motions that are rigid and planar. and planar. This allows us to construct This allows us to construct

biologicical motion where biologicical motion where rigidity alone, does not.rigidity alone, does not.

Because of the biological Because of the biological make up of our joints, our make up of our joints, our bones swing on planes.bones swing on planes.

Rigidity and Planarity Rigidity and Planarity together require only 2 together require only 2 points to construct a 3D points to construct a 3D object versus the 4 needed object versus the 4 needed with Ullman’s theorem. with Ullman’s theorem.

Rigidity and Planarity will Rigidity and Planarity will usually work with other usually work with other rules too such as rules rules too such as rules about smoothness or about smoothness or dynamical constraintsdynamical constraints

Motion Picture and Motion Picture and Gaming: need compelling Gaming: need compelling graphics or people won’t graphics or people won’t want it.want it.

Ok…enough dots already.Ok…enough dots already.

Let’s Move on…Let’s Move on…

Dan Kersten and Dan Kersten and Collaborators.Collaborators.

When you’re When you’re clever:clever: Interpreting a Interpreting a

smooth diagonal smooth diagonal movement along movement along the checkerboard the checkerboard along with the along with the shadow.shadow.

When you’re not so When you’re not so bright:bright: Interpreting a Interpreting a

complicated movement of complicated movement of varying elevation and varying elevation and direction just because of a direction just because of a different position of the different position of the shadow even though the shadow even though the solid sphere is moving solid sphere is moving along the same exact along the same exact path!path!

Studied how we construct shapes, motions, shadows, and light sources.Studied how we construct shapes, motions, shadows, and light sources.

But…motion in 3D depends not only on But…motion in 3D depends not only on the sphere, but also on how you interpret the sphere, but also on how you interpret the relation between it and its shadow.the relation between it and its shadow.

The sphere’s move along the same path, but the shadow The sphere’s move along the same path, but the shadow moves along in arcs. moves along in arcs.

Why?Why? Because of Because of Rule 35: Light Sources Rule 35: Light Sources

Move Slowly.Move Slowly. We don’t like to create light sources that move fast, We don’t like to create light sources that move fast,

we’d rather form an interpretation about the objects.we’d rather form an interpretation about the objects. Eg. After class if you walked outside and it was completely Eg. After class if you walked outside and it was completely

pitch black…you may think the world was coming to an end. pitch black…you may think the world was coming to an end.

Ooh all the Pretty Ooh all the Pretty CCoolloorrss!! Tom Albright and Tom Albright and

Karen Dobkins claim Karen Dobkins claim a strong interaction a strong interaction between the between the magnocellular magnocellular (motion, luminance, (motion, luminance, depth, and coarse depth, and coarse form) and form) and parvocellular (color, parvocellular (color, fine form). P. 166fine form). P. 166 Shows that we prefer Shows that we prefer

objects of the same objects of the same color to move together.color to move together.

““Birds of a feather Birds of a feather flock together.”flock together.”

Can motion affect how you Can motion affect how you create color?create color?

““Indeed it can.”Indeed it can.” Benedict Prevosts “Flicker Colors.”Benedict Prevosts “Flicker Colors.” Rediscovered by C.E. BenhamRediscovered by C.E. Benham

Benham’s Top.Benham’s Top. Spin counterclockwise and see an array of colors, Spin counterclockwise and see an array of colors,

spin it clockwise and you see another array of colors spin it clockwise and you see another array of colors but in reverse order. (red to green to blue to violet but in reverse order. (red to green to blue to violet and vice versa).and vice versa).

Phenomena of flicker colors shows how Phenomena of flicker colors shows how your creation of motion affects your your creation of motion affects your creation of color.creation of color.

But Hoffman too was a But Hoffman too was a clever one!clever one!

Dynamic Color SpreadingDynamic Color Spreading

Creating motion through color with Creating motion through color with stationary dots.stationary dots.

Dr. Hoffman and Dr. Carol Cicerone

More examples…More examples…

Computer Vision SystemsComputer Vision Systems

It is still very limited, but it’s It is still very limited, but it’s growing and may not be far beyond growing and may not be far beyond our reach!our reach!

Eg. Richman’s car tracking systemEg. Richman’s car tracking system

Chapter 7Chapter 7The Feel of a PhantomThe Feel of a Phantom

Phantom LimbsPhantom Limbs

The feel that an actual limb is there with The feel that an actual limb is there with real feelings and sensations when in fact real feelings and sensations when in fact it is not there.it is not there.

Case of F.A.Case of F.A. Found 2 Systematic maps of phantom hand.Found 2 Systematic maps of phantom hand.

First: About 6cm above stump First: About 6cm above stump Second: About 13cm above stump.Second: About 13cm above stump.

He could feel the sensations in his hand when those He could feel the sensations in his hand when those specific areas were touchedspecific areas were touched

Maps found on faces in some cases as with Maps found on faces in some cases as with patient V.Q. p.175patient V.Q. p.175

The Somatosensory The Somatosensory CortexCortex

The portion of your The portion of your brain devoted primarily brain devoted primarily to touch and sensationsto touch and sensations

Penfield map:Penfield map: Notice how the hand and Notice how the hand and

face are nearby?face are nearby? The face and arm regions The face and arm regions

of cortex “invade” the of cortex “invade” the cortex that normally cortex that normally processes the hand.processes the hand.

Proven with magnetic Proven with magnetic source imaging – a non-source imaging – a non-surgical process used to surgical process used to measure brain activity. measure brain activity. Tony T. Yang (1994)Tony T. Yang (1994)

Are we just making it up?Are we just making it up?

Do we construct everything we see and Do we construct everything we see and experience (hear, smell, taste, and feel)?experience (hear, smell, taste, and feel)? Hoffman argues that all your sensations and Hoffman argues that all your sensations and

perceptions are constructions.perceptions are constructions.

You don’t just construct simple feelings.You don’t just construct simple feelings. Exp. Diane Ackerman “Our palette of feelings is Exp. Diane Ackerman “Our palette of feelings is

elaborate.”elaborate.” You construct complex ones as well.You construct complex ones as well.

What if you have all your What if you have all your limbs, is it possible to have limbs, is it possible to have trouble with constructiion?trouble with constructiion?

Yes. If you’re constructive processes Yes. If you’re constructive processes are not intact, you will have are not intact, you will have problems with touch.problems with touch.

Case of E.C.Case of E.C. Stroke in areas 39 and 40 of left hemisphere. Stroke in areas 39 and 40 of left hemisphere.

Numbness in the right hand.Numbness in the right hand. Another stroke in temporal and occipital cortex.Another stroke in temporal and occipital cortex. Outcome: Trouble remembering visual objects Outcome: Trouble remembering visual objects

(RH) and her right hand had tactile agnosia, (RH) and her right hand had tactile agnosia, (LH). Hand still had sensations, it just couldn’t (LH). Hand still had sensations, it just couldn’t recognize objects.recognize objects.

So what about if we don’t So what about if we don’t have any physical problems have any physical problems like Phantoms or Strokes?like Phantoms or Strokes?

Frank Geldard and Carl Sherrick (1970’s)Frank Geldard and Carl Sherrick (1970’s) 3 vibrators on arm that accidentally delivered 3 vibrators on arm that accidentally delivered

taps and created the feeling of extra taps taps and created the feeling of extra taps between the vibrators. Feeling of a rabbit between the vibrators. Feeling of a rabbit hopping up your arm. hopping up your arm. Can create this effect anywhere on body and with Can create this effect anywhere on body and with

as little as 2 vibrators. as little as 2 vibrators. Can create more than one rabbit. P. 181Can create more than one rabbit. P. 181 The second tap tells you the direction of where to The second tap tells you the direction of where to

send the rabbit, so you must create the rabbit send the rabbit, so you must create the rabbit afterward.afterward.

So what does all this So what does all this mean?mean?

It means that we create not just the It means that we create not just the “what” and the “where” of feeling, “what” and the “where” of feeling, but the “when” as well.but the “when” as well.

It confirms that you create what you It confirms that you create what you feel!!!feel!!!

Now…Now…

How does all of this relate to the How does all of this relate to the Mind/Body problem?Mind/Body problem?

Does this bring us any closer to Does this bring us any closer to solving the problem?solving the problem?

How have your thoughts changed How have your thoughts changed from the first day of class until now?from the first day of class until now?

Thank You!Thank You!