June 3, 2008

Vibeke Sorensen, Professor and ChairDepartment of Media StudyUniversity at BuffaloThe State University of New York

URL: http://mediastudy.buffalo.edu/ URL: http://vibeke.infoemail: vibeke@buffalo.edu

Synthetic Characters in an Imperfect World

Synthetic Characters in an Imperfect World

Synthetic Characters in an Imperfect World

• Synthetic characters are representations of living beings and their experiences: people, animals, and plants

• They express our understanding of what it means to be alive, which we perceive with a synthesis of our senses – “common sense”

• As human beings, our experience is objective and subjective, and synthetic characters embody both

Seeing and Vision

• Seeing is not only an optical phenomenon

• A rich combination of memory and vision:– sense experience– memory and memory residue, and their constant

transformations in the mind– impressions and emotions that result from the

associations of senses and memories of experiences

Mystery of the Senses: Vision

Show DVD 1

Metaphor and Character

• How we express our experience in order to communicate as a representation of it, a metaphor, an approximation that is poetic

• The metaphor is as close as we can come to the truth of what we experience as human beings

• It is as close as we can come to expressing who we are, what we know, think and feel about life

• Character is that part of our identity that embodies these feelings and thoughts, our values

• Our character can be represented in many ways, in many media, and by a wide range of proxies, or ‘stand-ins’, including photorealistic doll-like beings.

• But they can go beyond conformation with external appearances and instead use abstract visual form to represent normally invisible realities or qualities that are beyond normal human perception.

• The use of symbolic visual form to represent nature and cosmological information is common across cultures, spanning most of human history.

• In Native American tradition, everything is animated by the forces of nature, and full of deep symbolism. This is true of the simplest items of everyday use.

• The images themselves are thought to be repositories for spirits that move in nature, thus inhabited by them, possessing metaphysical properties.

• In the case of the Hopi and Navajo people, the images, masks, and sculptures have a ritual purpose of helping to put people back into harmony with nature and the universe.

• Katsinas are considered spirits of the individual life forces.

• They are supernatural beings who mediate between the gods and the people who live on earth. As divine messengers, they live among the people between the winter and summer solstice where their task is to create order.

• During this time, they are the focus of ceremonies and rituals. They relay the wishes of the people to the gods. After the harvest, they return to the mountains from where they came.

KatsinaThe spirit of the invisible forces

of life

Avatshoya, the Spotted Corn Katsina

Squash Katsina,The 3rd member of indigenous triad of food plants, corn, beans and squash

Early Morning Katsina

http://www.hao.ucar.edu/Public/education/archeoslides/slide_10.html

To see movies of Katsinas in 3-D from the Indian Arts Collection at the School of American Research (IARC) Collection

http://www.sarweb.org/iarc/katsinamovies/katsinam.htm

Koyaala Katsina

Synthesis and Life

• Synthesis means several things– To synthesize or put together, like the “common sense”– To create from unnatural or otherwise inanimate sources, by using

light or electronic signals or some other non-organic or unusual materials

• To synthesize life is animation: – To ‘give life to’ and ‘to give soul to’– To transfer organic gesture to an inanimate object so that it appears

to be alive – living things and their gestures are the best reference– It is magical because we recognize the human and living in it, and

this can change the way we see ourselves and others– This helps to develop an interest in and understanding of life, and a

greater sense of empathy towards people and other living things

Realism

• Realism means to communicate deeply the real, lived experience of a creature and communicate its character.

• So ‘synthetic’ is not only something computer generated, artificial, or devoid of human qualities

• Synthetic is not necessarily photorealistic

• Photorealism is not the same as Realism

Gulliver’s Travels (Max Fleisher 1939)

• Uses rotoscope of a live actor for Gulliver and animated characters for Lilliputians

• Monster has the more photo-realistic movement

• But the more believable characters are the synthetic, because there is still room for our imagination

• We need always to make more room for the memory and imagination of the viewer

Click here to see video

Gulliver’s Travels

• With photography and compositing, we can make a kind of photo-surrealism

• This is also a kind of truth, a documentary of the subjective, internal reality and experience of a person or being, combining memories and perception

Jerry UelsmannPhotographer

Jerry UelsmannPhotographer

Photographer Vibeke Sorensen

Photographer Vibeke Sorensen

Some of my images

• It is possible to combine external and internal realities

– A unique possibility today using computers and peripherals as they allow us to digitize physical media and gestures, integrate and transform them in completely new ways

Fantasy vs Fantastic

• Because we can use live action but also alter it frame by frame, we can change photographic images so that they express our real feelings about what is seen.

• What we see can appear fantastic but it is not exactly fantasy.

• It’s not invented experience so much as internal, real feelings and memories, externalized and made visible.– Visualization of cognitive phenomena and understanding– Even visualization of new kinds of thinking that reflect

human intellectual abilities enhanced or extended by computers, ultimately reflect the human body and mind

Objectivity and Subjectivity

• Imagine a photograph of a city or nature scene. Then think of a painting of the same scene. Which tells us more about the artist or our emotional or intellectual relationships to it? In general, painting is subjective and photography is objective.

• The esthetic transformation of ‘reality’ not only draws our attention through emphasis, but expresses ideas, thoughts, abstract knowledge, and other internal and cultural realities.

• Thus truth of human experience is a combination of objectivity and subjectivity:

photography and painting

‘The Painting of Modern Life’

• Exhibition at London’s Hayward Gallery in 2007 showed the symbiotic relationship between painting and photography since 1840

• 100 works by 22 artists including Richard Hamilton, Malcolm Morley Gerhard Richter, and Andy Warhol.

“Painting, in the very moment of its technological and cultural eclipse, could face the new challenge. It should seize the day or, more precisely, the everyday, since the character of modern life could be best grasped through its smallest details.

Through the mass media and the amateur it was photography that seized the everyday. But in permeating culture so thoroughly it almost relinquished its claim to the critical distance of art. If any distance were to be recovered, the photograph’s bald familiarity would have to be estranged. This is part of what Pop art achieved and what subsequent generations of painters do when they work from photography: they estrange it enough to make its deepest impact thinkable.”

• David Campany, Frieze Magazine, January-February 2008

Andy Warhol, Big Electric Chair (1967) depicts the Sing Sing hot seat where convicted spies Ethel and Julius Rosenberg were executed.

“...Richard Hamilton’s Swingeing London 67 (1967–8), his reworking of a press photo depicting gallerist Robert Fraser and Mick Jagger handcuffed together in a police van following a drugs raid. This image has become iconic owing to the endless reproduction not of the original photo but of the painting. Hamilton grasped early on that art itself would not escape mass dissemination and hype. But for those who make the effort to see the work itself the artist has a neat surprise: the handcuffs are real aluminium, protruding from the flat image in a way that slyly escapes photographic reproduction of the art work. Fraser and Jagger revealed their handcuffs while shielding their faces.”

• David Campany, Frieze Magazine, January-February 2008

Gerhard Richter Woman With Umbrella

1964

Jacqueline Kennedy grieving after her husband’s

assassination

From Time Magazine Review

by Alex Altman, October 24, 2007: “When photography first emerged in the 19th century, art critics predicted a collision with painting. It turned out there was no need to squabble over cultural territory: most photographers document the world, while painters are out to interpret it.”

“These works are gestures of respect for photography's power to create iconic images, but they are also declarations of confidence in the transformative capacity of painting. Photographs freeze the decisive moment; these paintings infuse it with distinctive meaning.”

The same is true for moving images and animation. With the addition of motion, we obtain greater e-motion. Instead of banal,

they become imbued with poignancy and pathos.

Lord of the Rings• Peter Jackson – New Zealand

• Gollum, a synthetic character– A real actor was used for the voice

– Jackson discovered that the voice, face and body movements could not be separated

– The real actor became the reference for all aspects of the synthetic character

– The more realistically human was used to depict a monster (as in Gulliver). The addition of poetic invention and exaggeration makes a more believably human character.

Lord of the Rings

Show DVD 2

Computer use

• To make the 3-D model in the computer– References are drawings and scultpure

• To add “realistic”motion to the model– Motion capture - of actual movements of actor– Artificial Intelligence - to give each model individual decision

making ability, adding motion capture library– Keyframe technique – to follow live action reference

• Sometimes expression and movement had to go beyond the limitations of the computer and actor to seem “real”– In this case, the animators used the keyframe technique– Living person was the best reference

Relationship to the Environment

• The live action actors in Lord of the Rings moved in the real environment, and this was a constant reference.

• In real life, an organism responds to the environment and adapts. When the environment is unhealthy or “imperfect,” the organism often responds by becoming diseased or altered.

• These can be considered “imperfections” or “distortions”

• Thus, the more imperfect, the more perfect!

• These distortions reflect the realities and problems not only of the physical environment, but the social, pschological, and cultural as well.

• We use metaphor and esthetics to reflect this actuality

• Representations of actuality and reality are ethical issues: truth and ethics is also esthetics

• We can combine documentary and metaphorical animation to reflect and contemplate, to criticize, and understand the experience of life and world better

• A good example of how these ideas and techniques were combined in a 3D digital environment

• The real person, real experience is combined with a somewhat literal representation of the degeneration of the character is strong and touching

• By being ‘less realistic’ it is ‘more realistic’

• Psycho-realism, documentary and animation

• By combining real life story and many ‘imperfections’ including in the models, it is far more believable, real, and interesting

RYAN

• Used a video reference, photographs and drawings, storyboarded

• Maya, Paint Effects, Discreet Combustion

• Used keyframe animation, but added in nuance that reflected the unexpected gestures made by the character that had nothing to do with the content but were part of the character’s identifying qualities

• Landreth tried to rid the animation of the spliney look by using straight ahead animation techniques for important parts such as an arm or hand, and then going back and animating the rest of the character

• For textures, stitched together photographs in Adobe Photoshop and applied them to 3D models.

• For Derek and Felicity, used Paint Effects to mimic strokes in Larkin’s drawings and to create the animated CG models that looked hand drawn

Show DVD 3

• In Ryan, we are moving us away from ‘fantasy’ and towards the ‘real fantastic’ or ‘magical realism’

• Dreaming together, sharing and caring for each other

• This is one of the main possibilities at the intersection of animation and documentary worldwide, given the importance of cross-cultural communication through visual communications media

Magical Realism

• 6.6 billion people in the world today

• 106 billion all together since human beings appeared

• Difference between first and last human being 0.000001 %

• In this percentage you can put all different races and cultures

• Humanistically saying it is impossible for us to calculate it all

• Difference between human beings and synthetic characters is tiny and narrow today (technologically saying)

• But for us, our eyes, it is a monstrosity, because our eyes can see the difference, it is disturbing.

• Max Fleisher in Gulliver’s Travels used 100% rotoscope cel for movement. He used traditional cell animation for the small characters. Gulliver seems like a monster but the synthetic characters seem natural.

• Photorealism needs to be put into context. It is very important for scientific and historical visualization.

• But it is not by itself poetic and does not generally touch the human soul.

• It does not necessarily allow for cultural variation and diversity, individual interpretation, individual experiences or subjective points of view. It is not generally interesting for art, which is always expressing a critical or larger view of human experience.

• We need imagination and metaphor, transformation, and some empty space for imagination, memory and diversity, especially for a global audience.

• Several areas are positioned strategically for this given their rich cultural histories: Latin America given its use already of ‘Magical Realism’

• Asia and Australia are rich in cultural traditions, especially painting, sculpture, dance and puppet theater, and so can have a huge impact on the visual and gesture language of global visual and moving image culture. The content, meaning, or ideas enter into a huge global dialog about contemporary life. Len Lye of New Zealand pioneered this.

The Legend of Shangri-la

Bring not just the images but also the meaning of the images, and this comes through the stories, told by the people from those cultures who know it better than anyone else.

Being Indigenous.org

Tusalava by Len Lye

• Environments with a rich multicultural identity, including indigenous communities independent from Hollywood, are strategically positioned to develop important innovative and alternative approaches to global, cross-cultural communication by integrating physical and digital technologies that connect their cultures to the digital world. Most of the world’s cultural heritage is in physical and material culture.

• Imagine a state of the art digital ethnographic animation using documentary and digital media, connecting the people, their land, and their art – in space, time, and motion.

• As creators you know how to touch the souls of people around you, and all over the world.

• You transform people and the world

• You have a magical ability to create worlds and populate them, and connect them with the real world.

• There reflects on world cultures as well as the potential of digital technology to simulate life and human intelligence. It integrates metaphysics and religion, with physical reality.

• This is also a place for animation and documentary.

• With so many cultures and religions, sensitivity becomes very important.

• As I am sure you know, in some religions you are not supposed to make an image of the Gods. It is something to reference but not depict. God is therefore an open image, a mental construction. It may be better to use an abstraction. Spirits can take many forms.

• We need to find ways to integrate the real and imagined in ways that enhance our collective human heritage for the benefit of the entire world, to help build bridges of understanding.

• This is the potential that we have as artists and technologists, as we create new worlds and characters to inhabit them. We are referencing all of human history and affecting its future.

• The use of images in a global context is the larger issue.

Films, animation, and websites

• Al Gore’s An Inconvenient Truth• Richard Linklater’s Waking Life• Godfrey Reggio’s Qatsi: Koyaanisqatsi, Naqoyqatsi, Powaqqaatsi• Stacey Steers’ Watunna• Caroline Leaf’s The Owl Who Married a Goose• Erika Russell’s Feet of Song• Michael Dudok deWit’s Father and Daughter• The Story of the Weeping Camel (documentary) by Byambasuren

Davaa • Ashes and Snow, Gregory Colbert (photography – ashesandsnow.org)• Frida by Julie Taymor• Being Indigenous website (beingindigenous.org)

Thank you!

This is a global artform, so please use your knowledge well, to make the world a better place. With this, you will find that good things will come to you and everyone else!