narrative structures for new media

Leonardo

Narrative Structures for New Media: Towards a New DefinitionAuthor(s): Pamela JenningsReviewed work(s):Source: Leonardo, Vol. 29, No. 5, Fourth Annual New York Digital Salon (1996), pp. 345-350Published by: The MIT PressStable URL: http://www.jstor.org/stable/1576398 .

Accessed: 26/02/2013 04:41

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

The MIT Press and Leonardo are collaborating with JSTOR to digitize, preserve and extend access toLeonardo.

http://www.jstor.org

This content downloaded on Tue, 26 Feb 2013 04:41:13 AMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=mitpress

http://www.jstor.org/stable/1576398?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


Narrative Structures

for New Media

Towards a New Definition

Pamela Jennings

ARISTOTLE'S POETICS IS AN INADEQUATE NARRATIVE MODEL

for the creation of computer interactive art, contrary to the thesis laid down by Brenda Laurel's

Computers as Theatre. Although she opened up space for a rather tantalizing dialogue in this area, it is to

non-Western cultures that one should look for narrative structures that fit the sophistication of Western new

technologies. The theories and processes of African oral literature provide the groundwork for such

a narrative model. A comparison of contemporary postmodern thought and African oral literature shows

that they share many attributes. Through a description of her work, which challenges the notion of book arts

and narrative theory, as well as through an exploration of the advanced mathematical theory of fuzzy logic, the

author opens the door for a discussion of narrative that crosses cultural, aesthetic, and academic boundaries.

In this concept of narrative, nuance, indeterminacy, and polyventiality are major players.

The Book of Languages. This is a large, thick book with a blue-green cover that rainbow-hates in the light. More a box than a book, it opens in unorthodox fashion, with a door in its front cover. Inside is a collec- tion of eight smaller books arranged like bottles in a medicine case. Behind these eight books are another eight books, and so on. To open the smaller books is to let loose many languages, words and sentences,

paragraphs and chapters gather like tadpoles in a pond in April or star-

lings in a November evening sky. -Peter Greenaway [1]

he "interactive book" is a phrase heard daily in reference to the latest experiments in CD-ROM and other digital media. But how to translate the

concept of the book into a medium that has no paper and no pages remains a challenge for the artist. Is not a book first of all an object one holds in one's hands-the cover affected over time by acids and oils from the user's skin, and the pages turned down and yellowed, torn or marked

up? Artists have repeatedly challenged the notion of the traditional book in this century, and this challenge has accel- erated with the growing accessibility of new interactive and information technologies.

Ever since the release of Peter Greenaway's film Prospero's Books, I have wanted to design and build a living book-one that is sensitive to human touch and responds accordingly and that incorporates Umberto Eco's idea of an "open work," a work in movement [2]. My first step toward creating a truly computer-interactive book was a CD-ROM project, Solitaire: dream journal (Fig. 1). Mixing the metaphor of the game board and the book, Solitaire: dream journal whisks the user through a

haunting journey in quest of peace with oneself and connection with others. Solitaire: dream journals interface uses a three- dimensional solitaire game as the engine to move through this

journey. The solitaire board is tetrahedral (a three-faced pyra-

Pamela Jennings, 471 Bergen Street #3, Brooklyn, New York, NY 11217, U.S.A. Email: [email protected].

LEONARDO, Vo1.29, No. 5, pp. 345-350 1996 345 ? 1996 ISAST



Fig. 1. The triangular game face from "the book

of melancholy" in Solitaire: dream journal.

mid), and its three faces correspond to the three thematic areas of melancholy, flight, and balance. A move made on a side of this pyramid will randomly open up a chapter of the corresponding "book": "the book of melancholy," "the book of flight," or "the book of balance" (Fig. 2).

Sound is a crucial element in Solitaire: dream journal. To create nuanced layers of

tone, I used combinations of my voice,

sampled sounds, and algorithmically controlled serial interludes.

Solitaire: dream journal led into my next

computer-book project, the book of ruins and desire (Figs. 3 and 4), a kinetic, inter-

active, mixed-media sculpture that

explores issues of desire and communication. Imagine you are standing before a simple table on which rests a sculp- tural object with hinged metal leaves or

pages. From its case a barely audible voice murmurs "Touch me." This draws you closer and gives you permis- sion to interact with the book of ruins and desire. When you turn its metal pages, you begin an exploration into its multi-

ple layers, composing your own collage of my imagery and sound.

The book of ruins and desire incorporates some unusual materials; its pages, for

example, are etched metal plates. The

degree of a turned page as well as its

velocity are calculated in a fuzzy logic inference engine programmed into a

microprocessor. The results are fed to a Macintosh computer running the MIDI

programming language Opcode MAX [3]. MAX is programmed as a video and audio switching device that determines which images and sounds are played back via a small LCD monitor and embedded

speakers, as well as qualitative dynamics of the media such as volume, video

direction, and playback speed. MAX is also programmed to create MIDI interludes via freeware serial music libraries.

Just as importantly, the book of ruins and desire incorporates a new approach to narrative. With the advent of com-

puter-based interactive art and interactive information systems, many issues arise concerning the use of alternative

engines through which narrative information may be created. Our culture is

presently experiencing a shift in the

organization of knowledge away from the linear motif. As our Western culture works hard to develop new links between the medium and the message, we must include in our research and theories structures that may appear to be new for Western cultures but that have existed in non-Western and pre- Western cultures for centuries.

Notes on Narrative The problem with computers is that there is not enough Africa in them. This is why I can't use them for very long. Do you know what a nerd is? A nerd is a human being without

enough Africa in him or her. I know this sounds sort of inversely racist to say, but I think the African connection is so important.

-Brian Eno [4]

Ado bibi nagha bol ikio ikio eki Ado bibi na

[One ignorant of the Benin language may yet understand it by common sense.]

African Proverb [5]

Apart from the overtly racist lump- sum categorization of an entire continent of peoples and their collective relation-

ship to an abstract machine, Brian Eno has tapped the surface of what I believe are rather poignant connections between the linear and nonlinear synapses of the

computer as a medium of the mind and the body. However, without theoretical

grounding for such juxtapositions of cultures and technologies we are left with

empty echoes of the tendency to exoti- cize distant cultures. This dilutes what otherwise might be a powerful definition of the relation between technology and our future forms of communication.

Narrative is a function of delivering information over time. Written and oral cultures traditionally view the continu- um of time differently and therefore

have created narrative structures based

upon seemingly opposing metaphors: the unidirectional line and the iterative circle. The written culture's notion of narrative derives from the theory of dramatic progression expounded by Aristotle in his Poetics [6]. The Poetics

presents a strict guideline for the drama to follow from beginning to end: the narrative increases in intensity to the cli- max and then gradually reaches an end

parallel in tone to its beginning. By con-

trast, primarily oral cultures treat the elements of narrative as individual

agents that can communicate despite their placement within a story. As Walter J. Ong describes it, these elements are "boxes within boxes created

by thematic recurrence" [7]. It is a pro- cess analogous to recursion.

Brenda Laurel's book Computers as Theater proposes to use the theories laid down by Aristotle in the Poetics as a foundation from which software

designers and computer engineers can create effective human computer inter- faces [8]. Laurel tries to present a model for building computer-based environ- ments that have the structural elements inherent in artistic disciplines, especially drama and theater. She argues that no

Fig. z. Dream four from "the book of flight"

in Solitaire: dream journal.

346 Pamela Jennings, Narrative Structures for New Media



one has provided a theory of drama that is as comprehensive, well-integrat- ed, and widely accepted as Aristotle's. In my opinion, however, the model of the Poetics is inadequate to the creation of computer-based interactive work, especially in its fixed definition of plot. The Poetics is an arbitrary set of rules enforced for convenience and brevity in the communication of ideas. Precisely because it encourages linearity and trun- cation of thought, it is not a good model for interactive art.

Richard Schechner's analysis of con-

temporary Western performance theory explores why the Aristotelian model is

inapplicable to experimental dramas, happenings, and the many plays not modeled on the Poetics. I will add to his list human-computer genres of communication. He draws his examples from the work of Beckett, Genet, Ionesco, and others whose work revolves around life rhythms: eating, breathing, sleeping- waking, night-day, the seasons, the

phases of the moon, and so on. These

rhythms do not have the neat begin- nings, middles, and ends required by Aristotelian drama. One rhythmic cycle is completed only to begin again; noth-

ing is resolved. Drama modeled on life

rhythms contains episodes of varying length (usually short), in which tensions

increase, explode, and return to the

original situation [9]. Challenging the Aristotelian notion

of narrative introduces the quest for other concepts of time and narrative structure. This journey must take us outside of the neatly packaged Aristotelian methods of linear information processing. The mono-orgasmic narrative structure must unfold into an endless time frame where multiple cli- maxes can occur.

African Oral Literature Isidore Okpewho's book African Oral

Literature: Backgrounds, Character, and

Continuity provides a rare and much- needed analysis of the oral storytelling traditions of Africa. The word-of-mouth medium of presentation implies that oral literature makes its appeal first

through the sound of the words that reach the ears of the audience and only secondarily through the meaning or log-

ic contained in those words. The styles and techniques of presentation veer

away from the literal into a realm of circular mind-mapping. Combined with the rich mixture of voice, song, rhythm, and dance, this creates the ultimate multimedia extravaganza.

The internal structuring of such sto- ries depends on the use of repetitive episodes within a tale. The narrator controls the movement of his or her

imagination by putting successive

episodes of the story into convenient

frames, allowing events that have a certain amount of similarity between them to be described in roughly similar terms. Ronald Rassner refers to these periodic repetitions of similar symbols and situa- tions as narrative rhythms [lo]. They mark off successive stages of the plot and help give the story a certain balance

Fig. 3. An installation view of the book

of ruins and desire.

and regularity, much as rhythmic beats structure a musical performance.

The oral narrator manipulates the audience's sensations by controlling the

pattern of narrative beats (upward, downward, etc.). By intricate variation of these rhythmic patterns, a good narrator will create a complete aesthetic

experience for the audience. Unlike literature based upon the Poetics, African oral literature may contain numerous crises or peaks tangential to the nuances of the story, reflecting the environment it is told in and the responsiveness of the audience. Audience responsiveness is a crucial element of the African oral tradition. The audience is expected to

participate physically and verbally through such techniques as call and

response, much like the tradition of African-American churches. The oral narrator is receptive to this constant feedback and will alter his or her narra- tion appropriately.

The Open Work If the openness of the work is a key-

stone of traditional oral storytelling, a similar openness has become a staple of

postmodern art. Umberto Eco notes that all forms of communication, inter-

pretation, and understanding are by their nature tentative and hazardous acts of inference. Drawing examples from mid to late twentieth century Western aesthetic practices, Eco describes the

"open work" as the work in movement. There is no single prescribed point of

view, and there is the possibility of numerous different personal interven- tions. The invitation to participation offers the reader, interpreter, or per- former the opportunity to enter into and complete something that nonethe- less always remains the work intended

by the author. An author may not know the exact fashion in which her work will be concluded but is aware that once

completed the work in question will still be her own. In a fundamental sense, it will not be a different work.

While every performance offers a

complete and satisfying version of the

work, at the same time it remains

incomplete because it cannot simultane-

ously give all the other performances. The image the open work gives is one of discontinuity. It offers a transcenden- tal scheme that allows the participant to

comprehend new aspects of the world. The open work does not narrate; rather it offers the participant an infinite

potential for exchange rich in unfore- seeable discoveries [11].

Eco states that the theory of open work is none other than a poetics of serial thought. Serial thought aims at the production of a structure that is at once open and polyvalent, in music as well as in painting, in the novel as well as in poetry and theater. Serialism dom- inated European musical thinking in the

years following World War II. Pierre

Boulez, the leader of the development of this new musical language, declared that the neoclassicism of such com-

posers as Stravinsky was dead because it was too rooted in the music of the past. "Classical tonal thought is based on a world defined by gravitation and attrac- tion; serial thought, on a world that is

perpetually expanding" [12]. Serialism

Pamela Jennings, Narrative Structures for New Media 347



Fig. 4. A video still from the book of ruins and desire.

offered a language in which all familiar

patterns and preconstituted scales could be abolished and replaced by 12-tone melodies. In such melodies, all 12 notes between octaves hold equal potential for influencing the melodic qualities of the music [13]. If one were to study closely the traditional Western art of classical music composition and coun-

terpoint, one would find the collision of dissonance and consonance creating a

greater whole called harmony. However, traditional Western counterpoint is also

very restrictive to the overall plot of a musical composition as attributes like

repetition and parallelism are kept tight- ly in check [14].

The critic Ihab Hassan notes that

postmodernism, as an artistic, philo- sophical, and social phenomenon, sup- ports such open (in time and structure), playful, and disjunctive forms of discourse [15]. Immanence and indetermi-

nacy are the two attributes he considers definitive of postmodern culture. Immanence is the mind's ability to become its own environment, a virtual existence of symbols intervening with nature. Indeterminacy is an evasion of direct connections; it is discontinuity, pluralism, deconstruction, displacement, rupture, and silence. Immanence and

indeterminacy are neither antithetical nor complementary; rather, they function like strange attractors whose influ- ence on each other is often

unpredictable yet highly interdependent. They can be related both to the polyva- lency of serialism and the iterative structures of African oral literature.

Indeterminacy and immanence bring into question the hierarchy in Western

thought that places the mind's computa- tional power above the body's experien- tial knowledge. A long line of classical

scientific views-ultimately tracing bac to Aristotle-presuppose the existene of a rationally structured objective reali for which there is one correct descril tion that we arrive at step by ste] Postmodern thinking, by contrast, do( not assume that there is a single wor with a knowable structure; rather, the: are as many possible worlds as hums

beings can invent and experience. Und the discourse of postmodernism, not]

ing is meaningful until it has been exp rienced either by the body or by t}

body in and through the mind.

Nuance and Fuzzy Logic Conceptually, indeterminacy is ve:

closely related to what is now know as chaos theory. In Turbulent Mirr

[16], John Briggs and F. David Pe

analyze the similarities between cha(

theory and artistic creation by consi(

ering the phenomenon of the nuanc As a disjunctive connection made I the mind between seemingly unrelate events or thoughts, the nuance is t} seed of artistic creativity. For t}

artist, nuances reveal themselves

interlocking metaphors or "refle,

taphors," which are based

upon irony, metaphor, U simile, pun, paradox, and mf

synecdoche. The strength 41 of the reflectaphor lies in } its ability to create loat an irresolvable tension fo .float between similarities and fl differences. This tension

(a form of indeterminacy) induces an intense state for ( of wonder, doubt, and tt uncertainty in the audi- t{

ence, lending the moment a sense of unpredictability and randomness balanced

by an organic illusion of familiarity. Unfolding the nuance, Briggs and Peat

argue, is analogous to an iterative mathematical equation balancing on the boundary of finite order and infinite chaos.

Another area where we find a kind of creative indeterminacy is the field of

fuzzy logic (Fig. 5), developed by Lotfi Zadeh in the 196os [17]. Traditional

(crisp) logic arises from the ideas of

Aristote and Pythagoras, who believed that matter was essentially numerical and the universe could be defined as numerical relationships. They are credit- ed with the foundation of geometry and the mathematics of tone in Western music. By the tenth century, Aristotelian logic formed the basis of

European and Middle Eastern thought, and it has persisted for two basic rea- sons. It simplifies thinking about prob- lems, and it makes certainty or truth easier to prove and accept. In the com-

puter programming world we know this form of logic as Boolean logic, the world of ones and zeros.

Because many concepts are better defined by words than by numbers, Lotfi Zadeh was interested in a mathematics that was linked to language. His

fuzzy logic (and its expression in fuzzy sets) provides a discipline that can build better models of reality than pure mathematics can. Fuzzy logic systems are used today for estimating outcomes, for

decisionmaking, and for control of

many different types of mechanical systems, such as air conditioning, automo-

biles, and subways.

- r^USl'5ual l uCj vlo afwuwi i^.n |_W , :Iindex -= oxO; index I & 0x38) >> ?)

' utps iindex(mfLindex] = MAX(fL Iinde"

d( i i9 tt(int outsndex,float inputs

mp2 outm- Otmf _ndex nputmfdex

z im- 91orme on1 j{_ndem dex]

Fig. 5. A video still of fuzzy logic code from

the book of ruins and desire.




Fuzzy logic attaches a certain per-

centage or weight to its values, which are defined qualitatively. Fuzzy logic words can be organized under several

headings: quantification (e.g., all, most,

many); usuality (e.g., always, frequently, often); and likelihood (e.g., certain,

unlikely, certainly not). Inputs from the real world are defined within the system, and a rule-based matrix is created from the possible combinations of

inputs. The entire knowledge base (con- sisting of the defined inputs and desired

outputs) is then processed through an inference engine.

Designing a fuzzy logic system is

relatively simple, but the system itself

generally requires a great deal of simu- lation and fine-tuning before it is oper- ational. Despite these difficulties, fuzzy logic has been used since 1987 in the

city of Sendai, Japan, to keep its trains

rolling, braking, and accelerating at the

right times. Sony manufactures a fuzzy television set that automatically adjusts the quality of the screen image, and Nissan makes cars with fuzzy logic automatic transmissions and fuzzy logic antilock brakes.

A New Interactive Art Iteration, serialism, open structures,

fuzzy logic, language, desire, and interactive media can be a potent combina- tion in the hands of today's artists. The development of new tools-or a

rethinking of how we use the tools we

already have-will be crucial to the

development of interactive designs that push the horizon of storytelling and imagemaking in new media. It is a waste of energy and resources to make

applications that merely imitate media that exist in other forms, such as

print, television, and film. The early television industry, for example, quick- ly learned that radio plays don't work on TV. But just as photography prepared the ground for the moving image, today's mature media are the forebears of this new age of interactive communications.

Arthur Jafa, cinematographer of Julie Dash's film Daughters of the Dust, uses the word "polyventiality" when explor-

ing a possible definition of an Africa- centric style of visual production in

filmmaking. The tool Jafa describes to create this new cinema is a hand- cranked camera-a more appropriate tool to realize motion that has a certain level of plasticity to it. He draws an

analogy between the hand-cranked camera and the taut strings of the African

talking drum [18]. This leads me to consider the dilem-

ma faced by many artists and producers: how to gain the same knowledge of and access to the cutting edge of technology that engineers and computer scientists have. When I was working on the book

of ruins and desire, I had a hunch that

fuzzy logic was just what I needed for the project. It took me a three years of

research, Web surfing, a visit to a fresh- man mechanical engineering class at Rensselaer Polytechnic Institute that was studying the classical problem of the inverted pendulum [19], and finally a class at Steven's Institute of

Technology on microprocessors to make it happen.

Basically, to create art using these

technologies requires either the patience and capacity to learn the technology or

collaboration with engineers. As an artist desiring to explore the relationship of technology and art, I think it is

important to learn the tools. This gives artists a choice: either to develop the

technological component themselves, if

they have the ability to do so, or to col- laborate intelligently with computer engineers-or both.

There is a future for advanced tech-

nologies such as fuzzy logic and micro-

processors in industries focused on entertainment and education. Perhaps the incorporation of such technologies may move these industries beyond cre-

atively and intellectually stagnant games of violence and one-dimensional mer-

chandising catalogs. Eco's open work

challenges us as artists to begin explor- ing the possibility of polyventiality and

indeterminacy in computer interactive work. Subtleties of tonality, imagery, and digression amount to much more than flotsam surrounding the linear narrative model of the Poetics, they are the

engine of the ambiguities of life-an

engine where ritual, play, games, and

sports are as important elements as the more traditional narrative. The creation of products that more closely emulate the complex patterns of human

thought, desire, and emotion should be our goal as artists and producers of interactive work. The narrative tools to be used have been available for cen-

turies, and the technical tools are there in the halls of progressive art programs and engineering institutions.

References and Notes 1. Peter Greenaway, Prospero's Books: A Film of

Shakespeare's The Tempest (New York:

Four Walls Eight Windows, 1991) p. 21.

2. Umberto Eco, The Open Work (Cambridge, MA: Harvard University Press, 1989).

3. MIDI (for musical instrument digital

interface), developed in 1982, is an international

specification used by musical instruments that

contain microprocessors to communicate with

other microprocessor-controlled instruments or

devices. MIDI data organized into MIDI

"messages" flows over cables with standardized

5-pin DIN (Deutsche Industrie Norm) connectors serially at a rate of 31.25 kilobits per second. MIDI communicates performance data, not actual sound. (Adapted from Christopher

Yavelow, Macworld Music and Sound Bible

(San Mateo, CA: IDG Books Worldwide, 1992)

P. 34.)

4. Quoted in Kevin Kelly, "Gossip Is

Philosophy," Wired 3.o5 (May 1995) p. 148.

5. Quoted in Isidore Okpewho, African Oral

Literature: Backgrounds, Character and Continuity

(Indianapolis: Indiana University Press, 1992) p. 228.

6. George Dickie and Richard J. Sclafani,

Aesthetics: A CriticalAnthology (New York: St.

Martin's Press, 1977) p. 207.

7. Walter J. Ong, Orality and Literacy: The Technologi#ing of the World (New York:

Routledge, 1982) p. 144. 8. Brenda Laurel, Computers as Theatre

(Reading, MA: Addison-Wesley, 1993) p. 36.

9. Richard Schechner, Performance Theory

(New York: Routledge, 1977) p. 26.

lo. Okpewho [5] p. 224. 11. Eco [2] pp. 15-19. 12. Steven R. Holtzman, Digital Mantras: The

Languages of Abstract and Virtual Worlds

(Cambridge, MA: MIT Press, 1994) p. 85.

Pamela Jennings, Narrative Structures for New Media 349



13. Claude Levi-Strauss has offered a useful

comparison of the key differences between

structural and serial thought (Eco [2] p. 220).

In structural thought, communication occurs to

the extent to which a given message is decoded

according to a shared, preestablished code. It is

assumed that every code is based on a more

elementary code, and there is a search for this

underlying ur-code. In serialism, every message calls the code into question. At the extreme, each message posits its own code. Moreover, even though it is possible for communication to

be rooted in an ur-code that underlies all cultural

exchange, the main goal of serial thought is to

allow codes to evolve historically and to discover

new ones, rather than to trace them back to the

underlying code.

14. Counterpoint is the study of voice leading. It is the foundation of composition from which

most traditional classical composers have created

their musical masterpieces. The aim of

counterpoint is to develop the ability to hear,

understand, and control the fundamental

relationships that arise when two or more

melodic lines combine into a meaningful whole.

(Adapted from Felix Salzer and Carl Schachter,

Counterpoint in Composition (New York: Columbia

University Press, 1989) p. 3.)

15. Ihab Hassan, "Toward a Concept of

Postmodernism," in Postmodernism: A Reader

(New York: Columbia University Press, 1993)

pp. 152-154.

16. John Briggs and F. David Peat, "Prologue: Tension Forever New," in Turbulent Mirror (New York: Harper & Row, 1989) pp. 191-200.

17. F. Martin McNeill and Ellen Thro, Furgy

Logic: A Practical Approach (New York:

AP Professional, 1994) p. 9. 18. Arthur Jafa, "69," Black Popular Culture

No. 8, 253 (1992).

19. The basic inverted pendulum architecture is

also known as the cart-and-pole problem. A cart

is fixed to a linear path. Attached to the cart is a

free-falling pole. The object of the exercise is to

create a fuzzy logic engine that can calculate the

pole angle, angular velocity, cart velocity, and

cart position to keep the pole balanced within a

predetermined angle. (This is a variant on the

old spinning-the-dish-on-the-end-of-a-pole-on-

your-nose trick.) I was inspired by a group of

undergraduate students at Rensselaer Polytechnic Institute to use a modification of this concept for the operation of the book of ruins and desire.

Pamela Jennings is an electronic media artist

based in New York City. She was a 1993 MacDowell Artist's Colony fellow; recipient of

New York State Council on the Arts Media Arts

grants in 1992, 1994, and 1996; and artist-in-

residence at the Banff Centre for the Arts

(Alberta, Canada) in 1990 and 1992. Her video

Sleep Now Variations is currently touring with the

American Federation for the Arts and Museum

of Modern Art traveling exhibition "Video Art:

The First 25 Years." More information on her

projects is available on the World Wide Web at

http://www.sva.edu/alumni2/pamela/MFP/

HTML/homepage.html.




narrative structures for new media

Documents