notes on audio-vision and related concepts

Notes on: Visual Music, Audio-Vision,

Synaesthetic Cinema,Audiovisual Composition"

Nuno N. Correia, 2013 [email protected]

Visual Music"•  When defining visual music, Ox and Keefer (2008)

distinguish between four “differently formed visual structures”:"–  A visualization of music, which translates sound into visuals,

“with the original syntax being emulated in the new visual rendition”. According to Ox and Keefer, this can also be defined as intermedia."

–  A time-based visual composition, which is similar to the structure of a kind or style of music – “as if it were an aural piece”. It can have sound, or exist silent."

–  A direct translation of image to sound – “literally, what you see is also what you hear”. Some of Norman McLaren’s works (where scratchings on film produce simultaneously image and sound) fit in this category."

–  A static visual composition, “as in Klee”."

Michel Chion’s Audio-Vision"

Audiovisual Contract"•  Sound does not correspond "naturally" to an image."!•  Audiovisual contract:"–  "a kind of symbolic contract that the audio-viewer

enters into, agreeing to think of sound and image as forming a single entity" (Chion 1994, p.216)."

"

Added Value"•  Added value: "–  "the expressive and informative value with which

a sound enriches a given image "– so as to create the definitive impression, in the

immediate or remembered experience one has of it, "

–  that this information or expression 'naturally' comes from what is seen, "

– and is already contained in the image itself" (Chion 1994, p.5)."

Added Value"•  Added value works reciprocally: "–  on the one hand, "sound shows us the image differently

than what the image shows alone"; "–  on the other hand, image "makes us hear sound

differently than if the sound were ringing out in the dark" (Chion 1994, p.21)."

•  Added value is the most important of the relations between sound and image (Chion 1994, p.5). "

Synchresis"•  Based on the idea of synchronization (and also

synthesis), Chion created the notion of synchresis:"–  "the forging of an immediate and necessary

relationship between something one sees and something one hears at the same time" (Chion 1994, p.224)."

Synchresis"•  According to Chion, synchresis allows for numerous

combinations of possible sounds with possible images: "for a shot of a hammer, any one of a hundred sounds will do" (Chion 1994, p.63)."

•  But random associations may not generate synchresis: "play a stream of random audio and visual events, and you will find that certain ones will come together through synchresis and other combinations will not" (Chion 1994, p.63)"

Senses as Channels"•  Chion states that there is no "sensory given" that is

isolated from the start: "the senses are channels, highways more than territories or domains”."

•  He clarifies this, stating that "when Kinetic sensations organized into art are transmitted through a single sensory channel", they can convey all the other senses via that one channel. (Chion 1994, p.137). "

•  He exemplifies with the inherent visuality of concrete music, and the implied sound behind silent movies. "

Gene Youngblood’s Expanded Cinema"

Expanded Cinema"•  Expanded cinema has been expanding for a long

time. "•  Since it left the underground and became a popular

avant-garde form in the late 1950's the new cinema primarily has been an exercise in technique, "–  the gradual development of a truly cinematic language

with which to expand further man's communicative powers and thus his awareness. "

•  If expanded cinema has had anything to say, the message has been the medium."

•  Youngblood 1970, p. 75"

Expanded Cinema"•  Slavko Vorkapich: "–  "Most of the films made so far are examples not of

creative use of motion-picture devices and techniques, but of their use as recording instruments only.” "

–  “There are extremely few motion pictures that may be cited as instances of creative use of the medium, "

–  and from these only fragments and short passages may be compared to the best achievements in the other arts”."


Synaesthetic Cinema"•  The new cinema has emerged as the only aesthetic

language to match the environment in which we live."•  Emerging with it is a major paradigm: a conception

of the nature of cinema so encompassing and persuasive that it promises to dominate all image-making in much the same way as the theory of general relativity dominates all physics today. I call it synaesthetic cinema."


Synaesthetic Cinema"•  The new artist and the new scientist recognize that

chaos is order on another level, and they set about to find the rules of structuring by which nature has achieved it. "

•  That's why the scientist has abandoned absolutes and the filmmaker has abandoned montage."


Synaesthetic Cinema"•  Synaesthetic cinema is the only aesthetic language

suited to the post-industrial, post-literate, man-made environment with its multi- dimensional simulsensory network of information sources. "

•  It's the only aesthetic tool that even approaches the reality continuum of conscious existence in the nonuniform, nonlinear, nonconnected electronic atmosphere of the Paleocybernetic Age."


Mick Grierson’s Audiovisual Composition"

Audiovisual Composition"•  Conventionally, audiovisual composition may be

taken to mean the process of putting sound and music to visual material in order to create a 'soundtrack' to an existing film or animation. "

•  The study of this type of practice is often referred to as 'sound on film', 'audio for video' or 'sound design'. "

•  The theoretical debates which surround this practice are informed in particular by Michel Chion's Audio-Vision:Sound on Screen."

•  Grierson 2005, p. 9"

Audiovisual Composition"•  However, Chion also introduces concepts which

point to a more complex type of audiovisual practice. With this in mind, it is illuminating that Chion takes exception to the idea of the 'soundtrack'. According to Chion, there is no such thing as a soundtrack."

•  Grierson 2005, p. 9"•  Chion's justification for this view is bound up with his

notion of added value. This is, in essence, the idea that through the combination of audio and visual elements, a third audiovisual element is generated."

•  Grierson 2005, p. 10"

Audiovisual Composition"•  Therefore, the audiovisual work is more than a

combination of its component parts. "•  As such, the practice of audiovisual composition is

not simply the production of audio with video. "•  It is the process of composing audiovisual works

which exploit added value."•  Grierson 2005, p. 10""

Audiovisual Composition"•  So in audiovisual composition and analysis, added

value shifts the focus from separate audio and visual components, to the relationship between audio and visual components. "

•  This is particularly the case where a work has been designed from the outset with great attention to detail regarding the composition of both the sonic and visual material and their effect together, "

•  and even more so if the work attempts to exploit ideas about the formal relationships between sonic and visual material."

•  Grierson 2005, p. 10""

Bibliography"•  Chion, M., 1994. Audio-Vision: Sound on Screen, New

York: Columbia University Press."•  Grierson, M., 2005. Audiovisual Composition. Kent:

University of Kent. Available at: http://www.strangeloop.co.uk/Dr.%20M.Grierson%20-%20Audiovisual%20Composition%20Thesis.pdf."

•  Ox, J. & Keefer, C., 2008. On Curating Recent Digital Abstract Visual Music. In Abstract Visual Music. The New York Digital Salon. Available at: http://www.centerforvisualmusic.org/Ox_Keefer_VM.htm."

•  Youngblood, G., 1970. Expanded Cinema, New York: P. Dutton & Co. Available at: http://www.vasulka.org/Kitchen/PDF_ExpandedCinema/ExpandedCinema.html."

!

""""

notes on audio-vision and related concepts

Design