Analyzing Modular Smoothness in Video Game Music

Elizabeth Medina-Gray (Oberlin College)

Society for Music Theory, Annual Meeting, November 2014 Figure 1 The modular process

Figure 2 Definitions Sequential seam: One module sounds after another module with no overlap.

Simultaneous seam: Two modules sound at the same time (includes crossfades).

Smoothness: Two modules fit well together; agreement; merging/continuity.

Disjunction: Two modules do not fit well together; disagreement; separation.

Figure 3 Smoothness scale for modular seams

Entirely Primarily Half Partially Smooth Smooth Smooth Smooth

Partially Half Primarily Entirely Disjunct Disjunct Disjunct Disjunct

A single seam can be smooth in some ways and disjunct in others, because multiple aspects

of the music involved in the seam—including meter, timbre, pitch, volume, and

abruptness—can be either smooth or disjunct.

Sounding Music

Assembly

Modules + Rules

Medina-Gray, 2

Figure 4 Analytical rubric for sequential seams Smooth Disjunct Meter: Same Grouping or Grouping and

(agrees) displacement displacement Completely (continuous) dissonance dissonance discontinuous Timbre: All instruments Some instruments Very same same Different different Pitch: All PCs present Most PCs present Few PCs present No PCs present in macroharmony in macroharmony in macroharmony in macroharmony Volume: Same or similar Changes significantly during throughout seam seam (including to silence) Abruptness: Gradual fade-in and/or Abrupt entrance or fade-out, or pitches seem to exit mid-sound; enter and decay naturally cut off Sample additional contributions:

Gradual change across seam Significant difference in trends in meter, timbre, or pitch of consonance or dissonance Continuation of rhythmic patterns Significant difference in textural thickness Buffer during seam (silence > 250 ms, or a masking sound effect) Difference in sound location

Notes on using this rubric:

For each aspect (row) in each possible seam, compare the music in the 1 second after the seam (i.e., at the beginning of the second module) with the music in the 5 seconds before the seam (i.e., at the end of the first module). Show the results as probabilities in a grid.

Meter: Look for layers of pulse streams continuing across the seam. Grouping and displacement dissonance (after Krebs 1999) involve continuity in at least one pulse stream and conflict/discontinuity in others.

Timbre: Comparisons in this aspect can involve familiar instruments and/or synthesized timbres. “Very different” = different attack and quality of sound; e.g., guitar, bowed strings, and oboe are all “very different,” whereas guitar and pizz. strings are merely “different”. Piano and flute after just piano (or vice versa) is mildly smooth (“some instruments same”).

Pitch: Look to see how many of the pitch classes after the seam are also present in the

macroharmony before the seam. (Macroharmony = all pitch classes in a given span of music,

after Tymoczko 2011.)

Medina-Gray, 3

Figure 5 Analysis for a single OceanOutset Island sequential seam in Wind Waker (Nintendo 2003)

Figure 6 Analysis for all possible OceanOutset Island sequential seams in Wind Waker

5 seconds

1 second

(pizz.)

Medina-Gray, 4

Figure 7 Analytical rubric for simultaneous seams

Smooth Disjunct Meter: Grouping or Grouping and

Same displacement displacement Completely (agrees) dissonance dissonance disagrees Timbre: No instruments Some instruments All instruments Very different different different different Pitch: No new Soft One Two Three >Three intervals Consonance diss. hard hard hard hard (octaves only) only only diss. diss. diss. diss. Volume: Same or similar Significantly different (not counting rests or decay) (Only w/ Gradual fade-in and/or Abrupt entrance or continuity) fade-out, or pitches seem to exit mid-sound; Abruptness: enter and decay naturally cut off Sample additional contributions: Additional (dynamic, ongoing) Same rhythm/onset pattern process affects one module but not the other Additional (dynamic, ongoing) process affects both modules equally Difference in sound location Notes on using this rubric:

For each aspect (row) in each possible simultaneous combination, examine the two modules on a moment-by-moment basis. Show the results as probabilities in a grid.

Meter: Grouping and displacement dissonance (after Krebs 1999) involve alignment/agreement in at least one pulse stream and conflict in others.

Timbre: Piano on top of piano and percussion is very smooth (“no instruments different”). Piano and percussion on top of piano and flute is mildly smooth (“some instruments different”).

Pitch: Examine the intervals created by the two modules’ pitch classes in combination (i.e., the new intervals that are not already present in either module by itself). M2 and m7 are soft dissonances; m2, M7 and tritones are hard dissonances. E.g., {C,F#} + {C,G} = one hard dissonance (between F# and G).

Medina-Gray, 5

Select Bibliography

Bregman, Albert S. 1990. Auditory Scene Analysis: The Perceptual Organization of Sound. Cambridge, MA: The MIT Press.

Childs, IV, G. W. 2007. Creating Music and Sound for Games. Boston: Thomson Course Technology PTR.

Collins, Karen. 2008. Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. Cambridge, MA: The MIT Press.

Hasty, Christopher. 1981. “Segmentation and Process in Post-Tonal Music.” Music Theory Spectrum 3: 54–73.

Hoffert, Paul. 2007. Music for New Media: Composing for Videogames, Web Sites, Presentations, and Other Interactive Media. Edited by Jonathan Feist. Boston: Berklee Press.

Huron, David. 2006. Sweet Anticipation: Music and the Psychology of Expectation. Cambridge, MA: The MIT Press.

Jørgensen, Kristine. 2009. A Comprehensive Study of Sound in Computer Games: How Audio Affects Player Action. Lewiston, NY: Edwin Mellen Press.

Krebs, Harald. 1999. Fantasy Pieces: Metrical Dissonance in the Music of Robert Schumann. New York: Oxford University Press.

Marks, Aaron. 2009. The Complete Guide to Game Audio: For Composers, Musicians, Sound Designers, Game Developers. 2nd ed. New York: Focal Press.

Paul, Leonard J. 2013. “Droppin’ Science: Video Game Audio Breakdown.” In Music and Game: Perspectives on a Popular Alliance, edited by Peter Moormann, 63–80. Wiesbaden: Springer VS.

Sheffield, Brandon. 2008. “Staying In Tune: Richard Jacques On Game Music’s Past, Present, And Future.” Gamasutra. http://www.gamasutra.com/view/feature/3695/staying_in_tune_richard_jacques_.php.

Tymoczko, Dmitri. 2011. A Geometry of Music: Harmony and Counterpoint in the Extended Common Practice. New York: Oxford University Press.