Information Theory.

Claude Shannon and Weaver, W. - The Mathematical Theory of Communication (Urbana, 1949). Norbert Wiener – Cybernetics (1948). Louis Brillouin – Science and Information Theory (1956) Abraham Moles – Information Theory and Esthetic Perception (U of I Press, 1958). Information as a measurable quantity. Art/music as a communication process: messages are transmitted through channels that have a certain, measurable capacity. Information is regarded as a value, not considered for its significance. The efficiency η of the channel is given by:

η = (Itransmitted – Ireceived) / Itransmitted The value of information is related to the unexpected, unforeseeable/unpredictable (i.e. probability), original. Originality and significance are distinct and intelligibility is the inverse of originality ( 1 / originality). Originality is a function of the improbability of the message: H = K log I where I = improbability = 1 / ω (ω = probability of occurrence). Logarithms. The logarithm of a number to a given base is the power or exponent to which the base must be raised in order to produce the number. Here are some examples: 100 = 1 log10 1 = 0 20 = 1 log2 1 = 0 101 = 10 log10 10 = 1 21 = 2 log2 2 = 1 102 = 100 log10 100 = 2 22 = 4 log2 4 = 2 100.69897 = 5 log10 5 = 0.69897 10-1 = 0.1 log10 0.1 = -1 Since log 1/ω = log 1 – log ω = 0 – log ω, the expression above becomes: H = -K log ω which mirrors Boltzmann's formula for entropy (second law of Thermodynamics): S = k log ω where S = measure of disorder and ω = probability of a given state In the case of binary choices (yes/no questions): H = -K log ½ = K log2 it defines the unit of information, bit, binary choice: H = -log2 ω. Abraham Moles: “The quantity of information transmitted by a message is the binary logarithm of the number of choices necessary to define the message without ambiguity”. H = -Nt ∑ pi log2 pi with ∑ from i=1 to i=n pi = probability of occurrence of symbols drawn from a repertoire n = symbols in the repertoire Nt = length of sequence in which the symbols are assembled N (replacing Nt) could also be the rate of events / second or M = Nt = the total number of elements in the sequence Complexity. Both organisms and machines have structure. “To understand them is to understand the structure” (Moles). The only universal dimension of structure independent of its elements is complexity, the multiplicity of possible situations. In perceptual terms, it is related to variety – and linked to probability. Maximum information. ∑ pi log pi might be considered a weighting (averaging) which implies that the maximum originality or information occurs when

pi = p = 1/n (equiprobability of occurrence) then H = -log2 1/n and ∑ pi log2 pi = n pi log2 pi (with ∑ from i=1 to 1=n) and H = k log2 n for n → ∞, H → ∞ Maximum number of possible choices implies maximum entropy or information. Relative information and redundancy. If H1 is the information content of a given message and Hm = -log2 1/n is the maximum entropy for the same message, the ratio H1 / Hm is the relative information of the message independent of its length. Redundancy is the complementary magnitude 1 - H1 / Hm Redundancy is a measure of “wastage” of symbols in transmittal of information. Redundancy is 0 when H1 = Hm and is (almost) equal to 1 when H1 is very small or 0 We can now draw the following comparisons: H entropy, disorder 1 - H1 / Hm structure, order information redundancy originality banality unforeseeability foreseeability aperiodicity periodicity novel output intelligible form complex simple Repetition → regularity → degree of coherence/predictability. Hence, “the amount of periodicity is then a degree of order” (Moles). Consider pitch vs. noise and meter vs. aperiodic rhythm. Semantic and esthetic information. Semantic messages are expressible in symbols and determine translatable, logical decisions. Esthetic messages determine interior states and are untranslatable. In the 1960s it was thought that information theory is a tool that can deal with perception and with esthetics in a rational, precise way. “Information Theory invades a territory reserved for philosophy” - Louis Brillouin. Musical applications. Xenakis uses ataxy (entropy) as another dimension of his screens (see his Markovian music), a dimension of the musical vector space. Also, consider the following: complex chords: the more elements they have, the more information, more difficult to understand → perceived as dissonant enharmonic modulation/chords: create an ambiguous situation, unexpected resolution to a far tonality, high entropy (information); redundancy produced by the common language (only certain chords allowed) and voice leading. The resolution releases information. complex polyphony and polyrhythms: difficult to follow every stream in works by Ives or integral serialists. Xenakis' criticism – too much information, beyond a certain threshold, music becomes incomprehensible as such. His solution: move to a different (higher order) level, textures governed by stochastic distributions. tone-row music: new non-repeating pitches → novelty, information; repetition of intervals → redundancy; combinatoriality adds to both structure and complexity. Debussy – creates a gentle, sensual background → redundancy paired with novel scales, chords → information and complex timbres. Schoenberg – originality in tone-row idea, klangfarbenmelodie, sprechstimme → information; redundant background (traditional rhythmic structure, form, orchestration). Webern – very high originality, strict structure → very high redundancy; extremely complex. Because of high redundancy,

intelligible. Cage – huge entropy, no organization; there is a limit to the attention (apperceptual limit) above which the channel shuts off. Why does that not happen here ? 1. Our mind projects form, structure, organization even when there is none; 2. events are few and spaced in time giving us time to assimilate the sounds (which is what Cage asks us to do anyway). Timing and pacing of information becomes very important. Wagner – leitmotives are introduced at the beginning of the opera in an one-to-one reltionship with the character, object or action they represent: someone tells a story and we hear them in clear association. In later acts they might be in counterpoint with what a character says on stage (Lohengrin) or depicting unexpected mental associations (Parsifal). Herbert Brün: the role of the composer is to retard the decay of information. As one listens to a piece it slowly reveals itself, its structure, the rules of the game; repeated auditions accelerate the process – the more one knows about it, the less information is derived. There is nothing one can do about this decay but the composer can find strategies that will insure a slowing down, a retardation of this process. Complexity plays a role: the information delivered by a pop tune decays much faster than the information of a Beethoven string quartet.