renzo alves dantas torrecuso processing of syntax … · renzo alves dantas torrecuso . processing...
TRANSCRIPT
RENZO ALVES DANTAS TORRECUSO
PROCESSING OF SYNTAX IN MUSIC: AN EEG STUDY
Master thesis presented to the Post Graduation
Program in Neuroscience of the Federal University
of Rio Grande do Norte as a requirement to
obtain the title of Master in Neuroscience.
SUPERVISOR: PROF. DR. DIEGO ANDRÉS LAPLAGNE
CO-SUPERVISOR: PROF. DR. SIDARTA TOLLENDAL GOMES RIBEIRO
NATAL
2
Universidade Federal do Rio Grande do Norte - UFRN Sistema de Bibliotecas - SISBI
Catalogação de Publicação na Fonte. UFRN - Biblioteca Setorial Árvore do Conhecimento - Instituto d Cérebro - ICE
Torrecuso, Renzo Alves Dantas. Processing of syntax in music: an EEG study / Renzo Alves Dantas Torrecuso. - Natal, 2017. 46f.: il. Universidade Federal do Rio Grande do Norte. Instituto do Cérebro. Pós graduação em Neurociências. Orientador: Prof. Dr Diego Andrés Laplagne. Coorientador: Prof. Dr Sidarta Tollendal Ribeiro. 1. Sintaxe. 2. Música. 3. Eletroencefalografia. I. Laplagne, Prof. Dr Diego Andrés. II. Ribeiro, Prof. Dr Sidarta Tollendal. III. Título. RN/UF/BSICe CDU 612.8
3
2017
RENZO ALVES DANTAS TORRECUSO
PROCESSING OF SYNTAX IN MUSIC: AN EEG STUDY
Master thesis presented to the Post Graduation
Program in Neuroscience of the Federal University of Rio Grande do Norte as a requirement to
obtain the title of Master in Neuroscience.
Approval date: ___/ ___/______. EXAMINATION COMMITTEE
__________________________________
Prof. Dr. Diego Andrés Laplagne – President (UFRN)
________________________________________________________
Prof. Dr. Emilie Katarina Svahn Leão – Internal examiner (UFRN)
________________________________________________________
Prof. Dr. Peter Maurice Erna Claessens – External examiner (UFABC)
4
ACKNOWLEDGMENTS
Brain Institute, UFRN.
Prof Dr Diego Laplagne, Prof Dr Sidarta Ribeiro, Prof Dr Kia Leão, Prof Dr Peter Claessens
(UFABC), Prof Dr Kerstin Schmidt , Prof Dr Draulio Araujo, Prof Dr Tarciso Velho, Prof Dr
Marcos Costa, Prof Dr Richardson Leão, MSc Daniel Almeida , MSc Annie Costa, Dr Rodrigo
Pavão, Dr Nivaldo Vasconcelos, MSc Bryan Costa, Dr Vitor Santos, Dr Ernesto Soares, Rafael
Pedrosa, MSc Dardo Ferreiro, Dr Ignacio Sanches Gendriz, Dr Sergio Rolim, Dr Raphael Bender
Dr João Bacelos, João Patriota, Ana Raquel Torres, MSc Thawann Borges, MSc Barbara Boerner
MSc Diego Coelho.
Bergen University
Prof Dr Stefan Koelsch, Dr Sebastian Jentschke.
Freie University
Dr Moritz Lehne, Dr Stavros Skouras, Dr Martin Rohrmeier, Dr Joana Piperi, MSc Kamila
Borowiak.
EMUFRN
Prof MSc Juliano Ferreira, Prof Dr Ticiano Maciel Damore, Prof Dr Marcus André.
EC&T- UFRN
Prof MSc Diego Pena, Prof MSc Leonardo Mafra.
5
“Es Denkt In Mir”, Nietzsche.
6
ABSTRACT
In order to make sense out of a sequence of sounds in music, our brain must meaningfully fit
and recombine acoustic events into a hierarchic online stream. Although these information units are
auditively delivered in sequences with local connections (one after the other), it is assumed that
long-term dependencies are established counting on memory traces to sustain recursiveness in time.
Despite theoretical and empirical consensus, there is yet no clear physiological evidence of the
temporal dimension of syntactic relations in music.
We investigated whether there is quantifiable neural activity suggesting the existence of a
mental representation for fundamental music syntax rules like tonic-dominant-tonic chords. For
such, we compared brain electric activity in 24 subjects (12 musicians, 12 non-musicians) aroused
by original and harmonically modified versions of J.S. Bach chorales, using electroencephalography
(EEG). Chorales were built by two phrases: initiated by a tonic and arriving in a dominant chord
two bars away (first phrase), and concluded in a tonic chord three bars after the dominant (second
phrase). Modified versions were created either by elevating or lowering the first, therefore keeping
the second phrase intact. We compared the brain electric response for the last chord in both
versions.
Our data produced event related potentials (ERP) which suggest that the subjects’ brain
processed modified versions differently than originals. We observed an amplitude difference in the
negativities peaking around 210 ms after the last chord onset. This finding replicates previous
studies on harmonic disruptions processed in the same latency, but differs regarding brain source
contributions: our data revealed a more posterior than anterior effect. Considering that last chords
were the same acoustic event in both versions, we hypothesize that the amplitude differences
plausibly indicate that the syntactic expectancy (long-term dependency between the tonic, dominant
and tonic chord) may relate to brain mechanisms of processing non local connections, establishing a
hierarchical storage of acoustic events in memory.
We interpret that our parietal findings parallels with literature on mathematical sequence
processing which suggest posterior brain regions to be engaged on complex calculation requiring
the storage of numbers magnitude in memory for further computation.
Our study overcame an obstacle not to date surpassed: observe and separate brain electric
activity raised by long-term syntactic disruption in music from the overlap of local mismatch
detection.
7
INDEX
1. Introduction……………………………………………………………………………….....7
1.2. EEG wave Polarity…………………………………………………………….........10
1.3. Auditory Event Related Potential (ERP) Components………………….…….......11
1.4. N100………………………………………………….....…………...…………........12
1.5. MMN..........................................................................................................................12
1.6. LAN and ELAN to ERAN..........................................................................................16
2. Objectives...............................................................................................................................22
3. Methods..................................................................................................................................23
3.1. Stimuli..........................................................................................................................23
3.2. Subjects........................................................................................................................24
3.3. Procedure.....................................................................................................................25
3.4. EEG Recordings..........................................................................................................25
4. Results.....................................................................................................................................26
4.1. ERAN (Early Right Anterior Negativity)....................................................................26
4.2. Statistical analysis.........................................................................................................26
4.2. Musicians and Non Musicians.....................................................................................30
4.3. Brain regions...............................................................................................................31
5. Discussion...............................................................................................................................32
6. Bibliography...........................................................................................................................40
LIST OF ABREVIATIONS
ERAN: Early Right Anterior Negativity
MMN: Mismatch Negativity
LAN: Left Anterior Negativity
ELAN: Early Left Anterior Negativity
ERP: Event Related Potential
SLTDM: Syntactic Long-Term Dependencies in Music
8
1. Introduction.
Together with language and mathematics, music reveals our ability as humans to recognize
discrete elements, meaningfully fitting and recombining them into a hierarchical sequence. The
group of principles that guides this sequence is called syntax (Anderson 1967; Patel et al. 1998;
Winograd 1972) and recursion is its core property (Hauser et al., 2002; Lerdahl and Jackendoff,
1985). In the same way that the phrase The man threw his handkerchief is built by fundamental
hierarchic elements (subject, verb and object) which can be recursively stretched out to longer and
more meaningful structures (e.g. The man [ who was the first suspect of the robbery][for which the
charges have been removed] threw [with a blooded hand] his handkerchief) (figure 1, middle), a
western musical phrase can also expand its fundamental hierarchic elements (1st, 4th,5th and 1st
chords) to a chorale or a whole symphony movement (Jackendoff, 2009; Rohrmeier, 2011)(figure 1,
top). A citation from the XX century composer Arnold Schoenberg, stated in Theory of Harmony
(Schoenberg 1978), illustrates this hierarchy conception: “we can consider the chorale, as well as
every larger composition, a more or less big and elaborate cadence” (pg. 290). Cadence in its
elementary form is a particular chord sequence that represents a phrase end or suspension
(Bharucha and Krumhansl 1983; Tillmann and Bigand 2004). In a similar way, a mathematical
equation is organized in hierarchical operations necessary to find the correct solution:
multiplications and divisions precede sums and subtractions, and common factors can be identified
resuming calculation steps (figure 1, bottom).
Even though theorists have developed in the past fifty years musical grammars sustaining
that harmony is governed by recursiveness or long-term dependencies (Hofstadter 2000; Lerdahl
and Jackendoff 1985; Rohrmeier 2011; Schoenberg 1978; Steedman 1984) there is still no
consensus. Authors from music analysis (Clarke 1987; Konečni 1984; Marvin and Brinkman 1999),
musical psychology (McAdams and Bigand 1993) supported by some sparse empirical evidences in
experimental psychology (Bigand et al. 2003; Tillmann and Bigand 2004), argue that the listener
does not perceive music taking account long-term dependencies, but on the contrary, that the local
models are the basis of musical discourse and perception (Tillmann and Bigand 2004)
As most above mentioned experimental psychology works are based on subjects' ratings on
the perception of music harmonic irregularities, and there are still few studies approaching this issue
based on brain measurements analysis on naturalistic stimuli (real music), we wish to shed light on
the topic by an experiment design which combines electroencephalography (EEG) measurements
with subjects ratings using western music short models, J.S. Bach chorales.
9
Figure 1. Music, language and math as sequences based on non-local dependencies. Top: Music
score depicting, in the first two bars( vertical lines), the long-term dependencies between the G Major chord ( 1st
chord of the tonality´s scale, hence I ) and the D Major chord( 5th chord of the tonality´s scale, hence V) which
establishes an expressive tension. In the last three bars, the long-term dependency between the D Major chord,
5th (dominant) and the G Major chord, 1st (tonic), which resolves the expressive tension. Middle: Phrase
exemplifying long-term dependence in language between subject, verb and object. Note that, as in the music
score, sentences can be recursively inserted between subject and verb, and also between verb and object. Bottom:
in blue “1/2” is separated for later calculation while multiplying 3x5 arrives to a two steps fraction sum.
Intermediary steps are calculated until 1/2 is finally computed as a simple sum producing the final result.
We also intend to investigate if musical expertise may alter brain regions involved in
processing the perception of music long-term dependencies. In order to do so, our 24 subjects were
separated in musicians and non-musicians. We designed an experiment in which 64 EEG electrodes
were positioned in the subjects' scalps while they listened to a pseudo random sequence of J.S. Bach
chorales. The chorales had their harmonic structure modified and subjects were made unaware of
10
the harmonic modifications by engaging in a task relevant paradigm during a presentation of a silent
movie.
As we also intend to compare neural responses to perception ratings of long-term
dependencies, after the scanning session ended subjects were presented again to a pseudo random
sequence of some of the chorales listened previously but this time subjects received sheets and were
asked to rate the stimuli according to its conclusiveness.
This work is based on the past 50 years of research using EEG to identify auditory event
related potentials (ERP) associated to the processing of irregularities in a sound sequence, oddball,
mismatch negativity (MMN), of grammatical irregularities with the so called left negativity (LAN)
and the early left anterior negativity (ELAN), and musical syntactic disruptions with the early right
anterior negativity (ERAN)(Loui et al. 2005; Maess et al. 2001a; Poulin-Charronnat, Bigand, and
Koelsch 2006; Sammler, Koelsch, and Friederici 2011). It has also been observed that subjects'
perception of harmonic integration can be verified by a late negativity around 500 ms (N5)(Stefan
Koelsch 2012; Steinbeis and Koelsch 2008). Previous studies have mainly focused on local
harmonic dependencies, e.g. the succession of one chord to another. Here, we propose to investigate
if non local (long-term) syntactic musical disruptions (e.g. the non fulfillment of the relation
Dominant to last Tonic, separated by several chords) can also be detected by ERP components.
One may ask what is the relevance of proving that our brain processes long-term
dependencies in music without focused attention. First, if it does, this provides evidence that our
brain indeed does perceive music as a code based on recursiveness (an automatic pre attentional
hierarchic organization of acoustic features: in other words, we may contemplate the assumption
that before one consciously focus attention on a piece of music our brain organizes and attributes
different weights to the incoming acoustic features). This also contributes to theoretical and
experimental studies proposing that our brain keeps track of concomitant streams of information
governed by hierarchical rules (Dehaene 2015; Jackendoff 2009; Sussman, Ritter, and Vaughan
1999; Tillmann and Bigand 2004). Second, considering that recursiveness is a core aspect of
language and math, our findings may also contribute to studies which point to a common neural
architecture in processing of sequences of information (Maess et al. 2001b; Poulin-Charronnat,
Bigand, and Koelsch 2006; Sammler, Koelsch, and Friederici 2011; Dehaene 2015). This may also
be taken in consideration by studies in evolutionary theories that consider the close emergence of
music and language in human history (Kivy 1959). Third, as will be discussed in next section, most
event related potentials (ERP) studies using EEG have found evidences about pre-attentive brain
mechanisms. By using this technique to investigate the pre-attentional neural traces aroused by a
complex stimuli as polyphonic music, the present research will provide considerably precise time
information about when does our brain engage in cognitive computations without our awareness.
11
1.2 EEG waves' polarity.
The neuron is a brain cell characterized by a morphological regularity (comprising soma,
axons and dendrites) and by its electrical signaling property. The action potential is a well-
documented voltage variation phenomenon responsible for propagating neuronal information.
Nevertheless, the intricate mesh of brain tissue, with thousands of neuronal projections, and the
depth of a neuron's location prevents the action potential of a neuron to be detected by a scalp
electrode.
Figure 2. Location of afferences influence wave polarity (Kandel 2014). Schematic view of
afferents' input on apical dendrites of pyramidal cells (in layers 2 and 3) producing a current sink of cations
which contributes to the detection of a negative deflection by electrodes in the scalp (right). Conversely, similar
neurons with afferents in deeper layers, near the soma, produce a local current source and therefore may
contribute to a positive deflection measured in scalp electrodes (left). .
Therefore, superficial electrodes from EEG machines can only measure the summation of electrical
activity originated by assemblies with thousands (or hundreds of thousands) of neurons. Most
cortical neurons are pyramidal and are positioned in parallel to each other. This geometric disposal
together with the location of synaptic afferents in the apical dendrite contributes to the polarity that
a scalp electron may register. Kandel (Kandel, n.d.) points that excitatory thalamic afferents if near
the current source, around layer 5, will give rise to a positive deflection, while excitatory potentials
12
originated by axons coming from contralateral cortex and reaching the apical dendrite on layer 2
will give rise to a negative deflection because the action potentials will produce a current sink and
therefore generate a negative electric field (figure 2). Conversely, inhibitory potentials produce
negativities in deeper layers and positive deflection in superficial layers. Hereon, we will refer to
negativities according to this rationale.
1.3 Auditory Event Related Potential (ERP) Components
ERP components (e.g. N1, MMN, LAN, ELAN, ERAN) are time labels to multiple brain
processes that occur at given latencies. It is therefore misleading to attribute brain processes to the
emergence of a particular ERP component: a N1 is not necessarily produced by sensorial stimuli, a
MMN wave is not an undeniable evidence of the brain perception of a sound object, nor its absence
excludes the possible occurrence of the processes attributed to this component (e.g. sound object
recognition, comparison to environment, auditory stream discrimination, among others)(Schröger
2007). We stress that ERPs are time symptoms of given brain mechanisms but ERP\s may not be
alone causally linked to certain brain mechanisms, as diagnosis. Being that so, when one reads, for
instance, “N1 reveals frequency specificity”, it means that, according to the experiment design used,
it is within the 100ms latency after stimulus onset that the brain processes, among other stimuli
properties, frequency specification.
The earliest well known works that used EEG to relate a subjects' specific task to his
electrical brain activity took place at the Psychology and Physiology departments of Carlifornia´s
University in Los Angeles, during the 1960´s. Donchin, Lindsley, Haider and colleagues focused on
to what extent cortical activity could be time-locked to events in a reaction time sequence
and how attention could modulate these potentials (Donchin and Lindsley 1966; Haider, Spong, and
Lindsley 1964; Lindsley 1952). The studies used a similar experiment design: three subjects had
brain activity measured with four silver cup electrodes (Oz, T3, and Cz from the international "10-
20" system) of an electroencephalograph Grass model 6, on a frequency modulated tape recorder.
Scanning was conducted while subjects performed a visual task of identifying deviant luminosity
flashes of light from a sequence of standard flashes presented with regular inter stimulus
interval(ISI). Their preliminary results isolated a negative wave peaking around 160 to 200
milliseconds with an average amplitude of 5 micro volts which was time-locked to the deviant
stimuli, a light flash with diminished or increased luminosity compared to a sequence of standard
flashes. This component was obtained by averaging brain activity throughout a task that would be
repeated for around 100 minutes.
13
The technological improvements of the past five decades, mainly the use of computer,
impedance reduction and association with other techniques (e.g. Magnetoencephalography, PET and
fMRI), enabled researches to identify earlier components and divide them in subcomponents
according to their functionality and neural generators (Garrido et al. 2009; Stefan Koelsch 2012;
Risto Näätänen, Jacobsen, and Winkler 2005).
1.4 N100.
The above mentioned negativity, for instance, is nowadays observed in earlier latencies,
around 100ms, reason why it became named N100 or N1. For the purpose of this work the N1 will
not be discussed in depth because our questions relate to the processing of long-term dependencies
in music, which literature has suggested to be observed in latencies between 180-200ms (Stefan
Koelsch et al. 2000) or even later 125-300ms (Friederici 2004). The N1 has been interpreted as
divided into 6 sub components according to their different neural sources and refraction time. The
first three are considered as the “true” N1 and the other ones as an overlap with the MMN (R.
Näätänen and Picton 1987), discussed below.
Regarding its functionality, the “whole” N1 component can be summarized as the first
prominent and stable brain response sensible to the physical energy variances (intensity and
frequency) of a stimuli, even though after a short time repetition has its amplitude diminished (May
et al. 1999). This means that N1 can be used as a neurophysiological parameter of the brain's
response to different sound frequencies (R. Näätänen et al. 1988). In the auditory modality, the N1
is an ERP component more implicated to the sensorial neural response of a sound than its
perceptual trace (Schröger 2007).
1.5 Mismatch Negativity (MMN).
While in N1's latency (80-120ms) we can observe a sensorial brain response, it is beyond
this latency that EEG experiments have identified correlates of many cognitive processes such as
sound patterns and stream discrimination. Therefore the many different labels (MMN, ERAN,
ELAN, EAN) that are attributed to the negativities peaking from 120 to 400ms are findings
suggesting that during this time window a group of specific cognitive tasks are processed and can
be broadly localized in the brain ( “broadly” is used here due to EEG´s typical unreliability for brain
source location). Moreover, taking advantage on EEG´s good temporal precision, studies have
investigated the time threshold of subjects’ conscious auditory perception by linking
electrophysiological responses to unperceived auditory stimuli (Loui et al. 2005; R. Näätänen and
14
Winkler 1999; Sussman, Ritter, and Vaughan Jr. 1998; Winkler, Karmos, and Näätänen 1996).
Among all ERP components studied so far, the MMN has accumulated the biggest literature
which established a landmark in the research on latencies and neural sources of pre attentional
cognitive processing (Escera et al. 2000; Garrido et al. 2009; Risto Näätänen, Jacobsen, and
Winkler 2005; Schröger 2007). The MMN is defined as a negative wave that peaks between 100 -
200ms generated in the secondary auditory cortex in response to an auditory change compared to a
previous homogeneous sequence (Winkler et al. 1990). Schröger argues that in order to process
sound features of an ongoing sound stream the brain must realize at least 3 basic processes: 1-
separate the incoming sound in different features (pitch, intensity, duration, timbre) enough to
isolate a distinct sound object; 2- create and retain a model built with the very recent past sound
information apprehended from environment; 3- compare the incoming stream to the sound object
just built (Schröger 2007). It is widely accepted that all these processes occur before the subject
consciously focus attention on the deviant sound object (K. Alho et al. 1989; Kimmo Alho et al.
1992; Escera et al. 2000; Jääskeläinen et al. 2004; R. Näätänen and Winkler 1999) which leads to
the assumption that sensory memory is active pre attentively, storing spectral, temporal and spatial
information, during this latency after stimulus onset (Alain and Woods 1997).
Investigating MMN´s independence to attention, Näätänens et al., created an experiment in
which subjects were requested to detect intensity and frequency deviants of a sound sequence
delivered in one ear while they received in the other ear another sequence with same type of
deviants. Rather than observing that stimuli coming from both ears (attending or ignoring) elicited
MMN, their results showed that the MMN's amplitudes were not affected by attention for frequency
deviants, but only for intensity deviants. This finding contributed to later studies of the fading out
effect when one focuses attention in a single sound source in a party, the so-called cocktail party
effect (Conway, Cowan, and Bunting 2001). In order to test the possibility that it was the lack of
attention which reduced the MMN´s amplitude for unattended intensity stimuli, another experiment
was conducted where subjects received the same stimuli but this time completely ignored it while
reading a book. Results showed that MMN amplitudes' for unattended intensity stimuli were
smaller than the amplitudes when subjects were completely ignoring all sound during a reading
section. In other words, when subjects are not paying attention at all to the stimuli, their MMNs
amplitudes are higher than the response to one of the ears receiving stimuli while focusing attention
to the other. Based on these findings Näätänens et al., proposed that is was due to a probable
increase in the general demand of brain resources (resulting from the deliberate focus of attention)
that caused a diminished activation of MMN mechanisms, but that it was not attention per se that
influenced the component (R. Näätänen, Paavilainen, et al. 1993; Sams et al. 1985).
15
Following this research line, Sussman et al., investigated how early sensory memory could
segregate sound streams and, also, to which extent it was influenced by attention (Sussman et al.,
1998, 1999). The first work (published after the second) consisted on a sequence of 50ms
alternating high and low tones connected in groups of 6 tones: H1-L1-H2-L2-H3-L3. The deviant
stimuli, presented on a 16% rate, were either low or high deviant, H1-L3-H2-L2-H3-L1 or H3-L1-
H2-L2-H1-L3, respectively. In order to create a sound stream segregation effect, stimuli were
presented to subjects with short or long ISIs, 100 or 750ms respectively. The short ISI produces the
perception of two separated sound streams using the same frequencies as the long ISI. For the
control condition only the low tones were used separately, L1-L2-L3. Subjects were instructed to
ignore the stimuli while reading a book (Figure 2). Authors observed MMNs for the alternating
tones and for control condition, what is to say, the results suggest that subjects did discriminate
sound streams without attention focus.
Figure 3. ISI influenced MMN emergence. (Sussman et al., 1999) (a) with long ISI subjects
presented no MMN to deviant tones; (b, c) the tones presented with short ISI segregate to low- and high-tone
sequences pre attentively creating the memory trace underlying the MMN system maintaining the two
sequences; (d, e) the deviant sequences presented in the high or low streams was offset by one cycle so that the
low- and high-tone deviants did not occur within the same cycle of six tones. The low-tone deviant always
preceded the high-tone deviant .
The second study of Sussman et al. replicated the same experiment but this time with
subjects focusing attention on stimuli in a detection task. The results were in line with Näätänen´s
1993 study, and authors similarly suggested that it is not attention itself that modulates cognitive
16
processes in this latency, but the drain of neural resources caused by attention for a concurrent
stimuli.
Both studies produced important secondary findings: 1. MMNs were found exclusively for
the low tones either when presented alone or embedded in the alternating sequence; 2. all subjects
that presented MMNs for the control condition did not present MMN during the alternated
condition; 3. The control condition produced MMN only in four of nine subjects. An important
observation is that the authors did not make any distinction of musical expertise among the subjects.
Most likely, musical expertise can have influenced subjects to present MMN exclusively for the low
deviants. Western music is based on harmony which relies on the lowest notes of chords to establish
musical context, and harmony is implicit even in a one voice melody (Rohrmeier 2011). We
therefore speculate that subjects trained in music or even long exposed to it may have listened to
Sussman et al. experiments' stimuli differently, inattentively considering more the lowest note,
which holds fundamental information about the harmonic direction.
Converging the explanation of classical ERP studies to the theoretical grounding of this
research, our hypothesis that long exposure or intense practice of music may alter brain plasticity is
based on another of 1993 Näätänen´s study (R. Näätänen, Schröger, et al. 1993). In that work, they
asked whether MMN´s could produce evidences of the development of a memory trace for complex
sounds. The experiment consisted on the delivery of sequences containing complex spectral sound
patterns, a train of 180 standard blocks 365 ms long composed by the succession of 8 segments
with the frequencies 720, 500, 638, 1040, 1175, 565, 815 and 920 Hz. The 20 deviant blocks stimuli
pseudo randomly distributed within the standards were an alteration of the sixth segment (from 565
to 650Hz). Subjects were reading a book during 3 sessions ( early, middle and late) of unattended
Figure 4. Conscious contact with stimuli leads to
development of sensorial memory trace maintained in
further unattended tasks (Näätänen et al. 1993). Between
'early', 'middle' and 'late' sessions subjects tested if they could detect
deviant stimuli. Note increase of MMN´s amplitude(orange shade)
after more detection sessions. Bottom, in blue, depiction of the 8
frequencies used to build the complex spectral stimulus. The deviant
was produced by changing the sixth frequency (arrow).
17
listening, between these sessions they were required to listen to a short session of the stimuli
previously presented and try to identify the deviants. The results showed that subjects' MMN
presented increased amplitudes in progression with the detecting sessions, even though during the
early, middle and late sessions they were not attending to the stimuli (Figure 4).
The control group did not submit to the detection sessions. The results for this group showed
small amplitude MMN and with no differences between the sessions early, middle and late. This
result was interpreted as a consequence of the lack of sessions in which subjects consciously
attended to the stimuli: control subjects that did not go through detecting sessions were prevented
from forming a memory trace for the acoustic pattern, which from there on, would not be
incorporated to their long-term memory. This experiment showed how MMN amplitude increases
may be linked to the formation of sensorial memory traces for complex spectral stimuli. We suggest
that by Näätänen´s study results (R. Näätänen, Schröger, et al. 1993) it is very likely that musicians
ERPs to syntactic musical perception may differ from non-musicians due to the former´s intense
exposure to music.
1.6 Left Anterior and Early Left Anterior to Early Right Anterior Negativity (LAN and
ELAN to ERAN).
Authors differentiate ERP components by the latencies and their suggested functionality, that
is, which cognitive processes are being computed during that specific time window. While the
properties of sound features extraction and retention in sensory memory and comparison with a
built up model are the core distinction between MMN and N1, for the next latencies it is the
topography and time related aspects of long-term memory that establishes the distinguishing line.
Specifically, the border discussion is around studies that have been relating MMN to abstract, and
therefore not to a sensory stimuli (Paavilainen, Jaramillo, and Näätänen 1998; van Zuijen et al.
2006). By abstract the authors referred to a sequence of auditory stimuli which were frequent
ascending standard tone pairs with randomly deviant descending tones: as all tones were presented
in regular intervals, the descendent deviant tone elicited a MMN not due to the acoustical novelty (
since it was presented in the previous tone) but to a generalized abstract rule. Including the
processing of abstract features to the MMN´s latency (100-200ms), such as implicit rule extraction
from a sound stream, leads to the assumption that this component relates to long-term memory
based processes, since subjects necessarily needed time to implicitly learn the abstract rule
governing the auditory stream and then react to it pre-attentively. We argue that even though some
studies suggest that MMN does emerge for irregularities in musical scales, which can point to some
18
kind of syntax memory, the latencies observed in the mentioned studies (Paavilainen et al., 1998;
van Zuijen et al., 2006) happen to occur later than most MMNs, at 250 and 275ms, respectively. In
other words, it is likely that there is within this “late” MMN an overlap of waves generated by
neuronal populations implicated in long-term memory activity.
We deduce the above by examining the past twenty years literature of ERP studies focused
on latencies beyond 200ms. The first ERP studies to migrate from sensorial stimuli to explore pre
attentional long-term memory based processing came from language research. The early left
anterior negativity (ELAN) and left anterior negativity (LAN) were the first labels to comprise the
post MMN latency (between100-300ms, 300-500ms, respectively) and were attributed to morpho-
syntactic violations such as gender and number agreement, pseudo words and verb-argument
incongruities (Friederici, Pfeifer, and Hahne 1993; Gunter, Friederici, and Schriefers 2000; Hahne
and Friederici 1999; Münte, Heinze, and Mangun 1993; Neville et al. 1991). The widespread
experiment design used in the majority of LAN and ERAN studies can be found in Osterhout,
Bersick, and Mclaughlin 1997. A protocol in which phrases containing syntactic (gender)
irregularities were visually presented to subjects during EEG scalp recording. Here the phrase The
man prepared himself/herself to the operation elicited a LAN peaking around 300ms after the onset
of the gender definition violation. In a similar protocol (Hahne and Friederici, 1999) found an
ELAN peaking at 200ms and 400ms after past participle violations (Figure 5).
LAN ELAN
Figure 5. Language syntactic errors eliciting LAN(Osterhout et al., 1997) and ELAN (Hahne
and Friederici, 1999). Left, LAN observed 300ms after the presentation of nouns of phrases which made
definitional gender match (solid line) or mismatch (doted): in response to a final word in sentence which had a
reflexive pronoun (e.g 'herself', 'himself') that matched or not with the phrases' subject. Right, ELAN observed by syntactically incorrect sentences containing a phrase structure error: a preposition
19
appeared after the auxiliary and was directly followed by a past participle: The goose was in the fed . ELAN
peaking around 200ms for 20% of deviants per block of phrases and around 400ms for blocks with 80% of
phrases containing syntactic violations.
Following this line of research Stefan Koelsch et al.(2000) used a similar experiment design
to address the same language question but here applied to music: the authors proposed to test
whether it would be possible to observe ERP components in response to the violation of an
automatic built context in music, to the violation of the expectation on an overall previously learned
set of rules. Even though the word syntax is not mentioned in the article, a previous long-term
memory acquired by implicit learning was indispensable (as happens in language) given the fact
that the ERP was not elicited by an acoustical novelty effect. Such was their suggestion that there
was among the subjects a brain ability to learn a musical syntax by implicit learning, that the
authors named the article “Brain indices of music processing: 'Non musicians' are musical”.
The experiment consisted in auditively presenting non-musician subjects to 172 sequences
of five chords. Among these chords 10% had timbre deviants inserted to create a sound detecting
task. Subjects were instructed to identify these timbre deviants. As shown in the figure 6, the five
chords sequence were harmonically concatenated according to western music theory rules in order
to produce syntactically stable phrases (a, left); slightly unstable phrase with a Neapolitan chord in
the third position(a, middle); and a unstable phrase with a Neapolitan chord in the fifth position( a,
right). The Neapolitan chord is widely used in western tonal music for modulation and expressive
purposes because it is a consonant minor chord (when played isolated) with notes out of the tonality
but which can be inserted in major harmonic contexts (progressions) and sound like a subdominant
chord (Bharucha and Krumhansl, 1983; Krumhansl and Kessler, 1982). This harmonic resource
enabled the authors to access a negativity lateralized to the right hemisphere and peaking around
200ms (figure 6, c). This occurred with the highest amplitude when the Neapolitan chords were
presented in the fifth position (figure 6 c, left) and with a much smaller amplitude when in the third
position(c, middle). These results confirm the theoretical works (Piston, 1948)(page 265) which
explain the Neapolitan to be acceptable as a subdominant, even though it’s out of tonality notes
make it an exotic subdominant, reason why we hypothesize the even though present significantly
smaller negativity.
20
Koelsch and colleagues justify that although the latency was still within the MMN, the
experiment design and stimuli pointed to different brain processing, much related to an automatic
hierarchy recognition than to acoustical events. Observing a high resemblance with LAN and
ELAN components from language, which were elicited for syntactic disruptions, the authors
decided to name this component early right anterior negativity (ERAN) to brain responses to
musical syntactic violations.
Considering that the ERAN was elicited by a brain mechanism involved in grasping musical
syntax pre attentionally, authors hypothesized that intense practicing and exposure to music could
contribute to improve the neural architecture demanded for musical syntactic computing. Using the
exact same protocol as the previous study, Koelsch, Schmidt, and Kansok (2002) investigated the
effects of musical expertise in the emergence of the ERAN. The results showed that indeed
professional musicians with declared daily practice presented ERANs with higher amplitude when
compared to novices (figure 7, left). Replicating the Koelsch 2000 study but seeking to understand
how adulthood influences ERAN, Koelsch et al., (2003) tested children aged 5 and 9 years (figure
7, right). The results suggest that children do process music based on hierarchical rules and that
older kids produce ERANs with higher amplitudes. The study also revealed that no ERAN was
produced when children listened to the Neapolitan chord in the third position (the position in which
adults showed a very small amplitude ERAN). Authors interpreted that adults amplitudes (even
though smaller) on the third position were due to their overall higher specific representation of
music regularities: since children were less exposed to music their mental representation of
harmonic rules is less specific.
Figure 6. Syntactically incorrect chords produce ERAN (Koelsch et al., 2000). a.(left) syntactically correct chord progression without Neapolitan chord, ( middle) slightly incorrect, “expressive”, chord progression with Neapolitan in the third position, (right) syntactically incorrect chord progression ending with Neapolitan on last chord(fifth); (b) sequences of 'a'(left, middle and right) were connected in order to simulate a musical harmonic progression; (c) ERAN peaking around 180ms triggered by the Neapolitan in the incorrect (fifth) position(left), and on the third position(middle). FT8, location of the electrode with strongest ERAN effect (right).
21
Figure 7. ERAN is influenced by expertise and age (Koelsch et al., 2002a; Koelsch et al., 2003). Left: Subjects with music expertise presented ERAN peaking around 200ms with higher amplitudes compared to
non-musicians (novices) for Neapolitan chords presented at 5th position. ERAN was followed by a N500. Right: 5
and 9 year old subjects' ERAN (solid line) elicited by Neapolitan chord in 5th position around 220ms. Dotted line
for in syntactically correct chord (no accidents) in fifth position.
Adding some debate to the ERANs studies, Brattico et al., (2006) extrapolated the concept
of syntax in chords and proposed to examine which ERPs could be elicited by out of tune and out of
key notes in homophonic melodies(without chords, simultaneous sounds). A negativity peaking in
the common latency of both ERAN and MMN was observed for both out of key and out of tune
notes, with higher amplitude for the latter. Authors argued that the neural response to the deviant
stimuli could not be attributed to acoustical or novelty properties since there were no repeated notes
enough to create a standard pattern. The findings relate to the source of this ambiguous ERP:
ERAN are observed with higher peaks at frontal and fronto temporal scalp regions(Koelsch et al.,
2000, 2002a) and the negativity of this study had higher peaks at superior temporal lobe which are
known MMN peaking regions. These results point to a reconsideration about how early (around
200ms) long-term memory based processes of syntactic integration are computed in the brain.
Independently of the labels, many authors within the period of this discussion seeked
support for the existence of a memory based automatic musical syntax recognition by investigating
the brain regions involved in music and language processing (Besson and Schön 2001; Stefan
Koelsch et al. 2002; Maess et al. 2001a; Patel et al. 1998; Schön, Magne, and Besson 2004;
Tervaniemi et al. 2000). Combining different brain imaging techniques, fMRI, MEG and PET,
Koelsch and colleagues gathered important findings which support the overlap of brain regions
involved in music and language recognition (Besson and Schön, 2001; Ehrlé et al., 2001; Halpern et
22
al., 2004; Koelsch, 2005; Koelsch and Friederici, 2003; Koelsch et al., 2002b, 2005; Maess et al.,
2001). There is a general acceptance among the above-mentioned studies that musical and language
processing share common networks in the brain. These evidences strengthen the hypothesis that the
brain has an automatic mechanism able to process music and language according to their inherent
syntactic rules.
23
2. Objectives
General
i. Identify whether there is a brain processing of SLTDM.
ii. Identify if musicians and non-musicians present different EEG brain
measurements when exposed to disruptions of SLTDM
Specific
i. Determine the latency and predominant topographical distribution of the
eventual ERP raised by SLTDM.
ii. Observe eventual differences of this ERP for SLTDM with previous works on
MMN.
iii. Identify the amplitudes and latencies differences among musicians and non-
musicians.
24
3. Methods
3.1. Stimuli
Willing to investigate the perception of real music stimuli we selected two J.S. Bach
chorales, BWV 302 and 373. These chorales are built by two phrases separated by a half cadence on
the last chord of the first phrase and obey the traditional harmonic sequence of 1st - 5th - 1st chords ,
also called Tonic – Dominant – Tonic chord progression.
We created a sequence containing original, modified and timbre deviant chorales. Modified
chorales had the first phrase transposed a second up for BWV 302 (figure 8) or a fourth down(BWV
373). This manipulation was planned in such way that local harmonic relations were preserved
while in the same choral the long-term hierarchy were disrupted. In order to preserve local
dependencies we based the choral selection and manipulation criteria on harmonic rules (Rohrmeier
2011; Schoenberg 1978) as follows: the following chord from the second phrase should be different
from both the tonic (which would conclude the sequence since the previous was the dominant) and
the seventh( which would reinforce the dominant); and should also not produce longer intervals
then a fifth in the lowest notes, since it would create a discrepancy in the general pitch of the second
phrase, and therefore break the local nest.
Note that our manipulation created identical acoustical stimuli but with a long-term
dependence incongruity. As illustrated in figure 7, while in the original version ( A) the 1st -5th -1st
global dependency is fulfilled by the chords D major(tonic)- A Major(dominant)- D Major(tonic), in
the modified version (B) this hierarchy is disrupted since the expected E Major chord as the
conclusion of the choral, fulfilling the E Major( tonic) – B Major ( dominant)- E Major( tonic), does
not occur. Importantly, the final chord is the same in both versions.
Score manipulation were realized using Sibelius 6.2 software( Avid Tech Inc.). In musical
instrument digital interface(MIDI) the two versions were transposed to all 12 keys. Summing an
amount of 48 stimuli ( 2 chorales x 2 versions x 12 keys). Each stimuli was presented 5 times all
stimuli summed 53 minutes. In order to assure that second phrases were acoustically the same in
both conditions, the latter were copied and pasted using sound edition software Audacity 2.0
(audacity.sourceforge.net). Tempo was 100 beats per minute and the timbre was piano sound
for original and modified versions and for timbre deviant version two seconds of bassoon timbre
were inserted in one of the voices of the chorales, in 15% of trials. This insertion was used as a task
relevant paradigm to keep subjects aware and focusing attention on the stimuli but not specifically
in the harmonic disruptions.
25
Figure 8. Chords long-term dependency manipulation of the chorale Ein feste Burg ist unser
Gott (J. S. Bach, BWV 302). (a) The chorale has 2 phrases: the first ( D to A) begins with a D Major chord
in the 1st position (tonic) of the tonality scale and ends on a A Major chord in the 5th position (dominant)
producing a half cadence, suspension. The second phrase ( A to D) begins with the chord in the 5th position
(dominant) and ends on the final chord in the 1st position (tonic). This progression holds the long-term musical
syntactic dependency known as '1st - 5th - 1st progression' where the 5th chord establishes tension relation with
the 1st chord at the beginning and a relaxation, tension resolution with the 1st chord at the end; (b) The long-term
dependency is altered by shifting the whole first phrase( D to A) one tone up ( where previously there was a tone
D went up to E, C to D, E to F, and so on) making it an E to B phrase. This manipulation kept the syntactic
coherence in the first phrase but altered the long-term dependency for the last chord ( D), therefore breaking the
'1st - 5th - 1st progression'.
3.2. Subjects
We gathered 12 musicians and 12 non-musicians, 6 females in each group, age-range 23–39
years, M = 27:7. Musicians were recruited from Universität der Künste (Berlin) and had at least 10
years of formal instrument study. Non musicians stated that had no instrument classes besides
regular school education. All subjects were right handed and none had hearing disabilities nor
absolute pitch. Study was approved by the ethic committee of Freie Universität Berlin and was
runned according to Declaration of Helsinke.
26
3.3. Procedure
Subjects were comfortably siting while listened to the sequence of stimuli delivered through
headphones and watched “The Penguin’s March” in a monitor. Subjects were required to hit the
space bar whenever identified a deviant timbre among the stimuli.
After scan session, all subjects were presented again to twelve of the chorales used
previously( 6 from each version) and were required to fill a questionnaire with a 9 level rating
about how conclusive was the last chord, its valence, pleasantness, and arousal.
3.4. EEG recording
Recording used a 64 silver electrode distributed according to 10-20 system, referenced to the
left mastoid, four extra electrodes were placed around the left eye to capture electrooculogram
(EOG). Impedance was kept under 5 k ohms.
27
4. RESULTS
This study proposed to investigate if there is a mental representation for the
theoretical assumption that we perceive music based not only on non-local relations (one
chord/note after the other) but establishing long-term connections between sounds. As a
consequence, according to basic harmony rules, in order to unfold in time, music requires
our brain to organize incoming notes of a music into a hierarchy (e.g. first and fifth chords
have a bigger weight) and hold this pitch information in memory until it reappears assuming
different harmonic functions (e.g. a bridge to neighbor tonalities or the music´s conclusion).
To test our hypothesis we used harmonically modified J.S.Bach chorales which kept local
relations intact while long-term tonic to dominant chords dependencies were broken. We
measured and compared brain electric activity ERPs from the last chords (the concluding
tonic chords reappearance) in the original and modified version.
4.1. ERAN
As shown in Figure 9, the final chord evoked a negativity peaking around 210 ms after the
chords’ onset with statistically different amplitudes for modified and original versions. The effect
had anterior and posterior distribution with left and right lateralization respectively, figure 10.
Following this effect another more persistent negativity was evoked around 600 ms (figure 9 up).
4.2. Statistical analysis.
Statistical relevance was calculated in matlab as follows. First step, a 64 x 1000 x 120 data
matrix was created (channels x points (1000 pts = 2 seconds) x trials (original or modified)).
Second step, means were computed on the amplitudes of both versions within 100 and 300ms after
the last chord onset. For each channel these time locked two versions amplitudes’ means were
subtracted and with max function assessed the biggest value from this subtraction together with the
vector’s index for this value. Third step, centered on this index, data from a 40ms window ( figure 9
middle) was extracted to run a Wilcoxon signed-rank paired test comparing conditions (original and
modified) for each one of the 24 subjects ( subjects had his original version data compared to his
modified version data), figure 9 middle. The Wilcoxon signed-rank test was used because when we
submitted our data to a Kolmogorov- Smirnov test it resulted zero (not normally distributed), we
therefore discarded using ANOVA since besides not normally distributed we had a low sample, 12
subjects per condition.
28
Figure 9. Brain electric responses (mean from 24 subjects) evoked by the final chord measured in channel F1 (up) and PO8 (bottom). Yellow areas depict statistical relevant difference between Original (blue) and Modified (red) conditions calculated by finding the highest amplitude from subtraction of the 120 trials means taken from a window comprising 100 to 300 ms after stimulus onset. Dotted gray line for the onset of the final chord in both BWV 302 and BWV373 chorales. Early negativity (first yellow area) peaking 210ms after last chord onset, later negativity peaking 610ms. Middle, depiction of data window used for statistical analysis.
29
For statistical analysis of the ERP's, four multiple electrodes clusters with regions of interest (ROI)
were calculated: left anterior (AF3, F1, F3, FC3,F5, C1,C3, AF7,C5); right anterior (AF4, F2, F4,
FC4,F6, C2, C4, AF8, C6), left posterior (O1, P7, CP1, CP3, CP5, P3, P5, PO3, PO7), and right
posterior (O2, P8, CP2, CP4, CP6, P4, P6, PO4, PO8), figure 12. The electrode exclusion criteria
was a visual inspection for noisy electrodes; 2 out of 11 electrodes, of each cluster were excluded,
figure 11.
Figure 11. Two out of 11 electrodes were excluded from each of 4 clusters, ruled out as noise and indistinguishable ERP .
Figure 10. Electrodes where ERP presented
statistical relevant differences between
Original and Modified versions. Head electrode
map, system 10-20, depicting electrodes where the
difference between brain electric responses for
Original and Modified conditions was statistically
relevant.
30
31
Figure 12. Cluster of electrodes for ROI's comparing Regions, Musicians and Non-Musicians. Left, mean of 9 electrodes from regions Left and Right Anterior, Left and Right Posterior calculated on a 200ms window (refer to section 4.2. Statistical analysis); Central, depiction comparing amplitude electrical response raised by Original and Modified last chords considering all 24 subjects; Right, Same as “Central” but here separating Musicians from Non-Musicians brain electrical responses. The former contributes with more homogeneous amplitude differences when compared to the latter.
4.3. Musicians and Non Musicians
As shown in figure 12, when means from electrodes clusters of brain regions (Left and Right
anterior, Left and Right Posterior) were calculated, divided in musicians and non-musicians, the
former presented higher amplitude ranges in the ERAN latency at the Right Posterior and Left
Anterior regions, and lower amplitudes ranges at the Right Anterior. Left Posterior ranges were not
statistically differentiable. Amplitude range was assessed by the distance from the peak and valley
in the window from 100 and 300 after stimuli onset on the original condition. A non-paired
nonparametric Wilcoxon sum rank test was run and concluded that inter group comparisons
amplitude ranges for musicians and non-musicians were different, figure 12.
Figure 12. Musicians and Non Musicians region electrodes clusters differ. Each colored bar represents mean of 9 electrodes from the depicted regions (Left- Right anterior and posterior) of 12 musicians and 12 non-musicians. Note that parietal accounts for the widest amplitude range difference.
32
4.4. Brain Regions.
Despite the lack of topographic precision of EEG measurements we attempted to analyze the recruitment of brain areas in the processing of the last chord. For such, we plotted the amplitudes originated by the last chord unfolding in brain areas on a sequence of time windows, figure 13.
Figure 13. Progression of last chord's processing depicted in brain topographic maps. Top,
time progression of original and modified versions. Note similar response for sensorial processing, N100 (80-100ms), with
bilateral temporal-frontal activation of region near auditory cortex. Conversely, in ERAN latency range, a right temporal
activation is observed in 190ms, quickly suppressed and reappears in parietal-occipital region at 210-230, disappearing in
240, while in original version is observed a bilateral (predominant left) temporal activation. Bottom, mean from 64
channels, in 120 trials of 24 subjects for the modified version’s last chord.
33
5. Discussion.
Our results are in line with previous works which observed the ERAN, a fronto lateral
negativity peaking around 190 to 230 ms, related to harmonic disruptions (Brattico et al. 2006;
Fadiga, Craighero, and D’Ausilio 2009; Stefan Koelsch and Friederici 2003; S. Koelsch et al. 2001;
Maess et al. 2001b). However, in our experiment design we identified that our ERP with statistical
relevance had a predominantly parietal distribution therefore at odds with the ERAN's anterior
description: 11 parieto-ocipital electrodes ( Pz, POz, PO3, Oz, P4, PO4, O2, CP6, P6, PO8, P8)
against only 6 fronto-temporal electrodes( F1, F3, FC1, FC5, F4, C5). Also, the statistically relevant
frontal electrodes were predominantly lateralized to the left. Therefore, the ERP identified in this
study to musical syntactic disruptions differed from previous works regarding predominant effect
source. Although we did find a negativity around 200ms in one statistically relevant right anterior
(frontal) electrode, F4, hence an Early Right Anterior Negativity, we found the effect on 4 left
anterior and 7 right posterior electrodes, what could not sum up to a label since the effect was both
present in anterior and posterior regions although with a predominantly posterior electrode
distribution ( 6 anterior versus 11 posterior, figure 10). We interpret that this discrepancy is due
to our innovative paradigm. Here we used harmony deviant stimuli which were present in the same
chord as the standard, what is to say that the exact same acoustic stimuli relates to harmonically
different sequences stored in memory and sustained until the presentation of the last chord, which is
the same for deviant and standard chorale versions (figure 8). By doing so, we assured that the
ERAN observed in the last chord could not relate to a local disruption perception, like a MMN. This
means that the last chord was presented to the listener in a syntactically acceptable context ( no
local disruption) but the brain computed a long-term comparison to an event, the first chord,
presented around 9 seconds earlier( 9.062ms for the BWV 302 and 9.597ms for the BWV 373).
Locally accepted but globally not.
This long distance harmonic controlled paradigm differs from previous studies (Stefan
Koelsch 2005; Stefan Koelsch et al. 2000, 2002; Loui et al. 2005; Maidhof and Koelsch 2010;
Sammler, Koelsch, and Friederici 2011; Steinbeis and Koelsch 2008) which predominantly used a
similar stimuli design. The first ten years of ERAN studies used chord sequences where the deviant
stimuli, even though inserted in a harmonic progression, cannot be ruled out as a possible local
effect (Figure 6a, figure 11). Our experimental design also differs from a variation of this chord
sequence paradigm (Brattico et al. 2006; Kalda and Minati 2012): instead of chords, subjects were
presented to irregularities in tones of a simple one voice melody (no chords used). The ERAN
observed was attributed to the familiarity with musical scale knowledge, acquired by implicit
34
learning (Brattico et al. 2006; Kalda and Minati 2012; Leino et al. 2007; Rohrmeier and Rebuschat
2012) and therefore a mismatch of a pitch presented out of the subjects musical tonality expectancy.
The result does stand for an ERAN raised by a musical cognitive process (the expectancy to listen
to pitches inside the tonality field) and can be ruled out as an acoustical mismatch. However, since
the deviant event (an out of key or mistuned pitch) was not expected in the event by event
comparison, the data cannot stand for the segregation of the overlapped brain mechanisms recruited
for “something went wrong” from “something went harmonically wrong”, figure 14.
Figure 14. Local mismatch even though it is a strictly harmonic disruption (Koelsch 2005). The deviant last chord belongs to the neighbour tonalities and is presented in an inappropriate position in the harmonic sequence, therefore a purely harmonic violation. However, considering the chord-by-chord processing, the local effect(novelty) is still embedded in the last chord/ target. Observing that our protocol overcame previous experimental design which could not
successfully segregate brain computations for local and global mismatch, we hypothesize that the
more parietal contributions from our data may relate to the theoretical conception of hierarchic
connections between chords. As proposed by (Bharucha and Krumhansl 1983), musical syntax is
based on the expectation for the arrival of a stable chord, which can be understood as the return to
the central tonality of the music, subjectively reported as a relaxing feeling. The first chord of a
music establishes the tonality, the tonal center from which the other chords will come and from
where the neighbour tones relations are created. A basic rule of western tonal music is that the tonal
center of an entire music or phrase is presented in the beginning, hold off by “traveling” through
other neighbour tonalities, and finally resubmitted at the end of the piece/phrase, as a form of
determining its conclusion (Piston 1948).
This syntactical processing requires from the listener a memory process to store the first
chord, make online hierarchical organization with the incoming chords and finally refer to it in the
conclusion. The rationale to hypothesize the parietal contributions to this memory processing comes
from PET and fMRI works investigating arithmetic calculations (Stanislas Dehaene and Cohen
1997; Delazer et al. 2003; Fulbright et al. 2000; Menon et al. 2000) and from studies considering
music and math to hold similarity as sequences based in hierarchic connections (Stanislas Dehaene
35
et al. 2015; Schmithorst and Holland 2004). The establishment of a hierarchy among the incoming
elements of a sequence, e.g. new chords of a chorale or new numbers and operations of an
arithmetic computation, assumes that a constant up date is done in relation to a memory trace
formed by the first event. This memory trace is the very condition for a chord or number to unfold
in time and become a mental representation of a sequence. Not only a number is the temporal
instant unit of a computation but also what is referred to as arithmetic fact, which can be simple
operations repeated enough to become a lexicon, accessed automatically like a word without mental
manipulation(e.g. 2+2=4, 3³=27, cosine(0)=1)(Ashcraft 1992; Stanislas Dehaene 1992)
Neuroanatomical studies on arithmetic processing have come to a concept of a fronto-
parietal network in which arithmetic facts, exact calculations, are linked to the activation of frontal
and prefrontal brain areas while the parietal lobe has been suggested to be engaged in the processing
of complex calculations (e.g. approximation, 2 digit multiplication, 3 operand addition/subtraction
problems)(Chochon et al. 1999; S. Dehaene et al. 1999; Menon et al. 2000; Zago et al. 2001). This
fronto-parietal network was grounded in Dehaene's (Stanislas Dehaene 1992; Stanislas Dehaene and
Cohen 1997) theorization of human number processing architecture model, the so called triple code
model. As later would be fit in different brain areas, this model assumed that number processing
recruits distinct areas for arithmetic facts implicated in the activation of the left perisylvian network
associated with language (Broca and Wernicke), hence verbal code; visual number form associated
to the ventral visual pathway( inferior temporal and fusiform gyrus) activation, visual code; and for
comparison, approximation based on magnitude representation, necessary for time demanding
mental computation, the bilateral inferior parietal areas were recruited, magnitude code (Delazer et
al. 2003; Klein et al. 2016; Pesenti et al. 2000; Schmithorst and Brown 2004), figure 15.
(Dehaene et al., 1995)
36
Figure 15. Fronto-parietal network recruited in arithmetic calculation (Schmithrost, 2004). Top, schematic diagram of the triple code model. Arithmetic facts processed in anterior regions while magnitude
representations and comparisons are processed in posteriorly. Note the articulatory loop connecting anterior and
posterior brain regions. Bottom, function activation in brain regions suggested to be associated to magnitude
code(a), verbal code(b), and visual code (c)(d).
We here propose that the identification of a parietal contribution for complex memory based
arithmetic processing is a plausible parallel to interpret our predominant parietal effect in face of
our stimuli controlled for long-term dependencies in music. Similarly to a number approximation
and a 2 digit multiplication, which requires the sustention of a number or partial sum in memory ,
the syntactic relation of a first chord is kept in memory and sustained until the fulfillment of the
expectancy of its representation, the return to the tonal center. Furthermore, a parallel can also be
drawn between the arithmetic fact and auditory oddballs: the latter, like the former, is an automatic
brain reaction to an already consolidated memory producing a binary response to a right or wrong
stimulus, hence a fact. Considering that oddball paradigms have largely replicated a MMN with
higher amplitudes in fronto temporal regions ( May et al. 1999; Escera et al. 2000; Garrido et al.
37
2009; Risto Näätänen, Jacobsen, and Winkler 2005; Schröger 2007) we are tempted to interpret the
similar topographical brain findings for arithmetic facts ( fronto temporal) as a plausible fit for brain
areas engaged in processing auditory oddballs.
Our assumption that brain pathways engaged in arithmetic processing may be generalized to
the processing of other high level cognitive sequences, such as music syntax, is also based on
fundamental neuroanatomical evidences for a specific brain area involving music perception. First
relevant pinpoint of a musical processing network came from brain lesion cases in which patients
reported lost of music aesthetic perception, difficulty in discriminate sounds, pitch, rhythm and lost
of music´s gestalt. In the words of one of the first patient studied, amusia lesions result on the
difficulty “in grasping music as I used to” (Mazzoni et al. 1993; Mazzucchi et al. 1982). The studied
subjects presented lesions in their right temporal cortex around the middle portion of the sylvian
fissure.
Further studies on subjects reporting difficulty in identifying pitch intervals of semitones (
the smallest pitch unit in Western music) led to the identification of a congenital amusia(CA) (Hyde
and Peretz 2004; I 1985; Peretz and Kolinsky 1993). It is estimated that 4% of world population
suffers from congenital amusia (Kalmus and Fry 1980), and are therefore born with a deficiency in
music perception and production(less oftenly). Studies have been relating one possible cause of the
disease to a reduced white mater in the right inferior frontal gyrus (rIFG) associated to an increase
in gray matter(Hyde et al. 2006, 2007). This tissue abnormality was hypothesized to generate an
imporvished communication between the right auditory cortex and the rIFG(Hyde et al. 2007). This
postulation received contribution from (Loui, Alsop, and Schlaug 2009) findings which identified
less fiber volume of the arcuate fasciculus in subjects with CA. The arcuate fasciculus(AF) is a
bundle of axons with an important role in the posterior – anterior communication of multimodal
neural sites, like Wernicke (BA22) to Brocas(BA 44,45), caudal dorsolateral prefrontal cortex (BA
6, 8) to the inferior parietal lobule (BA 39, 40)(Phillips et al. 2011). Congenital amusia or brain
fronto-temporal damage on either side can provoke musical deficits, but the nature of the deficiency
relates to different brain hemispheres: right relate to melody and left to rhythm impairments (Peretz
1985; Di Pietro et al. 2004; Peretz et al. 2009), figure 16.
38
Figure 16. Arcuate fasciculus connects posterior and anterior multimodal neuronal sites and its reduced volume has been related to amusia (Loui et al.2009). Left, representation of arcuate fasciculus connecting anterior( Broca BA44, 45) to posterior( Wernicke BA21,22) areas; Right, (a) tractography of normal subjects' Afs: left-superior(yellow), left-inferior(pink), right-superior(red), right-inferior(green). Note that in tractography of amusic subjects(b) the right volume is reduced.
Here we propose possible shared mechanisms to process mathematic and music hierarchic
sequences coming from brain lesion cases in the arithmetic fronto-parietal circuit which lead do
acalculia ( difficulty in performing arithmetic calculations)(Cohen et al. 2000; Stanislas Dehaene et
al. 2003; Girelli et al. 1996; van Harskamp and Cipolotti 2001) compared to amusia cases which
also relate to damages in brain structures integrating fronto and parieto areas ( arcuate fasciculus).
In a first glance our assumption may seem pointing to a different direction of the well-
established findings of Tervaniemi et al.(Tervaniemi et al. 2000). Using PET, they observed the
activation of areas BA 42 for deviant chords (when subtracted the activity of standard chords
presented alone to the activity of standard and deviant presented together.) (Figure 17).
39
Figure 17. Deviant chords activate BA 42 of Superior Temporal Gyrus (Tervaniemi et al.
2000). Right, back and top view from PET data showing neuroanatomical correlates for an odd-ball paradigm using A
major( standard) and A minor( deviant) chords. Corroborating with Tervaniemi, Maess et al., 2001 found anterior activations when
functionally deviant chords were presented in different positions of a harmonic progression (the
Neapolitan paradigm designed by Stefan Koelsch et al. (2000). In the former, authors suggest a
ERAN source in the pars opercularis of the inferior frontal gyrus(IFG), figure 18.
Figure 18. Left and right sagittal and axial (parallel to anterior commissure-posterior
commissure line) view (Maess 2001). Grand average of dipole solutions for both magnetic ERAN(blue) and magnetic P200(yellow) refer to two-dipole configurations(one dipole in each hemisphere) for deviant chords.
Although these findings suggest an anterior location of brain sources for musical syntax
40
processing we argue that they reinforce our parietal hypothesis sustained in the rationale “fronto
parietal network for arithmetic – arcuate fasciculus involved in amusia – frontal neuroanatomical
sources for mismatch detection in chord sequences” developed in this section. Note that we refer in
the latter to a mismatch. This is because we understand that due to the fact that previous
experimental designs have not been able to isolate the auditory MMN effect from the musical
syntactic MMN (which has been called ERAN) previous ERAN findings necessarily contain an
overlap of the general detection of an auditory mismatch. We consider that in the case of a local
disruption, there is a general mismatch detecting process of “something went wrong” irrespective
of the nature of this mismatch ( semantic, syntactic, sensorial), which literature has identified as a
fronto-temporal effect often stronger in the right hemisphere for tone paradigms and on the left for
language paradigms (Garrido et al. 2009). Once we removed the local component of mismatch by
using a presumed long-term memory based mismatch, we interpret that our data reflect the
attenuation of the general classical MMN and an increase in posterior regions due to the memory
based musical syntactic relations (figure 13). In figure 13 we can also infer a major difference
between the brain areas recruited in the processing of original version (a normal sequence of music)
and the modified version (a musical sequence which activated a comparison mechanism due to a
global dependence disruption). While the former activates temporal slightly left lateralized areas (
from 190 to 220ms), the latter engages first an activation of a right temporal area at 190ms (
identified in previous works as the ERAN latency) which disappears after 10ms and at 220ms
initiates what we here named a parietal ERAN peaking at 230ms.
Our results are also in line with previous studies which observed an overall wider EEG
amplitudes in musicians when compared to non-musician, not only for ERAN (Brattico et al. 2013;
Stefan Koelsch and Jentschke 2008; Stefan Koelsch, Schmidt, and Kansok 2002) but for ERP's in
other latencies (Besson and Faita 1995; Patel et al. 1998; Rüsseler et al. 2001) and also fMRI
studies (Stefan Koelsch et al. 2005; Minati et al. 2008; Schulze et al. 2011). The mentioned studies
meet a consensus that the intense exposure and practice of music promotes plastic modifications in
the musicians' brains which are presumably related to higher electric amplitudes and voxel intensity
when exposed to musical stimuli. Our parietal findings with enhanced amplitudes for musicians
when compared to non-musicians (figure 12) corroborate with this assumption since the stimuli
used in the present study isolated the activation of brain areas involved in the maintenance of the
first chord in memory not contaminated by a local novelty effect.
We assume the speculative nature of the comparison between parietal findings from
arithmetic studies and our parietal results with musical long distance dependencies stimuli and that
further studies are necessary to better sustain this assumption. We also must state that even though
our findings point to some areas implicated in processing of music syntax, these areas may very
41
likely not be the source of this processing but one of the projections or sum of neural activities
recruited in this cognitive mechanism. EEG's low spatial resolution is a limitation which inspires us
to consider future studies using similar paradigm with better spatial brain imaging techniques like
PET and fMRI.
BIBLIOGRAPHY.
Alain, Claude, and David L. Woods. 1997. “Attention Modulates Auditory Pattern Memory as Indexed by Event-Related Brain Potentials.” Psychophysiology 34 (5): 534–46. doi:10.1111/j.1469-8986.1997.tb01740.x.
Alho, K., M. Sams, P. Paavilainen, K. Reinikainen, and R. Näätänen. 1989. “Event-Related Brain Potentials Reflecting Processing of Relevant and Irrelevant Stimuli During Selective Listening.” Psychophysiology 26 (5): 514–28. doi:10.1111/j.1469-8986.1989.tb00704.x.
Alho, Kimmo, David L. Woods, Alain Algazi, and Risto Näätänen. 1992. “Intermodal Selective Attention. II. Effects of Attentional Load on Processing of Auditory and Visual Stimuli in Central Space.” Electroencephalography and Clinical Neurophysiology 82 (5): 356–68. doi:10.1016/0013-4694(92)90005-3.
Anderson, Robert H. 1967. “Syntax-Directed Recognition of Hand-Printed Two-Dimensional Mathematics.” In Symposium on Interactive Systems for Experimental Applied Mathematics: Proceedings of the Association for Computing Machinery Inc. Symposium, 436–459. New York, NY, USA: ACM. doi:10.1145/2402536.2402585.
Ashcraft, Mark H. 1992. “Cognitive Arithmetic: A Review of Data and Theory.” Cognition, Numerical Cognition, 44 (1): 75–106. doi:10.1016/0010-0277(92)90051-I.
Besson, Mireille, and Frederique Faita. 1995. “An event-related potential (ERP) study of musical expectancy: comparison of musicians with nonmusicians.” Journal of Experimental Psychology. Human Perception and Performance 21: 1278–96.
Besson, Mireille, and Daniele Schön. 2001. “Comparison between Language and Music.” Annals of the New York Academy of Sciences 930 (1): 232–58. doi:10.1111/j.1749-6632.2001.tb05736.x.
Bharucha, Jamshed, and Carol L. Krumhansl. 1983. “The Representation of Harmonic Structure in Music: Hierarchies of Stability as a Function of Context.” Cognition 13 (1): 63–102. doi:10.1016/0010-0277(83)90003-3.
Bigand, Emmanuel, Bénédicte Poulin, Barbara Tillmann, François Madurell, and Daniel A. D’Adamo. 2003. “Sensory versus Cognitive Components in Harmonic Priming.” Journal of Experimental Psychology: Human Perception and Performance 29 (1): 159–71. doi:10.1037/0096-1523.29.1.159.
Brattico, Elvira, Mari Tervaniemi, Risto Näätänen, and Isabelle Peretz. 2006. “Musical Scale Properties Are Automatically Processed in the Human Auditory Cortex.” Brain Research 1117 (1): 162–74. doi:10.1016/j.brainres.2006.08.023.
Brattico, Elvira, Tiina Tupala, Enrico Glerean, and Mari Tervaniemi. 2013. “Modulated Neural Processing of Western Harmony in Folk Musicians.” Psychophysiology 50 (7): 653–63. doi:10.1111/psyp.12049.
Chochon, F., L. Cohen, P. F. van de Moortele, and S. Dehaene. 1999. “Differential Contributions of the Left and Right Inferior Parietal Lobules to Number Processing.” Journal of Cognitive
42
Neuroscience 11 (6): 617–30. doi:10.1162/089892999563689. Clarke, Eric F. 1987. “Levels of Structure in the Organization of Musical Time.” Contemporary
Music Review 2 (1): 211–38. doi:10.1080/07494468708567059. Cohen, L., S. Dehaene, F. Chochon, S. Lehéricy, and L. Naccache. 2000. “Language and
Calculation within the Parietal Lobe: A Combined Cognitive, Anatomical and fMRI Study.” Neuropsychologia 38 (10): 1426–40. doi:10.1016/S0028-3932(00)00038-5.
Conway, Andrew R. A., Nelson Cowan, and Michael F. Bunting. 2001. “The Cocktail Party Phenomenon Revisited: The Importance of Working Memory Capacity.” Psychonomic Bulletin & Review 8 (2): 331–35. doi:10.3758/BF03196169.
Dehaene, S., E. Spelke, P. Pinel, R. Stanescu, and S. Tsivkin. 1999. “Sources of Mathematical Thinking: Behavioral and Brain-Imaging Evidence.” Science 284 (5416): 970–74. doi:10.1126/science.284.5416.970.
Dehaene, Stanislas. 1992. “Varieties of Numerical Abilities.” Cognition, Numerical Cognition, 44 (1): 1–42. doi:10.1016/0010-0277(92)90049-N.
Dehaene, Stanislas, and Laurent Cohen. 1997. “Cerebral Pathways for Calculation: Double Dissociation between Rote Verbal and Quantitative Knowledge of Arithmetic.” Cortex 33 (2): 219–50. doi:10.1016/S0010-9452(08)70002-9.
Dehaene, Stanislas, Florent Meyniel, Catherine Wacongne, Liping Wang, and Christophe Pallier. 2015. “The Neural Representation of Sequences: From Transition Probabilities to Algebraic Patterns and Linguistic Trees.” Neuron 88 (1): 2–19. doi:10.1016/j.neuron.2015.09.019.
Dehaene, Stanislas, Manuela Piazza, Philippe Pinel, and Laurent Cohen. 2003. “Three Parietal Circuits for Number Processing.” Cognitive Neuropsychology 20 (3–6): 487–506. doi:10.1080/02643290244000239.
Delazer, M, F Domahs, L Bartha, C Brenneis, A Lochy, T Trieb, and T Benke. 2003. “Learning Complex Arithmetic—an fMRI Study.” Cognitive Brain Research 18 (1): 76–88. doi:10.1016/j.cogbrainres.2003.09.005.
Di Pietro, Marie, Marina Laganaro, Béatrice Leemann, and Armin Schnider. 2004. “Receptive Amusia: Temporal Auditory Processing Deficit in a Professional Musician Following a Left Temporo-Parietal Lesion.” Neuropsychologia 42 (7): 868–77. doi:10.1016/j.neuropsychologia.2003.12.004.
Donchin, E., and D. B. Lindsley. 1966. “Average Evoked Potentials and Reaction Times to Visual Stimuli.” Electroencephalography and Clinical Neurophysiology 20 (3): 217–23. doi:10.1016/0013-4694(66)90086-1.
Escera, C., K. Alho, E. Schröger, and I. Winkler. 2000. “Involuntary Attention and Distractibility as Evaluated with Event-Related Brain Potentials.” Audiology & Neuro-Otology 5 (3–4): 151–66. doi:13877.
Fadiga, Luciano, Laila Craighero, and Alessandro D’Ausilio. 2009. “Broca’s Area in Language, Action, and Music.” Annals of the New York Academy of Sciences 1169 (1): 448–58. doi:10.1111/j.1749-6632.2009.04582.x.
Friederici, Angela D. 2004. “Event-Related Brain Potential Studies in Language.” Current Neurology and Neuroscience Reports 4 (6): 466–70. doi:10.1007/s11910-004-0070-0.
Friederici, Angela D., Erdmut Pfeifer, and Anja Hahne. 1993. “Event-Related Brain Potentials during Natural Speech Processing: Effects of Semantic, Morphological and Syntactic Violations.” Cognitive Brain Research 1 (3): 183–92. doi:10.1016/0926-6410(93)90026-2.
Fulbright, Robert K., David L. Molfese, Alexander A. Stevens, Pawel Skudlarski, Cheryl M. Lacadie, and John C. Gore. 2000. “Cerebral Activation during Multiplication: A Functional MR Imaging Study of Number Processing.” American Journal of Neuroradiology 21 (6): 1048–54.
Garrido, Marta I., James M. Kilner, Klaas E. Stephan, and Karl J. Friston. 2009. “The Mismatch Negativity: A Review of Underlying Mechanisms.” Clinical Neurophysiology 120 (3): 453–63. doi:10.1016/j.clinph.2008.11.029.
Girelli, L., M. Delazer, C. Semenza, and G. Denes. 1996. “The Representation Of Arithmetical
43
Facts: Evidence from Two Rehabilitation Studies.” Cortex 32 (1): 49–66. doi:10.1016/S0010-9452(96)80016-5.
Gunter, Thomas C., Angela D. Friederici, and Herbert Schriefers. 2000. “Syntactic Gender and Semantic Expectancy: ERPs Reveal Early Autonomy and Late Interaction.” Journal of Cognitive Neuroscience 12 (4): 556–68. doi:10.1162/089892900562336.
Hahne, Anja, and Angela D. Friederici. 1999. “Electrophysiological Evidence for Two Steps in Syntactic Analysis: Early Automatic and Late Controlled Processes.” Journal of Cognitive Neuroscience 11 (2): 194–205. doi:10.1162/089892999563328.
Haider, M., P. Spong, and D. B. Lindsley. 1964. “ATTENTION, VIGILANCE, AND CORTICAL EVOKED-POTENTIALS IN HUMANS.” Science (New York, N.Y.) 145 (3628): 180–82.
Harskamp, N. J. van, and L. Cipolotti. 2001. “Selective Impairments for Addition, Subtraction and Multiplication. Implications for the Organisation of Arithmetical Facts.” Cortex; a Journal Devoted to the Study of the Nervous System and Behavior 37 (3): 363–88.
Hofstadter, Douglas. 2000. Gödel, Escher, Bach: An Eternal Golden Braid (20th Anniversary Edition with a New Preface by the Author). Penguin Books Ltd. http://www.amazon.ca/exec/obidos/redirect?tag=citeulike09-20&path=ASIN/0140289208.
Hyde, Krista L., Jason P. Lerch, Robert J. Zatorre, Timothy D. Griffiths, Alan C. Evans, and Isabelle Peretz. 2007. “Cortical Thickness in Congenital Amusia: When Less Is Better Than More.” Journal of Neuroscience 27 (47): 13028–32. doi:10.1523/JNEUROSCI.3039-07.2007.
Hyde, Krista L., and Isabelle Peretz. 2004. “Brains That Are out of Tune but in Time.” Psychological Science 15 (5): 356–60. doi:10.1111/j.0956-7976.2004.00683.x.
Hyde, Krista L., Robert J. Zatorre, Timothy D. Griffiths, Jason P. Lerch, and Isabelle Peretz. 2006. “Morphometry of the Amusic Brain: A Two-Site Study.” Brain 129 (10): 2562–70. doi:10.1093/brain/awl204.
I, Peretz. 1985. “[Hemispheric asymmetry in amusia].” Revue neurologique 141 (3): 169–83. Jääskeläinen, Iiro P., Jyrki Ahveninen, Giorgio Bonmassar, Anders M. Dale, Risto J. Ilmoniemi,
Sari Levänen, Fa-Hsuan Lin, et al. 2004. “Human Posterior Auditory Cortex Gates Novel Sounds to Consciousness.” Proceedings of the National Academy of Sciences of the United States of America 101 (17): 6809–14. doi:10.1073/pnas.0303760101.
Jackendoff, Ray. 2009. “Parallels and Nonparallels between Language and Music.” Music Perception: An Interdisciplinary Journal 26 (3): 195–204. doi:10.1525/mp.2009.26.3.195.
Kalda, Tiina, and Ludovico Minati. 2012. “Detecting Scale Violations in Absence of Mismatch Requires Music-Syntactic Analysis: A Further Look at the Early Right Anterior Negativity (ERAN).” Brain Topography 25 (3): 285–92. doi:10.1007/s10548-011-0208-8.
Kalmus, H., and D. B. Fry. 1980. “On Tune Deafness (Dysmelodia): Frequency, Development, Genetics and Musical Background.” Annals of Human Genetics 43 (4): 369–82. doi:10.1111/j.1469-1809.1980.tb01571.x.
Kandel, Eric R. n.d. Principles of Neuroscience. Kivy, Peter. 1959. “Charles Darwin on Music.” Journal of the American Musicological Society 12
(1): 42–48. doi:10.2307/829516. Klein, Elise, Julia Suchan, Korbinian Moeller, Hans-Otto Karnath, André Knops, Guilherme Wood,
Hans-Christoph Nuerk, and Klaus Willmes. 2016. “Considering Structural Connectivity in the Triple Code Model of Numerical Cognition: Differential Connectivity for Magnitude Processing and Arithmetic Facts.” Brain Structure and Function 221 (2): 979–95. doi:10.1007/s00429-014-0951-1.
Koelsch, S., T. C. Gunter, E. Schröger, M. Tervaniemi, D. Sammler, and A. D. Friederici. 2001. “Differentiating ERAN and MMN: An ERP Study.” Neuroreport 12 (7): 1385–89.
Koelsch, Stefan. 2005. “Neural Substrates of Processing Syntax and Semantics in Music.” Current Opinion in Neurobiology, Cognitive neuroscience, 15 (2): 207–12. doi:10.1016/j.conb.2005.03.005.
———. 2012. Brain and Music. John Wiley & Sons. Koelsch, Stefan, and Angela D. Friederici. 2003. “Toward the Neural Basis of Processing Structure
44
in Music.” Annals of the New York Academy of Sciences 999 (1): 15–28. doi:10.1196/annals.1284.002.
Koelsch, Stefan, Thomas Fritz, Katrin Schulze, David Alsop, and Gottfried Schlaug. 2005. “Adults and Children Processing Music: An fMRI Study.” NeuroImage 25 (4): 1068–76. doi:10.1016/j.neuroimage.2004.12.050.
Koelsch, Stefan, Thomas C. Gunter, D. Yves v. Cramon, Stefan Zysset, Gabriele Lohmann, and Angela D. Friederici. 2002. “Bach Speaks: A Cortical ‘Language-Network’ Serves the Processing of Music.” NeuroImage 17 (2): 956–66. doi:10.1006/nimg.2002.1154.
Koelsch, Stefan, Tomas Gunter, Angela D. Friederici, and Erich Schröger. 2000. “Brain Indices of Music Processing: ‘Nonmusicians’ Are Musical.” Journal of Cognitive Neuroscience 12 (3): 520–41. doi:10.1162/089892900562183.
Koelsch, Stefan, and Sebastian Jentschke. 2008. “Short-Term Effects of Processing Musical Syntax: An ERP Study.” Brain Research 1212 (May): 55–62. doi:10.1016/j.brainres.2007.10.078.
Koelsch, Stefan, Björn-helmer Schmidt, and Julia Kansok. 2002. “Effects of Musical Expertise on the Early Right Anterior Negativity: An Event-Related Brain Potential Study.” Psychophysiology 39 (5): 657–63. doi:10.1111/1469-8986.3950657.
Konečni, Vladimir J. 1984. “Elusive Effects of Artists’ ‘Messages.’” In Advances in Psychology, edited by W. Ray Crozier and Antlony J. Chapman, 19:71–93. Cognitive Processes in the Perception of Art. North-Holland. //www.sciencedirect.com/science/article/pii/S0166411508623468.
Lamarche, Fabrice, and Stéphane Donikian. 2004. “Crowd of Virtual Humans: A New Approach for Real Time Navigation in Complex and Structured Environments.” Computer Graphics Forum 23 (3): 509–18. doi:10.1111/j.1467-8659.2004.00782.x.
Leino, Sakari, Elvira Brattico, Mari Tervaniemi, and Peter Vuust. 2007. “Representation of Harmony Rules in the Human Brain: Further Evidence from Event-Related Potentials.” Brain Research 1142 (April): 169–77. doi:10.1016/j.brainres.2007.01.049.
Lerdahl, Fred, and Ray Jackendoff. 1985. A Generative Theory of Tonal Music. MIT Press. Lindsley, D. B. 1952. “Psychological Phenomena and the Electroencephalogram.”
Electroencephalography and Clinical Neurophysiology 4 (4): 443–56. Loui, Psyche, David Alsop, and Gottfried Schlaug. 2009. “Tone Deafness: A New Disconnection
Syndrome?” Journal of Neuroscience 29 (33): 10215–20. doi:10.1523/JNEUROSCI.1701-09.2009.
Loui, Psyche, Tineke Grent-’t-Jong, Dana Torpey, and Marty Woldorff. 2005. “Effects of Attention on the Neural Processing of Harmonic Syntax in Western Music.” Cognitive Brain Research 25 (3): 678–87. doi:10.1016/j.cogbrainres.2005.08.019.
Maess, Burkhard, Stefan Koelsch, Thomas C. Gunter, and Angela D. Friederici. 2001a. “Musical Syntax Is Processed in Broca’s Area: An MEG Study.” Nature Neuroscience 4 (5): 540–45. doi:10.1038/87502.
———. 2001b. “Musical Syntax Is Processed in Broca’s Area: An MEG Study.” Nature Neuroscience 4 (5): 540–45. doi:10.1038/87502.
Maidhof, Clemens, and Stefan Koelsch. 2010. “Effects of Selective Attention on Syntax Processing in Music and Language.” Journal of Cognitive Neuroscience 23 (9): 2252–67. doi:10.1162/jocn.2010.21542.
Marvin, Elizabeth West, and Alexander Brinkman. 1999. “The Effect of Modulation and Formal Manipulation on Perception of Tonic Closure by Expert Listeners.” Music Perception: An Interdisciplinary Journal 16 (4): 389–407. doi:10.2307/40285801.
May, Patrick, Hannu Tiitinen, Risto J. Ilmoniemi, Göte Nyman, John G. Taylor, and Risto Näätänen. 1999. “Frequency Change Detection in Human Auditory Cortex.” Journal of Computational Neuroscience 6 (2): 99–120. doi:10.1023/A:1008896417606.
Mazzoni, M, P Moretti, L Pardossi, M Vista, A Muratorio, and M Puglioli. 1993. “A Case of Music Imperception.” Journal of Neurology, Neurosurgery, and Psychiatry 56 (3): 322.
Mazzucchi, A., C. Marchini, R. Budai, and M. Parma. 1982. “A Case of Receptive Amusia with
45
Prominent Timbre Perception Defect.” Journal of Neurology, Neurosurgery, and Psychiatry 45 (7): 644–47.
McAdams, Stephen, and Emmanuel Bigand. 1993. Thinking in SoundThe Cognitive Psychology of Human Audition. Oxford University Press. http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780198522577.001.0001/acprof-9780198522577.
Menon, V., S. M. Rivera, C. D. White, G. H. Glover, and A. L. Reiss. 2000. “Dissociating Prefrontal and Parietal Cortex Activation during Arithmetic Processing.” NeuroImage 12 (4): 357–65. doi:10.1006/nimg.2000.0613.
Minati, Ludovico, Cristina Rosazza, Ludovico D’Incerti, Emanuela Pietrocini, Laura Valentini, Vidmer Scaioli, Catherine Loveday, and Maria Grazia Bruzzone. 2008. “FMRI/ERP of Musical Syntax: Comparison of Melodies and Unstructured Note Sequences.” Neuroreport 19 (14): 1381–85. doi:10.1097/WNR.0b013e32830c694b.
Münte, Thomas F., Hans-Jochen Heinze, and George R Mangun. 1993. “Dissociation of Brain Activity Related to Syntactic and Semantic Aspects of Language.” Journal of Cognitive Neuroscience 5 (3): 335–44. doi:10.1162/jocn.1993.5.3.335.
Näätänen, R., P. Paavilainen, H. Titinen, D. Jiang, and K. Alho. 1993. “Attention and Mismatch Negativity.” Psychophysiology 30 (5): 436–50. doi:10.1111/j.1469-8986.1993.tb02067.x.
Näätänen, R., and T. Picton. 1987. “The N1 Wave of the Human Electric and Magnetic Response to Sound: A Review and an Analysis of the Component Structure.” Psychophysiology 24 (4): 375–425.
Näätänen, R., M. Sams, K. Alho, P. Paavilainen, K. Reinikainen, and E. N. Sokolov. 1988. “Frequency and Location Specificity of the Human Vertex N1 Wave.” Electroencephalography and Clinical Neurophysiology 69 (6): 523–31.
Näätänen, R., E. Schröger, S. Karakas, M. Tervaniemi, and P. Paavilainen. 1993. “Development of a Memory Trace for a Complex Sound in the Human Brain.” Neuroreport 4 (5): 503–6.
Näätänen, R., and I. Winkler. 1999. “The Concept of Auditory Stimulus Representation in Cognitive Neuroscience.” Psychological Bulletin 125 (6): 826–59.
Näätänen, Risto, Thomas Jacobsen, and István Winkler. 2005. “Memory-Based or Afferent Processes in Mismatch Negativity (MMN): A Review of the Evidence.” Psychophysiology 42 (1): 25–32. doi:10.1111/j.1469-8986.2005.00256.x.
Neville, Helen, Janet L. Nicol, Andrew Barss, Kenneth I. Forster, and Merrill F. Garrett. 1991. “Syntactically Based Sentence Processing Classes: Evidence from Event-Related Brain Potentials.” Journal of Cognitive Neuroscience 3 (2): 151–65. doi:10.1162/jocn.1991.3.2.151.
Osterhout, Lee, Michael Bersick, and Judith Mclaughlin. 1997. “Brain Potentials Reflect Violations of Gender Stereotypes.” Memory & Cognition 25 (3): 273–85. doi:10.3758/BF03211283.
Paavilainen, Petri, Maria Jaramillo, and Risto Näätänen. 1998. “Binaural Information Can Converge in Abstract Memory Traces.” Psychophysiology 35 (5): 483–87. doi:10.1017/S0048577298970895.
Patel, A. D., E. Gibson, J. Ratner, M. Besson, and P. J. Holcomb. 1998. “Processing Syntactic Relations in Language and Music: An Event-Related Potential Study.” Journal of Cognitive Neuroscience 10 (6): 717–33.
Peretz, Isabelle, Elvira Brattico, Miika Järvenpää, and Mari Tervaniemi. 2009. “The Amusic Brain: In Tune, out of Key, and Unaware.” Brain: A Journal of Neurology 132 (Pt 5): 1277–86. doi:10.1093/brain/awp055.
Peretz, Isabelle, and Régine Kolinsky. 1993. “Boundaries of Separability between Melody and Rhythm in Music Discrimination: A Neuropsychological Perspective.” The Quarterly Journal of Experimental Psychology Section A 46 (2): 301–25. doi:10.1080/14640749308401048.
Pesenti, Mauro, Marc Thioux, Xavier Seron, and Anne De Volder. 2000. “Neuroanatomical Substrates of Arabic Number Processing, Numerical Comparison, and Simple Addition: A
46
PET Study.” Journal of Cognitive Neuroscience 12 (3): 461–79. doi:10.1162/089892900562273.
Phillips, Owen R., Kristi A. Clark, Roger P. Woods, Kenneth L. Subotnik, Robert F. Asarnow, Keith H. Nuechterlein, Arthur W. Toga, and Katherine L. Narr. 2011. “Topographical Relationships between Arcuate Fasciculus Connectivity and Cortical Thickness.” Human Brain Mapping 32 (11): 1788–1801. doi:10.1002/hbm.21147.
Piston, Walter. 1948. Harmony. New York: W. W. Norton & Company. Poulin-Charronnat, Bénédicte, Emmanuel Bigand, and Stefan Koelsch. 2006. “Processing of
Musical Syntax Tonic versus Subdominant: An Event-Related Potential Study.” Journal of Cognitive Neuroscience 18 (9): 1545–54. doi:10.1162/jocn.2006.18.9.1545.
Rohrmeier, Martin. 2011. “Towards a Generative Syntax of Tonal Harmony.” Journal of Mathematics and Music 5 (1): 35–53. doi:10.1080/17459737.2011.573676.
Rohrmeier, Martin, and Patrick Rebuschat. 2012. “Implicit Learning and Acquisition of Music.” Topics in Cognitive Science 4 (4): 525–53. doi:10.1111/j.1756-8765.2012.01223.x.
Rüsseler, Jascha, Eckart Altenmüller, Wido Nager, Christine Kohlmetz, and Thomas F. Münte. 2001. “Event-Related Brain Potentials to Sound Omissions Differ in Musicians and Non-Musicians.” Neuroscience Letters 308 (1): 33–36. doi:10.1016/S0304-3940(01)01977-2.
Sammler, Daniela, Stefan Koelsch, and Angela D. Friederici. 2011. “Are Left Fronto-Temporal Brain Areas a Prerequisite for Normal Music-Syntactic Processing?” Cortex 47 (6): 659–73. doi:10.1016/j.cortex.2010.04.007.
Sams, M, P Paavilainen, K Alho, and R Näätänen. 1985. “Auditory Frequency Discrimination and Event-Related Potentials.” Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section 62 (6): 437–48. doi:10.1016/0168-5597(85)90054-1.
Schmithorst, Vincent J, and Rhonda Douglas Brown. 2004. “Empirical Validation of the Triple-Code Model of Numerical Processing for Complex Math Operations Using Functional MRI and Group Independent Component Analysis of the Mental Addition and Subtraction of Fractions.” NeuroImage 22 (3): 1414–20. doi:10.1016/j.neuroimage.2004.03.021.
Schmithorst, Vincent J., and Scott K. Holland. 2004. “The Effect of Musical Training on the Neural Correlates of Math Processing: A Functional Magnetic Resonance Imaging Study in Humans.” Neuroscience Letters 354 (3): 193–96. doi:10.1016/j.neulet.2003.10.037.
Schoenberg, Arnold. 1978. Theory of Harmony. University of California Press. Schön, Daniele, Cyrille Magne, and Mireille Besson. 2004. “The Music of Speech: Music Training
Facilitates Pitch Processing in Both Music and Language.” Psychophysiology 41 (3): 341–49. doi:10.1111/1469-8986.00172.x.
Schröger, Erich. 2007. “Mismatch Negativity.” Journal of Psychophysiology 21 (3–4): 138–46. doi:10.1027/0269-8803.21.34.138.
Schulze, Katrin, Stefan Zysset, Karsten Mueller, Angela D. Friederici, and Stefan Koelsch. 2011. “Neuroarchitecture of Verbal and Tonal Working Memory in Nonmusicians and Musicians.” Human Brain Mapping 32 (5): 771–83. doi:10.1002/hbm.21060.
Steedman, Mark J. 1984. “A Generative Grammar for Jazz Chord Sequences.” Music Perception: An Interdisciplinary Journal 2 (1): 52–77. doi:10.2307/40285282.
Steinbeis, Nikolaus, and Stefan Koelsch. 2008. “Shared Neural Resources between Music and Language Indicate Semantic Processing of Musical Tension-Resolution Patterns.” Cerebral Cortex 18 (5): 1169–78. doi:10.1093/cercor/bhm149.
Sussman, Elyse, Walter Ritter, and Herbert G. Vaughan. 1999. “An Investigation of the Auditory Streaming Effect Using Event-Related Brain Potentials.” Psychophysiology 36 (1): 22–34. doi:10.1017/S0048577299971056.
Sussman, Elyse, Walter Ritter, and Herbert G Vaughan Jr. 1998. “Attention Affects the Organization of Auditory Input Associated with the Mismatch Negativity System.” Brain Research 789 (1): 130–38. doi:10.1016/S0006-8993(97)01443-1.
Tang, Antony, and Alice Yip. 2010. “Collision Avoidance Timing Analysis of DSRC-Based Vehicles.” Accident Analysis & Prevention 42 (1): 182–95. doi:10.1016/j.aap.2009.07.019.
47
Teo, G., T. Schmidt, J. Szalma, G. Hancock, and P. Hancock. 2014. “The Effects of Individual Differences on Vigilance Training and Performance in a Dynamic Vigilance Task.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 58 (1): 964–68. doi:10.1177/1541931214581202.
Tervaniemi, M., S. V. Medvedev, K. Alho, S. V. Pakhomov, M. S. Roudas, T. L. Van Zuijen, and R. Näätänen. 2000. “Lateralized Automatic Auditory Processing of Phonetic versus Musical Information: A PET Study.” Human Brain Mapping 10 (2): 74–79.
Tillmann, Barbara, and Emmanuel Bigand. 2004. “The Relative Importance of Local and Global Structures in Music Perception.” The Journal of Aesthetics and Art Criticism 62 (2): 211–22. doi:10.1111/j.1540-594X.2004.00153.x.
Winkler, I., G. Karmos, and R. Näätänen. 1996. “Adaptive Modeling of the Unattended Acoustic Environment Reflected in the Mismatch Negativity Event-Related Potential.” Brain Research 742 (1–2): 239–52.
Winkler, I., P. Paavilainen, K. Alho, K. Reinikainen, M. Sams, and R. Näätänen. 1990. “The Effect of Small Variation of the Frequent Auditory Stimulus on the Event-Related Brain Potential to the Infrequent Stimulus.” Psychophysiology 27 (2): 228–35.
Winograd, Terry. 1972. “Understanding Natural Language.” Cognitive Psychology 3 (1): 1–191. doi:10.1016/0010-0285(72)90002-3.
Zago, Laure, Mauro Pesenti, Emmanuel Mellet, Fabrice Crivello, Bernard Mazoyer, and Nathalie Tzourio-Mazoyer. 2001. “Neural Correlates of Simple and Complex Mental Calculation.” NeuroImage 13 (2): 314–27. doi:10.1006/nimg.2000.0697.
Zuijen, Titia L. van, Veerle L. Simoens, Petri Paavilainen, Risto Näätänen, and Mari Tervaniemi. 2006. “Implicit, Intuitive, and Explicit Knowledge of Abstract Regularities in a Sound Sequence: An Event-Related Brain Potential Study.” Journal of Cognitive Neuroscience 18 (8): 1292–1303. doi:10.1162/jocn.2006.18.8.1292.