Language

Processing Stages Language makes it possible for us to

inform each other about the world. A sentence (string of words) is perceived. That string of forms is used to construct a

representation of something in the world. There are two main speech processing

stages Feature Analysis Comparison and Response Selection

Processing Stages Feature Analysis

The auditory system makes a distinction between speech and nonspeech sound for which there is no analogy in vision.

The purpose of the feature analysis stage is to provide sufficient information to construct a representation of the speech sounds that the speaker intended to produce.

The left hemisphere processes speech input. Comparison and Response Selection

Processing Stages Feature Analysis

The auditory system makes a distinction between speech and nonspeech sound for which there is no analogy in vision.

The feature analysis stage provides sufficient information to represent the speech sounds that the speaker intended to produce.

The left hemisphere has evolved specific mechanisms for producing and processing speech input.

Comparison and Response Selection

Foundations of Language 1

There must be perceptual and motor systems that make communication possible. Perceptual systems:

Specialized speech understanding area in the temporal lobe makes it possible to recover phonemic intent.

Motor (speech production) systems: Lowering of larynx makes it possible to create

more sounds. Specialization of speech area in left frontal lobe

(Insula) makes faster production of sounds possible.

Foundations of Language 2 There must be a

memory/reasoning/control system so that there is something to talk about. The semantic system makes word

meaning possible. Working memory makes the processing

and production of temporal auditory patterns (utterances) possible.

Foundations of Language 3 There must be social creatures

who have a need for communication. (Duane and Sue Savage-Rumbaugh teach Metata and Kinzu to speak)

Human Prehistory. Human line appears 7,000,000 years ago.

Motor learning through imitation (mirror neurons)

Increased size and complexity of social organization

Modern humans (homo sapiens sapiens) appear 150,000 – 200,000 years ago. Evolution of modern human language system However, little cultural change until 50,000

years ago

Kinds of Meaning 500,00-50,00 Years Ago

Emotional Utterances-- amygdala invests and detects meaning in utterances such as shrieks and cries. In normal speech this system makes sarcastic utterances possible because emotional content of voice always dominates word meaning.

Pointing Utterances-- This/that take the place of gestures

Semantic system-- logogens make naming possible through sound-image associations

Multiple Changes in System make Modern Human Language Possible

Semantic system makes word meaning possible

Lowering of larynx makes it possible to create more sounds

Specialization of speech area in left frontal lobe makes faster production of sounds possible

Specialized speech understanding areas in the temporal and prefrontal cortex makes it possible to recover phonemic intent

Invention of Syntax (Grammar)

Allows utterances to be independent of context.

Syntax invented 50,000 years ago. Our immediate ancestors emerge:

50,000 years ago. Rapid cultural change. First settlements

& use of fire: 10,000 years ago. All other humans wiped off the earth.

Processing Stages Feature Analysis

The auditory system makes a distinction between speech and nonspeech sound for which there is no analogy in vision.

The purpose of the feature analysis stage is to provide sufficient information to construct a representation of the speech sounds that the speaker intended to produce.

The left hemisphere processes speech input. Damage to the left hemisphere produces aphasia Left ear speech sounds dominate right ear speech sounds in

a dichotic listening task. Only left hemisphere produces speech in split-brain patients.

Comparison and Response Selection

Speech Input Input to feature analysis

stage is continuous waveform.

Breaks in speech spectrograph do not correspond to breaks in words

Small vertical segments are ambiguous and heard as different speech sounds in different contexts

Small horizontal segments are redundant and are heard as the same speech sounds

Feature Analysis Output Output of feature

analysis stage is are feature representations for possible sequences of speech sounds.

Small vertical segments are ambiguous and heard as different speech sounds in different contexts

Speech sounds that affect meaning are called phonemes.

Speech sounds that have meanings are called morphemes.

Processing Stages Feature Analysis Comparison and Response Selection

The purpose of speech perception is to construct a structural description of something in the world. It is called

a semantic representation the meaning of the sentence

The process of constructing a semantic representation is called comprehension.

Before the meaning of the sentence is constructed it is first necessary to construct a structural description of the sentence itself.

Structural Description of Sentence Feature Analysis Comparison and Response Selection

The purpose of speech perception is to construct a semantic representation from the sentence.

Before the meaning of the sentence is constructed it is first necessary to construct a structural description of the sentence itself.

A structural, i.e., syntactic, description of a sentence describes the relative locations of different categories of words that are identified through feature representations. (See next slide.)

Syntactic Description•A syntactic (structural) description of a sentence.

Function of Syntactic Description Feature Analysis Comparison and Response Selection

The purpose of speech perception is to construct a semantic representation from the sentence.

Before the meaning of the sentence is constructed it is first necessary to construct a structural description of the sentence itself.

A structural, i.e., syntactic, description of a sentence describes the relative locations of different categories of words that are identified through feature representations.

Syntactic descriptions make it possible to understand an infinite number of sentences.

Procedural and Declarative Aspects of Syntactic Description Before the meaning of the sentence is

constructed it is first necessary to construct a structural description of the sentence itself.

A structural, i.e., syntactic, description of a sentence describes the relative locations of different categories of words that are identified through feature representations.

Syntactic descriptions make it possible to understand an infinite number of sentences.

The categorical syntactic descriptions that are used to construct representations are part of procedural memory but the instances of sentence representations constructed with the syntactic descriptions are part of declarative memory.

Procedural Nature of Syntactic Description Can you produce a sentence

containing the following word string?

read to from up for

Procedural Nature of Syntactic Description A child has two story books; he likes the

stories in one, but dislikes the stories in the other. One evening, his father negotiates a deal in which the kid goes upstairs and gets ready for bed by himself, and the father will tell him a story. So the kid goes up, and a few minutes later, the father shows up in the kid's bedroom, carrying the book with the disliked stories. "Oh," says the kid, "what did you bring that book that I don't like to be read to from up for?"

Processing Stages Feature Analysis Comparison and Response Selection

The purpose of speech perception is to construct a semantic representation from the speech input.

Before the meaning of the sentence is constructed it is first necessary to construct a structural description of the sentence itself.

Often part of the speech input matches more than one structural description, i.e., is ambiguous.

Types of Ambiguity Phonemic

Often parts of the speech input match more than one phoneme.

Syntactic Sometimes sequences of phonemes

match more than one syntactic description.

Phonemic Ambiguity Often short segments of the speech input each

matches more than one phoneme. However, a longer segment containing them

corresponds to only one or two sequences of phonemes that match a sequence of words that matches a syntactic description; hence the ambiguity is reduced or resolved.

If the ambiguity is not resolved at the syntactic level then the phonemic representation that produces the most plausible semantic representation is selected.

Analysis by Synthesis Ambiguity is resolved during comparison and selection by

choosing the phonemic representation that produces the most plausible semantic representation.

The process of constructing alternative representations and selecting among them is called analysis by synthesis.

Hence, when the phonemic sequence is ambiguous it must be maintained in awareness until the ambiguity is resolved.

The listener perceives the selected representation as clear and unambiguous. When a logogen responds and a word representation is activated all the phonemes of the word are heard even if they were not all part of the input.

Phonemic restoration effect

Phonemic Restoration Start with passage: the legislature voted

… Delete phoneme: the legi ature voted ... Add noise to space: the legi * ature voted

... Listener also hears missing phoneme:

the legislature voted … *

Top Down Effect of Meaning on Speech Perception The *eel was the axle The *eel was on the orange The *eel was on the table The *eel was on the shoe

wheel

peel

meal

heel

Types of Ambiguity Phonemic

Often parts of the speech input match more than one phoneme.

Syntactic Sometimes long sequences of

phonemes match more than one syntactic description, but the semantic representation of one sentence is more plausible than the other.

Syntactic and Semantic Ambiguity and Working Memory Often the same phoneme sequence can be

part of more than one syntactic and/or semantic description. Rocks sink vs. Rock’s sink When the phonemic sequence is ambiguous it

must be maintained in awareness until the ambiguity is resolved.

Maintaining the phonemic sequence requires voluntary action.

The ability to understand language depends in part on the ability to maintain a phonemic sequence in awareness (working memory).

The size of working memory is correlated with the ability to comprehend language.

Syntactic and Semantic Ambiguity Often the same phoneme sequence can

be part of more than one syntactic and/or semantic description. When the phonemic sequence is ambiguous it

must be maintained in awareness until the ambiguity is resolved.

Most ambiguities are resolved prior to awareness because the context primes the more plausible description. However, sometimes long ambiguous sequences are not resolved by priming.

Garden Path Sentences An ambiguous input sequence must be

retained in working memory until the ambiguity is resolved.

A garden path sentence tricks the system into prematurely selecting the wrong description for an ambiguous input because the wrong description is the more probable one. The horse raced past the barn fell down.

Processing Stages Feature Analysis Comparison and Response Selection

The purpose of speech perception is to construct a semantic representation from the speech input.

When the phonemic sequence is ambiguous it must be maintained in awareness until the ambiguity is resolved.

Sometimes the semantic representation of a sentence can only be completed through the use of general knowledge.

Semantics and Pragmatics Filling in the details through integrating

sentence’s semantic representation with general knowledge (activating logogens in semantic network). Comprehension sometimes depends on

context. The notes were sour because the seam was

split. The haystack was important because the cloth

was ripped.