Thesis Proposal :

Connectionist modelling of Specific Language Impairment

Themelis Karaminis

Supervisors :Dr .Michael Thomas∧Dr . RichardCooper

July 9 ,2008

1. Introduction

The primary objective of this thesis is to investigate language development in children with Specific Language Impairment using connectionist modelling. Specific Language Impairment (SLI) is a developmental disorder diagnosed when children fail to develop age-appropriate language in the absence of factors which are usually concomitant with language learning problems (such as hearing impairment, frank neurological damage or low non-verbal IQ test scores) (Leonard, 1998). SLI has a prevalence of ~7% in the children population (Tomblin, Records, Buckwalter, Zhang, Smith & O’Brien, 1997); language development in SLI has been studied extensively for English, and to a lesser extent for other languages. Studies report a wide range of deficits in language use and learning (in phonology, morphology, grammar, syntax, semantics or pragmatics), indicating that the disorder is characterised by a greatly heterogeneous profile. This is possibly a confounding factor in empirical investigations of the causes of SLI, as different studies offer rather different theoretical explanations for SLI. As we will see, for some researchers, SLI (or certain subtypes of SLI they consider) stems from a deficit in the brain systems specifically involved in language processing. For others, SLI should be attributed to a more general processing deficit, which is not specific to language. There is, so far, no consensus on the question of whether SLI originates from a language-specific deficit, and no unified account for the many aspects of the heterogeneous profile of the disorder.

This research project addresses main issues in SLI research with the connectionist modelling methodology. Connectionist or artificial neural network models are based on computations performed by neuron-like units which are interconnected with modifiable weighted connections. The most interesting property of this class of models is their ability to learn. For example, when a supervised learning regime is used, connectionist models can be trained to learn complex mappings between the input and the output (target)

1

representations of a training set. Furthermore, the learning trajectory in connectionist models is traceable. Connectionist models allow monitoring of various features of the learning process, such as activation and weight patterns (representations), accuracy and error types on the mappings of the training set, as well as the ability of the model to generalise its knowledge on novel mappings. Finally, as connectionist models are parameterisable, one can study how different parameter settings affect the learning trajectory. The ability to demonstrate learning, as well as the possibility to manipulate the parameter settings and trace the learning process are considered to be powerful features of the connectionist modelling methodology. These features render connectionist models a valuable computational tool for addressing issues of (language) development (Thomas & Karmiloff-Smith, 2002, 2003).

The current project proposes a systematic investigation of language development in SLI using connectionist modelling. In particular, this project seeks connectionist explanations for SLI, so as to contribute towards a better understanding of the impairment. For this purpose, the project will contrast typical and atypical language development using a series of connectionist models for different aspects of language development. Search into the parameter space will be performed in order to detect parameter settings under which the models simulate the corresponding behavioural data (accuracy rates, error patterns, etc). Manipulations of the parameter settings that cause the models to shift from a typical developmental trajectory to an atypical one will designate possible connectionist explanations of SLI, which will be contrasted to theoretical accounts of SLI.

A second aim of this project is to address the heterogeneous profile of SLI. This will be achieved by focusing on a wide range of issues of language development. A series of models, addressing phenomena in phonology, morphology, syntax and semantics for both typical and atypical development will be implemented. The modelling work will therefore endeavour to capture a significant portion of the heterogeneity of symptoms in SLI. The intended outcome is a unified connectionist explanation for the variability in SLI.

Finally, this project aims to adopt a cross-linguistic perspective, considering SLI in English, and also in (Modern) Greek. The project will examine whether the same connectionist models could account for atypical language development in two languages which possess different characteristics (e.g. in their morphological richness). In order to achieve this, the certain models initially developed for English will be subsequently expanded to Greek.

This document briefly reviews the relevant SLI literature (behavioural data, theoretical accounts) and introduces connectionist modelling from the perspective of the study of atypical language development. This background information provides the wider context and the rationale for the specific modelling work to be undertaken. In the research methodologies section, a series of models to study various aspects of language development in SLI is proposed. Finally, a detailed timetable of how this work will be accomplished is presented.

2

1.1. SLI literature

In the literature, SLI has also been referred as ‘developmental dysphasia’, ’developmental aphasia’, ‘word deafness’, ‘delayed speech’, ‘deviant language’, ‘language disorder’, ‘delayed language’, ‘developmental language impairment’, and ‘specific language deficit’ (Leonard, 1998). The variety of these terms reflects the shaping of the diagnostic criteria for the impairment along the history of the study of SLI (cf. Leonard, 1998, pp.5-8), as well as the existence of different views of the profile of SLI. For example, a ‘delay’ view of SLI suggests that the impairment is characterised by “late emergence of language and ... slower than average development of language from the point of emergence to mastery” (Leonard, 1998, p.32). By contrast, a ‘disorder’/’deviance’ view proposes qualitative differences between the linguistic profiles of children with SLI and typically developing children. The term ‘Specific Language Impairment’ is nowadays most broadly used. As Bishop (1997, p. 21) comments, this term denotes that, apart from language, (nonverbal) cognitive development falls in the normal range, while being neutral regarding the delay/disorder question.

SLI is diagnosed based on both inclusion and exclusion (Leonard, 1998, p.10). The linguistic and non-linguistic skills of children are assessed on standardised tests. Exclusionary criteria (e.g. sensory/ hearing/ oral motor deficits, frank neurological damage, behavioural/ emotional problems, environmental causes), are considered to rule out cases in which the impaired linguistic profile may be due to other factors.

1.1.1. Variability of deficits in SLI

Numerous studies have investigated the cognitive profile of children with SLI in greater detail than that provided by the diagnostic tests. Such studies often use linguistic and non-linguistic tasks to compare the performance of Children with SLI and typically developing children (controls) matched on chronological age, mental age, mean length of utterance (MLU), expressive/receptive vocabulary or other criteria (Leonard, 1998). Extensive listing of findings from behavioural studies of SLI are provided in Leonard (1998, chapter 2) and Bishop (1997).

Auditory and phonological deficits are prevalent in SLI, as poor performance has been reported in tone discrimination (Tallal & Piercy, 1973a, b), phoneme discrimination and identification of phoneme constancy (Bird, Bishop, & Freeman, 1995), or non-word repetition tasks (Gathercole & Baddeley, 1990). Morphological/grammatical deficits, such as omission of noun and verb inflections and auxiliary verb forms in obligatory contexts are characteristic of most of children with SLI and are often considered as behavioural hallmarks for the impairment (cf. Rice, 2000). Children with SLI present immature sentence structure (Johnston & Kamhi, 1984) and many grammatical/syntactical deficits, for example in wh-question formation (e.g. Ingram,1972; van der Lely & Batel, 2003). Grammatical/ syntactical comprehension is also impaired, with deficits in anaphoric pronoun resolution

3

(van der Lely & Stollwerk, 1997), comprehension of passives (Bishop, 1979; van der Lely & Harris, 1990; Dick, Wulfeck, Krupa-Kwiatkowski, & Bates, 2004). Lexical abilities are limited, and are characterised by word-finding difficulties (e.g. McGregor & Leonard, 1995; German, 1987), naming errors (e.g. Casby, 1992 ; McGregor & Waxman, 1995), impoverished word-definitions (Dockrell, Messer, George, & Ralli, 2003) and problems in the learning of new words (Oetting, Rice, & Swank, 1995). Furthermore, certain studies have suggested pragmatics-related difficulties, such as deficits in producing speech acts (Gallagher & Craig, 1984) or reduced conversational participation (Craig, 1993). Finally, weaknesses in certain non-linguistic tasks (e.g. in mental imagery tasks (Johnston & Ramstad, 1983)) were also identified.

Therefore, a wide range of deficits, which spread along all the dimensions of language, characterises SLI. When individual performance is considered, it is not the case that all children present the same deficits. On the contrary, it is generally agreed that there is a great degree of individual variability within SLI. SLI is commonly characterised as a developmental impairment with a “heterogeneous” or “uneven” profile (e.g. Ulman and Pierpont, 2005; Thomas, 2005).

The heterogeneous profile of SLI, has led certain researchers to consider subgroups within SLI, using statistical sorting procedures or based on clinical judgements (e.g. Aram & Nation, 1975; Rapin & Allen, 1987; Wilson & Risucci, 1986). Leonard (1989) comments that although certain subtypes of SLI were consistently identified by most of these studies, further investigation is needed for them to be validated. Of particular interest is perhaps Grammatical-SLI (abbreviation: G-SLI), a subtype of SLI proposed by van der Lely (e.g. van der Lely, 1996). As its name implies, this subtype refers to deficits centred in the grammatical comprehension and production of language. Based on a coherent pattern of grammatical errors in this subtype, van der Lely (1996) has argued that this brand of SLI should be attributed to a selective deficit of grammar. Finally, Bishop (2000) suggested an overlap between Semantic-Pragmatic Disorder, SLI and the autistic spectrum.

1.1.2. Theoretical Accounts of SLI

Although the cognitive profile of SLI has been extensively studied, there is still no widely accepted account for the aetiology of the impairment.

Certain researchers consider SLI to be due to a language-specific deficit, in particular a deficit of the brain systems involved in the processing of grammar. Rice, Wexler, and Cleave (1995), propose the Extended Optional Infinitive (abbreviation: EOI) account for SLI. According to this account, the Optional Infinitive stage, i.e. a stage in language development characterised by inconsistent application of tense-marking in obligatory contexts, is protracted in SLI. Other language-specific hypotheses address a wider range of grammatical deficits in SLI. The ‘feature blindness hypothesis’ (Gopnic & Crago, 1991) suggests that the grammar of Children with SLI lacks morphophonological rules. van der Lely’s (1996) Representational Deficit for Dependent Relationships (abbreviation: RDDR) hypothesis considers that an “underlying deficit in the computational syntactic system” accounts for

4

the grammatical difficulties presented in G-SLI). Although supported by the language-specific profile of SLI, grammar-specific hypotheses are often criticised for not being able to accommodate linguistic deficits beyond grammar or address cross-linguistic data (cf. Leonard, 1998, p.235).

For other authors, language limitations in SLI are due to a general (non-specific-to-language) deficit. These authors consider that deficits like slow processing (e.g. Bishop, 1994) or capacity limitations in processing (Kail, 1994; Leonard et al., 1992) affect language development in SLI. Such accounts have the potential to address a wide range of linguistic (and non-linguistic, e.g. deficits in mental imagery tasks (Johnston & Ramstad, 1983)) deficits in SLI. Whether these accounts could address variability in SLI in a coherent manner needs to be further investigated (cf. Leonard, 1998, p.268).

Another group of studies also adopt an explanation of SLI which is not based on a linguistic-processing deficit. However, this explanation does not consider a general processing deficit but a specific deficit, localised in the phonological system, which spreads along development to other aspects of language processing, such as syntax. Tallal and Piercy (1973a, b) suggest that a perceptual/temporal processing deficit (evidenced from difficulties in tone discrimination tasks under rapid conditions) underlies SLI. Gathercole and Baddeley (1990) propose limitations in the phonological working memory in SLI, based on poor performance of children with SLI in non-word repetition. Such accounts were also supported by the connectionist modelling work of Joanisse (2004) and Joanisse and Seidenberg (2003).

Finally, Ulman and Pierpont (2005) propose the Procedural Deficit Hypothesis (PDH). This hypothesis considers that language relies differentially on two different memory systems in the brain: the procedural and the declarative one. A general deficit of the procedural system (which is mainly involved in the learning and the execution of grammar) explains the wide range of linguistic and non-linguistic deficits in SLI. The uneven profile of SLI is also a result of the attempt of the declarative system to compensate for the impairment of the procedural system. A connectionist implementation of this compensation mechanism was considered in Thomas (2005).

1.1.3. Greek SLI

Greek is a language with a rich inflectional morphology. For example, nouns have three genders and are inflected with respect to number and case, depending on the singular form of the noun in the nominative case (Leonard, 1998). Verb inflections include features for person, number, tense, and aspect (Clahsen & Dalalakis, 1999).

Behavioural studies on Greek SLI are relatively sparse. Various morphological/ grammatical deficits are reported, such as problems in marking subject-verb agreement, (although past-tense is correctly marked) (Clahsen & Dalalakis, 1999), and omissions of definite articles, clitic object pronouns, interrogative pronouns, and the case marker of

5

masculine noun phrases (Tsimpli & Stavrakaki, 1999). Stavrakaki (2001, 2002, 2006) has identified problems of Greek children with SLI in the comprehension of reversible relative clauses and the production of wh-questions. The same author reports limited verb vocabulary and problems in verb-retrieval (Stavrakaki, 2000).

1.2.Connectionist modelling

Connectionist models are architectures of interconnected units, able to perform parallel distributed computation (Rumelhart, McClelland, & PDP Research Group, 1986). Based on a simplified model of the real neuron (cf. O’Reilly & Munakata, 2000, pp. 23-27), each unit accumulates (integrates) weighted input activation signals from other units. An activation function quantifies the unit’s output level of activation (firing), which is in turn input to proceeding units. The (synaptic) weights of the connections are modifiable: a learning algorithm determines changes in their values, so as to achieve certain criteria (for example, the minimisation of the output error, in supervised learning with Back-Propagation (for a detailed description of the algorithm, see O’Reilly & Munakata, 2000, pp. 158-162). Therefore, when connectionist models are exposed to an environment (training set), representations of aspects of this environment are developed in the weight patterns. Offering the possibility to trace these representations during learning, connectionist models are a powerful tool for addressing phenomena of cognitive development.

For example, in the domain of language development, the influential model of Rumelhart and McClelland (1986) showed that a single-route connectionist architecture can learn both regular and irregular past-tense inflections and demonstrate developmental patterns observed in children (e.g. the U-shaped curve for the learning of irregulars). The model argued against dual-route symbolic accounts (e.g. Pinker, 1991), which consider separate pathways for the learning of rules and exceptions. Subsequent connectionist models addressed the acquisition of morphology in further detail (e.g. incremental training and type/token frequency effects (Plunkett & Marchmann, 1990, 1993), acquisition of noun and verb morphology (Plunkett & Juola, 1999)). Other connectionist models studied acquired language disorders, considering trained networks to which changes corresponding to lesions (e.g. pruning of connections, noise) were introduced. For example, the model of Joanisse and Seidenberg (1999) simulated the “double dissociation” between past-tense formation for irregulars and nonce words reported for Parkinson’s Disease/Anterior Lesion and Alzheimer’s Disease/Posterior Aphasia patients (correspondingly). Finally, some connectionist models addressed developmental language disorders (e.g. Thomas & Karmiloff-Smith 2002, 2003). These models examined how a typical learning trajectory was changing under certain constraints, introduced in the parameter settings. Such connectionist models for studying SLI are discussed separately and in more detail in subsection 1.2.1. .

6

It is true that connectionist models could not claim a significant degree of biological plausibility. A great deal of biological detail is omitted, as the architecture of the brain is too complex to mimic, given current computational power. Scalability issues also apply. Moreover, it is implausible that learning in the brain happens through the back-propagation algorithm (O’Reilly & Munakata, 2000, p.162). Therefore, connectionist models could be better described as biologically-flavoured models of cognition, which demonstrate statistical learning, i.e. learning based on the regularities embedded in the training set.

The statistical nature of learning in connectionist networks often raises criticism, mainly coming from symbolic-processing point of views. Connectionist models are often considered statistical learners which afford ”a huge number of additional ‘degrees of freedom’ ” (e.g. Green, 1998). They are thus able to fit to any data (training set), after proper parameterisation. Marcus (1998) presents examples of connectionist networks whose ability to learn certain tasks (involving variable-instances relations or recursive structures) is strongly contingent on the types of representations or the training regimes used. For him, and for other authors (e.g. Fodor & Pylyshyn, 1988), connectionist models are successful in learning cognitive tasks when they are essentially implementations of symbolic systems. They therefore challenge the appropriateness of performing inferences about cognitive development based on the learning patterns of connectionist.

Being aware of the limitations of the connectionist methodology, this project still posits that a systematic investigation of SLI with connectionist modelling would be beneficial for a better understanding of the impairment.

1.2.1. Connectionist models of Specific Language Impairment

Connectionist modelling studies on SLI are relatively sparse. The acquisition of verb morphology1 (and especially the acquisition of the past-tense) in SLI has been addressed in Hoeffner and McClelland (1993), Joanisse (2003), and Thomas (2005)/Thomas and Karmiloff-Smith (2003). Joanisse and Seidenberg (2003) also proposed a model for the processing of sentences with anaphoric pronouns in SLI. Finally, Thomas and Redington (2004) proposed a model for syntax comprehension in SLI. In this section, the existing connectionist approaches to SLI are presented.

Hoeffner and McClelland (1993) addressed SLI deficits in the production of uninflected and inflected (past tense, past participle, 3rd singular and progressive) verb forms. Their model, a network with bidirectional connections (attractor network), learned mappings between representations of the semantics of words and representations of their phonological forms. The model considered lower phonetic saliency for word-final stops and fricatives; therefore less strong phonological representations were used for these

1 For a non connectionist explanation of aspects of morphological acquisition, such as optional infinitive errors in typical and atypical development, see MOSAIC (e.g. Freudenthal, Pine, & Gobet , 2006).

7

phonemes. Furthermore, SLI was simulated as a non-language specific perceptual deficit, based on experimental findings (e.g. Tallal and Piercy, 1973 a,b). Thus, the impaired model employed much weaker phonological representations for all phonemes (and even more weak representations for word-final stops and fricatives). This simple manipulation captured major phenomena of the acquisition of verb morphology in SLI, described in behavioural studies: slower and poorer learning compared to typical development, increased percentages of zero-marking errors, and different degrees of impairment for different inflections. The model also predicted a greater degree of impairment for regular past-tense forms than for irregular ones; this prediction was challenged by the modelling work of Joanisse (2003) for past tense formation in SLI presented next.

A model by Joanisse (2003) also considered the perceptual (phonological) deficit hypothesis for SLI in an attractor network trained on mappings between representations of the semantics and phonological forms. However, here SLI was simulated by the addition of small amounts of random noise to the phonological representations. In line with data from behavioural studies of SLI, the impaired model was weaker in learning both regular and irregular past-tense forms, while performance in generalising the rule to novel words was very low. Therefore, the model of Joanisse (2003) challenged dual-route accounts of SLI, which consider the disorder a rule-leaning deficit. Instead, the model demonstrated the importance of phonology in the acquisition of the past-tense, especially in the generalisation to novel forms.

Another model, in Thomas (2005), and Thomas and Karmiloff-Smith (2003), addressed past-tense acquisition in the context of the Procedural Deficit Hypothesis (Ullman & Pierpont,2005). The Procedural Deficit Hypothesis, which considers SLI as a deficit in the procedural systems (rules), proposes a compensation mechanism to account for the existence of overgeneralisation errors for irregulars or the ability to generalise the past tense rule to novel strings in SLI. The authors examined whether a similar compensatory pattern could be demonstrated in a connectionist network. A feed-forward three-layered architecture was trained on mappings between phonological and lexical-semantics representations of the input and phonological representations of the output. Interestingly, when a low-discriminality activation function was used, the model qualitatively fitted the SLI profile in past-tense production (low overall inflection rates, reduced overgeneralisation errors for irregulars, poor generalisation to novel strings and increased frequency effects on regular verbs). Therefore, it was a domain-general deficit that simulated the compensation mechanism posited by the domain-specific Procedural Deficit Hypothesis. However, this model did not capture the prevalence of zero-marked errors in SLI, while it suggested a delayed rather than an atypical developmental profile for SLI.

In the domain of syntax comprehension, a model by Joanisse and Seidenberg (2003) examined anaphoric resolution, an aspect of syntax processing in SLI. A three layered feed-forward network was trained on mappings between sequences of phonological words to their meanings (semantics). SLI was simulated as a perceptual deficit, with the addition of random noise to the phonological representations. The domain-general perceptual deficit replicated the gradient pattern of performance for different types of sentences reported in

8

behavioural studies. For example, resolution of anaphoric sentences with reflexive pronouns (e.g. himself) was impaired to a greater extent than resolution of sentences with bound pronouns (e.g. him), while the degree of impairment was less, when gender information was provided by the context (e.g. Peter Pan says Wendy is tickling herself). The model suggested that the phonological-deficit account can explain grammatical/syntactic deficits in SLI. However, other authors note that not all children with SLI present a phonological deficit (cf. Ullman & Pierpont, 2005, p. 400).

Finally, a model by Thomas and Redington (2004) addressed the comprehension of complex syntactic structures (actives, subject clefts, passives and object clefts). The model was a Simple Recurrent Network (abbreviation: SRN) (Elman, 1990) which was trained on identifying an agent-patient or a patient-agent structure in a set of sentences presented sequentially (word-by-word) in the input. Except from an acquired deficit (random connection pruning), which was implemented to address behavioural data from Dick et al. (2001), a developmental deficit (fewer hidden units) was also considered. The developmental deficit, which corresponded to a ‘processing limitations’ account for SLI, produced a pattern of performance qualitatively different from the performance pattern of the acquired deficit case. In particular, the model generated the prediction that passives structures would be relatively less vulnerable in a developmental deficit. Interestingly, this prediction was verified by a subsequent study of Dick, Wulfeck, Krupa-Kwiatkowski, and Bates (2004).

In conclusion, although successful in capturing many aspects of atypical language development in the domains they addressed, connectionist studies on SLI were limited in the most prevalent deficits in SLI, i.e. the deficits in morphology and syntax. Findings from the existing modelling approaches could not be combined to address the heterogeneous profile of SLI. This project therefore considers that SLI is “undermodelled” and proposes further investigation of SLI with connectionist modelling.

2. Proposed research methodologies

As outlined above, this project proposes a systematic investigation of SLI with the connectionist methodology, aiming to consider a wide range of phenomena on language development, to adopt a cross-linguistic perspective, and to seek a unified explanation that could address the heterogeneous profile of the impairment. This section summarises connectionist work already underway or completed and sketches out directions for future research.

2.1. Multiple infections Generator- A model of Morphology (Completed)

9

The first model that this project proposes is a model for the acquisition of inflectional morphology. As discussed in the introduction, deficits in this domain (e.g. omission of inflections) are prevalent in SLI, and are often considered a behavioural hallmark for SLI (Rice, 2000). Furthermore, theoretical accounts of SLI (eg. the EOI account, Rice et al., 1995), as well as the consideration of G-SLI subtype (van der Lely, 1996), were based on such deficits. A detailed description of the acquisition of inflectional morphemes is provided in the literature for both typical development (e.g. Brown, 1973 ; de Villiers & de Villiers, 1985) and SLI (e.g. van der Lely, 1996).

The proposed model, the Multiple inflections Generator (abbreviation: MIG) combines features of previous models of morphology in order to implement a generalised inflectional system. Previous models have shown that a single connectionist architecture can accommodate different inflection types (different Part-of-Speech (POS) categories) (Plunkett & Juola, 1993) and different inflections within types (e.g. noun plural, noun genitive) (Hoeffner & McClelland, 1993). Based on these findings, MIG was developed as a model for the acquisition of noun (base forms, plural, genitive), verb (base forms, progressive, 3 rd

singular, past tense), and adjective (base forms, comparative, superlative) inflections.

The architecture of MIG is depicted in Fig.1. It is a three-layered feed-forward architecture, which learns mappings between multiple types of input information (multiple-cues model: lexical-semantics, grammar, phonology, and target inflection) and output phonological representations.

Fig.1. The architecture of Multiple Inflections Generator (MIG)

Results2

2 Pilot results from simulations were presented in the ‘Alston Child Language Meeting’, 9-11 May 2008.

10

HIDDEN LAYER

SEMANTICS PHONOLOGY INPUTGRAMMATI

CAL CATEGORY

TARGET INFLECTI

ON

PHONOLOGY OUTPUT

The model (implemented in MATLAB) was trained using a training set that consisted of 2000 words of an artificial language, encapsulating basic features of English. The training set was constructed based on measurements obtained from the Tagged Brown Corpus (abbreviation: TBC) (Francis & Kucera, 1979) with computational linguistics methods (Python). Although, the TBC is distant from child language, using measurements from this corpus was to a degree appropriate, as children are also exposed to adult language. Frequencies were measured for the following: POS categories, inflections types, allophones, irregular mappings types, and phonologically overlapping items of different POS categories. Approximations of the obtained measurements were used for constructing the training set of MIG. A table that summarises the basic features of the training set is provided in the Appendix. The training set also considered low and high frequency tokens.

MIG was trained3 until it reached ceiling performance (accuracy: 99.85%). The model captured several phenomena described in the literature for acquisition of inflectional morphology. For example, the order with which different inflections were acquired (Fig.2) was consistent to that described in the studies of Brown (1973), and de Villers and de Villers (1981). Accuracy in subtypes of regular inflections (e.g. past tense allophones, Fig.3), was depending on their frequency within the inflection. Furthermore, differences in the acquisition of low and high frequency items (Fig.4) were more pronounced in irregular inflections. Finally, the model predicted that zero-marking errors occur mainly in verb inflections (Fig.5).

An impaired version of MIG, which employed fewer hidden units, was also considered as an implementation of the general-processing- limitations account for SLI. As depicted in Fig.6 this version performed significantly worse than the normal model. Different inflections were affected differentially (Fig. 7), in line with the model of Hoeffner and McClelland (1993). Furthermore, the impaired model produced more zero-mark errors (omissions of inflections) than the normal model. Zero-mark errors of the two models in the past-tense inflection are plotted in Fig. 8. The pattern presented is qualitatively consistent with the EOI account of SLI (Rice et al., 1995).

A generalisation set consisting of items which rhymed with items of the training set (in two degradations: low and high similarity), and strings violating the phonotactic rules was used to assess the ability of the model to generalise the inflection on novel items. Generalisation results for the past-tense rule are illustrated in Fig. 9. Although accuracy (Fig.9, upper left) in generalising the past-tense rule was low for non-phonotactic novel words (blue line), the errors that the model made were mostly incorrect replications of an unfamiliar stem (Fig.9, bottom left). This yields that although the model could not reproduce the stem accurately, it learned to apply the rule (i.e. to apply a past-tense suffix) to most (>80%) of the novel items (Fig.9, upper right). These high rates of generalisation that MIG achieved are consistent with behavioural data, and are an interesting result, not demonstrated by previous connectionist models of morphology. Furthermore, the low rates

3 The exact parameters of training were: 250 epochs, BackPropagation with Cross-Entropy Error, learning rate 001, 75 hidden units, 95 units for input and output phonology, 2000 units for lexical semantics, 3units for grammatical category, and 10 units for the targeted inflection.

11

of generalisation that connectionist models of morphological processing exhibited, was a big criticism of Marcus to connectionism (e.g. Marcus, 1996).

Future directions

The pilot results from simulations with MIG are promising. Further work with MIG will aim to seek parameters under which a quantitative match to the typical development and SLI data (accuracy, error types) is achieved. Training under other atypical conditions (impairments in phonology and semantics, low-discrimination activation function) will also be considered. Finally, the model will be extended to explain acquisition of inflectional morphology in a morphologically rich language (Greek).

Fig.2. Normal model: Order of emergence of different inflection types

12

Fig.3. Normal model: Accuracy in regular past tense allophones

Fig.4. Normal model: Frequency effects in irregular past tense past-tense

13

Fig.5. Normal model: No-mark errors in different POS categories

Fig.6. Overall accuracy of normal and impaired model

14

Fig.7. Differential effect of impairment in different inflection types

Fig.8: No-mark errors in past-tense

15

Fig.9. Normal model: accuracy/errors in the generalisation of past-tense

2.2. Models proposed as future work

In this section, a series of connectionist models is proposed as future work for this project. These models are just sketched out, so as to demonstrate the empirical phenomena that will be targeted and the architectures that will be used. A great deal of implementation details is omitted.

2.2.1. Model of phonology

The second model proposed by this project is a model for phonological processing in SLI. As already discussed, deficits in phonology (e.g. in tone discrimination (Tallal & Piercy,

16

1973a, b), phoneme discrimination and identification of phoneme constancy (Bird et al., 1995) or non-word repetition (Gathercole et al., 1995)) are prevalent in SLI. According to the phonological-deficit account, such deficits affect many other aspects of language.

Fig.10. Proposed architecture for investigating phonological processing

To address these phenomena, this project proposes a model that learns phonological words, i.e. mappings between input and output phonological representations (Fig.10). The architecture is similar to a model proposed by Harm and Seidenberg (1999), which studied phoneme discrimination in dyslexics. The proposed model will investigate neighbourhood density and frequency effects in the acquisition of phonology and will also consider training under constraints of atypical development. Finally, the model will also be extended to Greek.

2.3. Model of semantics

Another major area of deficits in SLI is lexical semantics. As outlined above, children with SLI present word-finding difficulties (McGregor & Leonard, 1995; German, 1987), naming errors (Casby, 1992 ; McGregor & Waxman, 1995), impoverished word-definitions (Dockrell et al., 2003) and problems in the learning of new words (Oetting et al., 1995).

The connectionist architecture of Fig. 11 is proposed to target these empirical phenomena. The model is based on the model for semantic cognition of Rogers and McClelland (2004). The model will employ distributed feature-based semantics representations for a set of entities. The architecture will be trained on mappings between the input and the output representations Differences in the acquisition of prototype and atypical entities and differences in the ease of acquisition of different semantic features will be investigated.

17

Fig.11. Proposed architecture for investigating semantics.

Finally, training under conditions of reduced resources or other learning constraints will be considered, to simulate SLI.

2.4. Model of Syntax Comprehension

Another model proposed in this project will address SLI deficits in syntax comprehension. This model will focus in the comprehension of active, subject-cleft, passive, and object-cleft sentences. In particular, the model will expand on the previous modelling work of Thomas and Redington (2004), in which an SRN (Elman, 1990) simulated empirical phenomena on syntax comprehension from Dick et al (2001). This model generated predictions which were in turn verified by a subsequent study by Dick et al. (2004).

The proposed model will therefore use a similar recurrent architecture (Fig. 12) to target the empirical data from Dick et al.(2004) on the comprehension of complex sentence structures by typically developing children, children with SLI, and children with focal lesions. Due to the recurrent loop of hidden units SRNs are capable of processing sequential information and consequently sentence syntax. This model will therefore be trained to assign an AP (agent-patient) or a PA (patient-agent) tag (correspondingly) to active/subject-cleft and passive/object-cleft sentences presented sequentially in the input. Prediction of the next work - a common task for SRNs- will also be investigated.

Again, the aim will be to fit the behavioural data and to identify parameter settings which simulate the SLI profile. The model will be also extended to address relevant Greek data (Stavrakaki, 2002).

18

Fig.12. Proposed architecture for investigating syntax comprehension

2.5. Model of Syntax Production

Finally, a model for investigating syntactic production will be developed. The phenomenon that will be targeted is the formation of Wh-questions. In SLI, Wh-question formation is characterised by deficits (e.g. “What under the table?” “What John eat something?”) (Ingram, 1972 ; van der Lely & Batel, 2003).

To address the domain of question formation, an architecture (Fig.13) which learns to map the input semantic scheme (e.g. in an agent/patient/action format) and the input type information (e.g. declarative/interrogative) of sentences, to a sequence of output representations for the words (the targeted question) is proposed. This architecture is based on a Jordan network (Jordan, 1986) and employs recurrent connections from the output to the input, which enable a dynamic output sequence to be produced from a static input representation.

Training under limited processing resources or other learning constraints will be considered for simulating SLI. The model will pursue the thesis that error patterns in SLI represent the interference between canonical/ high frequency syntactic forms (e.g. declarative) and lower frequency forms (e.g. interrogative) under suboptimal conditions. Extension to Greek data (Stavrakaki, 2006) will also be considered.

19

Fig.13. Proposed architecture for investigating syntax production

20

3. Timetable

Time period TaskWinter term

07/08(Completed)

-Preliminary review of the literature

Spring term 07/08

(Completed)

-Development of the model for inflectional morphology (MIG)-Production and processing of pilot results

Summer term 07/08

-Completion of investigations with MIG (English)-Extension of MIG to Greek

Winter term 08/09

-Development of the model of phonology, generation / processing of results-Development of the model of semantics, generation /processing of results- Extension of the model of phonology to Greek, generation/processing of results

Spring term 08/09

-Development of the model of syntactical comprehension, generation / processing of results

Summer term 08/09

-Extension of the model of syntactical comprehension to Greek, generation / processing of results

Winter term 09/10

-Development of the model of syntactical production, generation/ processing of results

Spring term 09/10

-Extension of the model of syntactical production to Greek, generation / processing of results-Writing up of thesis

Summer term 09/10

-Writing up of thesis

Table.1. Timetable allocating research time

21

References

Aram, D., & Nation, J. (1975). Patterns of language behavior in children with developmental language disorders. Journal of Speech and Hearing Research, 18, pp. 229-241.

Bird, J., Bishop, D.V.M., & Freeman, N.H. (1995). Phonological awareness and literacy development in children with expressive phonological impairments. Journal of Speech and Hearing Research, 38, pp. 446-462.

Bishop, D.V.M. (2000). Pragmatic language impairment: a correlate of SLI, a distinct subgroup, or part of the autistic continuum? In D. V. M. Bishop & L. B. Leonard (Eds.), Speech and Language Impairments in Children: Causes, Characteristics, Intervention and Outcome Hove, UK: Psychology Press.

Bishop, D.V.M. (1997). Uncommon understanding: Development and Disorders of language comprehension in children. Hove, England: Psychology Press.

Bishop, D.V.M. (1979). Comprehension in developmental language disorders, Developmental Medicine and Child Neurology 21, pp. 225–238.

Brown, R. (1973). A first language: The early stages. London: George Allen & Unwin Ltd.

Casby, M.W. (1992). An intervention approach for naming problems in children. American Journal of Speech-Language Pathology, 1, pp. 35-42.

Clahsen, H., & J. Dalalakis 1999. Tense and agreement in Greek SLI: A case study. Essex Research Reports in Linguistics, 24, pp. 1-25.

Craig, H. (1993) Social skills of children with specific language impairment: Peer relations. Language, Speech, and Hearing Services in Schools, 24, pp. 206-215.

de Villiers, J.G., & de Villiers, P.A. (1985). Acquisition of English. In D. Slobin (Ed.), The crosslinguistic study of language acquisition: Vol. 1. The data (pp. 27-140). Hillsdale, NJ: Lawrence Erlbaum.

Dick, F., Wulfeck, B., Krupa-Kwiatkowski, & Bates (2004). The development of complex sentence interpretation in typically developing children compared with children with specific language impairments or early unilateral focal lesions. Developmental Science, 7(3), pp. 360-377.

Dick, F., Bates, E., Wulfeck, B., Aydelott Utman, J., Dronkers, N., & Gernsbacher, M.A. (2001). Language deficits, localization and grammar: evidence for a distributive model of language breakdown in aphasics and normals. Psychological Review, 108(4), pp. 759-788.

Dockrell, J.E., Messer, D., George, R., & Ralli, M. (2003). Beyond naming patterns in children with WFDS – definitions for nouns and verbs. Journal of Neurolinguistics, 16, pp. 191-211.

Elman, J.L. (1990). Finding structure in time. Cognitive Science, 14, pp. 179-211.

Fodor, J. A., & Pylyshyn, Z. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28, pp. 3-71.

Francis, W.N., & Kucera, H. (1979). Manual of Information to Accompany a Standard Sample

22

of Present-day Edited American English , for Use with Digital Computers. Original ed. 1964, revised 1971, revised and augmented 1979. Department of Linguistics, Brown University, Providence, R.I.

Freudenthal, D., Pine, J.M. & Gobet, F. (2006). Modelling the development of children's use of Optional Infinitives in English and Dutch using MOSAIC. Cognitive Science, 30, pp. 277-310.

Gallagher, T.M., & Craig, H.K. (1984). Pragmatic assessment: Analysis of a highly frequent repeated utterance. Journal of Speech and Hearing Disorders , 49 , pp. 368-377.

Gathercole, S. E., & Baddeley, A. D. (1990). Phonological memory deficits in language disordered children: Is there a causal connection? Journal of Memory and Language 29, pp. 336–360.

German, D. J. (1987). Spontaneous language profiles of children with word-finding problems. Language, Speech, Hearing Services in Schools, 18, pp. 217-230.

Gopnik, M., & Crago, M. (1991). Familial aggregation of the developmental language disorders. Cognition, 39, pp. 1–50.

Green, C.D. (1998) Are connectionist models theories of cognition? Psycoloquy 9(4).

Harm, M.W. & Seidenberg, M.S. (1999). Phonology, Reading Acquisition, and Dyslexia: Insights from Connectionist Models. Psychological Review, 106(3), pp. 491-528.

Hoeffner, J. H., & McClelland, J. L. (1993). Can a perceptual processing deficit explain the impairment of inflectional morphology in developmental dysphasia? A computational investigation. In E. V. Clark (Ed.), Proceedings of the 25th Child Language Research Forum. Palo Alto, CA: Stanford University Press.

Joanisse, M.F. (2004) Specific language impairments in children: Phonology, semantics and the English past tense. Current Directions in Psychological Science, 13, pp. 156-160.

Joanisse, M.F., & Seidenberg, M.S. (1999). Impairments in verb morphology following brain injury: A connectionist model. Proceedings of the National Academy of Science USA, 96, pp. 7592-7597.

Joanisse, M.F., & Seidenberg, M.S. (2003). Phonology and syntax in specific language impairment: Evidence from a connectionist model. Brain and Language, 86, pp. 40-56.

Johnston, J., & Kamhi, A. (1984). Syntactic and semantic aspects of the utterances of language-impaired children: The same can be less. Merrill-Palmer Quarterly, 30, pp. 65–85.

Johnston, J., & Ramstad, V. (1983). Cognitive development in preadolescent language impaired children. British Journal of Disorders of Communication, 18, pp. 49-55.

Jordan, M.I. (1986). Attractor dynamics and parallelism in a connectionist sequential machine. Proceedings of the Eighth Annual Conference of the Cognitive Science Society, Englewood Cliffs, NJ: Erlbaum, pp. 531-546.

Ingram, D. (1972). The acquisition of questions and its relation to cognitive development in normal and linguistically deviant children: A pilot study. U Papers and reports in child language development (br. 4, 13). Committee on linguistics Stanford University.

Kail, R. (1994). A method for studying the generalized slowing hypothesis in children with specific language impairment. Journal of Speech and Hearing Research, 37, pp. 418-421.

23

Leonard, L.B. (1998). Children with specific language impairment. Cambridge, MA: MIT Press.

Leonard, L.B., Bortolini, U., Caselli, M.C., McGregor, K.K., & Sabbadini, L. (1992). Morphological deficits in children with specific language impairment: The status of features in the underlying grammar. Language Acquisition, 2 , pp. 151-179.

Marcus, G.F. (1996) Why do children say “breaked”? Current Directions in PsychologicalScience, 5, pp. 81-85.

Marcus, G. F. (2001). The Algebraic Mind: Integrating Connectionism and Cognitive Science. Cambridge, MA: MIT Press.

McGregor, K.K., & Leonard, L.B. (1995). Intervention for word-finding deficits in children. In M. Fey, J. Windsor & S. Warren, (Eds.), Language intervention: Preschool through the elementary years , (pp. 85-105). Baltimore, MD: Paul H. Brookes.

McGregor, K.K., & Waxman, S.R. (1995). Multiple level naming abilities of children with word-finding deficits. Paper presented at the Boston University Conference on Language Development, Boston, MA.

O'Reilly, R.C., & Munakata, Y. (2000). Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain. Cambridge: MIT Press.

Oetting, J., Rice, M., & Swank, L. (1995). Quick incidental learning (QUIL) of words by schoolagechildren with and without SLI. Journal of Speech and Hearing Research, 38 , pp. 434–445.

Pinker, S. (1991). Rules of language. Science, 253, pp. 530-535.

Plunkett, K., & Juola, P. (1999). A connectionist model of English past tense and plural morphology. Cognitive Science, 23, pp. 463-490.

Plunkett, K., & Marchman, V. (1991). U-shaped learning and frequency effects in a multi-layered perceptron: Implications for child language acquisition. Cognition, 38, pp. 1-60.

Plunkett, K., & Marchman, V. (1993). From rote learning to system building: Acquiring verb morphology in children and connectionist nets. Cognition, 48, pp. 21-69.

Rapin, I. & Allen, D. (1983). Developmental language disorders: Nosologic considerations. In U.Kirk (Ed.), Neuropsychology of language, reading, and spelling (pp. 155–184). New York: Academic Press.

Rice, M.L. (2000). Grammatical symptoms of specific language impairment. In: D.V.M. Bishop & L.B. Leonard (Eds.), Speech and Language Impairments in Children: Causes, characteristics, intervention and outcome (pp.17-34). Hove, England: Psychology Press.

Rice, M, Wexler, K. & Cleave, P. (1995). Specific Language Impairment as a period of extended optional infinitive. Journal of Speech and Hearing Research, 38, pp. 850-863.

Rogers, T. T. and McClelland, J. L. (2004). Semantic Cognition: A Parallel Distributed Processing Approach. Cambridge, MA: MIT Press.

24

Rumelhart, D. E. & McClelland, J. L. (1986). On learning the past tense of English verbs. In J. L. McClelland, D. E. Rumelhart and PDP Research Group (Eds), Parallel distributed processing: Volume 2: Psychological and Biological models. (pp. 216-271). Cambridge, MA: MIT Press.

Rumelhart, D. E., McClelland, J. L., & PDP Research Group (Eds.) (1986). Parallel distributed processing: Volume 1: Foundations. Cambridge, MA: MIT Press.

Stavrakaki, S. (2006). Developmental perspectives on Specific Language Impairment: Evidence from the production of wh-questions by Greek SLI children over time. International Journal of Speech-Language Pathology, 8( 4) , pp. 384–396.

Stavrakaki, S. (2002). Sentence comprehension in Greek SLI children. In L. Kelly, F. Windsor & N. Hewlett (Eds.), Advances in Clinical Linguistics and Phonetics. Hillsdale, NJ:Erlbaum. Chapter 5, pp. 57-72.

Stavrakaki, S. (2001). Comprehension of reversible relative clauses in Specifically Language Impaired and Normally Developing Greek children. Brain and Language, 77, pp. 419-431.

Stavrakaki, S. (2000). Verb lexicons in SLI: some experimental data from Modern Greek. Journal of Greek Linguistics, 1, pp. 93-129.

Tallal, P., & Piercy, M. (1973a). Defects of non-verbal auditory perception in children with developmental aphasia. Nature, 241, pp. 468-469.

Tallal, P., & Piercy, M. (1973b). Developmental aphasia: Impaired rate of non-verbal processing as a function of sensory modality. Neuropsychologia, 11, pp. 389-398.

Tomblin, J.B., Records, N.L., Buckwalter, P., Zhang, X., Smith, E., & O’Brien, M. (1997). Prevalence of specific language impairment in kindergarten children. Journal of Speech, Language and Hearing Research, 40, 1245-1260.

Thomas, M. S. C. (2005). Characterising compensation. Cortex, 41(3), pp. 434-442.

Thomas, M.S.C. & Karmiloff-Smith, A. (2003). Modelling language acquisition in atypical phenotypes. Psychological Review, 110 (4), pp. 647-682.

Thomas, M.S.C. & Karmiloff-Smith, A. (2002). Modelling typical and atypical cognitive development. In U. Goswami (Ed.), Handbook of Childhood Development (pp. 575-599). Blackwells Publishers.

Thomas, M.S.C. & Redington, M. (2004). Modelling atypical syntax processing. In W. Sakas (Ed.), Proceedings of the First Workshop on Psycho-computational models of human language acquisition at the 20th International Conference on Computational Linguistics, pp. 85-92.

Tsimpli, I.M. & Stavrakaki, S. (1999). The effects of a morphosyntactic deficit in the Determiner System: The case of a Greek SLI child. Lingua, 108, pp. 31-85.

Ullman, M.T., & Pierpont, E. (2005). Specific Language Impairment is not specific to language: The Procedural Deficit hypothesis. Cortex, 41(3), pp. 399-433.

van der Lely, H. K. J. (1996). Empirical evidence for the modularity of language from Grammatical SLI children. Proceedings of The 20 annual Boston University conference on language development, Volume 2. Somerville, MA: Cascadilla Press, pp. 781-791.

25

van der Lely, H. K. J., & Battell, J. (2003). Wh-movement in children with grammatical SLI: a test of the RDDR hypothesis. Language, 79, pp. 153–181.

van der Lely, H. K. J., & Harris, M. (1990). Comprehension of reversible sentences in specifically language impaired children. Journal of Speech and Hearing Disorders, 55, pp. 101-117.

van der Lely, H. K. J. & Stollwerck, L. (1997). Binding theory and specifically language impaired children, Cognition, 62, pp. 245–290.

Wilson, B.C., & Risucci, D.A. (1986). A model for the clinical quantitative classification Generation 1—Application to language disordered preschool children. Brain and Language, 27 , pp. 281–309.

26

Appendix: The training set of MIG: features that approximate the measurements on the tagged Brown Corpus.

level0 level1 level2 level3 level4 level5count

s

TOKEN TYPES (1600)

NOUNS (800)

SINGULAR (600)

600

PLURAL (150)

REGURAL (770)

/s/ (140)

140

/z/ (500)

500

/ez/ (130)

130

IRREGULAR (30)

30

GENITIVE (50)

/s/ (150)

150

/z/ (500)

500

/ez/ (150) 150

VERBS (400)

BASE FORMS (130)

130

3rd SINGULAR

(70)

/s/ (130) 130

/z/ (200) 200

/ez/ (70) 70

PROGRESSIVE (80)

80

PAST TENSE (120)

REGULAR (330)

/t/ (65) 65

/d/ (180) 180

/ed / (85) 85

IRREGULAR (70)

IDENTITY (10)

10

VOWEL_CHANGE (50)

50

ARBITRARY (10)

10

ADJECTIVES (400)

BASE FORMS (320)

320

COMPARATIVE (40)

REGULAR (380)

380

IRREGULAR (20)

20

SUPERLATIVE (40)

REGULAR (380)

380

IRREGULAR (20)

20

27