assessment battery for communication — abaco: a new...

Assessment Battery for Communication — ABaCo: A new Instrument for the Evaluation of Pragmatic Abilities*

Sacco K.,1, 2, 3 Angeleri R.,1 Bosco F.M.,1, 3 Colle L.,1, 3

Mate D.,1 and Bara B.G.1, 3

1Center for Cognitive Science and Department of Psychology, University of Turin, Italy

2CCS fMRI, Neuroradiology at Koelliker Hospital & Department of Psychology at University of Turin, Italy3Institute of Neuroscience of Turin, Italy

[email protected]

The Assessment Battery for Communication (ABaCo) is a new clinicalinstrument for the evaluation of communicative abilities in patients withneuropsychological and psychiatric disorders, such as aphasia, right hemisphericdamage, closed head injury, autism and schizophrenia. ABaCo consists of 5scales, investigating comprehension and production of linguistic andextralinguistic acts, paralinguistic expressions, appropriateness with respect todiscourse and social norms, and management of conversation. Validitymeasures (content and construct validity) and reliability measures (inter-raterreliability and internal consistency) were computed. The experts’ contentvalidity evaluations indicate an excellent match between test items and themeasurement of pragmatic abilities, as well as the suitability of the battery forboth children and adults. Regarding the other psychometric measures, computedon 390 normal children in different age groups, factor analysis shows thevalidity of the underlying theoretical construct. Reliability analyses show a highinter-rater agreement, suggesting that the battery can be administered and scoredby any trained judge, and a good internal consistency, suggesting that thevarious items that make up each scale are coherent and contribute to themeasurement of communicative abilities.

Journal of Cognitive Science 9: 111 - 157, 2008.ⓒ2008 Institute for Cognitive Science, Seoul National University.* We thank the anonymous JCS reviewers for their useful comments and suggestionsto the first version of the paper.

Keywords: Assessment, Communication, Pragmatic abilities, Battery,Cognitive Pragmatics

1. Introduction

The aim of the paper is to present a new clinical instrument for theassessment of communicative abilities in patients with neuropsychological andpsychiatric disorders, due to congenital or acquired cerebral pathologies, suchas aphasia, right hemispheric damage, closed head injury, autism andschizophrenia. The battery is designed both for children and for adults, in twoslightly modified versions. Here we describe the battery and report some of itspsychometric properties, such as content and construct validity, and reliabilitymeasures, all based on a non-clinical sample. The validation of the battery onthe above-mentioned clinical populations is in progress, and will thus bepresented in future works.

A pragmatic approach to the assessment of language problems emerged inthe early 1980s (see, for example, Prutting, 1982), and went on to gainincreasing clinical popularity. Traditional language assessment models,encompassing phonetic, syntactic and semantic aspects, were not sufficient todetermine the impact of patients’ disabilities on their everyday life. Indeed,patients with different etiologies can perform similarly on standard languagetests but have vastly distinctive communicative profiles; even more strikingly,there are patients who present little or no deficit on formal language tests butwho have substantial problems in the social use of communication. Forexample, right-hemisphere-damaged individuals have difficulty inunderstanding the main points of a conversation (Hough, 1990) and indistinguishing lies from jokes (Winner, Brownell, Happe, Blum & Pincus,1998). Traumatic-brain-injured patients’ deficits include tangential and sociallyinappropriate speech (Hartley & Jensen, 1991) and inability to meet theinformational needs of the listener (McDonald, 1993). Autistic patientsperform poorly when required to distinguish between inappropriate andappropriate utterances, i.e. utterances that avoid redundancy, are informative,truthful, relevant and polite (Surian, Baron-Cohen & Van der Lely, 1996;Tager-Flusberg, 2000), and so on.

112 Sacco K., Angeleri R., Bosco F.M., Colle L., Mate D., and Bara B.G.

Our goal is to create a theoretically grounded, wide-range instrument,capable of assessing and comparing both comprehension and production ofvarious kinds of pragmatic phenomena, using different means, such aslinguistic, extralinguistic and paralinguistic communication. The aim is not todiagnose a specific pathology, but rather to assess specific communicativedeficits in patients with different pathologies.

A detailed review of instruments for the clinical assessment of pragmaticswas published by Penn (1999). Given the variety of formats, administrationprocedures and items assessed by the different instruments, in Table 1 we

Assessment Battery for Communication — ABaCo 113

Table 1. Most commonly-used instruments for communication assessment in clinical practice

R-type Test DescriptionCommunication categories

investigated

Pragmatic Protocol — PP (Prutting & Kirchner,1987)

Profile of CommunicativeAppropriateness — PCA (Penn, 1985)

Children’sCommunicationChecklist — CCC (Bishop, 1998)

Right HemisphereCommunicationBattery — RHCB(Gardner & Brownell,1986)

The examiner evaluates a 15-minutesample of on-line or videotapedspontaneous conversation with a familiarpartner on a 30-item grid. Items arescored as appropriate, inappropriate or notobserved.

The examiner evaluates a 20-minutesample of prompted conversation(description of a procedure e.g. makingtea, history of the patient’s disease, a topicof common interest) on a 36-item grid.Items are scored for appropriateness on a5-point Likert scale.

Two independent raters (e.g. teacher andspeech-language therapist) compile aqualitative questionnaire about the child’scommunicative abilities.

The examiner presents the patient with 11tests measuring pragmatic abilities:picture/sentence sequence completion forhumorous stories, repetition of jokes,recognition of emotions from prosody,comprehension of indirect requests,

Verbal aspects (speechacts, topic, turn-taking,lexical selection, stylisticvariations), paralinguisticaspects (intelligibility andprosodics), non-verbalaspects (kinesics andproxemics).

Response to interlocutor,semantic control, cohesion,fluency, sociolinguisticsensitivity, non-verbalcommunication.

Inappropriate initiation,coherence, stereotypedconversation, use ofcontext, rapport (plus othernon-pragmatic categories).

Humor, emotion, non-literal language, integrativeprocesses.


Table 1. Continued


investigated

Right HemisphereLanguage Battery — RHLB (Bryan, 1995)

The Awareness ofSocial Inference Test— TASIT (McDonald et al.,2003)

Promoting Aphasics’CommunicativeEffectiveness — PACE (Davis & Wilcox,1985)

FunctionalCommunicationProfile — FCP (Sarno, 1969)

matching pictures and explainingmetaphoric phrases, making inferencesfrom sentence pairs, understandingsarcastic pieces of conversation, detectionof different meanings of words, recallingand understanding stories.

The examiner presents the patient with 6tests measuring linguistic and pragmaticabilities: matching words to pictures,comprehension of metaphors usingwritten and pictorial material, inferringmeanings from short texts, recognition ofthe humorous punch-line of short writtenstories, comprehension of linguistic stressusage.

The examiner shows the patient shortvideos of everyday interactions andevaluates the patient’s ability to makesocial inferences which requireinterpretation of facial expression andintonation, as well as taking into accountof physical context and speakers’knowledge and beliefs.

The examiner involves the patient in aconversation, using a set of cards (cardsshowing everyday objects, verb cards andstory-sequence cards) as topics. In turn,examiner and patient must describe oneof their cards in order to make the otherunderstand which card is being described.

The examiner involves the patient in aconversation, and rates hiscommunication behavior using 45 itemsand a 9-point scale where 8 is ‘normal’,i.e. the same as pre-morbidly.

L e x i c o n / s e m a n t i c s ,metaphors, inferences,humor, emphatic accent.

Recognition of facialexpressions of emotions,recognition of sarcasm,recognition of lies vs.sarcasm.

Effectiveness in sendingand receiving messages:number of identified cards,number of exchangesneeded to reachunderstanding, number ofcrucial pieces ofinformation produced, useof compensatory strategies(gestures, paraphrases),ability in using feedbacks.

Understanding, reading,writing, speaking, other.

report some of the most representative ones by way of example. For the sake ofsimplicity, we adopt the convention that the examiner is a female and thepatient is a male.

In the reference literature, a general distinction is often drawn betweenpragmatic assessment and functional assessment.

[a] The goal of pragmatic assessment is to identify and measure singlecognitive processes underlying a range of communication behaviors, in orderto define profiles of impairment. To mention some examples, the PragmaticProtocol (PP; Prutting & Kirchner, 1987) and the Profile of CommunicativeAppropriateness (PCA; Penn, 1985) are pragmatic assessment checklists basedon the speech act theory, evaluating the appropriateness of specific pragmaticcomponents during conversation. In the field of developmental age, theChildren’s Communication Checklist (CCC; Bishop, 1998) is a questionnaire


Table 1. Continued


investigated

FunctionalAssessment ofCommunicationSkills — FACS(Frattali et al., 1995)

CommunicativeAbilities in DailyLiving — CADL (Holland, 1980;Holland et al., 1998)

The examiner rates the patient’sperformance on 43 functionalcommunication tasks (e.g. namingfamiliar people, expressing feelings, etc.),using the following two scales. The scaleof communication independence is a 7-point scale rating the level of assistanceneeded to complete a task. The scale ofqualitative dimensions of communicationis a 5-point scale rating adequacy,appropriateness, promptness andcommunication sharing.

The examiner evaluates videotaped clipsof conversation prompted by role-playingsimulating everyday activities (e.g. ‘at thegrocery store’) on a 68-item grid. Itemsare scored for functional appropriatenesson a 3-point scale. The number ofmessages measures communicativeattitude, and the comprehensibility of themessages measures communicativeefficiency.

Social communication;communication of basicneeds; reading, writing, andnumber concepts; dailyplanning.

Social conventions,utilizing context, speechacts, role playing ability,non-verbal symbolmanagement, reading-writing, humor sensitivity,repetition, reachingconclusions from multipleinformation.

compiled by a care-giver, investigating communication abnormalities inchildren, such as inappropriate initiation, discourse incoherence, socialinadequacy. There are also tests of pragmatic skills aimed at particularpathologies, such as the Right Hemisphere Communication Battery (RHCB;Gardner & Brownell, 1986), the Right Hemisphere Language Battery (Bryan,1995; standardized for the Italian population in Zanini & Bryan, 2003; Zanini,Bryan, De Luca & Bava, 2005), and the Awareness of Social Inference Test(TASIT; McDonald, Flanagan, Rollins & Kinch, 2003), the first two designedfor patients with right hemisphere damage and the latter for those withtraumatic brain injury.1 Finally, tests based on the referential communicationframework2 (Clark & Wilkes-Gibbs, 1986) are observational techniques usedfor conversational analysis in which the emphasis is on the patient’s ability toconvey a precise message and use feed-back information.

[b] The goal of functional assessment is to measure a person’s ability tocommunicate efficiently in real-life situations without directly identifying thecomponential abilities underlying communication. The first test of this type tobe developed was the Functional Communication Profile (FCP; Sarno, 1969),which rates the effectiveness of communicative behavior in an informalconversation. Other tests of functional assessment, with more precise scoringprocedures and psychometric properties, are the American speech-languagehearing association Functional Assessment of Communication skills for adults(ASHA FACS; Frattali, Thompson, Holland, Wohl & Ferketic, 1995), whichuses functional communication tasks such as naming familiar people,expressing feeling and so on, and the Communicative Abilities in Daily Living(CADL; Holland, 1980; Holland, Frattali & Fromm, 1998), which makes use


1 In the Italian context, a new pragmatic tool — the Right Hemisphere LanguageSanta Lucia Battery, BLED — has been recently developed. It evaluates deficits of apragmatic-verbal nature in patients with brain damage. Available at www.giuntios.it.One of the subtests (Metaphors) was administered to right brain-damaged patients ina previous study (Rinaldi, Marangolo & Baldassarri, 2004).

2 The setting used in these tests is similar to that of the rehabilitative tool calledPromoting Aphasics’ Communicative Effectiveness (PACE; Davis & Wilcox, 1985;Carlomagno, 1994); for details about how it can be used for assessment seeCarlomagno, Blasi, Labruna & Santoro, 2000.

of role-playing, reproducing everyday social situations. In clinical practice, an assessment instrument should serve two main

purposes: therapy planning and therapy outcome evaluation. Generallyspeaking, instruments for the assessment of pragmatics are useful tools forplanning specific intervention paths: indeed, on the basis of their well-specifiedtheoretical framework,3 they identify distinct pragmatic components, and arethus able to clearly identify breakdown behaviors, which can then be the mainobjects of rehabilitation. However, such measures are inadequate for evaluatingthe outcome of the therapy through test and re-test procedures: since most arebased on observation of a conversation, there is a lack of control of thesampled behaviors and, therefore, replicability is not guaranteed. On the otherhand, functional assessment instruments are helpful in outcome evaluation:since it is quite easy to reach agreement on a patient’s overall success on acertain functional task, they usually have adequate psychometric properties forevaluating inter-rater and intra-rater reliability, as well as internal validity,necessary for comparing data across time. However, such instruments are oftenless informative about therapy planning: because the various skills involved inthe examined communicative behaviors are not analytically analyzed, it isdifficult to program specific tasks for rehabilitation.

There are of course some exceptions to this rough classification. Forexample, the CADL is a functional assessment test: like other measurements ofeffectiveness it shows a high level of scoring reproducibility and is thussuitable for test-retest assessments; at the same time, it includes the possibilityof plotting and analyzing the component abilities tapped by each task, and thisenhances the test’s usefulness as a therapy planning tool. Specularly, referentialcommunication tests (Clark & Wilkes-Gibbs, 1986), as observationaltechniques used for conversational analysis, are helpful for developing therapyprograms; at the same time, thanks to their psychometric approach, they alsoappear to be valid for obtaining repeated measurements. However, the CADL,like other instruments typical of the functional approach, is not linked to aparticular theory of conversation; thus, performance cannot be interpreted with


3 Most of these instruments find their theoretical basis in the speech act theory(Austin, 1962; Searle, 1969), while others are an application of the conversationalanalysis of language (Sacks, Schegloff & Jefferson, 1978).

reference to the cognitive processes underlying communicative competence.Besides, this test does not include formal definitions for coding, and is affectedby the influence of contextual variables such as familiarity with the topics ofconversation and the type of relationship between interlocutors. As far asreferential communication tests are concerned, although able to describe theeffectiveness of sending and receiving messages and to detect residual abilitiesand compensatory strategies, they are limited to just a few communicativephenomena, and thus fail to provide an adequate description of the pragmaticcomponents. The latter tests can therefore be used for diagnosing differenttypes of aphasia, but are of little informative value in other pathologies, such astraumatic brain injury and dementia.

ABaCo is a pragmatic assessment instrument as, thanks to its solidtheoretical grounding, it can be used to evaluate the various abilities involvedin communication. At the same time, in order to overcome the limits ofpragmatic assessment tools, we designed an objective coding system and awell-specified administration/scoring training procedure, in order to achievereplicability. ABaCo also assesses a wide range of communicative phenomena,which can be selectively impaired in different pathologies. Indeed, its goal isthe pragmatic assessment of communicative abilities in patients with variouskinds of cognitive impairments, such as developmental disorders (e.g. autism,specific language impairment, Down syndrome), brain pathologies (e.g. closedhead injury, right hemisphere damage, aphasia), psychiatric disorders(schizophrenia), disorders of old age (e.g. dementia). To sum up, ABaCoattempts to satisfy the following theoretical and clinical needs:

— Theoretical grounding. The theoretical basis of the battery derivesfrom the Cognitive Pragmatics theory (Airenti, Bara & Colombetti1993a), a model of the cognitive processes underlying communication,from a speech act perspective (see next paragraph).

— Objective scoring. Scores are attributed according to a series ofwell-specified criteria, so that evaluations are not affected by examiners’subjectivity.

— Precise training. The training needed to administer and score thebattery is set out in clear steps. Training does not require the personalintervention of the authors as all the material is included in the battery


package.— Completeness. The battery assesses both the comprehension and the

production of a variety of communication acts, expressed using linguisticand extralinguistic modalities, and involving paralinguistic aspects as wellas discourse and social norms.

— Ecologic validity. The patient either deals with videotapedcommunicative interactions set in everyday contexts, or he is personallyengaged by the examiner in short exchanges/conversations.

— Flexibility. The battery comprises five different scales, which can beadministered separately, according to the patient’s needs.

— Broad extent. The battery can be administered to a variety ofpatients, characterized by different types of communicative impairments.

2. Theoretical Grounding

The theoretical basis of the battery derives from Cognitive Pragmatics(Airenti et al., 1993a), a theory of the cognitive processes underlying humancommunication. Cognitive Pragmatics stems from the speech act theory(Austin, 1962; Searle, 1969) and the cooperative principle of conversation(Grice, 1975; 1989). The soundness of the theory is demonstrated by a series offactors. First, it has been formalized by the computational method (Airenti etal., 1993a; Airenti, Bara & Colombetti, 1993b). Second, its relevance for thestudy of both the normal and the abnormal mind-brain system has been pointedout (Bara, Bosco & Bucciarelli, 1999a; Tirassa, 1999; Bara & Tirassa, 2000;Becchio, Adenzato & Bara, 2006). Third, its predictions have been tested in aseries of experimental works on normal children (Bucciarelli, Colle & Bara,2003; Bosco, Bucciarelli & Bara, 2004; 2006), children with autism (Bara,Bucciarelli & Colle, 2001), patients with traumatic brain injury (Bara, Tirassa& Zettin, 1997; Bara, Cutica & Tirassa, 2001), patients with right and left focalbrain lesions (Cutica, Bucciarelli, Bara, 2006), patients with Alzheimer’sdisease (Bara, Bucciarelli & Geminiani, 2000). Fourth, some of itsfundamental assumptions have been tested using neuroimaging techniques(Walter, Adenzato, Ciaramidaro, Enrici, Pia & Bara, 2004). A comprehensivepresentation of the theory, along with the results derived from its experimentalvalidations, can be found in Bara (2008).


We now summarize the main assumptions of the Cognitive Pragmaticstheory, that have been used to construct the assessment battery. The theory seescommunication as a social activity involving at least two persons whointentionally and overtly attempt to share meanings and to affect each other’smental states. Individuals can express their communicative intentions both withlinguistic and extralinguistic means. Linguistic communication is performedusing a system of compositional symbols, i.e. language, and its accompanyingspontaneous gesticulations; extralinguistic communication is the use of a set ofassociable symbols, i.e. gestures such as facial expressions, hand and bodymovements, intentionally performed to construct and share a communicativemeaning.4 Given that the theory holds for both linguistic and extralinguisticcommunication, we adopt the term communication act instead of speech act,and the terms actor and partner instead of speaker and hearer. Further, for thesake of simplicity, we refer to dyadic interactions, and we follow theconvention that actor A is always a female and partner B is always a male.

As intentional communication is a process of meaning construction, itrequires cooperation between the two agents. Within an interaction,cooperation is granted by the mutual knowledge of: a) a behavior game, i.e. thestereotypical scheme of interaction within which communication acts can beinterpreted; b) the conversation game, i.e. the rules governing the structure of


4 The Cognitive Pragmatics theory adopts the distinction between linguistic andextralinguistic communication rather than the traditional distinction between verbaland non-verbal communication (Hinde, 1972). Indeed, the latter contained a series ofpitfalls. For example, the traditional category of non-verbal communicationencompasses all non-acoustic communicative inputs, and thus includes signlanguages which are instead characterized by syntactic and semantic rulessubstantially identical to those governing spoken language. The distinction betweenlinguistic and extralinguistic communication, based on the modality of processingdata instead of on the input channel, make it possible to include sign languages as aform of linguistic communication. See Bara & Tirassa (1999) for a theoreticaldiscussion on this issue, and Cutica (2005) for neurological evidence in favor of suchdistinction. Further support for these concepts is provided by studies on deaf signlanguage users, adopting neuropsychological (Hickok, Bellugi & Klima, 1996) andneuroimaging (Emmorey, Grabowski, McCullough, Damasio, Ponto, Hichwa &Bellugi, 2003) techniques.

conversation. The behavior game governs the global structure of the interaction, being the

stereotypical scheme of interaction mutually shared by the participants. Inorder to understand the actor’s communicative intentions, the partner has tofind a meaningful connection between the actor’s expression act and thebehavioral game they are playing. The degree of complexity of the inferentialchain which connects the expression act to the behavior game it refers toallows us to operate a distinction between simple and complex communicationacts. In the case of simple communication acts, the passage from the expressionact to the behavior game is immediate: what the actor communicates(expressed content) immediately refers to the behavior game bid by the agents.Instead, in the case of complex communication acts, the passage requires aninferential chain of variable length: what the actor communicates does notimmediately refer to the behavior game; the comprehension of what the actormeans (speaker’s meaning) requires an inferential process capable ofconnecting the expressed content to the behavior game bid by the agents.Cases of simple communication acts are directs [1] and conventional indirects[2] (see Gibbs, 1994). Consider for example an interaction between two agentshaving dinner. If actor A says [1] “Please pass me the salt” or [2] “Can youpass me the salt?”, partner B simply has to refer the utterance to the game [ask-for-object] in order to understand that A intends to obtain the salt. In contrast,non-conventional indirects are complex communication acts [3]. For example,if actor A says [3] “My soup lacks salt” a more complex inferential process isnecessary: the partner needs to share with the actor the belief that if the souplacks salt it is not good to eat, and that if there is some salt on the table andsomebody proffers an utterance such as [3], she probably wants it. Only thencan the partner attribute the value of a move of the game [ask-for-object] to theutterance.

The conversation game governs the succession of stages by which acommunication act is understood. Assuming that actor A produces anutterance addressed to partner B, it is possible to distinguish the followinglogically-connected steps in B’s mental processes:

1. Expression act: B recognizes what A communicated, starting fromthe locutionary act.


2. Speaker’s meaning:5 B comprehends and reconstructs A’scommunicative intentions, i.e. what A implies or presupposes.

3. Communicative effect: B attributes private mental states (e.g. beliefs)to A.

4. Reaction: B produces the intentions he will communicate in hisresponse.

5. Response: B produces an overt communicative response.

The conversation game ensures that all the standard inferences, from 1 to 5,follow smoothly in succession without any blocks occurring. If one stage doesnot achieve its goal, then the conversation game blocks the default rulesspecific to that stage, activating an alternative cognitive process. Thus,standard communication acts are those whose production respects theinferential rules of communication, i.e. what the actor communicates is in linewith her private knowledge and with both agents’ shared knowledge, andwhose comprehension must follow an inferential chain based on default rules.Non-standard communication acts are those whose production violatescommunication rules, in that what the actor communicates is not in line witheither her private beliefs or with the two agents’ shared knowledge, and whosecomprehension involves detachment from the standard inferential chain(Bucciarelli, Colle & Bara, 2003). The most important non-standard acts aredeceits and ironies. In the case of deceit, the violation consists in the expressionof something different from what one privately entertains. For instance, actor Ahas broken a vase but does not want to be punished. Partner B asks “Whobroke the vase?”. A replies [4] “It was the dog”. In order to comprehend adeceit it is necessary to recognize the difference between the mental states theactor expresses and those that she privately entertains. Differently, in the caseof irony, the discrepancy between what the actor expresses and the knowledgeshe shares with the partner is exploited. For instance, partner B tries on a suitwhich is clearly too tight for him. Actor A says [5] “Your last diet workedreally well …”. In order to recognize the ironic meaning it is necessary to


5 For the sake of simplicity, the term speaker’s meaning is maintained even though itrefers to both linguistic and extralinguistic communication.

recognize the contrast between the expressed mental states and the scenarioprovided by the knowledge shared by the interlocutors that the diet has notworked at all. Standard and non-standard communication acts, both linguisticand extralinguistic, can be simple and complex (Bara, Bosco & Bucciarelli1999b; Bosco & Bucciarelli, 2007). All these types of communication actshave been included in our battery.

A special subset of simple standard acts are basic communication acts(Kasher, 1981), which comprise simple assertions, questions, requests andcommands. Assertions express a state of affairs, which can be judged as true orfalse; questions aim at obtaining information; requests aim to induce thepartner to perform a specific action; commands aim to force the partner toperform a specific action. Indeed, basic communication acts represent theprototypical categories of communication acts, and thus they can be considereda sort of baseline. For this reason, in the construction of the battery, we decidedto investigate them separately.

Table 2 schematizes the various types of linguistic and extralinguistic acts.Linguistic and extralinguistic communication acts are accompanied by

paralinguistic aspects that emphasize or possibly modify the expressedcontent. These aspects are not endowed with autonomous meaning, but addinformation to the message, contributing to its interpretation. Generally, the


Table 2. Types of communication acts

COMMUNICATION ACT

STANDARD NON-STANDARDNo conflicting representations YES conflicting representations

Default rules Non-default rules

BSA SIMPLE COMPLEX SIMPLE COMPLEXPrototypical simple Inferential Inferential Inferential Inferential

act load: low load: high load: low load: high

Source: Bucciarelli, Colle & Bara (2003, modified). The dotted line means that BSA are asubset of simple standard communicative acts but, being prototypical, they are consideredseparately.

term ‘paralinguistic’ refers to tributary language structures, in particular toprosodic cues, such as intonation, rhythm, and the voice’s tone, pitch, intensityand quality, accompanying speech. In actual fact, extralinguisticcommunication is also accompanied by modifiers, such as kinesics andproxemics. Kinesics include for instance head signs, facial expressions, bodymovements and ocular movements; proxemics refer to posture andinterpersonal distance. All these aspects can be considered as tributarycommunication structures, as they do not possess an autonomous meaning, butare rather qualifiers of communicative actions. Thus, we consider the term‘paralinguistic’ to include all aspects that accompany, qualify and structurelinguistic and extralinguistic communication.

Paralinguistic aspects are generally used to convey: a) a propositionalattitude, viz. the relationship between the actor and the communication act; e.g.an assertion presupposes a propositional attitude of belief;

b) an emotional state. In our battery, we include both cases: paralinguisticaspects indicating the actor’s propositional attitude towards basiccommunication acts (assertion, question, request, command), and paralinguisticaspects conveying an emotional state (fear, anger, happiness, sadness …).Paralinguistic aspects can also be tuned or not tuned to the semantic contentexpressed through language or extralinguistic gestures. When the semanticcontent and paralinguistic components diverge, a paralinguistic contradictionemerges. In our battery, we introduced items characterized by a paralinguisticcontradiction, i.e. items in which the agent’s expressed content is contradictedby the paralinguistic indicators revealing a different mental state. For example,in one of the videos, a boy receives a tie as a gift and he says “Oh, thanks, it’svery nice!”, while his voice and his attitude reveal that he doesn’t like it.

Finally, communication acts have to be appropriate with respect to thecontext of enunciation as well as to the social context. As far as adequacy inrelation to discourse norms is concerned, Grice (1975) formulated the‘cooperative principle’, which states that participants in a conversation expecteach other to make a“conversational contribution such as is required, at thestage at which it occurs, by the accepted purpose or direction of the talkexchange”. In terms of the Cognitive Pragmatics theory, the cooperativeprinciple can be viewed as a declination of rules belonging to the conversationgame, as it regards the respect of the principles which regulate the local


structure of conversation. During conversation, participants assume that theperson expressing a communication act is being cooperative. For this reason, inorder to evaluate in/adequacy in relation to discourse norms we tested theability to comprehend the conversational maxims deriving from thecooperative principle: quantity, quality, relation and manner. On the other hand,adequacy in relation to social norms concerns the praxis of organizingcommunicative action in a way which is considered appropriate to the socialsituation in which one is involved. Such a praxis includes the ability to select acommunication act which is appropriate to the context, and that of evaluatingits degree of acceptability with respect to the partner’s expectation. Thecapacity to behave in an adequate way is framed by the common knowledge(Clark, 1996) shared by a given social community, which determines the styleof interaction depending on social roles, hierarchical positions, contexts offormality and informality. These capacities are tested in our battery, both interms of detecting transgressions of social norms, and in terms of producingcommunication acts respectful of the social context.

The various communicative abilities delineated above have to be integratedwithin a conversation, in which the agents must coordinate their reciprocalparticipation. Pickering and Garrod (2003) refer to the notion of ‘coordination’as meaning that interlocutors are coordinated in a successful dialogue just asparticipants in any successful joint activity are coordinated (e.g., ballroomdancers, or lumberjacks using a two-handed saw). The Cognitive Pragmaticstheory views the global structure of dialogues as deriving from the sharedknowledge of an action plan, i.e. the behavior game. The behavior gamegoverns the interaction as a whole, while the conversation game is responsiblefor the harmonious local development of the dialogue. The global structuredetermines the flow of conversation, in particular, the way in which thedifferent phases of the conversation are connected sequentially, where asequence is a block of exchanges tied together by strong semantic andpragmatic coherence. Thus, the primary ability related to this level is topicmanagement, that is compliance with discourse topics. On the other hand, thelocal structure regards the alternation of turns, each of which is a sequence ofspeech acts uttered by the same actor, and also manages the relationshipbetween speech acts within the same turn; a turn may thus be composed ofmore than one speech, the set of these acts being characterized by coherence.


At this level the primary ability is thus turn-taking. The evaluation of theseabilities is included in the battery.

In this paragraph we presented the theoretical basis of our instrument, andspecified the different cognitive processes underlying the various investigatedphenomena. In the following two paragraphs we describe the scales that makeup the battery, the types of items assessed, the administration procedure and thescoring system.

3. Battery

The battery includes 5 scales (see below) and is composed of 180 items:6 72are based on the examiner’s prompts, and 108 on videotaped scenes eachlasting 20~25 seconds. Administration of the full battery takes about one and ahalf hours. However, the battery is modular, and it is possible to administereach scale separately to facilitate clinical usage. Appendix 1 shows the numberand type of items for each investigated phenomenon; Appendices 2 and 3report the dimensions to be evaluated in scoring; Appendix 4 contains someexamples of battery items, a patient’s responses, and a judge’s ratings.

The battery includes the following five scales:

1. Linguistic scale2. Extralinguistic scale3. Paralinguistic scale4. Context scale5. Conversation scale

1. Linguistic scale. This assesses the comprehension and production of


6 Some items in the battery were created in two different forms in order to makethem as ecological as possible for both children and adults. For example, we havemodified the situations proposed in the context scale (i.e., talking with a teacher forchildren and talking with a notary for adults) and some contents of the basic speechacts (candies/cigarettes; school/hospital; parents/consorts). In this way, we createdtwo different but still similar versions of the battery, so that it can be administeredboth in the developmental stage and in adulthood.

communication acts expressed primarily through the linguistic modality.7 Itconsists of:

— Basic communication acts (assertions, questions, requests,commands)

— Standard communication acts (simple and complex)— Non-standard communication acts (simple and complex irony;

simple and complex deceits)

2. Extralinguistic scale. This assesses the comprehension and production ofcommunication acts expressed through the extralinguistic modality only. Itincludes the same communication acts investigated in the linguistic scale.

We use the following tasks to assess the comprehension of linguistic andextralinguistic acts:

— Basic communication acts. In the linguistic scale, the examiner asksthe subject to evaluate the truthfulness of assertions, to answer simplequestions, to perform actions on request, to execute orders. In theextralinguistic scale, the examiner shows the subject short videos wherean actor makes an assertion, asks a question, makes a request or issues acommand through the use of gestures. The subject has to understand theact produced by the actor.

— Standard and non-standard communication acts. The examinershows the subject short videos where two agents are engaged in acommunicative interaction: the actor asks her partner a question and thepartner replies. The subject has to understand the communication actproduced by the partner. In the linguistic scale the agents communicateverbally; in the extralinguistic scale the agents communicate throughgestures only.


7 It has to be noted that the Linguistic scale is actually an ecological scale, because itextends beyond language to also encompass extralinguistic and paralinguisticelements. However, the linguistic modality is here the primary source of information.Thus, for the sake of simplicity, we decided to call it Linguistic scale.

We use the following tasks to assess the production of linguistic andextralinguistic acts:

— Basic communication acts. The examiner asks the subject toproduce assertions, questions, requests and commands; the examinerprovides the semantic content of the requested act. For example, theexaminer asks the subject “Tell me that you are cold”, or “Order me to bequite”. In the linguistic scale the subject has to produce linguistic acts; inthe extralinguistic scale the subject has to produce gestural acts.

— Standard and non-standard communication acts. The examinershows the subject short videos where two agents are engaged in acommunicative interaction: the actor asks her partner a question. Thesubject has to produce a communication act in reply. In the linguistic scalethe question asked by the actor is linguistic and the subject has to replyverbally. In the extralinguistic scale the question asked by the actor isgestural and the subject has to reply through gestures.

3. Paralinguistic scale. This assesses the comprehension and production ofthose aspects that generally accompany a communication act, such asgesticulation, facial expressions, prosody. The paralinguistic scale comprises:

— Basic communication acts— Communication acts expressing an emotion— Acts characterized by a paralinguistic contradiction (assessed only

in comprehension).

Basic communication acts are assertions, questions, requests and commands.Communication acts expressing an emotion include expressions of basic

emotions, such as anger, sadness, happiness, fear.Acts characterized by a paralinguistic contradiction are acts in which the

expressed content is in contrast with the paralinguistic indicators utilized in itsproduction. For example, saying “I like it very much” while one’s voice andattitude reveal that one doesn’t like it at all.

We use the following tasks to assess the comprehension of paralinguisticaspects.


— Basic communication acts. The examiner shows the subject shortvideos where an actor, speaking an invented language, makes anassertion, asks a question, makes a request or gives a command. Thesubject has to understand the type of act produced by the actor, throughthe paralinguistic indicators. The examiner verbally provides four possibleanswers, only one of which is correct.

— Communication acts expressing an emotion. The examiner showsthe subject short videos where an actor, speaking an invented language,expresses a basic emotion. The subject has to understand the emotion,through the paralinguistic indicators. The examiner provides four possibleanswers, only one of which is correct.

— Acts characterized by a paralinguistic contradiction. The examinershows the subject short videos where two agents are engaged in acommunicative interaction: the actor verbally expresses something that isin contrast with the paralinguistic indicators. The subject has tounderstand the actor’s mental state, detectable through the paralinguisticindicators.

We use the following tasks to assess the production of paralinguistic aspects.

— Basic communication acts. The examiner asks the subject toproduce assertions, questions, requests and commands, paying specialattention to the paralinguistic indicators; the examiner provides thesemantic content of the requested act. For example, the examiner tells thesubject to “Ask me whether it is sunny today” or “Tell me that it is sunnytoday”.

— Communication acts expressing an emotion. The examiner asks thesubject to produce communication acts colored by a specific emotion ormood; the examiner provides the semantic content of the requested actand the emotion with which it has to be expressed. For example, theexaminer asks the subject to “Tell me that you have received a letter. Tellme that in an happy way”.


4. Context scale. This assesses:

— The adequacy/inadequacy of a communication act with respect todiscourse norms (assessed only in comprehension)

— The adequacy/inadequacy of a communication act with respect tosocial norms: the ability to recognize whether and why a communicationact is appropriate with respect to a given context or situation(comprehension); the ability to produce communication acts which areappropriate in a given context or situation, according to rules of formalityand informality (production).

We use the following tasks to assess the comprehension of discourse andsocial norms:

— Discourse norms.8 The examiner shows the subject short videoswhere two agents are engaged in a communicative interaction: the actorasks her partner a question; the partner replies either according to thenorms of discourse or giving a generic, false, irrelevant or ambiguousanswer. The subject has to detect and explain the adequacy/inadequacy ofthe partner’s reply. For example, in an item representing inadequacy withrespect to the Gricean maxim of quantity, the actor asks “Where are yougoing precisely?” and the partner replies “I’m going out”.

— Social norms. The examiner shows the subject short videos wheretwo agents are engaged in a communicative interaction: the actor asks herpartner a question; the partner replies either according to the norms ofsocial appropriateness or in a manner which is not appropriate in the givensocial context. For example, the actor asks “Could you lend me yourpen?” and the partner replies in a very impolite way “I don’t want to bedisturbed!”


8 Here ‘discourse norms’ coincide with Grice’s maxims; indeed, Gricean maximsestablish the most important norms of discourse as they serve as rules for a rationaland effective communication (see for example Brown & Yule, 1983).

We use the following tasks to assess the production of communication actsin accordance with the norms of social appropriateness.

— Social norms. The examiner asks the subject to producecommunication acts requiring different levels of formality/informality; theexaminer provides the semantic content of the requested act.

5. Conversation scale. This assesses the ability to appropriately participate ina conversation, complying with the topics of the discourse and turn-taking.Examiner and subject are engaged in a conversation, where the examinerintroduces two topics, for a total duration of 4-6 minutes.

4. Scoring

Scores are kept on specific score sheets, while watching the subject’s video-recorded performance. Performance is evaluated for each item on the basis of aseries of dimensions, derived by the Cognitive Pragmatics theory, underlyingthe investigated communicative phenomena. Appendices 2 and 3 show thedimensions assessed in the linguistic, extralinguistic, paralinguistic and contextscales, in comprehension and production respectively. On such scales,dimensions can be seen as the steps necessary to comprehend or produce therelative communicative phenomena. Thus, the more complex the pragmaticphenomena, the more steps it will comprise. Dimensions for the conversationscale are instead independent from one another as they assess different anduncorrelated aspects of the ability to entertain a conversation, and thus theywill be treated separately. The reader may refer to Appendix 4 for clarifyingexamples on how to rate performance on the various dimensions.

Dimensions are useful for different purposes. First, as they are provideddirectly by the Cognitive Pragmatics theory, they give the battery atheoretically grounded basis from which specific and well-defined assessmentcriteria have been derived. Secondly, dimensions guide the rater in accuratescoring: each dimension corresponds to a specific question that the examinerasks the patient during administration (see Appendix 4): based on the patient’sanswers to each of these questions, the rater can ascertain whether the patienthas passed the corresponding dimension. This helps to overcome some of the


uncertainty that raters may experience when evaluating patients’ performancefor each item on a single dichotomic level, which is often difficult to achieve.Instead, breaking down a pragmatic phenomenon and evaluating its variousunderlying dimensions is a way of making sure that scores are attributed moreaccurately and objectively. Thirdly, the use of dimensions enables two differentdata analysis procedures to be performed. (A) In a quantitative data analysis,patients get 1 mark for each item where all dimensions have been passed, and 0marks for each item where not all dimensions have been passed. This type ofanalysis reveals the categories of pragmatic phenomena in which the patient isimpaired. (B) A qualitative data analysis can also be computed considering atwhich level, i.e. which step in the inferential chain, the patient has difficulty.

4.1 ComprehensionThe dimensions in the comprehension of communication acts are listed

below (Appendices 2 and 4). Expressed Content. The subject passes the dimension if he has recognized

what the actor9 expressed, i.e. the content of the expression act through whichthe actor grasped her partner’s attention. Simply repeating what the actor said(echo) is not sufficient to demonstrate comprehension of the expressed content;at least a paraphrase is necessary to prove effective comprehension. Forexample, supposing the actor says “It wasn’t my fault” in order to avoid apunishment (see example 3 in Appendix 4); the examiner asks the subject“What did the actor mean?”: if the subject answers ‘It wasn’t my fault’ (echo),this is not sufficient to establish he has actually recognized the expressedcontent; thus an evaluation must be made on the basis of the following in-depthquestion. In contrast, if the subject answers ‘That it wasn’t his fault’(paraphrase), he demonstrates that he has recognized the expressed content andthus passes the dimension. On the extralinguistic scale, simple repetition refersboth to the repetition of the same gesture performed by the actor and to thelinguistic echo. For example, supposing the actor asks “Will I see you later?”and the partner performs the “OK” gesture in reply: if the subject simply


9 Even though, technically, it is always the partner’s communication act that has to beunderstood, we use ‘actor’ for the sake of simplicity.

repeats the same gesture or orally says “OK” these are considered mererepetitions; by paraphrasing a gesture we refer to a minimal explanation of thegesture, such as “Yes, I’ll see you later”.

Speaker’s meaning. The subject passes the dimension if he has understoodwhat the utterance/gesture implies or presupposes.

Violation of Cooperation (truthfulness). The subject passes the dimension ifhe has understood that what the actor says is not true/not serious or, in the caseof some ironies, that the actor is communicating more than what she says.

Purpose of Violation. The subject passes the dimension if he has understoodthe reason why the actor produced her act, for example if he understands thatthe actor expressed something false in order to hide her guilt (deceit) or as ajoke (irony).

Expressive Modality. The subject passes the dimension if he has understoodthe type of communicated act or emotion. As the actor communicates in aninvented language, the recognition of the communication act necessarily passesthrough the correct interpretation of the intonation/mimic with which the act isexpressed.

Norm In/adequacy. The subject passes the dimension if he has recognizedthat there is something inappropriate in the proposed interaction. In contrast, oncontrol scenes for this dimension, the subject passes the dimension if he hasrecognized that everything is appropriate in the interaction.

Reason for Inadequacy. The subject passes the dimension if he hasunderstood what is inappropriate in the interaction. In particular, for ‘socialnorm’ type items, the subject should have understood that the inadequacy isreferred to the external context/situation, i.e. he must mention elements ofimpoliteness or of lack of respect for the actor’s social status; for ‘discoursenorm’ type items, the subject should have understood that the inadequacy isreferred to the norms regulating good communicative exchange, that areviolated when the information is too generic, false, not relevant, or ambiguous.In cases when the subject considers as appropriate a communication actviolating discourse norms of good communication, but provides a satisfactoryjustification, for example he justifies information that is too generic as the actornot wanting to say more than what she said, the subject passes both the NormIn/adequacy and Reason for In/adequacy dimensions.


4.2 ProductionThe dimensions for the production of communication acts are listed below

(see Appendices 3 and 4). Expressed Content. The subject passes the dimension if he produces a

communication act which is congruent with respect to the test question. Theact produced must be an utterance in the linguistic scale, and a gesture in theextralinguistic scale.

Expressive modality. The subject passes the dimension if the paralinguisticindicators are appropriate for the type of communication act. In particular,intonation and mimics must be appropriate with respect to the type ofcommunicated act or emotion.

Speaker’s meaning. The subject passes the dimension if he produces acommunication act which is plausible with respect to the communicativecontext, and if there is a logical connection between the patient’s answer andthe context shared by the participants in the interaction. In other words, the actmust be unambiguous and has to be easily understood by the interlocutor, i.e.the rater can find an answer to the test question in the act. For example,supposing that, in reply to the question ‘What would you like to do thisafternoon?’ the subject answers ‘I’d like to go to Mars’: this answer is consistentwith the question (expressed content), since indicating a place where one wouldlike to go is consistent with a question about what one would like to do, but it isunclear and the interlocutor cannot understand what the subject intended tocommunicate; thus the subject passes the expressed content dimension, but notthe speaker’s meaning dimension. In contrast, ‘It’s a long time since I’ve beento the cinema’ is a complex communication act which is both consistent andplausible. In the extralinguistic scale, sharedeness also concerns, for example,the amplitude of the gesture: the performed gesture has to be sufficiently ampleand clear so that the interlocutor comprehends its meaning.

Violation of Cooperation and Purpose of Violation. In assessing theproduction of communication acts, the Violation of Cooperation (truthfulness)and the Purpose of Violation (deceiving or ironic) are considered conjointly.The subject passes the dimension if he produces a communication act fulfillingthe requested goals for the communicative phenomenon in question. In thecase of deceit, the subject has to say (in the linguistic scale) or communicatewith a gesture (in the extralinguistic scale) something that is not true, with the


purpose of hiding his guilt/ deceit. In the case of irony, the subject has to say orcommunicate something with the aim of joking or making fun.

Norms In/adequacy. The subject passes the dimension if he produces acommunication act appropriate to the context or situation. In particular theproduction has to respect the characteristics of formality and informalityrequired by the context/situation; the intonation and gestures must respect thetype of act and the social setting.

In the linguistic, extralinguistic, paralinguistic and context scales,dimensions occupy hierarchical positions: passing one dimension presupposeshaving passed all the previous dimensions. This is based on the assumptionthat the comprehension and production of a given communication act require aseries of sequential inferential steps (see page 9 for a description of thelogically-connected steps in the comprehension of a communication act); forexample, comprehending the speaker’s meaning implies having previouslyrecognized the expressed content. Thus, if the subject does not understand theexpressed content, the rater also considers all the following dimensions as notpassed; on the other hand, if the subject immediately gives an answer thatdemonstrates an understanding of the speaker’s meaning, then the rater alsoconsiders the expressed content as passed.

4.3 Conversation ScaleThe dimensions for the conversation scale are listed below. Topic management

— Topic maintenance. The subject gets 1 mark if he respects theproposed topic and maintains the thread of the speech.

— Topic introduction/initiation. The subject gets 1 mark if heintroduces new themes or hints for widening the conversation.

— Topic shift. The subject gets 1 mark if he manages to follow thetopics of conversation without perseverations.

Turn taking— Taking one’s turn. The subject gets 1 mark if he intervenes in the

silent pauses left by the examiner and s/he does not impose on theexaminer’s voice.

— Allowing the other person to have his turn. The subject gets 1 markif he allows the examiner time to speak.


— Reference to interlocutor. The subject gets 1 mark if he considersthe contents expressed by the examiner, using lexical and cohesivedevices. For example, in order to express a contrary opinion, if he startswith ‘In contrast, I think …’

5. Methods and Results

5.1 Sample and administration procedures ABaCo is designed to have strong ceiling effects and low variability within

adult subjects with normal communicative abilities. A pilot study on 50 normaladults confirmed this assumption: subjects’ mean performance ranged from82% of correct responses in the comprehension part of the Paralinguistic scaleto 97% in the comprehension part of the Linguistic scale, with standarddeviations below 0.97. For this reason, measures of validity/reliability based oncorrelations computed on adult subjects are inappropriate. Thus, we decided totest both construct validity and instrument reliability on a sample of normallydeveloping children (5 to 8;6 years of age), expecting a greater variability ofscores. In particular, we expected to find a trend of improved performance forall subscales with the increase in age. Indeed, developmental progressionswithin such age range have been demonstrated in the comprehension of thestandard (Letts & Leinonen, 2001) and the non-standard meaning (Spector,1996), of paralinguistic aspects such as prosody (Cohen et al., 1990) and facialexpressions (Herba & Phillips, 2004), as well as in the detection of discourseimplicatures (Noveck, 2001).

The battery was administered10 to a total of 390 healthy children, equallydivided into three age groups: 130 children ranging from 5 to 5;6 (mean age: 5years and 3 months; standard deviation: 2.2 months), 130 children rangingfrom 6;6 to 7 (mean age: 6 years and 8 months; standard deviation: 2.2months) and 130 children ranging from 8 to 8;6 (mean age: 8 years and 2months; standard deviation: 2.1 months).11 Each age group included an equal


10 The use of the battery with both normal and clinical populations was approved bythe Ethical Committee of the Department of Psychology, University of Turin.11 Each group was separated by 6 months of age from each other in order to avoid“almost-overlapping” age ranges.

number of males and females. Children were recruited from pre-schools andprimary schools in Turin (Italy) and the city’s suburbs. All were Italian nativespeakers. The children completed the battery tasks individually with one of theauthors or a research assistant in a quiet room at school. Parental consent wasobtained. Parents also returned a questionnaire aimed at determining bothsocio-economic conditions and any physical or mental problems affecting thechild. The children came from families ranging from the working to middleand upper classes; children with physical disabilities and neurological/psychiatric disorders were excluded from the study.

Subjects were video-recorded during administration of the battery, to allowpost-test scoring. Participants’ responses were then rated by two independentjudges, who were not involved in the administration of the battery and wereblind to the aims of the research.

Each standard and non-standard scene was recorded in two versions, onesimple and one complex. Thus we devised two experimental batteries, A andB. Each battery contains only one version for each simple and complex scene;the total number of simple and complex scenes is balanced in batteries A andB. In each battery the scenes are represented in a different balanced order. Halfof the participants dealt with battery A, while the other half dealt with batteryB. Each subject was randomly assigned to battery A or B.

5.2 Behavioral resultsIn table 3 we report the mean percentages and standard deviations of correct


Table 3. Participants’ performance on the Battery

Age groupSCALE

5-5;6 yr 6;6-7 yr 8-8;6 yrOverall

Linguistic .70 (.11) .78 (.09) .81 (.08) .76 (.11)Extralinguistic .52 (.16) .67 (.12) .72 (.08) .64 (.15)Paralinguistic .53 (.18) .71 (.16) .80 (.11) .68 (.19)Context .47 (.2) .58 (.17) .67 (.15) .58 (.19)Conversation .94 (.13) .96 (.18) .97 (.12) .96 (.22)

Mean scores and standard deviations of correct responses overall for children and forsingle age groups are reported for each scale.

responses on each scale. For all scales, we found a significant correlation withage in the total sample of children (Pearson’s correlation: .41 < r < .59; p <.0001), indicating that performance improves with the increase in age. Only inthe conversation scale the correlation was not significant at statistical level,although the data were in the expected direction.

5.3 Validation of the batterySome questions must be answered when using a new clinical tool. The first

main question concerns the reliability of the instrument. In particular:

(a) Is each scale/subscale of the Battery composed of congruent items,i.e. items that are intrinsically related to one another? This questionconcerns internal consistency and was answered by calculating thecohesion within each subscale (Cronbach alpha).

(b) Is the scoring system sufficiently clear and objective to be used byany trained examiner/rater? This question concerns inter-rater reliabilityand a measure of agreement between the ratings given by twoindependent judges (Cohen’s kappa) was computed to satisfy such a goal.

The second main question concerns the validity of the instrument. Inparticular:

(c) At item level, do the test questions match the test objectives, i.e.does their content precisely address the subject area they are intended toassess? (b) Are the items appropriate for the age group of the subjects theinstrument is intended for? In the case of our battery, are the contents ofthe items suitable for both developing children and adult subjects? Thesetwo questions concern content validity, and were answered through itemevaluation by independent pragmatic experts.

(d) At a more general level, is the instrument actually measuring what itis assumed to measure, i.e. are the five scales of the battery referable topragmatic abilities? This question concerns the construct validity, and wasdealt with by computing a factor analysis.

In the present paper we intend to test the validity of the battery’s contents


and construction. The validation of the battery on clinical populations, e.g. thesensitivity of the instrument in differentiating various profiles of impairment indistinct brain pathologies or the external validity measured by the correlationwith other assessment instruments, is beyond the scope of this work.

Note that the current version of the battery actually derives from a series ofprevious attempts: a number of items present in previous versions have beeneliminated and replaced in view of normal adults’ poor performance (≤ 80% ofcorrect responses) or misunderstanding, low internal consistency (α ≤ .50), lowinter-rater reliability (k ≤ .50), or the experts’ negative evaluations (≤ 2 on the 5point Likert-scale). All the behavioral and validation results reported in thispaper clearly only refer to the administration of the current version of theBattery as described above.

5.3.1 Reliability measuresTwo measures of reliability were considered: internal consistency and inter-

rater reliability. In computing such measures, we again considered‘comprehension’ and ‘production’ separately as both item similarity and raters’agreement are required within each subscale: we thus used a total of 9variables. Statistics were calculated using the SAS statistical package.

Internal consistency measures the correlation among the items belonging toa subset (e.g. a subscale); it was assessed using Cronbach’s alpha (Cronbach,1951). Given the high concordance between the two judges’ ratings (see inter-rater reliability in the next paragraph), one rating was randomly selected tocompute internal consistency estimates. The value of Cronbach α can varyfrom 0 to 1; a satisfactory level of consistency is indicated by values higherthan 0.5 (De Vellis, 1991) or 0.6 (Nunnally, 1978). Except for the productionitems of the Context scale, which have an α = .52, the internal consistency ofthe scales that make up the ABaCo ranges from α = .626 to α = .91. As can beseen from Table 3, most of the subscales show a good to perfect internalsimilarity (> .7).

Inter-rater reliability (IRR) measures agreement, indicating how well theratings given by the two independent judges agreed; it was assessed usingCohen’s kappa (Cohen, 1960). The value of Cohen’s k can vary from ≤ 0 (noagreement among the raters) to 1 (perfect agreement). In table 3 the mean kvalue for each subscale is reported. Following the criteria defined by Landis


and Koch (1977), inter-rater correlations led to an excellent agreement (k >.75) within each sub-scale, ranging from k = .76 to k = .96.

5.3.2 Content validityVarious methods were used to evaluate content validity. First, four experts in pragmatic language were recruited to assess whether

the various parts of the battery measure the pragmatic abilities they areintended to address. The experts were given an electronic form to fill inindividually, in which they were required to evaluate each item in the battery.For each item, the experts were presented with the item, including the teststimulus (videotaped scene or examiner’s prompt) and test questions; theywere given a written statement claiming the capability of the item to measurethe target domain (e.g. for items in the Linguistic scale — comprehension, thestatement was: “The following items measure the ability to comprehendlinguistic acts”); and they were asked to rate the statement on a five-pointLikert-type scale (strongly disagree; disagree; neither agree or disagree; agree;strongly agree) (Likert, 1932). The items were grouped in nine sets,corresponding to the subscales12 that make up the battery, and, as for eachsubscale, the appropriate statement was presented. Scoring was computed by


Table 4. shows the results of reliability measures.

INTERNAL CONSISTENCY INTER-RATER RELIABILITYSCALE

(Cronbach’s alpha) (Cohen’s kappa)

Linguistic — Comprehension α = .63 k = .87Linguistic — Production α = .85 k = .92Extralinguistic — Comprehension α = .76 k = .88Extralinguistic — Production α = .90 k = .92Paralinguistic — Comprehension α = .62 k = .96Paralinguistic— Production α = .91 k = .94Context — Comprehension α = .84 k = .91Context — Production α = .52 k = .77Conversation α = .67 k = .76

Internal consistency measures the correlation among the items belonging to each scale.Inter-rater reliability indicates how well the ratings given by the two independent judgesagreed.

attributing 1 to 5 marks for each item (from 1 for strongly disagree to 5 forstrongly agree); item scores were averaged, obtaining a score for each subscale.Mean values across experts, and relative standard deviations, are reported inTable 4 (Appropriateness of items): all values are > 4, showing that the expertsjudged the items to be appropriate.

Secondly, in order to assess whether the items were appropriate for thedevelopmental age as well as for adult subjects, four developmentalpsychologists and four adult neuropsychologists were recruited. Following thesame procedure used with the pragmatic experts, the developmentalpsychologists had to rate whether each item was suitable for children aged 5 to8;6 years, while the neuropsychologists had to rate whether each item wassuitable for adult brain injured patients, with regard to wording and thesituation that was depicted. Mean values across experts, and relative standarddeviations, are reported in Table 4 (Suitability of items), for children and foradults separately: all values are > 4, showing that the experts judged the items


Table 5. summarizes the results supporting content validity.

Appropriateness of Suitability of items Suitability of itemsSCALE items for children for adults

Mean SD Mean SD Mean SD

Linguistic —Comprehension 4.75 .30 4.73 .19 4.59 .52Linguistic —Production 4.75 .30 4.76 .22 4.51 .69Extralinguistic — Comprehension 4.69 .38 4.74 .20 4.52 .36Extralinguistic —Production 4.77 .29 4.78 .20 4.52 .62Paralinguistic — Comprehension 4.71 .32 4.80 .22 4.72 .31Paralinguistic —Production 4.83 .28 4.79 .28 4.54 .75Context —Comprehension 4.62 .52 4.66 .23 4.62 .49Context —Production 4.75 .29 4.87 .25 4.12 .63Conversation 4.87 .14 4.94 .12 4.62 .75

Item evaluation by experts, computed on 5 point Likert scales. Mean (across experts anditems) and standard deviations (SD) are reported for each subscale.

12 As each scale (except Conversation) is divided into ‘comprehension’ and‘production’, we have a total of 9 subscales.

to be suitable.

5.3.3 Construct validityConstruct validity concerns the capability of the battery to measure

pragmatic abilities and was assessed by factor analysis. The underlyingassumption is that all items relate to a single factor. Indeed, given that all scalesmeasure different aspects of the same phenomenon, i.e. pragmatic competence,we expected to obtain a unitary factor structure able to explain a significantamount of the variance. In order to avoid the extraction of meaningless factors,e.g. factors emerging from strong correlations of a limited number of items,before performing the factor analysis we assembled the items according totheir theoretically assumed homogeneity, i.e. on their belonging to each of the5 scales that make up the battery. In other words, factor analysis was computedon five variables corresponding to the five pragmatic scales: the participants’scores obtained on each scale were used as input variables, referable to thehypothesized? unique factor. An exploratory factor analysis using principalcomponent analysis as the extraction method was performed (SAS statisticalpackage). The results confirmed our hypothesis, showing a one-factor solutionthat accounted for 63% of the variance (eigenvalue of the first factor = 3.14;eigenvalues of the other factors < 1). The emergence of a one-factor solutionindicates that the scales making up the battery overlap in their underlyingdimension. This suggests that a more general ‘communication ability’ ispresent in each scale, going beyond language or gestures per se. Suchpragmatic component is the underlying basis for the entire structure of thebattery.

6. Conclusions

In this paper we present ABaCo as an integrated clinical tool for assessingcommunication. ABaCo is an instrument for the assessment of pragmatics and,as such, it can be used to define a profile of the patient’s impaired vs. residualcommunicative abilities. We have attempted to create an instrument with twofundamental characteristics: a solid theoretical framework, and psychometricvalidity.

Indeed, each investigated phenomena is clearly derived from the theory, and


is investigated according to the underlying cognitive processes assumed by thetheory. Besides, factor analysis indicated that one common ability underlinedthe entire structure of the battery, suggesting that the theoretical construct iswell-conceptualized and operatively well-defined. It can be argued that theemergence of a unique latent factor lessens the importance of using fivedifferent scales; however, with respect to our objectives, that is not the case:although the five scales are supported by an underlying dimension, i.e.pragmatic ability, they still measure different communicative modalities.Testing the various means through which communication can be achieved isnecessary to detect modality-specific pragmatic deficits, and also to allow forthe administration of parts of the Battery to patients who are completelycompromised in one specific modality, such as for example some aphasics whoare unable to deal with the linguistic scale.

As far as the other psychometric measures are concerned, the experts’judgments provided an independent validation of the content of the instrumentand of its suitability for both children and adults. Besides, the battery hasshown to be a reliable tool for the evaluation of communicative abilities: thescales comprise coherent items and the scoring system proved correctly usableby different raters. All these features, plus the possibility of investigating awide range of communicative phenomena according to the patient’s needs andpossibilities, make ABaCo a promising clinical tool.

Work is currently underway to split the battery according to statisticalcriteria, in order to obtain two parallel versions; the aim is to make theinstrument suitable for testing patients at different times and to obtain anefficient measure of the efficacy of rehabilitation. Validity studies on patientswith different neurological pathologies are also in progress; in such studies,correlations with basic cognitive processes involved in communication, such astheory of mind and executive functions, are also being investigated (e.g.Angeleri et. al. (2008), reports data on traumatic brain injured patients). Finally,data are being collected for a normative study on the Italian population, so thatcut-off scores can be defined.

Acknowledgments

We thank Gianluca Bo, Marco Del Giudice and Marco Zuffranieri for


statistical assistance, and Ivan Enrici for participating in the initial stages of thisproject, and all the participants in this study. This research has been supportedby Fondazione Cassa di Risparmio di Torino, CRT Foundation, Italy.

References

Airenti, G., Bara, B.G., & Colombetti, M. (1993a). Conversation and behaviorgames in the pragmatics of dialogue. Cognitive Science, 17, 197-256.

Airenti, G., Bara, B.G., & Colombetti, M. (1993b). Failures, exploitations anddeceits in communication. Journal of Pragmatics, 20, 303-326.

Angeleri, R., Bosco, F.M., Zettin, M., Sacco, K., Colle, L., Bara, B.G. (2008).Communicative impairment in traumatic brain injury: A complete pragmaticassessment. Brain and Language, 107, 229-245.

Austin, J.L. (1962). How to Do Things with Words. Oxford: Clarendon Press.Bara, B., Bucciarelli, M., & Colle, L. (2001). Communicative abilities in autism:

Evidence for attentional deficits. Brain and Language, 77, 216-240.Bara, B.G. (2008). Cognitive Pragmatics. Cambridge, MA: MIT Press.Bara, B.G., Bosco, F.M., & Bucciarelli, M. (1999a). Developmental pragmatics in

normal and abnormal children. Brain and Language, 68, 507-528.Bara, B.G., Bosco, F.M., & Bucciarelli, M. (1999b). Simple and complex speech

acts: What makes the difference within a developmental perspective. InProceedings of the XXI Annual Conference of the Cognitive Science Society, M.Hahn & S.C. Stoness, Mahwah, NJ, pp. 55-60.

Bara, B.G., Bucciarelli, M., & Geminiani, G. (2000). Development and decay ofextra-linguistic communication. Brain and Cognition, 43, 21-27.

Bara, B.G., Cutica, I., & Tirassa, M. (2001). Neuropragmatics: Extralinguisticcommunication after closed head injury. Brain and Language, 77, 72-94.

Bara, B.G., & Tirassa, M. (1999). A mentalist framework for linguistic andextralinguistic communication. In Proceedings of the 3rd European Conferenceon Cognitive Science, S. Bagnara, Roma, pp. 285-29.

Bara, B.G., & Tirassa, M. (2000). Neuropragmatics: Brain and communication.Brain and Language, 71, 10-14.

Bara, B.G., Tirassa, M., & Zettin, M. (1997). Neuropragmatics: Neuropsychologicalconstraints on formal theories of dialogue. Brain and Language, 59, 7-49.

Becchio, B., Adenzato, M., & Bara, B.G. (2006). How brain understands intention.Different neural circuits identify the componential features of motor and priorintention. Consciousness and Cognition, 15, 64-74.

Bishop, D.V.M. (1998). Development of the children’s communication checklist


(CCC): A method for assessing qualitative aspects of communicativeimpairment in children. Journal of Child Psychology and Psychiatry, 39, 879-891.

Bosco F.M., Bucciarelli M. (2008). Simple and Complex Deceits and Ironies.Journal of Pragmatics, 40, 583-607.

Bosco, F.M., Bucciarelli, M., & Bara, B.G. (2004). Context categories inunderstanding communicative intentions. Journal of Pragmatics, 36, 436-488.

Bosco F.M., Bucciarelli M., & Bara B. G. (2006). Recognition and recovery ofcommunicative failures: a developmental perspective. Journal of Pragmatics,38, 1398-1429.

Brown G., & Yule G. (1983). Discourse Analysis: The sociolinguistic analysis ofnatural language. Cambridge, England, UK: Cambridge University Press.

Bryan, K. (1995). The right hemisphere language battery. London: Whurr.Bucciarelli, M., Colle, L., & Bara, B.G. (2003). How children comprehend speech

acts and communicative gestures. Journal of Pragmatics, 2, 207-241.Carlomagno, S. (1994). Pragmatic approaches to aphasia therapy. London: Whurr.Carlomagno, S., Blasi, V., Labruna, L., & Santoro, A. (2000). The role of

communication models in assessment and therapy of language disorders inaphasic adults. Neuropsychological Rehabilitation, 10, 337-363.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational andPsychological Measurement 20, 37-46.

Clark, H.H. (1996). Using language. Cambridge, England, UK: CambridgeUniversity Press.

Clark, H.H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process.Cognition, 22, 1-39.

Cohen, M., Prather, A., Town, P., & Hynd, G. (1990). Neurodevelopmentaldifferences in emotional prosody in normal children and children with left andright temporal lobe epilepsy. Brain and Language, 38, 122-134.

Cronbach, L.J. (1951). Coefficient alpha and the internal structure of tests.Psychometrika, 16, 297-334.

Cutica, I. (2005). Neuropsychological evidence for linguistic and extralinguisticpaths in communication. In Proceedings of the XXVII International Conferenceof Cognitive Science, Stresa, Italy.

Cutica, I., Bucciarelli, M., & Bara, B.G. (2006). Neuropragmatics: Extralinguisticpragmatic ability is better preserved in left-hemisphere-damaged patients than inright-hemisphere-damaged patients. Brain and Language, 98 , 12-25.

Davis, G.A., & Wilcox, M.J. (1985). Adult aphasia rehabilitation: Appliedpragmatics. Windsor, UK: NFER-Nelson.

De Vellis, R.F. (1991). Scale development: Theory and applications. Thousand


Oaks, CA: Sage.Emmorey, K., Grabowski, T., McCullough, S., Damasio, H., Ponto, L.L., Hichwa,

R.D., & Bellugi, U. (2003). Neural system underlying lexical retrieval for signlanguage. Neuropsychologia, 41, 85-95.

Frattali, C.M., Thompson, C.K., Holland, A.L., Wohl, C.B., & Ferketic, M.M.(1995). American speech-language hearing association functional assessmentof communication skills for adults. Rockville, MD: ASHA FulfillmentOperations.

Gardner, H., & Brownell, H.H. (1986). Right hemisphere communication battery.Boston: Psychology Service, VAMC.

Gibbs, R.W. (1994). The poetics of mind: Figurative though, language, andunderstanding. Cambridge, England, UK: Cambridge University Press.

Grice, H.P. (1975). Logic and Conversation. In P. Cole & J.L. Morgan (Eds.), Syntaxand Semantics (pp. 41-58). New York: Academic Press.

Grice, H.P. (1989). Studies in the ways of words. Cambridge: Harvard UniversityPress.

Hartley, L.L., & Jensen, P.J. (1991). Narrative and procedural discourse after closedhead injury. Brain Injury, 5, 267-285.

Hickok, G., Bellugi, U., & Klima, E.S. (1996). The neurobiology of sign languageand its implications for the neural basis of language. Nature, 381, 699-670.

Hinde R.A. (1972). Non-verbal Communication. Cambridge, UK: CambridgeUniversity Press.

Holland, A.L. (1980). Communicative abilities in daily living. Baltimore: UniversityPark Press.

Holland, A.L., Frattali, C., & Fromm, D. (1998). Communication activities of dailyliving (CADL-2). Austin, TX: Pro-Editor.

Hough, M.S. (1990). Narrative comprehension in adults with right and lefthemisphere brain-damage: Theme organization. Brain and Language, 38, 253-277.

Kasher, A. (1981). Minimal speakers and necessary speech acts. In F. Coulmas(Eds), Festschrift for Native Speaker (pp. 93-101). The Hague: Mouton.

Landis, J.R., & Koch, G.G. (1977). The Measurement of Observer Agreement forCategorial Data. Biometrics, 33, 159-174.

Letts, C., & Leinonen, E. (2001). Comprehension of inferential meaning inlanguage-impaired and language normal children. International Journal ofLanguage and Communication Disorders, 36, 307-328.

Likert, R. (1932). Techinque for the measurement of attitudes. Archives ofPschology, 140, 1-55.

McDonald, S. (1993). Pragmatic language loss following closed head injury:


Inability to meet the informational needs of the listener. Brain and Language,44, 28-46.

McDonald, S., Flanagan, S., Rollins, J., & Kinch, J. (2003). TASIT: A new clinicaltool for assessing social perception after traumatic brain injury. Journal of HeadTrauma Rehabilitation, 18, 219-238.

Noveck, I.A. (2001). When children are more logical than adults: Experimentalinvestigations of scalar implicature. Cognition, 78, 165-188.

Nunnaly, J. (1978). Psychometric theory. New York: McGraw-Hill. Penn, C. (1999). Pragmatic assessment and therapy for persons with brain damage:

What have clinicians gleaned in two decades? Brain and Language, 68, 535-552.

Penn, C. (1985). The profile of communicative appropriateness: A clinical tool forthe assessment of pragmatics. South African Journal of CommunicationDisorders, 32, 18-23.

Pickering M., & Garrod S. (2003). Toward a Mechanistic Psychology of Dialogue:Behavioral and Brain Sciences, 27, 169-226.

Prutting, C.A. (1982). Pragmatics as social competence. Journal of Speech andHearing Disorders, 47, 123-134.

Prutting, C.A., & Kirchner, D.M. (1987). A clinical appraisal of the pragmaticaspects of language. Journal of Speech and Hearing Disorders, 52, 105-119.

Rinaldi, M.C., Marangolo P., & Baldassarri, F. (2004). Metaphor comprehension inright brain-damaged patients with visuo-verbal and verbal material: adissociation (re)considered. Cortex, 40, 479-490.

Sacks, H., Schegloff, E.A., & Jefferson G. (1978). A simplest systematics for theorganization of turn taking for conversation. In J.N. Schenkein (Ed.), Studies inthe organization of conversational interaction (pp. 7-55). New York: AcademicPress.

Sarno, M.T. (1969). The functional communication profile. New York: Institute ofRehabilitation Medicine, NYU Medical Center.

Searle, J.R. (1969). Speech acts. Cambridge: Cambridge University Press.Spector, C.C. (1996). Children’s comprehension of idioms in the context of humor.

Language, Speech, and Hearing Services in Schools, 27, 307-313.Surian, L., Cohen, S.B., & der Lely, H.V. (1996). Are children with autism deaf to

Gricean maxims? Cognitive Neuropsychiatry, 1, 55-71.Tager-Flusberg, H., (2000). Language and understanding minds: connection in

autism. In S. Baron-Cohen, H. Tager-Flusberg, & D.J. Cohen (Eds.).Understanding other minds. Perspective from development cognitiveneuroscience (124-149). Oxford, NY: University Press.

Tirassa, M. (1999). Communicative competence and the architecture of the


mind/brain. Brain and Language, 68, 419-441.Walter, H., Adenzato, M., Ciaramidaro, A., Enrici, I., Pia, L., & Bara, B.G. (2004).

Understanding intentions in social interactions: The role of the anteriorparacingulate cortex. Journal of Cognitive Neuroscience, 16, 1854-1863.

Winner, E., Brownell, H., Happe, F., Blum, A., & Pincus, D. (1998). Distinguishinglies from jokes: Theory of mind deficits and discourse interpretation in righthemisphere brain-damaged patients. Brain and Language, 62, 89-106.

Zanini, S., & Bryan, K. (2003). La batteria del linguaggio dell’emisfero destro —BaLED. EMS, Bologna, Italy.

Zanini, S., Bryan, K., De Luca, G., & Bava, A. (2005). Italian Right hemispherelanguage battery: the normative study. Neurological Science, 26, 13-25.



Appendix 1. Number and type of items for each investigated phenomenon

NUMBER OF ITEMS TYPE OF ITEM NOTES

LINGUISTIC SCALEComprehensionBasicAssertion 4 Examiner’s prompt- Question 4 Examiner’s prompt- Request 4 Examiner’s prompt- Command 4 Examiner’s promptStandard simple 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery BStandard complex 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery BNon-Standard simple- Irony 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery B- Deceit 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery BNon-Standard complex- Irony 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery B- Deceit 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery BProductionBasic- Assertion 4 Examiner’s prompt- Question 4 Examiner’s prompt- Request 4 Examiner’s prompt- Command 4 Examiner’s promptStandard 4 Videotaped sceneNon-Standard- Irony 4 Videotaped scene- Deceit 4 Videotaped sceneEXTRALINGUISTIC SCALEComprehensionBasic- Assertion 4 Videotaped scene- Question 4 Videotaped scene- Request 4 Videotaped scene- Command 4 Videotaped sceneStandard simple 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery BStandard complex 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery BNon-Standard simple- Irony 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery B- Deceit 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery BNon-Standard complex- Irony 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery B- Deceit 4 Videotaped scene 2 Items in Battery A; 2 Items in Battery BProductionBasic- Assertion 4 Examiner’s prompt- Question 4 Examiner’s prompt


Appendix 1. Continued

- Request 4 Examiner’s prompt- Command 4 Examiner’s promptStandard 4 Videotaped sceneNon-Standard- Irony 4 Videotaped scene- Deceit 4 Videotaped scenePARALINGUISTIC SCALEComprehensionBasic 4 Videotaped scene 1 for each Basic Communication ActEmotion 4 Videotaped sceneContradiction 4 Videotaped sceneProductionBasic 4 Examiner’s prompt 1 for each Basic Communication ActEmotion 4 Examiner’s prompt Each requires the expression

of two emotionsCONTEXT SCALEComprehensionDiscourse norms 4 Videotaped scene 1 for each Gricean Maxim+2 control itemsSocial norms 4 Videotaped scene + 2 control itemsProductionSocial norms 4 Examiner’s prompt 2 prompts, each requires formal

and informal expressionsCONVERSATION SCALE 4 Examiner’s prompt

Assessment Battery for Communication — ABaCo 151A

ppen

dix

2.D

imen

sion

s fo

r ass

essi

ng th

e co

mpr

ehen

sion

of c

omm

unic

atio

n ac

ts

Scal

eL

ING

UIS

TIC

AN

DE

XT

RA

LIN

GU

IST

ICPA

RA

LIN

GU

IST

ICC

ON

TE

XT

SCA

LE

Com

mun

icat

ion

Act

Bas

icSt

anda

rdSt

anda

rdN

on-S

tand

ard

Non

-Sta

ndar

dB

asic

Em

otio

nC

ontra

dict

ion

Dis

cour

seSo

cial

sim

ple

com

plex

sim

ple

com

plex

norm

sno

rms

Dim

ensi

on

Exp

ress

ed C

onte

nt (E

C)

XX

XX

XX

Spea

ker’

s m

eani

ng (S

M)

XX

Vio

latio

n of

Coo

pera

tion

(VC

)X

XX

Purp

ose

of V

iola

tion

(PV

)X

XE

xpre

ssiv

e M

odal

ity (E

M)

XX

Nor

m In

/ade

quac

y (N

I)X

XR

easo

n fo

r Ina

dequ

acy

(RI)

XX

The

tabl

e re

pres

ents

the

dim

ensi

ons

inve

stig

ated

for e

ach

prag

mat

ic p

heno

men

a. E

ven

thou

gh e

ach

repo

rted

dim

ensi

on re

gard

s al

l typ

es o

fco

mm

unic

ativ

e ph

enom

ena,

for

eac

h ty

pe o

f ph

enom

enon

onl

y th

e di

men

sion

s w

hich

spe

cifi

cally

cha

ract

eriz

ed i

t w

ere

cons

ider

ed f

orev

alua

tion.

For

exa

mpl

e, c

ompr

ehen

ding

a s

tand

ard

sim

ple

act

only

inv

olve

s un

ders

tand

ing

the

expr

esse

d co

nten

t. O

n th

e ot

her

hand

,co

mpr

ehen

ding

a c

ompl

ex a

ct i

nvol

ves

unde

rsta

ndin

g th

e sp

eake

r’s

mea

ning

, as

thi

s go

es b

eyon

d th

at o

f th

e ex

pres

sed

cont

ent,

and

com

preh

endi

ng a

non

-sta

ndar

d ac

t inv

olve

s re

cogn

izin

g a

viol

atio

n of

the

com

mun

icat

ion

rule

s an

d th

e re

ason

for

suc

h vi

olat

ion.

The

‘X

’in

dica

tes

that

the

corr

espo

ndin

g di

men

sion

is e

valu

ated

for t

hat c

omm

unic

ativ

e ph

enom

enon

.


Appendix 3. Dimensions for assessing the production of communication acts

Scale LINGUISTIC AND EXTRALINGUISTIC PARALINGUISTIC CONTEXT SCALE

Communication Act Basic Standard Non-Standard Basic Emotion Social normsDimension

Expressed Content (EC) X X X X

Speaker’s meaning (SM) X X

Violation and Purpose (V) X

Expressive Modality (EM) X X

Norm In/adequacy (NI) X

The ‘X’ indicates that the corresponding dimension is evaluated for that communicative

phenomenon.


ppen

dix

4.E

xam

ples

of

batte

ry it

ems,

pat

ient

’s r

espo

nses

, and

judg

e’s

ratin

gs. A

n ex

ampl

e is

rep

orte

d fo

r ea

ch in

vest

igat

edph

enom

enon

VID

EO

TA

PED

SC

EN

ET

EST

QU

EST

ION

SPA

TIE

NT

’S R

ESP

ON

SE

SCO

RIN

G

On

each

Tot

al s

core

dim

ensi

onon

the

item

LIN

GU

IST

ICSC

AL

E

Com

preh

ensi

on

Bas

ic (Q

uest

ion)

----

--W

here

do

you

live?

In T

urin

EC

= y

es1

[1] S

tand

ard

sim

ple

Man

: “T

his

past

a is

ver

y go

od.

1. W

hat d

id th

e w

oman

say

?1.

Tha

t she

coo

ked

itE

C =

yes

1

Wom

an: “

I did

”ID

Q. W

ho c

ooke

d th

e pa

sta?

[2] S

tand

ard

com

plex

Girl

: “D

id y

ou g

o to

the

gym

?”1.

Wha

t did

the

boy

say?

1. H

e w

as ti

red

EC

= y

es1

Boy

: “I h

aven

’t fe

lt so

tire

d fo

r age

s!”

2. W

hat d

oes

it m

ean?

IDQ

. Did

the

boy

go to

the

gym

?2.

He

wen

t to

the

gym

SM =

yes

0

[3] N

on-S

tand

ard

sim

ple

The

chi

ld k

nock

s a

vase

ove

r.1.

Wha

t did

the

child

say

?1.

It w

as B

obi

EC

= y

es

(Dec

eit)

Mum

: “W

ho k

nock

ed th

e va

se o

ver?

”ID

Q. W

hat d

oes

it m

ean?

IDQ

. Bob

i bro

ke th

e va

seV

C =

no

Chi

ld: “

It w

as B

obi”

2. D

id th

e ch

ild te

ll th

e tru

th?

2. Y

es

PV =

no

3. W

hy d

id th

e ch

ild a

nsw

er li

ke th

at?

[4] N

on-S

tand

ard

com

plex

The

girl

is w

earin

g a

dres

s tha

t is t

oo ti

ght.

1. W

hat d

id th

e bo

y sa

y?1.

The

die

t is w

orki

ngE

C =

yes

0

(Iro

ny)

Girl

: “H

ow d

oes

it fit

me?

”2.

Wha

t doe

s it

mea

n?2.

The

die

t is

heal

thy

SM =

no

Boy

: “Y

our d

iet i

s w

orki

ng w

ell!”

3. D

id th

e bo

y m

ean

it se

rious

ly?

3. Y

esV

C =

no

4. W

hy d

id th

e bo

y an

swer

the

girl

4. B

ecau

se th

e gi

rl w

asPV

= n

o

like

that

?th

inne

r tha

n be

fore

154 Sacco K., Angeleri R., Bosco F.M., Colle L., Mate D., and Bara B.G.A

ppen

dix

4.C

ontin

ued

VID

EO

TA

PED

SC

EN

ET

EST

QU

EST

ION

SPA

TIE

NT

’S R

ESP

ON

SE

SCO

RIN

G

On

each

Tot

al s

core

dim

ensi

onon

the

item

[5] B

asic

(Que

stio

n)--

----

Ask

me

whe

ther

I’ve

got

chi

ldre

nD

o yo

u ha

ve a

ny k

ids?

EC

= y

es1

[6] S

tand

ard

Hus

band

and

wife

are

sitt

ing

on th

e so

fa.

1. W

hat c

ould

the

man

ans

wer

?I’

d lik

e to

go

for a

ride

EC

= y

es1

Wife

: “W

hat w

ould

you

like

to d

o SM

= y

es

this

afte

rnoo

n?”

[7] N

on-S

tand

ard

(Iro

ny)

Bro

ther

and

sis

ter a

re h

avin

g br

eakf

ast.

1. W

hat c

ould

the

girl

answ

er to

Y

ou’v

e al

read

y go

t the

jam

EC

= y

es0

He’

s pu

t his

elb

ow in

the

jam

.m

ake

fun

of th

e bo

y?SM

= y

es

Bro

ther

: “C

an y

ou p

ass

me

the

jam

, ple

ase?

”PV

= n

o

EX

TR

AL

ING

UIS

TIC

SCA

LE

Com

preh

ensi

on

[8] B

asic

(Com

man

d)Th

e guy

per

form

s a g

estu

re m

eani

ng “G

o ou

t”1.

Wha

t did

he

tell

you?

Go

out!

EC

= y

es1

[9] S

tand

ard

sim

ple

The

wife

has

got

a d

ish

of s

team

ing

soup

.1.

Wha

t did

the

man

say

?1.

Yes

EC

= y

es1

She

nods

to h

er h

usba

nd w

ith a

ges

ture

ID

Q. W

ill th

e m

an g

o to

eat

?ID

Q. Y

es

mea

ning

“A

re y

ou c

omin

g?”

The

hus

band

nod

s ye

s.

[10]

Sta

ndar

d co

mpl

exT

he b

oy p

erfo

rms

a ge

stur

e to

ask

the

girl

1. W

hat d

id th

e gi

rl sa

y?1.

I ca

n’t h

ave

a co

ffee

EC

= y

es1

“Do

you

wan

t som

e co

ffee

?”2.

Wha

t doe

s it

mea

n?2.

No,

she

has

to g

o ou

tSM

= y

es

The

girl

look

s at

her

wat

ch w

ith a

ges

ture

ID

Q. W

ill th

e gi

rl dr

ink

the

coff

ee?

mea

ning

“It’

s la

te”


ppen

dix

4.C

ontin

ued

VID

EO

TA

PED

SC

EN

ET

EST

QU

EST

ION

SPA

TIE

NT

’S R

ESP

ON

SE

SCO

RIN

G

On

each

Tot

al sc

ore

dim

ensi

onon

the

item

[11]

Non

-Sta

ndar

d si

mpl

eB

oy a

nd g

irl a

re ta

stin

g so

me

disg

ustin

g1.

Wha

t did

the

boy

say?

1. T

he so

up is

per

fect

, may

be

EC

= y

es0

(Iro

ny)

soup

.ID

Q. W

hat d

oes

it m

ean?

it’s k

ind

of th

ick

but i

t’s g

ood

VC

= n

o

The

boy

sm

acks

his

lips

with

a g

estu

re

2. D

id th

e bo

y m

ean

it se

rious

ly?

2. Y

esPV

= n

o

mea

ning

“It’

s ve

ry g

ood!

”3.

Why

did

the

boy

answ

er li

ke th

at?

3. J

ust t

o sa

y so

met

hing

[12]

Non

-Sta

ndar

d T

he b

oy p

erfo

rms

a ge

stur

e w

ith w

hich

he

1. W

hat d

id th

e gi

rl sa

y?1-

2. T

hat t

he c

andi

es w

ere

EC

= y

es0

com

plex

(Dec

eit)

asks

for s

ome

cand

ies.

2. W

hat d

oes

it m

ean?

not g

ood

to e

atSM

= y

es

The

girl

doe

sn’t

wan

t to

give

him

any

can

dy.3

. Did

the

girl

tell

the

truth

?3.

Yes

VC

= n

o

So, s

he lo

oks

at th

e ca

ndie

s w

ith a

dis

gust

ed 4

. Why

did

the

girl

answ

er li

ke th

at?

4. B

ecau

se th

ey w

ere

terri

ble

PV =

no

expr

essi

on w

hich

mea

ns “

The

y ar

e aw

ful!”

Prod

uctio

n

[13]

Bas

ic (C

omm

and)

----

---

Ord

er m

e to

be

quie

tTh

e pa

tient

mov

es h

er

EC

= y

es1

open

han

d up

and

dow

n to

mea

n “B

e qu

iet!”

[14]

Sta

ndar

d A

man

nee

ds h

elp

in th

e st

reet

. He

sees

W

hat g

estu

re c

an th

e m

an p

erfo

rm?

The

patie

nt li

fts h

er h

and

inE

C =

yes

1

a ca

r com

ing.

orde

r to

sto

p th

e ca

r

[15]

Non

-Sta

ndar

d Th

e bo

y th

row

s a d

ish

of v

eget

able

s in

the

bin.

The

boy

doe

sn’t

wan

t to

be s

cold

ed.

The

patie

nt tu

rns

her

inde

x E

C =

yes

1

(Dec

eit)

Mum

com

es in

and

per

form

s a g

estu

re m

eani

ng W

hat g

estu

re c

an h

e pe

rfor

m?

finge

r on

her

che

ek in

a

SM =

yes

“Hav

e yo

u al

read

y fin

ishe

d yo

ur v

eget

able

s?”

gest

ure

mea

ning

PV

= y

es

“Ver

y go

od!”

156 Sacco K., Angeleri R., Bosco F.M., Colle L., Mate D., and Bara B.G.A

ppen

dix

4.C

ontin

ued

VID

EO

TA

PED

SC

EN

ET

EST

QU

EST

ION

SPA

TIE

NT

’S R

ESP

ON

SE

SCO

RIN

G

On

each

Tot

al sc

ore

dim

ensi

onon

the

item

PAR

AL

ING

UIS

TIC

SCA

LE

Com

preh

ensi

on[1

6] B

asic

(Req

uest

)T

he m

an m

akes

a re

ques

t in

an in

vent

ed

The

man

I don

’t kn

ow

EM

= n

o0

lang

uage

- sai

d w

hat h

e th

inks

- mad

e a

requ

est (

targ

et)

- gav

e an

ord

er- t

old

a lie

-

[17]

Em

otio

nT

he w

oman

scre

ams a

nd g

estic

ulat

es a

ngril

yT

he w

oman

isSh

e’s a

ngry

EM

= y

es1

- ang

ry (t

arge

t)- s

ad- h

appy

- em

barr

asse

d

[18]

Con

tradi

ctio

nG

irl: “

Did

you

like

the

cake

?”1.

Wha

t did

the

boy

say?

1. I’

ll ha

ve o

ne m

ore!

EC

= n

o0

Boy

, with

dis

gust

ed e

xpre

ssio

n:

IDQ

. Wha

t doe

s it

mea

n?2.

Yes

VC

= n

o“Y

es, v

ery

good

”2.

Did

the

boy

like

the

cake

?ID

Q2.

How

did

you

und

erst

and

that

?

Prod

uctio

n[1

9] B

asic

(Req

uest

)--

----

-A

sk m

e to

giv

e yo

u a

pen

Cou

ld y

ou g

ive

me

a pe

n?E

M =

yes

1

[20]

Em

otio

n--

----

-A

sk m

e w

here

the

doct

or is

The

patie

nt p

erfo

rms

the

EM

= y

es1

- act

ing

sad

task

cor

rect

ly


ppen

dix

4.C

ontin

ued

VID

EO

TA

PED

SC

EN

ET

EST

QU

EST

ION

SPA

TIE

NT

’S R

ESP

ON

SE

SCO

RIN

G

On

each

Tot

al sc

ore

dim

ensio

non

the

item

CO

NT

EX

TSC

AL

E

Com

preh

ensi

on[2

1] D

isco

urse

nor

ms

Sist

er: “

Whe

re d

id y

ou p

ut m

y di

ary?

”1.

Is th

e an

swer

OK

?1.

No

NI =

yes

1B

roth

er, i

n fr

ont o

f a re

d ch

est o

f dra

wer

s:

2. W

hy?

2. T

here

are

lots

of d

raw

ers.

RI =

yes

“In

the

red

draw

er”

Whi

ch o

ne?

[22]

Soc

ial n

orm

sH

ead

offic

e: “

Mis

s, c

an y

ou ty

pe th

is le

tter

1. W

as th

e se

cret

ary

polit

e?1.

No

plea

se?

“2.

How

did

you

und

erst

and

that

?2.

She

sho

uld

say

som

ethi

ngN

I = y

es1

Secr

etar

y, a

ngril

y: “

I can

not d

o it

now

! lik

e “I

’m s

orry

, I c

an’t

doR

I = y

esI’

ve g

ot s

o m

uch

wor

k!”

that

now

”

Prod

uctio

n[2

3] S

ocia

l nor

ms

----

---

Imag

ine

you

are

late

for a

n ap

poin

tmen

t- I

’m s

orry

EC

= y

es0

- with

you

r law

yer

- I’m

sor

ryN

I = n

o- w

ith a

frie

nd o

f you

rsH

ow w

ould

you

apol

ogiz

e for

bei

ng la

te?

CO

NV

ER

SAT

ION

SCA

LE

——

-C

onve

rsat

ion

on s

pare

tim

eTh

e pa

tient

has

a c

onve

rsat

ion

TM

= 0

-3(s

ee te

xt fo

r det

aile

d sc

orin

g)w

ith th

e ex

amin

er (5

min

utes

)T

T =

0-3

The

fir

st s

cori

ng c

olum

n in

dica

tes

the

mar

ks f

or e

ach

dim

ensi

on, w

hile

the

sec

ond

scor

ing

colu

mn

indi

cate

s th

e to

tal

scor

e fo

r th

e ite

m. D

imen

sion

s ar

eab

brev

iate

d as

fol

low

s: E

C=

expr

esse

d co

nten

t, SM

= sp

eake

r’s

mea

ning

, V

C=

viol

atio

n of

coo

pera

tion,

PV

= pu

rpos

e of

vio

latio

n, V

= vi

olat

ion

and

purp

ose,

EM

= ex

pres

sive

mod

ality

, N

I= n

orm

ina

dequ

acy,

RI=

rea

son

for

inad

equa

cy.

For

the

conv

ersa

tion

scal

e: T

M=

topi

c m

anag

emen

t; T

T=

turn

taki

ng. I

t has

to b

e no

ted

that

, as

dim

ensi

ons

are

hier

arch

ical

(se

e te

xt),

whe

n th

e su

bjec

t doe

s no

t pas

s on

e di

men

sion

, the

re is

no

poin

t in

cont

inui

ng w

ithth

e re

mai

ning

test

que

stio

ns; h

owev

er, i

n so

me

case

s th

e ex

amin

er d

oes

cont

inue

with

the

ques

tions

in o

rder

to g

et a

dee

per

unde

rsta

ndin

g of

the

patie

nt’s

resp

onse

s, w

hich

can

be

cons

ider

ed f

or q

ualit

ativ

e da

ta a

naly

sis.

The

exa

mpl

es p

rovi

ded

in t

he ‘

patie

nt’s

res

pons

e’ c

olum

n ar

e re

spon

ses

give

n by

atr

aum

atic

bra

in in

jure

d pa

tient

.

assessment battery for communication — abaco: a new...

Documents