1

Assessing Taiwanese Learners’ Italian Proficiency: an Explorative Study

Francesco Nati

Department of Italian Language and Culture

Fu Jen Catholic University, Taiwan

049405@mail.fju.edu.tw

Abstract

The Department of Italian Language and Culture at Fu Jen Catholic University, created in

1996, is the only academic entity specifically devoted to the teaching and promotion of Italian

language and culture in Taiwan.

Growing relationships between Taiwan and the European Union in recent years and the

subsequent active promotion of the Common European Framework of Reference (CEFR)’s

guidelines, especially for what concerns Language Proficiency Tests, did have some impact in

Taiwan, especially since the adoption of the CEFR by the Ministry of Education in 2005. Locally

much has been said and discussed in relation to the CEFR’s levels and their suitability for the

Taiwanese EFL context, and different opinions exist as to whether it should be adopted for teaching

and testing; as discussion related to other European languages ensued, we believe that it’s

worthwhile to carry out an evaluation regarding Italian in Taiwan.

The purpose of this study is to carry out an explorative comparison between the B1 level of

the four officially recognized Italian Language Certifications (CILS, CELI, PLIDA and IT),

introducing their design characteristics and investigating their appropriateness in measuring our

students’ language ability. For this research, we decided to start from the analysis of passive skills

(listening and reading) because they can be easily graded in loco, while active skills are usually

graded in Italy by each testing center and thus require a more specific training.

Research methods for this study are mainly quantitative: statistical data regarding the results

of the four Italian language proficiency test will be presented (an experimental group of 8 university

students was selected, all sharing the same basic characteristics: around 20 years of age; 2 years and

a half of Italian learning; mother tongue Chinese speakers), alongside with an analysis of university

scores from this Department regarding the students’ performance in our Language courses;

moreover, students’ questionnaires will be presented regarding their perception of the four language

tests and their opinions on language courses taken within our Department in comparison with

official Italian Language Certifications.

The results of this study will shed further light on the possibility of developing an Italian

Language Proficiency Test for Taiwanese users, and at the same time will give us an insight into

our students’ language performance in comparison with international standards.

Keywords: Language assessment, Italian, Chinese learners, Taiwan

2

ASSESSING TAIWANESE LEARNERS’ ITALIAN PROFICIENCY: AN EXPLORATIVE STUDY

Francesco Nati

Fu Jen Catholic University

3

1. Introduction

The Department of Italian Language and Culture at Fu Jen Catholic University, created in

1996, is the only academic entity specifically devoted to the teaching and promotion of Italian

language and culture in Taiwan; every year we accept around 60 students (most of whom are

Chinese mother tongue speakers) who will obtain, hopefully within four years, a bachelor’s degree

in Italian language and culture. General linguistic goals for each year are largely set according to

the standards of the CEFR, with the first year expected to reach an A1-A2 level, the second year an

A2-B1 level and the third year a B1-B2 level; since the fourth year has almost no compulsory

language courses, it is very difficult to determine the linguistic level of the students at the time of

graduation. We are also an authorized center for CILS exams (one of the two main officially-

recognized Italian certifications - see next section for more information), held twice a year in our

University, with an average of 14 examinees registering at every session, a good proportion of

which are students from our Department1.

In this study we carried out an explorative comparison between the four officially

recognized Italian Language Certifications (CILS, CELI, PLIDA and IT), concentrating on passive

competences (reading and listening tasks), mainly because they can be easily graded in loco without

specific training. For our research, 8 volunteers sharing the same study background were found and

all four language tests’ Listening and Reading exercises were administered; subsequently, exam

sheets were graded and results were compared to the students’ scores within our Department.

Additionally, questionnaires were given to the testees in order to both do self-evaluation and to

provide opinions on the tests’ perceived difficulty.

The results of this research are meaningful not only in the perspective of better

understanding our students’ language ability and verifying if their proficiency will be up to our

expected standards (from another point of view, it may also tell us something about the

appropriateness of choosing one or another certification in order to measure their linguistic level),

1 Statistical data referred to the years 2001-2011, gathered from Tulli (2011).

4

but also in seeing if and how different testing styles for the same language level (in this case, B1)

may lead to different performances; we believe that the importance of this last point cannot be

overlooked because, in the 21st century, owning an official language certificate may sometimes

make the difference in accessing universities, finding jobs, calculating scores for public selections

etc., hence candidates could and should fully understand the differences between different tests

before registering in one of them.

2. An historical overview of the officially recognized Italian proficiency tests.

Language assessment in Italy is a rather new area of research that has witnessed remarkable

progress in the last 15-20 years; it has been pointed out2 that only in 1982, on the occasion of a

national convention regarding the status of the Italian language in the world, did Italian scholars and

government representatives recognize the importance of developing a clear policy for the promotion

of Italian in the world, and at the same time the necessity of establishing an officially recognized

Italian proficiency test, catching up with other European languages who were already starting to

take their first steps in this direction.

After some years of researching, based on different theoretical starting points, in the middle

of the nineties three academic entities came up with their own Italian language assessment models:

on one side was the newly founded University of Roma Tre, where the Department of Language

Sciences designed a proficiency test called “IT”; on the other side were the two Universities for

Foreigners of Siena and Perugia, which developed respectively the CILS (Certificato di Italiano

come Lingua Straniera - “Certificate of Italian as Foreign Language”) and the CELI (Certificato di

Lingua Italiana - “Certificate of Italian Language”) exams. The dissimilarity between their

approaches derived from these universities’ different history and background: while the Department

of Language Sciences at Roma Tre was more centered on pure and applied linguistics, the

Universities for Foreigners in Siena and Perugia enjoyed a much higher degree of

2 Vedovelli M. et al. (on print), p. 28-40 and Novello (2009), p. 99-128.

5

internationalization because of their conspicuous number of foreign students coming from every

part of the world, and thus had more opportunities to calibrate their tests according to the testees

actual performance and needs.

The last Italian test to appear on the market was developed by the Dante Alighieri Society,

an officially recognized non-profit organization founded in 1889 which aims at promoting Italian

language and culture around the world; their language certificate was conceived within their project

called “PLIDA” (Progetto Lingua Italiana Dante Alighieri - “Dante Alighieri Italian Language

Project”) and was ratified by the Italian Ministry of Foreign Affairs, starting examinations from

1999. Not being an academic institution, the Dante Alighieri Society created a scientific committee

which would supervise every aspect of the tests, including design, creation, validation, marking etc.

All Italian language tests did undergo some kind of modification since the middle of the

nineties, in a moment when the six CEFR levels were taking shape by means of the research carried

out within the Council of Europe: for example, the CILS, which had started as a four-level

certificate, was finally structured in six levels in 1998; the CELI also was gradually changed to six

levels after the middle of the nineties; the IT, which had originally started as a two-level certificate

(as we said before, the theoretical bases were different from the Universities for Foreigners), was

finally changed into a four-level structure which included A2, B1, B2 and C2 levels. The PLIDA,

being the newest certificate, followed the six CEFR levels from the beginning.

At the beginning of the new century, two years after the publication of the CEFR by the

Council of Europe in 2001, the Manual for Relating Language Examinations to the Common

European Framework of Reference for Languages was made available to the public; subsequently,

many language certificates around the world were linked to the 6 CEFR levels and in Italy this work

was mainly carried out in the Universities for Foreigners of Siena and Perugia, respectively

members of EALTA and ALTE3.

3 See Grego Bolli (2008) and Barni et al. (2010) for specific reports on linking the CELI and the CILS to the CEFR.

6

Presently, the two Italian language tests with the highest number of candidates are the CELI

and the CILS (summing up to more than 150,000 candidates until 20094); except for the IT, which

is held annually, all Italian certification exams are held twice a year.

3.1 Characteristics of the testees.

Eight of our students volunteered to take part in this project, all sharing the same basic study

background: Chinese mother tongue speakers, majoring in Italian, attending the third year of studies

(therefore, with approximately 2 and a half years of Italian study experience, that is a little more

than 5 semesters), university-aged (20-23), with university-level English proficiency5. The only

discriminating factor was that 5 of them had spent one semester as exchange students in Italy, while

the rest had always studied in Taiwan.

Before starting the tests, students were asked to fill in a questionnaire regarding their self-

perceived language ability (questions and results can be found in Appendix 1). For every self-

assessed ability, the first four questions correspond to descriptors for A2 level, while questions 5 to

8 are descriptors for B1 level, nevertheless students weren’t informed of this and were just asked to

indicate the extent to which they agreed with each statement.

Averagely, students tended to agree more with statements corresponding to the A2 level,

while some of the B1 descriptors yielded inconsistent answers: for example, in the listening section,

while all students agreed on being able to “understand phrases and expressions related to areas of

most immediate priority provided speech is clearly and slowly articulated”, they showed less

confidence in being able to “follow the main points of extended discussion […] provided speech is

clearly articulated in standard dialect” (3 students disagreeing). For what concerns reading, more

uncertainty prevailed for abilities related to the B1 level: for instance, while almost 6 students

strongly agreed with being able to “understand short, simple texts as long as they are on familiar

4 See Vedovelli, M. et al. (on print), p. 28-40.

5 Students weren’t asked to provide any proof of English language proficiency; anyway, since they were all Chinese

speakers who grew in Taiwan, it was assumed that they all had non-mother tongue English ability derived by their

compulsory education (about 12 years of study from elementary school to university).

7

matters of a concrete type which consist of high frequency vocabulary”, most of them didn’t feel

able to “identify the main conclusions in clearly signaled argumentative texts and […] recognize

significant points in straightforward newspaper articles on familiar subjects” (5 students

disagreeing).

Nevertheless, as we will see, the students’ self-perception of their own listening ability is

somewhat different from the skills required to perform well in language proficiency tests: even

though in class they are usually exposed to standard and controlled everyday conversation, most of

the listening tasks for the B1 level are adapted from radio programs and thus are much more

difficult to understand, actually requiring specific preparation in order to be fully accomplished.

3.2 Characteristics of the four tests.

We will now introduce the basic structure of the four tests’ Listening and Reading tasks for

B1 Level; the presentation will also include the specific texts and topics pertaining to the test

sessions that were chosen6.

Table 1 - Basic Characteristics - Listening

Test Tasks # Items # Total time Max score Passing score

CILS 3 23 30 min 20 11

PLIDA 2 22 20 min 30 18

IT 4 20 30 min 20 12 CELI 4 33 20 min 40 24

Table 2 - Basic Characteristics - Reading

Test Tasks # Items # Total time Max score Passing score

CILS 3 32 50 min 20 11

PLIDA 2 22 30 min 30 18

IT 5 21 60 min* 25 15

CELI 5 37 60 min* 40 24

* The actual time assigned is 2 hours in total for reading and writing.

6 Specifically: CILS June 2009, PLIDA May 2009, It 2011 and CELI June 2007. These tests, at the time, were all

available on the Internet as free examples for download.

8

As we can see from the tables, the PLIDA stands out immediately not only because of its

lower overall time, but also because of the generally low number of tasks: a low number of texts

will allow the testee to concentrate on less inputs, and at the same time a limited number of tasks

means more time to complete each item; on the other hand, the CELI has the highest number of

tasks and items, requiring much more concentration and efficiency on the students’ side, and we

may infer that it is the most anxiogenous among the four tests.

One interesting thing to notice is that the CILS is the only certification with a passing mark

of 55%, slightly below the passing mark of other proficiency exams7; as we will see in section 4.2,

2 of the students who reached the passing level on the CILS had barely obtained a 55%, which

would have been considered a failing score in another exam.

Table 3- Characteristics of the tasks - Listening

Test Type of tasks Length of texts Topics Type of text

CILS Dictation; multiple choice (1 out of 4); information retrieval

dictation: 60 words; listening: 2:00; 2:20

photography exhibition; plans for future studies; tourism (architecture exhibition)

radio news; conversation

PLIDA Multiple choice (1 out of 3); information retrieval

3:30; 3:00 literature; wine fair radio interview; radio news

IT Multiple choice (1 out of 3); multiple choice (1 out of 2); True or False; sentences completion

3:50; 5 sec x 6; 1:10; 1:10

advice for travelers; ecology (shopping bags); sales season

radio interview; radio programs (beginning and ending); radio news

CELI Multiple choice (1 out of 3), multiple-text; multiple choice (1 out of 3), multiple-text; sentences recognition; information retrieval

25-30 sec x 4; 30-35 sec x 4; 3:00; 2:00

daily life, shopping, health etc.; housing and social trends; perfumes, culture

radio news; radio interview; radio commercial

7 As pointed out in Novello (2009) at p. 137, passing scores for different language proficiency tests in Europe vary a lot,

from 50% in CIEP’s exams (CIEP stands for “Centre International d’Etudes Pédagogiques” or International Center for

Pedagogical Studies, responsible for the assessment of French) to 70% in the Dele (Diploma of Spanish as a Foreign

Language). Also, differently from all other tests in Europe, each section of the Italian language proficiency exams

(reading, listening, writing etc.) must be over the passing threshold in order to obtain a Certificate.

9

Table 4- Characteristics of the tasks - Reading

Test Type of tasks Length of texts Topics Type of text

CILS Multiple choice questions (1 out of 4); information retrieval; text reconstruction

400-450 words x 2; around 250 words for scrambled text

Young Italian writers; special rates for electricity bills; vacation on the Alps

Informative article; fiction/diary

PLIDA Information retrieval; multiple choice cloze (1 out of 4)

300 words, 230 words

Excerpt from a novel; elementary school regulations

fiction; instructions

IT Multiple choice questions (1 out of 3); word retrieval (synonyms); True or false; referents retrieval; linking sentences and texts

300 words x 2, 500 words

New train service; online shopping; literature

newspaper article; book review

CELI Multiple choice questions (1 out of 3); information retrieval; multiple choice gap-filling (1 out of 4); multiple choice cloze (1 out of 3); free gap-filling (pronouns)

660 words, 215 words, 60 words (5 sentences), 140 words, 60 words (5 sentences)

Daily life Newspaper article; magazine article

When looking at tables 3 and 4, it is evident that multiple choice and information retrieval8

exercises are usually present in all tests; anyhow, it’s not difficult to notice that every test is

somewhat different from the others. For listening, the CILS is the only test including a dictation

exercise, while the IT contains a completion exercise where the testees need to fill sentences with

information according to what they hear on the recording; the CELI is the only test with a sentence

recognition exercise, where the candidates can listen to a text only once and must choose which of

the written sentences are present in the recording. Topics all conform to the B1 level and text types

are mostly excerpts from radio programs; texts are simplified, professionally recorded and read in a

clear and slow manner, except for the IT test, whose recording quality was not as good as the others

and where only authentic recordings were used. Texts are always played twice (except for the

aforementioned sentence recognition task in the CELI exam).

8 Information retrieval exercises are similar to True or False, but the candidate is asked to read a list of sentences and

decide which of them contains information that is present in the article. The candidate may or may not be informed on

the number of sentences with correct information.

10

As regards reading assessment, multiple choice and information retrieval exercises are still

widely adopted; specific situations are as follows: the CILS is the only test with a text

reconstruction exercise, while the CELI is the only one to have gap-filling exercises; the IT also has

some specificity, including synonyms retrieval, referents9

retrieval and linking texts with

corresponding sentences. Topics all conform to the B1 level and text types are usually adapted

excerpts from newspaper articles or fiction.

Once again, when compared to the others tests, the PLIDA seems to be the easiest to deal

with, including only the most common types of exercises, a low number of texts and very common

topics related to daily life.

One other differentiating aspect among the tests is in how students are informed about the

way their exams will be scored: for example, both the CILS and the CELI’s scoring methods are

never explicitly stated within the exam sheet and even though negative scores are assigned for

wrong answers, this information is only provided when downloading sample tests with keys from

the Internet; also, total scores for each task aren’t written on the exam sheet, so the students aren’t

aware of which exercises are more “valuable” to the tester. On the other hand, the PLIDA exam

sheet provides the tasks’ total scores and grading method10

, while the IT is the simplest in that no

negative scores are given for wrong answers and total scores for each task are written at the

beginning of the exam paper.

4.1 Method of administration.

After selecting the students to be tested, who all participated voluntarily in the experiment,

they were informed that four test sessions would be organized, corresponding to the four officially

recognized Italian proficiency exams, and that they would be tested in their listening and reading

ability for the B1 level of the CEFR. Students weren’t informed about the structure of the tests

9 In this case, “referents” indicates those words which are referred to, within the text, by means of pronouns and other

elements that the testees have to recognize. 10

It even tells the students how many answers they should provide in the information retrieval exercise, warning them

that every exceeding answer will have a negative score; amusingly, one of our examinees failed to pass the PLIDA

reading test because of forgetting to carefully read the instructions, thus choosing an excessive number of answers.

11

themselves and were asked not to undergo specific training for any of the sessions; from the point

of view of the students’ previous knowledge, the only discriminating element could be that they

were previously tested on a CILS-like simulation for the A2 level in June 2011, but still we don’t

believe that this had a significant influence on their performance because language levels were

different and some of the students were studying in Italy at that time, so they didn’t take the

simulation test.

The four tests were taken between March 21st and March 30

th 2012, with a time span of 2-5

days between each session. At the beginning, students were briefly informed about the sequence of

the exercises and then given the exam sheets, where they had to write their examinee progressive

number (1 to 8); subsequently, they were first tested in listening (by means of an MP3 file played

through a desktop computer with professional speakers installed in the classroom) and then in

reading.

After each session, students were asked to fill in a questionnaire regarding their perceived

difficulty and validity of the test. Finally, exam papers were anonymously graded11

and

questionnaire results were calculated.

4.2 Analysis of post-test questionnaire results.

In this paragraph, we will analyze the results of questionnaires given to the students after

each session. Specific questions and detailed results can be found in Appendix 2.

The first question was about the intelligibility of written instructions within the tests, i.e.

what students were asked to do in the exam papers. Most tests scored positive opinions, with the

only exception of the IT, whose listening test created some confusion both because of the aforesaid

recording quality and also because the second exercise (see p. 8) had students listen to excerpts

from radio programs and choose if they were at the beginning or at the end of them12

.

11

CILS, CELI and PLIDA all provide exercise keys along with their test samples, while IT didn’t provide any. 12

During informal conversations after the test, the students admitted to choosing answers randomly in that exercise

because they hadn’t understood what they were required to do.

12

Regarding the level of familiarity towards the topics (questions 2), most answers fall in the

categories of little or no acquaintance, with only the PLIDA and the CELI scoring 3 completely

positive answers out of 8; among the tests, the most unfamiliar was the IT’s listening, with half of

the students stating that they weren’t acquainted at all with the subjects. Question 4, which is

slightly related to the second one, asked about the tests’ similarity with our language courses

content, and here answers generally show more positive results, with only the IT being the

exception. This shows that, even though we usually talk about topics related to all aspects of Italian

daily life, students still show some level of unfamiliarity towards them (see next session for

possible explanations for this).

Questions 3 and 5 were more related to the students’ perceived difficulty of the tests. Here,

it is evident the majority feels comfortable with the time allotted for completing all exercises, while

also most of the testees perceived a moderate-to-high level of difficulty for the CILS, the PLIDA

and the IT, the only exception being the CELI, where most students ranked it in the moderate-to-

low difficulty range13

.

Finally, the last question was investigating on the examinees’ perceived appropriateness of

the tests in measuring their ability, i.e. its “face validity”14

; the highest level of agreement was

reached with the CILS’s listening, the PLIDA’s listening and the IT’s reading exams while some

disagreeing answers (2 out of 8) were scored only for the IT’s listening (probably for the reasons

stated above) and the PLIDA’s reading.

13

When comparing this with the tests results in the next section, we’ll see that there was a great deal of

misunderstanding on the students’ side regarding the complexity of the CELI: indeed, even though its perceived

difficulty was rather low when compared to other tests, results show that the students’ average performance in the CELI

was inferior to all other tests. Our assumption, not yet verified, is that, even though the CELI’s listening and reading

texts seem to be simpler, their exercises are more difficult to complete, including misleading distractors in multiple

choice exercises, subtler meanings in the information retrieval sentences etc. 14

Much has been said over the concept of face validity: cf. Bachman (1990), Alderson et al (1995), Brown (2003) etc.

While face validity may not have scientific bases, as it is purely an opinion by non-experts, our interest was just to

understand if and how the students perceive the tests’ as efficient in measuring their language ability.

13

4.3 Analysis of the tests results.

Test results are shown in Table 3, while in Table 4 we can see a comparison with our

Department’s average scores (along with criteria adopted for their calculation).

Table 3 - Test results15

Testee # Listening Reading Listening

Average

Reading

Average

Proficiency tests

Average CILS PLIDA IT CELI CILS PLIDA IT CELI

1 73 60 60 59 37 80 56 52 63 56 59

2 60 80 55 22 43 73 68 45 54 57 56

3 25 67 30 41 46 53 80 36 41 54 47

4 58 80 40 46 42 67 72 50 56 58 57

5 80 40 50 54 47 73 68 82 56 68 62

6 58 73 30 39 51 80 64 52 50 62 56

7 63 60 40 37 30 67 52 50 50 50 50

8 60 80 55 63 66 80 80 48 65 68 66

Average 59 68 45 45 45 72 68 52 54 59 57

Naturally, the first thing we notice is that Italian proficiency test results for the B1 level are

averagely under 60% and, even when we consider the average scores separately for listening and

reading, we can see that they are also under this threshold. Among the four proficiency tests,

students’ performed best in the PLIDA, where all the examinees managed to obtain passing scores

in listening and reading, with only one exception (testee nr. 3, who didn’t pass the reading test); on

the other hand, the hardest exam was the CELI, where only student n. 8 passed the listening test and

a different one, n. 5, managed to pass the reading test. In the middle, the CILS and the IT showed

controversial results: in the former, the listening test yielded better results (with only one person

failing and two students just above the passing score of 55%), while in the latter there was a better

outcome in the reading test, with only two students failing to pass16

.

When calculating average scores for all of the four proficiency tests, only two students

obtained a passing percentage, with only one of them (n. 8) managing to reach a passing average

both for listening and reading.

15

The scores have all been percentaged for the sake of comparison; marks in bold characters are over the passing score. 16

Nevertheless, if we examine the structure of the IT exam, we can see that it includes more grammatical-oriented

exercises (for example: synonyms, referents retrieval), where our students usually perform better, and also one

True/False exercise where the chance of correct guessing was 50%.

14

Table 4 - Comparison between test results and Fu Jen grades17

Testee # 1st year average 2

nd year average

3rd

year - 1st

semester average

Fu Jen average (1st to

3rd

year)

Proficiency tests

average

1 90 83 85 87 59

2 84 72 78 79 56

3 78 71 21 65 47

4 79 78 83 79 57

5 83 83 81 83 62

6 74 74 75 74 56

7 79 77 81 78 50

8 82 81 77 81 66

If we compare these results with scores obtained in our Department, some interesting

considerations can be done. First of all, average Fu Jen scores are higher than proficiency exams;

secondly, some of the students who delivered an unsatisfactory performance in the proficiency tests

are actually ranked in our top 25%, confirming that the skills they are being trained to develop may

not correspond to what these tests are requiring them to do; thirdly, Fu Jen rankings and proficiency

tests’ rankings are also different: for example, the first three in the Fu Jen average are number 1, 5

and 8, while the first three in the other category are the same people but in a different order: 8, 5

and 1. This means that not only required skills vary, but also the importance attached to them is

different (for example, in proficiency tests grasping the general idea of a text may have more

importance if compared to what we require in our exams).

The reasons for this discrepancy are manifold, and we will just try to list some of them

based on our teaching experience in the last 12 years:

1. First of all, our Department aims at bringing students to a B1-B2 level by the end of their

third year, and this means that right in the middle of it, not all students may be able to fully reach a

B1 level, as we can see in their self-assessment questionnaire presented on page 6. Also, there is a

higher correlation between our courses’ content and our exams, as we usually make adjustments

17

We took into consideration only scores related to our compulsory language courses: Italian Grammar, Italian Reading

& Composition, Italian Conversation and Italian Pronunciation and Phonetics; whenever these scores had been obtained

from Italian universities during exchange programs, they were treated as Fu Jen scores.

15

according to the students’ needs and interests, while international proficiency tests are calibrated

with a general “foreign/international examinee” profile in mind;

2. No matter which ability we take into consideration, our students are generally unfamiliar

with the topics dealt with in the proficiency tests (see questionnaire analysis on page 11); also,

textbooks we use in class may or may not deal with these subjects, and even when they do, some of

the students find it hard to feel concerned or interested about them because of the different cultural

background;

3. In our Department, and generally in Taiwan and most of East Asia, language training

techniques often differ from Western teaching styles: in order to understand foreign language

content, students rely a lot on written texts, show a great deal of attention for grammatical rules,

have a tendency to look up every word on the dictionary, and are frequently unable to synthesize or

grasp the gist of a long language input. Unless we insist on a change of perspective and learning

style, it’s not hard to anticipate that they will mostly be at a disadvantage when compared to

Western students18

;

4. As observed by Bachman19

, among the elements that influence test scores, there is always

a percentage of random factors. In this case, it may stand to reason that students were influenced by

some psychological factors like: fatigue or stress derived by school assignments, anxiety and

insecurity (i.e. the inability to listen to prolonged recorded content or a tendency to “give up” when

not being able to understand etc.), but also being aware that they were participating in an

experiment not having any concrete repercussion on their school scores or their personal career,

which may have led to random guessing on their side.

5. Conclusions.

18

For a deeper understanding of Taiwanese students’ learning styles and motivation (particularly in relation learning

Italian), see Nati (2006). 19

Bachman (1990), p. 164.

16

At the beginning of this article, we asked ourselves if our students’ language proficiency

would be up to our expected standards, but also if and how different testing styles for the same

language level could lead to different performances.

We’re afraid that the answer to the first question will be more on the negative side: it is

undeniable that our students struggled with language certifications’ Listening and Reading for the

B1 level, and even without testing them on the other abilities (like writing, speaking etc.), we can be

fairly sure that most of them wouldn’t obtain their certification on their first attempt. Of course, the

students’ performance on a certification exam cannot be considered a complete and accurate

evaluation of their language proficiency in a real-life situation; nevertheless, as we previously stated,

professional certificates (not only for what concerns foreign languages, but also in many other fields,

for example in medicine, law, accounting, music, transports etc.) do play an important role in

society, and being able to obtain one of them is undoubtedly an important step in a university

student’s career, not to mention how this could improve self-confidence and motivation for study.

As to the second question, the experiment seemed to provide a positive answer: passing

results were uniformly different from one test to another, with the PLIDA standing out as the most

“accessible” and the CELI being the hardest. From the users’ perspective, this could just mean that

choosing an “easier” certification would allow them to obtain their certificate with less effort; from

our perspective as teachers, though, this could also mean that the PLIDA’s approach and

interpretation of the CEFR levels could be more suitable for our students. Indeed, this is confirmed

by Vedovelli, stating that those who study Italian in the world are extremely diversified not only in

their geographical distribution, but also in their motivation; thus, “the plurality of the certification

authorities represents a valid strategy to create broader opportunities for foreigners to get in contact

with our language and culture”20

. Because every test is designed for a different kind of user, the

most logical choice for the examinees is to choose those proficiency tests that can be more suitable

to measure their language ability.

20

Vedovelli (in print), p. 20-21.

17

In conclusion, even though Certification exams should not be the only basis for evaluating

our students’ Italian proficiency, they can provide an interesting perspective on what they are able

to do with their foreign language: on one hand, it is evident that linguistic ability can’t be

disconnected from the knowledge of the country where it is used, therefore students should be

reminded that the Italian language is strictly related to its culture even when it is studied abroad,

outside its original context; on the other hand, since most of our students almost never end up

relocating to Italy, it would be advisable to adopt, or maybe even develop locally, language

proficiency tests that do not attach excessive importance to practical aspects of Italian everyday life,

incorporating cultural elements in a more traditional sense.

6. Bibliography

Alderson, J. C. (2000). Assessing reading. Cambridge: Cambridge University Press

Alderson J.C., Clapham C. & Wall D. (1995). Language Test Construction and Evaluation.

Cambridge: Cambridge University Press

Bachman, L. (1990), Fundamental considerations in language testing, Oxford: Oxford

University Press

Bachman, L.F. and Palmer, A.S. (1996), Language testing in practice. Oxford: Oxford

University Press.

Barni M., Scaglioso A.M, Machetti S. (2010), “Linking the CILS examinations to the CEFR:

the A1 speaking test”. In W. Martyniuk (ed.), Aligning Tests with the CEFR. Reflections on using

the Council of Europe’s draft manual, Cambridge: Cambridge University Press, 159-176.

Brown, H. D. (2003), Language Assessment Principles and Classroom Practices, San

Francisco: Longman

Buck, G. (2001). Assessing listening. Cambridge: Cambridge University

Figueras, N. et al (2005), Relating Examinations to the Common European Framework: a

Manual, Language Testing 22 (3), 1–19

Grego Bolli, G. (2008), “Progetti europei: nuove prospettive sulla scia del Quadro Comune

Europeo di Riferimento”. In Ciliberti, A. (ed.) Un mondo di italiano. Perugia: Guerra Edizioni, 25-

48.

Nati, F. (2006). “Riflessioni sull’uso di metodi comunicativi per apprendenti di italiano a

Taiwan”. Article presented at “The 3rd Cross-Strait Conference on Foreign Language Teaching &

18

Learning”, Fu Jen Catholic University, Taipei, Taiwan, 2 November 2006. Can be downloaded

from: www.svd.fju.edu.tw/fl/lang_teaching/papers/ita_3.pdf

Novello Alberta (2009), Valutare una lingua straniera: le certificazioni europee, Venezia:

Cafoscarina

Tulli, A. (2011), The Certification of Italian as a Foreign Language (CILS) in Taiwan and

its distribution in other Asian countries, presentation held at the Second International Symposium

on European Languages in East Asia, Taipei, Taiwan, 5-6 November 2011

Vedovelli M., Barni M., Bagna C., Machetti S. (in print), Vademecum. Le certificazioni di

competenza linguistica in Italiano come lingua straniera, Roma, FAPI, retrieved on May 11th at:

http://www.aislo.it/ImagePub.aspx?id=54704

19

Appendix 1 - Self-evaluation questionnaire on Italian proficiency (original questions were in

Chinese)

Please read the statements and indicate the extent to which you agree with each of them (4 strongly agree 1

strongly disagree).

Listening ability

4 3 2 1

1. I can understand phrases and expressions related to areas of most immediate priority (e.g. very

basic personal and family information, shopping, local geography, employment) provided speech

is clearly and slowly articulated.

5 3 0 0

2. I can generally identify the topic of discussion around me that is conducted slowly and clearly. 5 3 0 0

3. I can catch the main point in short, clear, simple messages and announcement.

I can understand simple directions relating to how to get from X to Y, by foot or public transport.

4 3 1 0

4. I can understand and extract the essential information from short recorded passages dealing

with predictable everyday matters that are delivered slowly and clearly.

4 2 2 0

5. I can understand the main points of clear standard input on familiar matters. 5 2 1 0

6. I can generally follow the main points of extended discussion around me, provided speech is

clearly articulated in standard dialect.

1 4 3 0

7. I can understand simple technical information, such as operating instructions for everyday

equipment and I can follow detailed directions.

2 4 1 1

8. I can understand slow and clear radio news bulletins and recorded material, as long as it is

about familiar subjects or those related to my field of specialization.

3 4 1 0

Reading ability

4 3 2 1

1. I can understand short, simple texts as long as they are on familiar matters of a concrete type

which consist of high frequency vocabulary.

6 2 0 0

2. I can understand short simple personal letters and recognize basic types of standard routine

letters (enquiries, orders, letters of confirmation etc.) on familiar topics.

3 4 1 0

3. I can find specific, predictable information in materials such as advertisements, prospectuses,

menus, reference lists and timetables, and I can understand everyday signs and notices such as

directions, safety warnings etc.

4 1 2 1

4. I can identify specific information in simpler written material such as letters, brochures and

short newspaper articles describing events.

3 3 2 0

5. I can read straightforward factual texts on subjects related to my field and interest with a

satisfactory level of comprehension.

1 5 2 0

6. I can understand the description of events, feelings and wishes in personal letters. 3 4 1 0

7. I can scan longer texts in order to locate desired information and I can find and understand

relevant information in everyday material.

0 5 3 0

8. I can identify the main conclusions in clearly signaled argumentative texts and I can recognize

significant points in straightforward newspaper articles on familiar subjects.

0 3 5 0

20

Appendix 2 - Perceived validity of each proficiency test (original questions were in Chinese)

1. I think that the instructions in

the test are:

a. clear b. a little unclear c. not clear

2. I am… with the topics: a. acquainted b. a little acquainted c. not acquainted

3. I think that the time given for

answering is:

a. too much b. just enough c. not enough

4. This test is… what I learned in

class:

a. similar to b. a little different

from

c. very different from

5. The overall difficulty of this

test was:

a. high b. moderate c. low

6. This test can measure my real

reading/listening ability:

a. I agree b. I have no opinion c. I disagree

Questionnaire results:

Question

Test 1 2 3 4 5 6

a b c a b c a b c a b c a b c a b c

CILS - listening 7 0 1 2 6 0 0 7 1 4 4 0 2 6 0 4 4 0

CILS - reading 8 0 0 0 5 3 1 3 4 3 4 1 7 1 0 3 5 0

PLIDA - listening 8 0 0 3 5 0 0 8 0 5 3 0 0 7 1 4 4 0

PLIDA - reading 8 0 0 3 4 1 0 7 1 3 4 1 3 5 0 3 3 2

IT - listening 0 4 4 1 3 4 0 8 0 1 6 1 8 0 0 3 3 2

IT - reading 6 2 0 1 5 2 2 6 0 3 5 0 2 6 0 4 4 0

CELI - listening 6 2 0 3 5 0 0 7 1 3 5 0 1 5 2 2 6 0

CELI - reading 7 1 0 5 3 0 1 7 0 5 3 0 0 4 4 3 5 0