TASK WEB APPLICATION

PILOTING

SYNTHESISREPORT

TASK - Teacher Assessing Key Competences in School: authentic task based evaluation methodology 2015-1-IT02-KA201-015399

Funded by the Erasmus+ Programme of the European Union The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerives 3.0 Unported (CC BY-NY-ND 3.0).

Released in June 2017 http://www.taskeuproject.com

CONTENTS

ABOUTTHISDOCUMENT......................................................................1

1.BACKGROUNDANDMOTIVATIONS...................................................21.1. Expected results and targets.............................................................................2

1.2. Actors involved....................................................................................................32.METHOD/PROCEDURE....................................................................52.1. Tools.....................................................................................................................5

2.3. Detail of the assesment per ISCED.................................................................83.RESULTS...........................................................................................93.1. Management of time needed for the assessment process completion.......9

3.2 Management of the assessment procedure..................................................10

3.3 Management of the assessment procedure with the teaching staff...........11

3.4 Management of performance evidences........................................................13

3.5 Items adaptation and contextualization..........................................................14

3.6 Control questions...............................................................................................15

3.7 Further aspects..................................................................................................16

ABOUT THIS DOCUMENT This document is a final report collecting and summarizing the evidences of the piloting actions carried out locally by the partner institutions testing with their students the TASK web application implemented to assess Key Competences in school. The logs of the web platform integrate and strengthen the information and data provided with the local piloting reports. The European Organizations involved in the piloting actions of the web application “TASK - Teacher Assessing Key Competences in School: authentic task based evaluation methodology” (2015-1-IT02-KA201-015399) are: European Schoolnet (EUN), IC Bozzano, IIS Majorana, Merkez Atatürk Ortaokulu, Collège Léo Drouyn and IES Ribeira. Collège Léo Drouyn, as Activity leading Organization, coordinated the whole piloting actions and released this report collecting the evidences from all partners, Learning Community, as independent educational research Institute, supervised the piloting actions and tools. This report is organized as indicated below:

• Background and motivations • Expected results and targets • Actors involved • Method / procedure • Tools • Results • SWOT analysis (synthesis of the six points) • Suggestions.

The evidences and highlights collected through the piloting actions and summarised in the present report offer the basis for the web Application fine tuning and the perfection of the TASK Guidelines.

2

1. BACKGROUND AND MOTIVATIONS TASK project aims to support teachers through the development of a methodology for the assessment of Key Competences in order to certificate them with national formats. The assessment is based on real tasks performance to be evaluated by collection of performance evidences, performance observation, control questions to evaluate the quality of the performance. The TASK web application provides a digital support to apply the TASK assessment framework (http://www.taskeuproject.com/wp-content/uploads/2016/06/TASK-FRAMEWORK-EN-FORMATTED.pdf) guiding step by step the assessment process inspired to the authentic evaluation, based on self reflection and supported by control questions. The Piloting actions aim to check the consistence, effectiveness and coherence of the online tool (task based) with the actual scenario of the daily life in school and with the profile of real learners.

1.1. EXPECTED RESULTS AND TARGETS

The staff of the secondary schools involved in school education on levels ISCED2 (lower secondary school) and ISCED3 (upper secondary school) levels is the target group of the piloting actions. Partners involved in this phase of implementation and testing were expected to engage 100 teachers in total with a suggested prevision of: 20 from Italy (10 IISS Majorana, 10 IC Bozzano); 20 from France (College Drouyn); 20 from Spain (IES Ribeira) ; 20 from Belgium (Eun Partnership); 20 from Turkey (Merkez Ataturk Ortaokulu). The objective of this phase was to engage teachers in the application of the TASK assessment procedure, supported by Task online tool, to assess the competences of their students. A suggested arrangement was to engage roughly 20-30 students per each teacher. The piloting activities have been implemented in the national language of the partners. The piloting focused on the evaluation of Key Competence 1 Communication in mother tongue, Key Competence 2 Communication in a foreign language, and Key Competence 4 Digital competence.

3

Partners have collected feedback from the professionals of European Education Institutions, have compared highlights and ideas about the consistence, effectiveness and coherence of the TASK web application with the actual scenario of the daily life in school and with the profile of real learners. These results are useful in order to perfect the TASK web Application and the TASK Guidelines.

1.2. ACTORS INVOLVED

The actors involved in the piloting are: IISS Majorana (Italy), EUN Partnership (Belgium), ISTITUTO COMPRENSIVO BOZZANO (Italy), MERKEZ ATATURK ORTAOKULU (Turkey), COLLEGE DROUYN (France), IES RIBEIRA (Spain), LEARNING COMMUNITY (Italy). On the expected target of 100 teachers, the piloting results shows the active engagement of 122 teachers, distributed as follows: Bozzano and Majorana schools involved 59 secondary school teachers. (36 ISCED2 and 23 ISCED 3) EUN involved 16 teachers (4 ISCED 2 and 15 ISCED 3). A few teachers are both ISCED 2 and 3. MERKEZ ATATURK ORTAOKULU involved 25 ISCED teachers. Collège Léo Drouyn involved 4 ISCED2 teachers. IES RIBEIRA involved 18 teachers. LEARNING COMMUNITY, as independent research Institute, supervised the piloting actions and tools. In addition to this, the logs of the TASK web application record the activities of:

• 226 teachers, • 776 students, • 684 evaluation exercises,

overall from 15 Countries:

1. Italy 2. France 3. Spain 4. Turkey 5. Belgium 6. Albania 7. Romania 8. Cyprus 9. Lithuania

4

10. Poland 11. Serbia 12. Brazil 13. Croatia 14. Hungary 15. Portugal

The different partners have tested the tool by themselves and with their students. This report collects their feedback and presents a summary of comments and/or suggestions of improvement.

5

2. METHOD / PROCEDURE 2.1. TOOLS

A balance between a common track for the piloting and the respect for the local requirements was needed. A piloting toolkit for the collection of data and observation of students’ activities, together with a checklist about the testing procedure, provided the Partners with the common basis allowing the integration and summary of the expected information. Teachers were asked to fill in a SWOT (Strengths, Weaknesses, Opportunities, Threats) analysis questionnaire to validate the effectiveness of the TASK assessment procedure, focusing on:

• management of time needed for the assessment completion, • management of the assessment procedure with the students, • management of the assessment procedure with the teaching staff, • assessment items adaptation/contextualization.

Nevertheless, they were let free to decide, according to their specific requirements, how many students to involve in the piloting, how many competences and areas of competences to investigate. Teachers could decide, according to the local requirements, if it was better to let students accomplish the task individually, at home or in classroom collecting by themselves the evidence of the performance, or if it was better to accomplish the task in a plenary session where evidence would be collected by the teacher or by the classmates; if it was better to give the assessment feedback to students individually or in a plenary session, etc. They were also invited to indicate the tutorship setting they preferred:

• one to one tutorship setting with the students, • one to many tutorship setting with the students, • peer to peer tutorship setting among the students, • cooperative setting with the teaching staff of the involved students’ classroom, • one alone setting.

6

Here is a table presenting the number of teachers involved in the piloting and how the different partners have chosen to work:

Partner

In a cooperative setting with the teaching staff of the involved students' classroom

In a one to many tutorship setting with the students

In a one alone setting

In a peer to peer tutorship setting among the students

In a one to one tutorship setting with the students

Merkez Atatürk Ortaokulu (Turkey)

-

-

-

-

25

Collège Léo Drouyn (France)

-

4

-

-

-

European Schoolnet (EUN)1

4

5

3

3

7

IC Bozzano and IIS Majorana (Italy)

27

29

1

2

IES Ribeira (ES) 18 Here is the profile of the teachers involved and the number of students involved:

Partner

Number of teachers from ISCED 2

Number of teachers from ISCED 3

Number of students involved

Merkez Atatürk Ortaokulu (Turkey)

25 - 28

Collège Léo Drouyn (France) 4 - 125

European Schoolnet (EUN) 13 21 22

IC Bozzano and IIS Majorana2 (Italy)

36 23 80

IES Ribeira (ES) 18 200

1Intotal,35teacherstestedthetoolbythemselves.Amongthese35teachers,18alsotestedthetoolwiththeirstudentsandgavefeedback.2TeachersfromMajoranaandBozzanotestedtheTASKwebapplicationgroupedinteachingteamworkingwiththeirclass.

7

Hereisanoverviewofthemethodsadoptedforthecollectionofrequiredfeedbacks.

Partner Evidence Merkez Atatürk Ortaokulu (Turkey)

Direct observation of the students by teachers.

Collège Léo Drouyn (France)

· Two Google forms were created for the students to answer directly online and to give their opinion on the tool

https://docs.google.com/forms/d/1flSmYPwmjSf_PiIEEx-SGlkEkyB7RlyhqfYcUFeMT8o/prefill https://docs.google.com/forms/d/19GNVn5mc51TgqksDWq0sAPwhs7SI2ni99WN-ChCKVRw/prefill

· An entire lesson in English based on relationships, social networks and TASK was made up in order to assess writing in foreign language

· paper versions of the web application to assess writing in foreign language before going in the computer lab (for younger students)

European Schoolnet (EUN)

· pictures, videos, products, exercises; as for example: · https://www.facebook.com/groups/1600371850190600/ · The Croatian teacher Kristina Kaučić published an article about the TASK

tool on her schools’ website and a newspaper: School web page: Srednja škola Donji Miholjac: http://ss-donji-miholjac.skole.hr/

· newspaper link: Glas Slavonije: http://www.glas-slavonije.hr/327061/4/Prvi-u-RH-koriste-novi-online-alat-za-samoprocjenu-znanja

IC Bozzano and IIS Majorana (Italy)

Classroom debate, creation of a portfolio for some teachers; collection of students' results on the local platform, on memory sticks, email; preparation and collection of cards for the detection and documentation of performance.

IES Ribeira (Spain)

Direct observation of the students by teachers.

8

2.3. DETAIL OF THE ASSESMENT PER ISCED

HereisanoverviewoftheassessmentexercisescompletedpereachKeyCompetenceandISCED

Competences

Assessments

EN IT FR ES TR Tot

Digital Competence 11 87 40 3 15 156

Mother Tongue -ISCED2

53 60 5 26 3 147

Mother Tongue -ISCED3

35 35

Foreign Language -ISCED2

63 252 4 13 14 346

Foreign Language -ISCED3

0

Totali 127 434 49 42 32 684

9

3. RESULTS

3.1. MANAGEMENT OF TIME NEEDED FOR THE ASSESSMENT PROCESS COMPLETION

First, a few statistics: for Turkey and EUN, this management of time was mostly a Weakness whereas it was more of an Opportunity for our Italian partners and of a Strength for our Spanish ones. French teachers were equally divided between Strength and Weakness.

Here is a summary of positive and negative comments that were given by all partners. Positive aspects: Time management can be seen as an asset because the web application gives students the opportunity to reflect on the performance of a task. It gives teachers the opportunity to observe their students, share opinions with them, and it gives teachers time to check their students’ work. Negative aspects: On the other hand, the assessment process is described as too long. As a consequence, pupils get bored and lack attention during the process. They end up

Strength.24%

Weakness23%

Opportunity45%

Threat8%

Timeforassessmentprocedure

Strength. Weakness Opportunity Threat

10

answering control questions hastily and at random. Some teachers underlined that the assessment process was long and it was then difficult to fit in a heavy curriculum.

3.2 MANAGEMENT OF THE ASSESSMENT PROCEDURE.

We were asked to give our opinion about the assessment procedure, and particularly about the fact that students and teachers had to move repetitively from the web application to the classroom context.

First a few statistics: Almost all partners mostly consider that the management of the assessment procedure is an Opportunity or a Strength. Only French teachers mostly think it represents a Weakness for Task. Positive aspects: Partners reported that the web application is a new tool which brings a new stimulus and a different way of working. They qualify the application as educational and useful.

Strength70%

Weakness2%

Opportunity27%

Threat1%

Assessmentprocedurewithstaff

Strength Weakness Opportunity Threat

11

Teachers say it gives them time to observe and give better instructions to students, and that the application allows them to work in serene conditions. French teachers agree with this but for groups composed of 15 students at most. Negative aspects: Teachers from Italy, France and EUN also highlight some negative aspects, such as the fact that texts and control questions are too long, and that the assessment process contains many steps. The combination of these two facts lead to students getting distracted, and to a lack of continuity in learning. French teachers reported that their younger students (11-12year old pupils) found it difficult to mentally connect the task performance and the assessment online. Teachers from various countries highlighted that the assessment procedure was a long and time-consuming procedure.

3.3 MANAGEMENT OF THE ASSESSMENT PROCEDURE WITH THE TEACHING STAFF.

Teachers were asked to reflect on the pros and cons of key competences being transversal / not linked to a specific, disciplinary subject. All European partners unanimously consider that this is an Opportunity or a Strength.

Strength70%

Weakness2%

Opportunity27%

Threat1%

Assessmentprocedurewithstaff

Strength Weakness Opportunity Threat

12

Most comments are consequently positive: Teachers report that the interdisciplinary of the key competences makes them more appealing to students. Besides, it encourages teamwork, exchanges, discussions between teachers, and all teachers can use the application successfully. Some teachers acknowledged that school work is fundamentally transversal anyway, just as real life is. French teachers remarked that the web application was as such relevant with their 2016 educational reform. A few teachers noticed that the fact that key competences are transversal enabled them to observe their students in different and new contexts, and to be more objective in the way they assessed them. Only one teacher regretted that cooperation between teachers was limited.

13

3.4 MANAGEMENT OF PERFORMANCE EVIDENCES.

Teachers were asked to give their opinion about the requirement of collecting evidences in the assessment process. All partners reported that the collection of evidences represents an Opportunity or a Strength for teachers and the assessment procedure.

Once again, most comments are positive: Teachers report that collecting evidences enables them to better observe their students work and progress, and that it enables them to get a deeper knowledge of them. Some teachers created portfolios to define educational paths for their students, and underline that it helps their students keep track of their work. In addition, students gain autonomy and self-confidence. Other teachers noticed that the collection of evidences encourages team-work among students, and that they get used to working this way quickly. Eventually, a few teachers highlight that the collecting of evidences is and has always been a necessary step when they assess.

Strength54%

Weakness2%

Opportunity42%

Threat2%

Performanceevidences

Strength Weakness Opportunity Threat

14

However, various teachers also noted that the process of collecting evidences is time-consuming. Other teachers said they didn't assess all their students at the same time so as not to spend too much time on the collection of evidences.

3.5 ITEMS ADAPTATION AND CONTEXTUALIZATION.

All partners mostly consider that the need to adapt or contextualize assessment items for the classroom context is an Opportunity or a Strength

Teachers report that this adaptation requirement encourages creativity. Most of them say that it has always been part of their work as teachers. Teachers adapted the assessment items to the different profiles of students that compose their classes. It enables them to individualise their lessons. On the other hand, as always, they note that it is a time-consuming process. A teacher suggested that a list of tasks for each level be given in advance.

Strength53%

Weakness11%

Opportunity34%

Threat2%

Itemsadaptation/contextualization

Strength Weakness Opportunity Threat

15

3.6 CONTROL QUESTIONS

Only France and Italy gave their opinion about control questions and they differ. Italian partners consider that it is an Opportunity whereas French teachers think it is more of a Weakness.

On the one hand, teachers report that control questions enable comparisons and reflection on performances for students. As a consequence, they are more aware of their action. On the other hand, control questions need clarifying. Teachers both from Italy and France regret control questions are too long and not adapted for (young) students. Questions are frequently repeated: students can get bored and give random answers.

Strength17%

Weakness23%

Opportunity55%

Threat5%

Controlquestions

Strength Weakness

16

3.7 FURTHER ASPECTS

EUN and our Italian partners gave their opinion about TASK tool in a general way too. Teachers from these two countries/organisations mostly consider the tool in a positive way. Italian teachers also point at its Weaknesses and potential Threats. Positive remarks: Teachers highlight the enthusiasm, excitement, and awareness that the Task tool generated among students. They congratulate the partners on the setting up of this new tool and way of working, and say that it can still by improved. They also appreciate the creation of a portfolio that helps using the tool. Negative remarks: Teachers also remark that the tool and the website can be improved, especially the control questions. The feedbacks are also too long and difficult to read and should be clarified too.