training trainers for migrants and roma grundtvig ... · wp 3 quality assurance final evaluation...

TRAINING TRAINERS FOR MIGRANTS AND ROMA

GRUNDTVIG MULTILATERAL PROJECT

WP 3 QUALITY ASSURANCE

FINAL EVALUATION REPORT

MARCH 2014

TTMIRO - Training Trainers for Migrants and Roma Lifelong Learning Programme Project number: 518256-LLP-1-2011-1-GR-GRUNDTVIG-GMP

2

This project has been funded with support from the European Commission. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

TABLE OF CONTENTS

1. Quality Assurance” (QA) in the TTMIRO Project

2. Quality Evaluation Methodology

3. Evaluation results according to framework standards: Overarching issues

4. Content Quality Assessment

5. Evaluation of the Quality of Implementation Procedures


3


1. “Quality Assurance” (QA) in the TTMIRO Project

The TTMIRO Project focuses on the development of Training Material for trainers of trainers and

teacher/trainers in the VET and in LLL, whose participants include migrants and/or Roma. This

Training Material can be used either as a whole in a relevant training course or in parts, in cases

that the Modules it consists of correspond to specific needs of the training of trainers and adult

education teachers.

Developing a training course or the training material for a course means in general looking at the

overarching educational system (European, national, regional, local), the conditions and settings

of the educational organizations providing VET, AE and LLL, the abilities, skills and competences

of the teachers and trainers and the situation of the learners / participants with regards to

assessment, learning modules etc. in the broadest sense. For this project and for its quality

assurance the specific abilities, skills and competences of trainers / teachers that are targeted

have to do with “mixed” classes. By the term we define the adult classes where members of the

dominant culture co-exist and learn together with individual carriers of other cultures, usually

perceived as different but also often understood as inferior ones. In this sense, what we are

targeting is not simply a multicultural group of learners and the interplay of cultures among

them but also the impact on learning in such an environment, where the power of

superimposing cultural norms to other cultures is also at play. Another important factor to

address via Quality Assurance is the potential of the existing knowledge with regards to “social

factors” such as the social exclusion ones, in the social services and its availability and/or

usefulness and transferability elements for the VET/LLL-sector classes. Last but not least, the

meeting of the needs of the learners and participants in VET/LLL with migration background /

Roma is another main factor that was addressed through the Quality Assurance procedures of

the TTMIRO Project .


4


The above three factors were at the centre of the quality quest of the Project. In order to reach

results/outcomes that would encompass (a) the impact of culture and (b) the social exclusion

aspect in relation to learning of (c) immigrant and Roma adults, the TTMIRO partnership has

adopted a quality assurance approach that run through the 2-year project’s implementation. It

was organized in the realm of a Work Package (WP 3) titled “Quality Assurance” that aimed at

• ensuring quality of the outcomes and deliverables and at

• ensuring the quality of the execution.

For its purposes and taking into account the overall implication of QA , the TTMIRO Project has

defined Quality Assurance as the Processes and procedures for ensuring that qualifications,

assessment and programme delivery meet certain standards. So, one of the goals of the work

done in the Work Package 3 was the design of standards for each deliverable's quality, design of

the tools that were used for the on-going evaluation of the Training Material and the workshops

preceding it. These standards as well as the procedures towards meeting them were depicted in

the Project’s “Quality Plan” that was agreed upon among the Project partners in their first

meeting ( Athens, July 2012) and validated.

Quality Plan

The Quality Plan (www.ttmiro.eu) had a particular role to play in assuring the quality of

implementation and outcomes of the TTMIRO Project. It contains the commonly agreed

description of the Project’s rules and procedures, the application of which would guarantee that

the results of the project have been produced based on high quality standards. Thus, it was used

for the overall monitoring of the work produced during the project’s lifespan in terms of

deliverables quality, timely execution of project tasks, safeguarding of the financial rules implied

by the European Commission and accordance to the terms and conditions of the Grant

Agreement (Agreement n. 2011 – 5000 / 001 – 001). It reflects and records the Project’s course


5


of action on behalf of the contracting authority which is the Education, Audiovisual and Culture

Executive Agency of the LLP EU Programme, called hereafter “the Agency”.

The Quality Plan of the TTMIRO Project was strongly connected to the methodological choices of

the partners. Taking under consideration the innovative character of the Project, the partners

acknowledged its strong research aspect and have decided for a methodology scheme that

includes research features, while at the same time ensuring the validity of outcomes and

providing for risks or dead-ends not unusual in the exploration of innovation. More specifically:

Deriving mostly from the action-research framework, the methodology adopted refers to a

modular, “step-by-step” development of the Project. Each stage and Work Package result to a

specific outcome that serves as the basis for the next one. This served the initial choice of the

partnership, to create the space and the opportunity for end-users, both teachers/trainers and

learners, to get involved in the Project’s development and have a say at its content. So, the

various stages of the Project refer to the development of draft versions of the Training Material

(Curriculum + Lesson Plans) that were tested by end-users. Their feedback was introduced into

the draft version(s) so that the final product included their experience and point of view.

The above choice set the need for assuring the quality of the whole procedure and its results. It

is for this reason that the partners adopted the “on-going evaluation” methodology that allows

for corrections along the way of implementation. The Quality Plan included therefore standards

and tools to be used for the on-going evaluation of the Training Course (January 2013) and the

workshops preceding it, all of them piloting the Training Material and thus ensuring the quality

of the Project’s outcomes and deliverables. The results collected at each stage of the Training

Material’s development supported the partners decision to validate the final product (s).


6


In order to ensure the quality of the Project’s execution the partners have also agreed upon a set

of procedures that referred to the monitoring and actual implementation of the Project, the

results of which are also discussed below. The quality management is therefore in integral part

of the project management process (including evaluation and QA) and played a vital role in all of

the project phases. The partnership has been sensitized that in TTMIRO the transfer potential of

the contributions for the lessons plans are very important as well as the needs of the identified

target groups.

Following the above, the partnership has approved its own quality policy, which was continually

improved and discussed in the group. It has committed itself to the following overarching quality

aspects during the runtime of the TTMIRO Project:

• Quality of the plan of actions: the organisation of the work is clear and appropriate to

achieving the objectives; tasks/activities are distributed among the partners in such a way

that the results will be achieved on time and to budget;

• Quality of the methodology: the tools and practical approaches proposed are coherent,

innovative and appropriate to address the identified needs for clearly identified target

groups.

• Quality of the consortium: the consortium includes all the skills, recognized expertise and

competences required to carry out all aspects of the plan of actions, and there is an

appropriate distribution of tasks across the partners.

The step by step process of the TTMIRO’s final product (Training Material) development

State of Play in the EU + Consultation with Peers, Stakeholders and End-Users =

Curriculum Outline (1st version) Development of 1st Draft Version of Lesson Plans

Consultation with Consultation with Peers, Stakeholders and End-User to evaluate the 1st

Draft Version of Lesson Plans


7


Development of the 2nd Draft Version of Lesson Plans + Curriculum Outline (2nd version)

Modification of the 2nd Draft Version of Lesson Plans and of the Curriculum Outline

(2nd version) into Training Course material

Pilot Training Course Development of final, full edition of the Training Material.

2. Quality Evaluation Methodology

The methods used were the following:

- Text Analysis of the drafts of the final material

- Participatory observation during the TTMIRO Steering Group meetings and validation

procedures

- Implementation of quality standards and materials as per Quality Plan (www.ttmiro.eu).

As mentioned also in the Quality Plan, the evaluation of the Project’s results was organized in

two levels. The first one refers to issues that the partnership considers as overarching both

implementation of the Project and the development of its Final Product/Training Material. The

partnership considered that the qualitative realization of these overarching issues is a

prerequisite for the successful development of the Project’s final product. For this reason, the

following framework standards have been approved to assess the quality of the overarching

issues:

- Participation standards

- Communication standards

- Design control standard

- Documentation standards

http://www.ttmiro.eu/�


8


- Intercultural check-up standard with regards to work flow

- Interface standard

- European added value preciseness and transferability/comparability standard.

It must also be noted that the issues we consider here as overarching reflect the synthesis of

means and objectives that the Project encompasses. For example, the modular structure of the

Project reflects its objective to produce Training Material that would reflect the concerns of both

trainers and end-users. Or, the innovation aspect of the TTMIRO Project led to its development

via research techniques the implementation of which secures that the final product (Training

Material) responds to the reasons that brought the Project into life.

The second level refers to the quality of the Training Material in itself. For its assessment the

partnership has agreed upon the following indicators:

1. Stakeholders input in the Training Material

2. The techniques suggested in the Training Material are effective

3. The Training Material includes topics missing but needed (innovation)

4. The Training Material refers to specific skills and capacities

5. The Training Material improves trainers intercultural capacities

6. The Training Material refers to immigrant and Roma learners.

As mentioned before, the evaluation of the Project’s implementation results from the quality

assessment of its development. It is also strongly connected to the quality assessment of each

final product (Training Material), given the dialectics that informed its design and methodological

choices. There are then some overlapping issues, recognized in the Quality Plan (www.ttmiro.eu)

and presented here under the “overarching issues” and “quality of content” chapters. In its strict

sense, for the quality assurance of the Project’s implementation were organized the following

procedures, agreed by the partners and expressed in the Project’s Quality Plan:

1. updating the objectives of the Project

2. updating the plan of the Project



9


3. describing the management structures of the Project

4. describing the management procedures of the Project.

These were used as quality evaluation standards regarding the implementation of the Project.

Evaluation results according to framework standards: Overarching issues

Participation standards

What is the standard about: As TTMIRO was a very challenging, multifaceted and meta-level

project, there was a specific need to ensure to have the competent members of staff working on

it. The personnel included needed to fulfill the competence requirements with regards to the

addressed training issues, but also the ability and understanding of the core idea of European

work.

Steering Group

The steering group committee is the decision making and taking body of TTIMIRO. The SC votes

and agrees on interim and final results, with regards to content and financial aspects of the work

packages, the overall project management and relevant issues, which might occur. Its members

need to have expertise with regards to project management issues and relevant and sufficient

knowledge with regards to the TTMIRO project.

The members of the steering group committee were:


10


NOSTOS : C. Andritsopoulou

PAKIV : G.Bumbu

PPP : D.Jansen

Y.E.S. Forum : A.Wiedermann

EU WAREHOUSE : K.Weertz .

Evaluation results: The Steering Group members representing each partner are all high-ranked

officials in their organizations; they all have rich experience and know-how on their field of

expertise as well as on management and EU Projects. The members remained the same for all

the duration of the Project. All partners took part in all meetings. The same persons represented

each organization in every meeting.

Also, some of them have worked together before and knew each other. This facilitated the

fulfillment of their tasks and secured a fruitful team spirit during the 2 years of our working

together.

The Steering Group met 6 times, two more than initially planned .

It is important to mention here that the Steering Group’s ability to lead the partnership and its

commitment to the Project were evidenced when the partnership received the 1st year

evaluation by the Agency: all members of the Steering Group took part in the Meeting with the

Agency (Monitoring Visit, Athens, June 2013) and leaded the efforts to meet the suggestions of

the Agency and the Evaluators, judging them important for the Project’s quality (Steering Group

Meeting, Athens, June 2013).

It must also be noted that the Project was disseminated in the realm of each participating

organization. Staff members knew of its existence and followed its implementation. We consider

that the interest raised in the inside of each partner organization resulted from the Steering

Group members work.

Work Group


11


What is the standard about: Quality of the consortium: the consortium includes all the skills,

recognized expertise and competences required to carry out all aspects of the plan of actions,

and there is an appropriate distribution of tasks across the partners.

In the European workgroup partners make sure that the participating cast has the adequate

experiences, skills and competences to work on the project goals, also with reference to the

language ability in order to tackle the various workshops.

Evaluation results: The work group assembled junior and more experienced staff members, thus

ensuring that expertise is combined with novel ideas and a fresh look on the issues the Project

touches upon. They were all competent speakers of English and there was never a need to use

interpretation during meetings or during the Training Course that brought partners and trainers

from all the participating countries together (Athens, January 2013).

Distribution of tasks among partners was aggreed upon when the project was designed –during

the preparation of the proposal. During its implementation there was never an issue raised

regarding this subject. The 1st year EACEA evaluation also recognized the rational distribution of

tasks among partners. The staff involved in the Project had all university degrees, most of them

had also post-graduate ones in fields relevant to the Project’s subjects.

Communication standards

What is the standard about: Communication determines as one vital factor the project

management process. The TTMIRO partnerships committed to

• understanding the need of a transparent and accurate communication, sharing relevant

information and work results in time and adequate quality;

• to the need being precise and of answering in time;

• communication will happen in English: partners communication in a foreign language

should take into consideration, that working in a foreign language has the potential for

misunderstanding (on several levels) the partners oblige themselves to stay open to that

fact and rather ask for clarification (active participation approach);


12


• partners communicating in their native language will actively try to adapt their (written

and oral) communication to the partnership needs and show intercultural sensitivity

through leaving “encouraging spaces” to the non-native speakers instead of taking over

dominant “speaker roles”, specifically in workshop situations.

Evaluation results: The above standards were fully met. Communication in the partnership was

effortless because all persons involved had good command of english language and were also

experienced in working in a transnational environment. The members of the coordinating

organization staff that were involved in the Project worked hard on explaining and clarifying

matters to each partner and to the partnership as a whole. Information flow among partners was

directed by the coordinator and it was clear, consistent and on time. During face-to-face

interactions (meetings, workshops etc) thanks to the experience and professionalism of the

participants, everyone was aware of the fact that communication was conducted in a foreign

language and took special care to understand and be understood. The EU Warehouse

representative Ms K.Weertz that was responsible for Quality Assurance helped a lot on this,

taking special care to remind to all present the need for intercultural awareness and often

mediating so that the topic discussed became clear for everyone present. It must also be noted

that there were never evidenced any instances of conflict or misunderstanding among partners.

Design control standard

What is the standard about: The documents prepared, produced and published would have a

common design. Internal working papers are have to show the data of the organization, the

TTMIRO name, title and project number, the work package and specified task.

NOSTOS would provide the basic framework, derived from the website, poster and pamphlet

design.

Evaluation results: All the above standards were fully met.


13


Documentation standards

What is the standard about: Beyond the “classic” documentation standards defined in the ISO

norm, we consider it an important standard not only to collect the local/regional/national

documents produced and used while working, but also the complementing European documents

with regards to law, policy developments and recent development in VET and LLL. The

documents need to be quoted correctly and made available to the group.

Evaluation results: The above standard was fully met. More specifically:

All papers and products of the TTMIRO Project

• are documented following current academic standards

• were kept up to date as regards developments in policy and adult education theory and

didactics during Project’s life-time

• shed light on aspects of VET and LLL that are related with social issues such as inclusion

and the fight against inequalities, that are only lately gaining recognition in the relevant

field of VET and LLL practices.

These efforts are depicted in the “State of Play” report that attempts to describe the current

situation in the participating countries and in the EU as regards the relation between LLL/VET

and socially disadvantaged groups. This report was constantly revised during the second year of

the Project’s implementation although it was due to be finalized during its first year.

Intercultural check-up standard with regards to work flow

What is the standard about: The partnership had to face a multiplied challenge with regards to

the several work layers of the TTMIRO project in combination with a transnational work

approach. That gives much space for intercultural (and other) misunderstandings, impacting on

the work-flow.

We agreed that we need a multiplied and multifaceted intercultural sensitivity: we have to

consider the necessity to agree on work-steps and make efforts, that we have understood each


14


other within our own (national) work approaches. In order to guarantee work flow, we needed

to be able to rely on agreements we have made on the way and therefore check the agreements

for possible intercultural lapses.

Evaluation results: Acknowledging the need to make permanently and carefully sure that we are

“on the same page” with regards to project goals, target groups and other requirements of the

Project, the partnership validated each outcome of the project’s development. Thus it was

secured that each intermediate outcome would add to the final product and that the additions

would correspond to the goals and requirements set at the phase of the Project’s preparation (as

a proposal) and refined at the beginnning of its implementation. This methodological choice is in

accordance with the Project’s design, that its final product would result from a series of a step-

by-step procedure, structured in stages; the outcome of each stage was used as the basis for the

development of the outcome of the next stage.

In order to address possible “gaps” between national approach(es) of the Training Material’s

content (the Project’s final product), the partnership took special care to (i) discuss thoroughly

and examine the topics that were finally included in the Training Material’s Lesson Plans. This

process took … months and was presented in workshops and meetings of ……… and is probably

the main reason that Work Package 2 (Development) took longer than originally planned (ii) to

validate the step-by-step outcomes, as mentioned also above.

It must also be noted that the inputs the partnership received from trainers, social workers and

end-users during the preliminary stages of the Project’s development, coming from 4 different

european countries converged on the same issues. These were consequently the ones included

in the draft Curriculum (WP1). The same goes for the evaluation results of the Project’s next

stage of development , provided by trainers, social service staff and end-users (WP2) (see also

the reports at www.ttmiro.eu).



15


Interface standard

What is the standard about: to relate work/deliverables/outcomes to the world of Training /

work, to relate work/deliverables/outcomes

One of our most challenging standard was the “interface standard” with regards to interface

management. The lessons plans to be developed as well as the quality event needed to

permanently bring horizontal and vertical issues and results together.

The partners had to permanently reflect on the need to relate and transfer to: The project goals,

the topic, the target group(s), a European/national/regional/local level, the transfer from the

“theoretical sphere” to the project sphere and the demands and needs of the target group(s) as

well as references to the possible inputs of the social services.

Evaluation results: The above demanded a great level of structure and clarity throughout the

project process. The partners were aware from the beginning that their national inputs needed

to be transferred on meta-level and from there again transferred into a common outcome. That

is why they chose for the development of the Project's final product to be based on the feed-back

received by its end-users; their active involvement in every stage of the implementation was embedded

in the Project's realization, given its step-by-step design. Also, all the partners provide trainers training,

VET, counselling and orientation to mainstream as well as to migrant and Roma population. Hence, the

Project combined their expertise to its objectives aiming at a bottom-up approach in the development of

its final product, i.e. Training Material. This modular and bottom-up approach secured that the content of

the Training Material reflected concerns and needs of the target group(s) and incorporated inputs by the

social services as documented in the various workshops and events organized in the realm of the Project.

Work and Steering meetings served as connection points/interfaces. During them the partners discussed

and decided on the feed-back that corresponded to the Project’s goals and aligned it to the content of the

Lesson Plans, thus transcending national or local particularities.


16


European added value preciseness and transferability/comparability standard

What is the standard about: This very unique and outermost standard refers to the need of

permanent work awareness that things have to be delivered into order to achieve European

added value. The partners needed to adapt their national work or particularities and reflect upon

results of each Project’s stage in order to achieve European added value. All the work presented

had to be permanently transported to meta-level and referred back to the aims described in the

TTMIRO project.

Evaluation results: The TTMIRO Training Material consists of (a) short descriptions of well

established and academically sound theoretical approaches that are recognized world-wide (b)

short argumentation on how these theoretical topics are treated in the realm of the TTMIRO

Project and (c) original exercises and techniques constructed in the realm for the Project’s

purposes. The material under b and c was included in the final product after deliberation with all

partners, thus ensuring reflection upon the suggestions that each partner offered. Thanks to the

modular structure of the project’s implementation plan, it was secured that the final training

material reflects a synthesis that overcomes national particularities.

Text analysis has shown that the 4 Modules of the TTMIRO Training Material do not refer to any

national institutional or other framework; they are applicable in any learning circumstances

where members of culturally diverse groups co-exist with learners that are carriers of the

dominant culture. This happens in adult classrooms all over Europe. Also, the content of the

Modules focuses on factors that apply to everyone but that are rarely addressed in training. By

focusing on the “intra” and tackling issues of self-awareness and the legacy of experiences and

on the impact of social factors on learning conditions and circumstances, the partnership has

added to an important european debate.

The techniques that accompany the short description of each Module item are absolutely

transferable in every kind of adult learning environment that is culturally “mixed” because their

implementation does not depend on national particularities neither needs adjustment according


17


to institutional prerequisites that vary per country. Their development took into account the

points and the degree of convergence among EU member countries as regards VET/AE/LLL (see

also State of Play revised report 2013), while it is also open-ended in the sense that trainers of

trainers can modify the suggested techniques according to their particular needs and choices.

Content Quality Assessment


18


Overall result: The Final Product of the TTMIRO Project, that is the Training Material available at

www.ttmiro.eu in english, greek, german, romanian and dutch, corresponds to the objectives set

for its development (as per initial proposal and grant agreement). More specifically:

- The Training Material consists of a curriculum and lesson plans that focus on the relation

between inequalities and culture as this is expressed in the realm of adult classroom. The

concepts of identity (micro-level, individual) and social exclusion (macro-level, society)

were applied in order to provide linkages between theory on macro-level (inequalities)

and interaction on micro-level (classroom).

- To provide these linkages the partnership explored the impact that social factors have on

learning processes through the lens of culture. The concepts of culture and cultural

identity served as the “red thread” that holds together the issues that the TTMIRO

Training Material touches upon.

- The TTMIRO Training Material targets the intercultural capacities of the trainers; it

upgrades them by adding to their content the importance of social factors such as the

reproduction of social inequalities on the micro-level of face-to-face interaction

- The end-users (learners and trainers) concerns are reflected in the content of the Training

Material following a series of consultation and dissemination activities that attracted

interest and input from all participating countries.

- The modular structure of the Project and its step-by-step development allowed for

constructing the conceptual framework of the Training Material and sharpening the

argument that the TTMIRO project adds in the current agenda on the quality and

inclusiveness of VET/LLL in Europe.

- The Final Product is divided in Modules and Module Items that describe a full course of

approx. 20 hours of teaching. All Modules follow the same format and include a short

description that documents the relation between culture and identity as experienced in

adult learning interaction, description of methods/techniques that includes 25

practical/experiential exercises and workshop scenarios, accompanied by instructions

and tips for the trainers of trainers that will use them.



19


More specifically:

Evaluation results according to indicators for the quality of the Training Material’s content

Indicator 1. Stakeholders input in the Training Material

Indicator 1.1. What is the indicator about: Adult education teachers and trainers, social service

counselors, coaches and social workers, Immigrant and Roma end-users offered input for the

issues to be included in the Draft Curriculum (WP 1).

Evaluation results: To achieve the participation of trainers, social service staff and end-users the

partners organized workshops and face to face meetings, including in some cases self-

completion of questionnaires. This work is part of the Work Package 1 and lasted from February

to April 2012. Its scope was to get the stakeholders contribution in investigating the relation

between VET and social inclusion. The issues discussed followed a pre-designed interview tool

(published in the “Consultation with Peers, Stakeholders and End-users” report, www.ttmiro.eu)

that was used as a guide to help the interviewer not lose sight of the topic at hand. The tool was

translated in each partner’s language and was accompanied by an ice-breaking scenario,

designed by the the partners in order to facilitate the trainers participation in the workshops.

Input was provided by

• Adult education teachers and trainers that had experience in teaching “mixed” classes:

20 Vocational Education teachers from Further Education Colleges in London, UK

22 teachers and trainers in Athens, Greece

30 teachers and trainers in Alba Iulia, Romania

20 adult teachers and trainers in Brussels, Belgium

• Social service counselors and social workers involved in services for migrants and Roma:

10 social workers in London, UK



20


10 social workers and coaches in Athens, Greece

30 social workers in Alba Iulia, Romania

10 social support professionals in Brussels, Belgium

• 35 Migrant and 50 Roma end-users in London, Athens, Alba Iulia and Brussels that had

participated at least once in a VET or LLL course.

The partnership handled their input as follows:

o Adult education teachers and trainers seem not to be familiar with sociology of

education findings that explain why and how social inequality is related to

education in general. This finding corresponds with findings of the state-of-play

survey that the partnership has conducted (see also www.ttmiro.eu). Using

mostly the pedagogical lens, the trainers often tend to ignore the impact of social

factors on learning processes. In trying to address these issues the partnership has

shaped the concept from which the Training Material derives and formulated its

argument (see the Training Material edtion at www.ttmiro.eu) .

o What trainers define as “problems” that they believe they have faced when

teaching “mixed” classes, for instance lack of motivation on the part of immigrant

or Roma trainees, may be the result of different perceptions of how motivation is

expressed, in relation to cultural codes and culturally regulated behaviours. It is

for this that the TTMIRO partnership has decided (London meeting, October 2012)

to include in the lesson plans material on the intercultural approach, emphasizing

its interplay with social factors. The relevant processes that partnership finally

reached are included in all the Modules of its final product (Training Material,

www.ttmiro.eu), functioning as the “thread” that holds together all parts of the

Training Material.

o Trainers expressed their interest to find out more about their immigrant and

Roma learners social background and history. For this reason the partnership has




21


added in the Curriculum its approach on how social and historical factors may be

reflected on the individual (Module 1.3.) and enriched the practices/techniques it

proposes with this perspective (Modules 3.1. and 3.2. See for more details

Training Material, www.ttmiro.eu).

o By voicing the concerns of the end-users of their services social service

counselors and social workers are closer to the social exclusion problematic and

arguments. The social exclusion aspect and its practical consequences on the

individual is something that social service counselors and staff in general are

familiar with. This makes them better equipped to explain behavioural patterns,

for example why the immigrant or Roma learner’s behaviour may be interpreted

by the trainer as less motivated when compared to the native learner. In order to

familiarize trainers with this approach and thus provide them with a tool to

interpret the learner’s behaviour or attitude, the Lesson Plans include the

theoretical argumentation and notions on the impact of social exclusion on the

individual learner both in the Project’s argument and in Module 3 (See for more

details Training Material, www.ttmiro.eu).

o Social services professionals remarked that in most cases information on the

beneficiary’s profile and needs is not transferred to the training organization,

even when the beneficiary/learner is a disadvantaged group member, directly

referred to the VET organization by the social service. The TTMIRO partnership

included this observation in its work, on the argument that the more the trainer

knows about the social profile of his/her learners, the more prepared he/she is to

explain and understand their behaviour. TTMIRO linked this with the prevailing in

education idea of equal treatment and relevant practices. It is for this reason that

the TTMIRO final product emphasizes the need that the trainer should

“individualize” his/her strategies (Modules 3.1. and 3.2.) in connection to the

learner’s social identity (Module 1.3.), also supported by the trainer’s own self-




22


awareness and reflexivity (Module 2) (See for more details Training Material,

www.ttmiro.eu).

• Immigrant and Roma end-users that have attended at least once a VET or LLL

program offered in the training material’s content their own explanation on how

they experience their attending “mixed” courses. Their input made us think that

what is often interpreted by trainers as a need for motivation could reflect

difficulties of the person to adjust to the “western way” of learning (and

teaching); at the same time the assumption that this “western way” of learning is

the only way of learning shifts the focus from social factors to cognitive, i.e.

individual ones. This is why TTMIRO included intercultural competences in its

Training Material (Module 1.2), as well as class management techniques (Module

3.2.) that support a respectful and tolerant learning environment, while at the

same time giving to the trainer the hunch to self-examine his/her own feelings

towards what is “right” or “wrong” (Module 2.1.) (See for more details Training

Material, www.ttmiro.eu).

o Immigrant and Roma end-users stressed the need for complementary measures

during VET courses, such as language support courses. They expressed the view that

the lessons they take part in should not “waste time” on “theories” but rather be job-

oriented. TTMIRO discussed these ideas with the trainers that took part in the Athens

Training Course that piloted the Training Material who agreed that this is a usual

concern of their learners. So, the TTMIRO included these remarks in the “Beyond the

Classroom” paper it has prepared (www.ttmiro.eu).





23


Indicator 1.2. What is the indicator about: Adult education teachers and trainers, social service

counselors, coaches and social workers, Immigrant and Roma end-users offered their evaluation

on the the 1st draft version of the lesson plans that TTMIRO project had produced up to this

stage of its development, so that their remarks and input would be included in the TTMIRO

Training Material

(WP 2).

Evaluation results: In order to achieve active involvement of stakeholders in the Project’s

content and at the same time ensure that their voice will be heard TTMIRO planned a series of

workshops and consultation meetings with adult education teachers, trainers, migrant and Roma

learners and social services counsellors in 4 European cities as part of the Work Package 2

(November – December 2012).

The meetings followed the same agenda that included

-a short presentation of the Project and its rationale

- a short introduction on the objectives of the meeting

- a presentation of the Lesson Plans (that is the outcome prepared at that stage of the

Project’s development)

- discussion and exchange of ideas, brainstorming following a pre-design interview

instrument (see also the Report on Evaluation Feed-back workshops and evaluation meetings at

www.ttmiro.eu) .

In the meetings and workshops took part

-44 Adult education teachers and trainers

-36 Social service counselors and social workers involved in services for migrants and Roma

-12 Migrant and 6 Roma end-users.

The partnership handled their input as follows:

o Adult education teachers and trainers found the Lesson Plans’ objectives relevant to



24


their work; they agreed that the topics reflect problems they have encountered when teaching

“mixed” classes

o Adult education teachers and trainers stressed the need for “grounding the theory”

and providing practical solutions to the problems they encounter in the classroom. Following

their suggestion, the partnership worked towards

- focusing the Training Material on what happens in the classroom (see for example

Module 3 of the final product)

- providing tips and practical advice in each Module they included in the Training

Material. For this all Modules follow the same format and the content of each one corresponds

to the pre-set format/prerequisites described during the Project’s development , for example

short description of the content item, methods to be used, etc.

- developping practical examples and “doable” exercises. So, the Training Material

includes 25 exericises and workshop scenarios along with instructions for their implementation.

o Social service counselors, coaches, social workers agreed that the topics suggested in the

Lesson Plans reflect existing issues arising when immigrant or Roma persons attend the same

course with the mainstream population. They stressed the importance of the social inclusion

approach reflected in the TTMIRO material.

o Social service professionals suggested the practice of the “contract” between the service

and its client as important for identifying and also mirorring the needs of the beneficiary/client.

Connecting this with the suggestion that the needs of the learner should be known to the VET &

LLL teachers, TTMIRO partnership developed the “learning contract” technique (Module 3.2.)

(See for more details Training Material, www.ttmiro.eu).

o Immigrant and Roma end-users acknowledged that the topics suggested in the Lesson

Plans reflect existing issues arising when immigrant or Roma persons attend the same course

with the mainstream population. Their input confirmed the TTMIRO’s assumption on the

importance of transforming the principles of intercultural educational approach into specific



25


techniques that can be applied in an adult classroom (see for more details the Report on

Evaluation Feed-back workshops and evaluation meetings at www.ttmiro.eu) .

Indicator 1.3. What is the indicator about: Adult education teachers and trainers, trainers of

trainers and end-users offer input to the Training Material’s final version (WP 2).

Evaluation results: 25 adult and vocational education trainers and trainers of trainers from the 5

partner countries (WP2, January 2013), tested the Training Material in a 5-days Training Course,

being at the same time learners and evaluators of what they learned. The realization of the

Training Course was a great encouragement for the partnership due to the warm interest and

the comments offered by the participants. It confirmed that the content of the Training Material

is useful for the training of trainers and that it actually offers new aspects in dealing with

multicultural groups of learners. Most of the participants remarks were included in the final

version of the Training Material, especially regarding the practicality of exercises and workshop

scenarios and the instructions to the future users, The Training Course gave to the participants

the opportunity to express their views on how the inclusion aspects of VET provision could gain

from its closer cooperation with social services. The partnership has processed these views into

the “Beyond the Classroom” paper that is published at the Project’s site and also served as a

means to further promote the need for more inclusive policies on national and European level

(WP 4.Dissemination).

Also 16 trainers and end-users, as well as high-rank European Committee officials took part in

the Multi-Person Quality Event (WP3, March 2013) evaluating and reviewing the Project's final

product. Their input had helped the partnership refine the relation between social exclusion and

social identity. It had also stressed the need for measures that will allow for “tailor-made’

approaches corresponding to assessment of needs combining the learning ones to the “profile”

of the individual learner. Thus, the need for cooperation between VET/LLL organizations and

social support services became highlighted.



26


Indicator 2. The techniques suggested in the Training Material are effective

What is the indicator about: The TTMIRO Training Material was developed in response to specific

needs that the partnership has identified early on. Actually, these needs were the motive for the

Project’s proposal. It was therefore imperative that the outcomes and final product

corresponded to the needs that brought the project into life.

Evaluation results: Adult education teachers and trainers that took part in the successive stages

of the Project’s development confirmed that the Training Material corresponded to its pre-set

aims. Also, the content of each Module (that the Training Material consists of) corresponds to

the pre-set aims for this Module. The suggested techniques per Module have been piloted

during the Athens Training Course (January 2013) and have been positively evaluated by the 25

trainers and trainers of trainers that participated in it. Also, their feed-back was included in the

final version of the Training Material (see also Training Material, www.ttmiro.eu).

The content of the Training Material takes into account the need that trainers expressed to

motivate and engage the learners coming from a culturally diverse group. TTMIRO was centered

on the impact of social factors on the disandvantaged groups’member learning and performance

in the classroom. In order to provide the trainers with tools that would help them gain

effectiveness in their teaching “mixed” classes, it addresses the impact of culture on identity in

the realm of the social interaction that includes the power game of social forces as it is

experienced in the classroom, proposing techniques that are interactive (as suggested by the

liberal adult learning approach. See also below) and at the same time respectful of the varying

perceptions of the so-called “modern” pedagogy that may embarrass learners coming from

different cultures than the North-western one.

On a more technical level, the initially proposed time for the teaching of each Module has been

revised, further to remarks of participants (trainers acting as trainees during the piloting of the



27


Training Material in the Athens Training course) and in vivo observation of their implementation,

so as to be applicable in real class environment.

Lastly, the TTMIRO Training Material adopts the liberal adult learning approach that is

academically acknowledged as the most effective for the goals of the TTMIRO Project. This

approach encourages critical thinking as both a means for and a result of learning. Furthermore,

all of the Project’s educational activities are organized so as to be learner-centric, with the

teaching person in the role of facilitator. The techniques and exercises that the Training Material

suggests are all based on group-work; there is also a special chapter (Module Item 3.3.) that

refers to the Project Method that is according to literature (see also Training Material at

www.ttmiro.eu) one of the most effective methods for teaching adults, especially persons that

have been out of education for a very long time or are intimidated by it, as is often the case with

immigrants or Roma. Also, the partnership had foreseen that participants in the piloting activities

of the Project would successively take up the role of learner and teacher, a practice seen here as

a method for upgrading their teaching skills by acquiring a better understanding of their tasks

and relevant skills. The same practices are encouraged in the Project’s Final Product in order to

secure its effectiveness.

Indicator 3. The innovation aspect: the final material includes topics missing but needed

What is the indicator about: The TTMIRO Project set as its starting point the exploration of a field

that is strongly connected with innovation: its development was based on the assumption that

linkages between VET and social services are currently missing. If they existed though, then the

social inclusion aspect of VET and LLL would be stregthened for the best interests of the socially

disadvantaged group members. To achieve the above, the TTMIRO partnership needed to focus



28


on topics missing up to now in the training of trainers available material but at the same time

related to the argument that the partnership gradually developed.

Evaluation results: The work that the partners conducted in the realm of Work Package 1 in

exploring the current state of play in their countries and in the EU (see the “State of Play” report

at www.ttmiro.eu) revealed that there is a rising research/academic interest in the inequality

aspects of VET provision, related to the impact of culture, family background and cultural capital.

This confirmed the partners’ initial assumption based on their common experience, that the

impact of social factors on learning processes is missing from the training of trainers. The

exploration that the partners proceeded to reveal the importance of to socio-economic

conditions, as well as of cultural ones on the learning outcome, while the material and resources

available for the trainers that wish to act upon what is reflected in their everyday work as

“difficulty” or “problem” is scarce.

The above were confirmed in the consultation meetings with stake holders and during the

piloting “Athens Training Course” (January 2013).

Another important aspect of the innovative nature of the TTMIRO Training Material is that it

focuses on what happens in the classroom attempting to provide linkages between theory on

macro-level (=inequalities ) and interaction on micro-level (=classroom). As reported in social

theory and social studies epistemology, the exploration of such linkages is at the forefront of

relevant research.

Indicator 4. Trainers skills and competences.

What is the indicator about: The lessons chapters/Modules that the Training Material consists of

are referring to different aspects of the interaction between trainer/teacher and the target

groups (migrant and / or Roma participants). The question then arises, which aspects exactly



29


does each Module address (a skill, a competence, the organizational level). Is it referring to an

ability, a skill or a compentence? Here competences are defined as the ability to use and

integrate knowledge, skills and attitudes to realise specific goals. By using the term attitude, it is

made explicit that teaching / training can/should have an important intrapersonal / intercultural

component.

Evaluation results:

MODULE 1 – CULTURAL DIVERSITY: SETTING THE FRAMEWORK (THEORETICAL INTRODUCTION)

provides a theoretical approach and conceptual tools that serve as common ground among the

trainers/trainees that would participate in a training course implementing the TTMIRO Training

Material. Its objectives are to familiarize the trainers with understanding aspects of their own

intercultural competence and upgrade them.

MODULE 2 – SELF AWARENESS OF THE TRAINER OF CULTURALLY DIVERSE GROUPS provides a

set of practical/experiential examples on Self-Awareness and Reflexivity. These focus more

directly on the development of skills and competences of professionals who are working with

vulnerable groups. The aim was to provide them with a set of skills that they can apply their own

experience to which will help them to better support their students.

MODULE 3 – IMPROVING LEARNING PROCESSES FOR CULTURALLY DIVERSE GROUPS IN ADULT

LEARNING (OUTCOMES) focuses directly on competencies and skills upgrading; it proposes

specific techniques covering aspects of the everyday interaction in the classroom, bringing into

the training of the trainers some of the knowledge and know-how of social services, adjusted to

the educational requirements of adult teaching. These refer mainly to the impact of inequalities -

that is a macro-phenomenon, on the micro-level of everyday interaction in an adult classroom.

MODULE 4 – CROSS CUTTING ISSUES (IMPACT OF CULTURE) AND EVALUATION refers to skills

that would support the trainer’s capacity to self-evaluate his/her work and have access to


30


resourses in order to develop his/her own training material or expand and adjust to specific

needs the existing one.

Indicator 5. Improvement of intercultural competences

What is the indicator about:

The conceptual framework and the argument built by the TTMIRO partnership are directly

related to the intercultural competences of the teachers/trainers in VET, Adult Education and LLL

in general. Having acknowledged this, the partnership has set the discourse on culture and

identity in the centre of its theoretical approach and aimed at reflecting this into the Modules

that consist the Training Material.

Evaluation results: The content of the TTMIRO Training Material as well as the specific aims of

each of its Modules refer to various aspects of intercultural competences in relation to TTMIRO’s

concept and approach, according to which these competences should include the social impact

of culture and identities in the adult classroom and techiques/practices on how to address it.

More specifically:

The theoretical framework of identity, stigma, prejudice and stereotyping described in Module 1

reflects the cognitive background of intercultural competences. It also informs the practical

examples and exercises in the whole Training Material as well as the practices described in

Modules 2 and 3. This information provides the basis for the development of the TTMIRO

approach that is further elaborated in the next 3 Modules and summarizes the academic

consensus in the field. It also contains information on aspects of immigrant and Roma identity

that are a pre-requisite for better understanding attitudes of the group(s) that trainers come in


31


cooperation with when in “mixed” classes. All the above are part of the conceptual framework of

the intercultural competences.

The practices of self-awareness and reflexivity included in Module 2 contribute to the

improvement of intercultural competences for targeted trainers by providing them with the

tools to begin understanding the historical effects of prejudices and stereotypes on our own

personal legacies and experiences and how these in turn affect the way in which we view the

world and our approach to teaching others, and in particular minority and vulnerable groups. By

beginning to understand this, the trainer will be able to better support the development of their

student’s identity as well as their skills and competences.

The content of Module 3 on and practices of class management and interaction between trainer

and learner(s) provide to the trainers the tools to handle everyday situations in “mixed” classes

that add up to intercultural competence

Module 4 outlines suggestions for trainers of trainers and trainers/adult education teachers on

how to build upon the knowledge they acquire in the field of intercultural competences.

Indicator 6. Relation of the Training Material content to the target groups of migrants and

Roma.

What is the indicator about: The TTMIRO Project was designed for the purpose of exploring the

inclusion aspects of VET and LLL as regards the social groups that mostly need inclusion policies

on the one hand. On the other, the design and the development of the TTMIRO Project took into

account the effects of social inequality on social cohesion. For all these reasons it focused its

development on immigrant and Roma population that takes part in mainstream VET programs

and looked forward to investigating gaps and divergences that may hinder the inclusive quality

of VET programs.


32


Evaluation results: The content of the TTMIRO Training Material addresses the target groups

(immigrants and Roma) in the following ways:

(a) its starting point and main objective is to enhance the chances of migrant and Roma adult

learners to benefit from their participation in VET and LLL programs. The TTMIRO Training

Material is in this sense an intervention towards the training of trainers so as to include in

it the impact of social factors (identity, culture and its role etc) on the learning outcome

of learners coming from disadvantaged social groups

(b) all the Modules of the Training Material are oriented towards enhancing the trainers’

skills and capacities to address a “mixed” or multicultural class, where migrants and

Roma learners co-exist with mainstream population. The lens that the TTMIRO used for

this is the intercultural approach, emphasizing the role of culture in the reproduction of

social inequality, the effects of which are imposed on disadvantaged groups

(c) Module 1.3. includes information on how stigma and prejudice has affected the identity

and the perception of the identity of immigrant and Roma groups in the western world

(d) all the Modules’ content derived from the end-users comments and feed-back, received

in various stages of the Training Material’ s development (see also Indicator 1 above). So,

the concerns and needs of the target groups are directly reflected into the Material,

besides the input offered by social workers and counselors involved in services for the

specific target groups.


33


Evaluation of the Quality of Implementation Procedures

The Quality Assurance System that the partnership adopted (Quality Plan, www.ttmiro.eu)

consisted of four (4) main sections:

1.updating the objectives of the Project

2.updating the plan of the Project

3.describing the management structures of the Project

4.describing the management procedures of the Project.

These were used as quality evaluation standards regarding the implementation of the Project.

More specifically:

1. The updating of the objectives of the Project involves monitoring of its operations and

their adjustment according to new needs that may arise.

Evaluation results: The partnership has accomplished the objectives set at the beginning of the

Project and produced the number of outcomes that was originally defined. The outcomes were

produced in accordance to the step-by-step, modular development of the Project’s Training

Material. The implementation of this methodology also assured the active involvement of peers

and stakeholders in the content of the Training Material through workshops, consultation and

other meetings, the pilot Training Course (Athens, January 2013) and the Quality Event (Brussels,

March 2013). So, this objective was also realized.

The Project’s objectives were updated exclusively in response to the input that peers and

stakeholders provided. It is our estimation that it was channelled into the training material’s

content in correspondence to the Project’s original aims and goals.



34


2. The updating of the plan of the Project refers to the managerial ability to monitor and re-

adjust operations so that the Project is realized in compliance to the set standards. It is

related to the actual description of its phases as regards the following:

the Objectives

the actions

the necessary resources

the timetable

the deliverables of each Action.

Evaluation results: There was no deviation as regards the objectives, the actions and the

deliverables of each action in the implementation of the Project. In retrospect, it seems that all

partners were not equally familiar with the Project’s modular structure and the action-research

techniques it involved. It also seems that the partnership had overestimated the readiness of

each partner to apply such methods without a formal kick-off meeting. The partnership took

though corrective measures as soon as this problem was realized, following the results of the 1st

year EACEA evaluation and the monitioring visit comments (EACEA/P3/SL/F2/13DO19702, June

2013), as well as observations of the partners themselves in the realm of the quality assurance

system that the Project had adopted. Among these measures we mention here the following:

- the partners decided in January 2013, as a result of the input received by trainers in the

Athens Training Course that they should work harder on reaching officials of the official

system for training trainers on national and European level, in the realm of the

dissemination activities (WP4). They had succeeded in raising interest in the Project by

inviting high rank officials in the Brussels Quality Event, networking and disseminating on

national level and presenting the Project and its results to CEDEFOP officials

- the partners decided and committed themselves to follow the recommendations

suggested by the EACEA after the monitoring visit


35


- the partners took particular care to explain the “work-in-progress” character of the

Project in order to attract and encourage the active involvement of peers and

stakeholders in its development through their dissemination and exploitation activities

(WP 4 and 5).

Also, the monitoring of the implementation procedure was done via the This allowed for

immediate actions and a reasonable degree of flexibility, so as to prevent any divergence from

the original plan and objectives.

As regards the management structures of the Project and its management procedures, these

were described in the Project’s Guidelines, the Project’ s Quality Plan and Work Plan, made

known and approved by all partners.

training trainers for migrants and roma grundtvig ... · wp 3 quality assurance final evaluation...

Documents