live&learn issue 31

24
Making better qualifications – conference report | 08 Tackling unemployment the active way | 14 Framing future skills with foresight | 18 My vocation – a Moroccan sous-chef in Italy | 20 Ukraine How to build skills for new times Report from Kiev ISSUE 31 – JUL 2014 NEWS AND VIEWS FROM ACROSS THE ETF COMMUNITY Live & Learn

Upload: european-training-foundation

Post on 01-Apr-2016

224 views

Category:

Documents


0 download

DESCRIPTION

Live&Learn, the quarterly magazine of the European Training Foundation (ETF), brings stories about vocational education (VET) and employment from the countries neighbouring European Union. In this issue: focus on Ukraine, report from ETF conference on qualifications, an explainer on active labour market policies, and more.

TRANSCRIPT

Page 1: Live&Learn Issue 31

Making better qualifications – conference report | 08

Tackling unemployment the active way | 14

Framing future skills with foresight | 18

My vocation – a Moroccan sous-chef in Italy | 20

Ukraine How to build skills for new times

Report from Kiev

ISSUE 31 – JUL 2014NEWS AND VIEWS FROM ACROSS THE ETF COMMUNITY

Live&Learn

Page 2: Live&Learn Issue 31

Live&Learn July 201402

INSIDE04 Country focus: Ukraine

08 Focus on better qualifications

10 Social inclusion and vocational schools

12 Faster, better, different – Innovation at the ETF

16 Reforming governance of vocational education and training in the Mediterranean

14 Tackling unemployment the active way

18 Framing the future with foresight

20 My vocation: Between literature, couscous and pizza

CONTACT US

Further information can be found on the ETF website: www.etf.europa.eu

For any additional information, please contact:

Communication Department European Training Foundation ADDRESS Villa Gualino, Viale Settimio Severo 65, I – 10133 Torino, Italy TELEPHONE +39 011 630 2222 FAX +39 011 630 2200 EMAIL [email protected]

To receive a copy of Live&Learn please email [email protected]

The European Training Foundation is the European Union’s centre of expertise supporting vocational and training reforms in the context of the European Union’s external relations programmes.

ISSN: 1725-9479 @ European Training Foundation, 2014

Cover photograph: A Ukrainian mother wearing the national flag holds her daughter wearing an EU flag as they attend a gathering in the iconic Maidan or Independence Square in Kiev, to celebrate the signing of a historic EU-Ukraine association agreement, on 27 June 2014.

GENYA SAVILOV/AFP/Getty Images

Please recycle this magazine when you finish with it.

22 New publications & Digital update

Page 3: Live&Learn Issue 31

Live&LearnJuly 2014 03

IN ThE STArTINg blOCkS

Bent Sørensen Head of Communication

Editorial

The 2014 European elections are over and 766 new and re-elected MEPs will be moving to Brussels shortly to begin their busy term. Many pressing challenges are ahead of the new Parliament. First and foremost the need to find a way out of the economic crisis, and a focus on how to ensure future growth through research, education and flexible labour markets.

The EU cooperation and neighbourhood partnerships can facilitate the creation of more jobs and strengthen EU efforts to promote peace, democracy and prosperity inside and outside Europe.

The role of the newly elected European Parliament is to maintain and further develop the programmes and instruments to support these developments. Education and training are key to all the EU external policies and it is important to keep developing human capital by focusing also on entrepreneurship, creativity and innovation.

The ETF looks forward to contributing to the work of European policy makers on these issues by further strengthening the relationship with MEPs, their committees and secretariats in the coming years.

However, not only is the European Parliament in the starting blocks. On 1 July 2014, Italy, the home country of the ETF, took over from Greece the six-month rotating presidency of the Council of the European Union with an ambitious agenda focusing on growth and employment. These topics are at the core of the ETF’s work and strategy, and

a number of meetings and joint activities with the Italian EU Presidency are foreseen in the coming months.

A new, political season in Europe has begun. ■

Italian Presidency of the Council of the European Union

Page 4: Live&Learn Issue 31

Live&Learn July 201404

A new opening, a fresh crisis, the old problems – as Ukraine is reliving its defining moments and the world is looking on, Live&Learn reports from Kiev, the country’s capital, on how people working on education and training reform try to keep it on policy makers’ busy agenda.

The windows in Viktoria Karbysheva’s office face one of the most eminent public places in Europe – Maidan Nezalezhnosti or Independence Square. Karbysheva is deputy director for vocational education and training at the Ukrainian Ministry of Education and Science.

Since November 2013 Kiev’s Maidan has been the focal point of the political life in Ukraine. Tens of thousands of people came here to demand political changes. They camped for weeks, they fought and died, and they overthrew the president. More than six months since the start of the demonstrations, behind barricades of car tires and in military tents, many protesters remain on the Maidan, keeping the revolutionary pressure on the new government.

Education reform needs politicians

But Kabysheva, a civil servant, tries not to talk about politics. ”Politicians do their work, and we do ours. It’s not easy now, priorities may change, but in principle

we all want education to develop and to respond to the demands of employers.”

Kabysheva believes Ukraine needs the politics, or political will, for the education reforms. “Politicians should be interested in this subject. In Ukraine there is not a big enough [qualified] workforce. We have too many higher education graduates who can only find work as simple workers.”

When asked about the problems of vocational education and training, Kabysheva names two issues: weak links with employers, and lack of investment in schools. The key to solve both of them may well lay with a better involvement of business.

Skills: not a side issue

Oleksiy Miroshnichenko, Vice President of the Federation of Employers of Ukraine, a powerful organisation representing Ukraine’s industry and business, says people’s skills are not a

side issue, even in times of crisis. His organisation has been instrumental in the ETF-supported setting up of a national qualifications framework.

“The question of education is always important for business and for Ukraine,” says Miroshnichenko. “Indeed, Ukraine is experiencing a crisis, political crisis in the east, annexation of Crimea, economic crisis. But business trusts in the future of Ukraine, and the future is based on two things: stability and education.”

The economic growth (2% in 2013) and unemployment statistics (8% in 2013) are not alarming at first sight, but if you look at the past trends and forecasts the picture becomes bleak. Ukraine’s economy today is just half of its size in 1992, and experts say many industries would shed thousands of jobs if not for the subsidies and regulation.

The Torino Process report, written by a team of Ukrainian experts, notes “a shrinking and aging population, a high

UkrAINE: SkIllS FOr NEW TIMES

Country focus Ukraine

Page 5: Live&Learn Issue 31

Live&LearnJuly 2014 05

level of premature mortality amongst people of working age and an increased migration outflow of wage workers with vocational and complete higher education…, and the decline in prestige of working trades has resulted in a shift from the training of skilled workers to the production of specialists with higher education.”

Vocational schools network – potential and drag

According to the data from the education ministry, in 2012 there were 990 vocational schools in Ukraine with 48,880 teaching staff providing education or training to some 380,000 students. This is an impressive network, but why can’t it meet the market’s demand? The same report says only 25-40% of the industry’s need for skilled workers will be met in 2015.

In the past two years ETF experts have worked with Ukrainian counterparts on two broad issues apart from the overall analysis of the system through the Torino Process: reform of qualifications and validation of non-formal and informal learning, and anticipation of skills needs and rationalisation of school network.

“This big network is really a problem, because [schools] have so limited resources,” says Timo Kuusela, who manages ETF activities in Ukraine. “If you cannot maintain schools and provide them with proper resources, you’re making the system worse.”

In Dnepropetrovsk, an important industrial centre in east of Kiev, the ETF had some success at the local level in aligning vocational school network to the anticipated skills demand.

“Ukraine has traditionally had a strong education system, with lots of institutions, the Academy of Pedagogical Sciences, but the problem is they were not able to ensure quality,” says Kuusela. “It has deteriorated because of the lack of resources and lack of decisions on systemic reforms.”

Qualifications, legislation, skills anticipation – ETF’s targeted support

So what is the future for the ETF’s work in the country? Kuusela says the ETF will urge Kiev to reform institutions and advocate more investment in education and training. At this stage the ETF’s limited resources will go to

support the implementation of a national qualifications framework, modernising legislation and anticipating skills demand. This is where the Ukrainians have an interest, and where the ETF’s targeted actions can make a difference. ■

Text and photos: Marcin Monko, ETF

The famous Maidan Nezalezhnosti or Independence Square is where the vocational education and training department of the Ukrainian Ministry of Education and Science has its offices.

Ukraine’s economy today is just half of its size in 1992, and experts say many industries would shed thousands of jobs if not for the subsidies and regulation.

Key facts

On 27 June 2014, Ukraine, along with Georgia and the Republic of Moldova, signed an Association Agreement with the EU. The pacts grant access to the EU’s internal market and encourage cooperation across a wide range of sectors.

Population: 44.3 million (2014 est.)

Area: 603,550 sq km

Capital: Kiev (Kyiv)

Main ethnic groups: Ukrainian 77.8%, Russian 17.3%

GDP: $337.4 billion (purchasing power parity, 2013 est.)

GDP per capita: $7,400 (2013 est.)

Unemployment: 8% (2013 est.)

Page 6: Live&Learn Issue 31

Live&Learn July 201406

With political instability and the budget crisis undermining its economy, Ukraine is seeking new ways to tackle joblessness. Helping people set up businesses is a priority. The state offers seed money and training to the unemployed who choose this option. But this support is not a panacea. The current regulatory framework is complex and makes it hard to start a business, especially for newcomers.

“Who suffers the most when the crisis escalates? Big companies for sure! Small businesses are much more flexible, they can adapt to difficult situations. So creating small businesses is a priority today,” says Victor Zubritsky, deputy head of the regional employment centre in Zhytomyr, a city in northwestern Ukraine.

He runs a state programme which helps unemployed people start their own businesses. It began more than ten years ago, but in late 2008, with a global economic crisis underway, the programme was put on pause.

Now that the Ukrainian economy is on thin ice again, the government is keen not to repeat past mistakes. “We have not cut funding. On the contrary, we are increasing it,” says Zubritsky. In 2012, 11,000 unemployed people across Ukraine got funding for new business ventures. In 2013 the number increased to 15,000 and this year there will be many more.

You won’t find any mention of the state programme when you visit Briscola Pizzeria, owned by Tanya Kostetska, who lost her previous job last summer. Only friends and relatives are aware of where the money came from.

But the pizzeria itself is well known by everyone in Andrushivka, a town of 9,000 inhabitants. Ten kinds of pizza is an ordinary menu in Ukraine’s big cities, but for small and poor Andrushivka, 170 km from Kiev, Briscola is impressive.

“My sister travelled abroad a lot, she knows how things work in Europe, she helped me organising and decorating it,” Kostetska says while setting a table. She is the cook, waitress and boss all-in-one. She aims to take someone on soon as working seven days a week is hard.

But Kostetska is not complaining. “This is my business,” she says proudly.

Fitting out the premises cost UAH 30,000 (about EUR 2,500 at the time). The state contributed UAH 10,000, a significant amount in a poor region with an average monthly salary of less than UAH 2,000.

Employment centres help individuals, usually providing up to seven months’ redundancy payments, a maximum sum of around EUR 2,000. In provincial Ukraine that could be enough to set up a small business venture. But funding is not the only state support.

Free business training is more important for some people, such as Tetyana Artemchuk, co-owner of a grocery in Stara Kotelnya, a village near Andrushivka.

Artemchuk is 54. She was made redundant two years before taking retirement. When their children promised to provide financial support, she and her husband Dmytro Petryshyn decided to start a family business.

UkrAINE bETS ON STArT-UpS TO CrEATE jObS

Tanya Kostetska (left) is one of 11,000 unemployed people

across Ukraine who got funding for new business ventures.

Country focus Ukraine

Page 7: Live&Learn Issue 31

Live&LearnJuly 2014 07

“There are some shops in the village, but all sell really cheap food. But we are human beings, why should we eat junk? That’s why we stock top quality goods,” says Petryshyn.

This seems to be a good strategy. Their shop, which does not even have a signboard – the owners have not yet decided on a name – is popular and is the only place in the region where you can buy expensive types of wurst or a bottle of Guinness.

Artemchuk and Petryshyn did business training last January. They had to drive 50 km to Zhytomyr every day, but they describe this period with great warmth.

“It was like going back to school – lessons, taking notes – there we were, with kids under 20 sat next to us,” laughs Artemchuk.

“I do refer to my notes from time to time,” Petryshyn continues, “they give plenty of advice on dealing with tax and fire inspections and so on. You know, it can be so tough...”

When talk turns to official inspections, their smiles vanish. “If we had known

what blood suckers they are, we’d have thought twice about starting the business,” Artemchuk says. They paid UAH 70,000 for their electricity supply, five times more than the seed capital they received from the state.

There is a huge amount of bureaucracy not only when you start a business, but also when you close it down.

Two years ago Oxana Lysyuk set up a children’s early development centre in Malyn, a town 100 km from the capital. It turned out to be a seasonal business as in winter rent and taxes were higher than income.

“Last November I decided to shut it down. Today, six months later, the process is still not finished. The tax office informed me recently that I have to sign something and to pay again,” says Lysyuk who also complains of a lack of state support or seed capital when she started.

“But we don’t promise people success,” replies Zubritsky, “it’s a business. Some go bust, others succeed. It improves our unemployment statistics. It means not only they are not looking for work

themselves but they may also hire other people, creating new jobs.”

Low score for business friendliness

Ukraine is not a business-friendly country. In 2012, it was 152nd of around 180 countries in the World Bank’s Doing Business index. In 2014 it had moved up 40 places to 112th of 189 countries by simplifying a few procedures such as registering companies and building permits. But on other criteria it ranks still very low.

For instance, Ukraine currently ranks 172nd on Getting electricity and 164th for Paying taxes. “This is exactly where we should implement European standards. Guess how long it takes to open a restaurant in Zhytomyr? Two to three months at the very least! Does that encourage people to start a business? Certainly not! That’s why unemployed people sometimes refuse to take the support and go back to the job market,” says Victor Zubritsky. ■

Text and photos: Sergiy Sydorenko, ICE

Free business training is more important than funding

for people like Tetyana Artemchuk, co-owner of

a grocery in the village of Stara Kotelnya, two hours

drive west of Kiev.

Page 8: Live&Learn Issue 31

Live&Learn July 201408

“Making Better Qualifications in ETF Partner Countries” was the title of an international conference, held by the ETF in Brussels on 8 and 9 April 2014. The event was a forum for peer learning and information sharing. It generated ideas to support partner countries in reforming their vocational qualifications systems. The ETF reports...

Three major events since 2009

Around the world in the past years qualifications frameworks have become the favoured tool to produce more relevant and higher-quality qualifications. Since 2009 the ETF has held three major international events about qualifications frameworks, the external dimension of the European Qualifications Framework (EQF) and its implications for ETF partner countries.

In 2009 the ETF focused on the concepts and principles of qualifications frameworks, in 2011 on approaches to implementation of qualifications frameworks, and this year the participants tackled the very content of the frameworks, qualifications themselves. The Brussels conference explored in particular development approaches for qualifications, quality assurance and the role qualifications play in migration.

What we learnt

First, the European Qualifications Framework has created a common language not only in the EU but in ETF partner countries. It remains central to progress in partner countries’ reforms of vocational qualifications.

Second, qualifications are a reform priority in almost all ETF partner countries and are likely to remain so for several years to come. Institutions’ capacity remains the biggest challenge in reforming qualifications.

Third, while qualifications have important functions for individuals and employers, they are only one of the elements contributing to an individual’s employability.

Fourth, in most partner countries there are at least some qualifications based on the outcomes of learning, specifying what a learner is expected to know, understand and do after a learning process. More and more qualifications are designed to meet the needs of the learner and the labour market.

What we concluded

Linking to the EU: ETF partner countries should consider developing EQF-compatible qualifications. This can increase transparency and comparability.

Qualifications for lifelong learning: Qualifications should aim at lifelong learning. They should offer “bite-sized learning” in units; allow the validation of knowledge and skills acquired in different informal and non-formal contexts; and cater also to adult learners.

Quality assurance: Assuring quality should focus more on improved assessment and institutionalised involvement of key stakeholders e.g., through the sector skills councils.

Support tools: Registers of qualifications are useful administration tools. Partner countries should also consider adapting tools used directly by learners such as the EU’s VET Certificate Supplement.

Internationalisation: Qualifications have increasingly international aspects. There are now regional frameworks for qualifications, such as the EQF, the Southern African Development Community Qualifications Framework and the Caribbean Vocational Qualifications Framework. ■

Text: Michael Graham, ETF

FOCUS ON bETTEr qUAlIFICATIONS

Report from conference

Jan Truszczyński, Director General for Education and Culture at the European Commission, opened the conference.

“2014: FOCUS ON ThE CONTENT OF qUAlIFICATIONS”

lEArN MOrE ON ETF qUAlIFICATIONS plATFOrM:http://ow.ly/yEBhy

Page 9: Live&Learn Issue 31

Live&LearnJuly 2014

... and a participant shares his views.

The ETF excels at bringing people together from the ‘vertical’ and the ‘horizontal’ axes of the VET sector. The vertical axis comprises representatives of government ministries, employer groups, trade unions, individual businesses, analysts, experts, VET providers and learners. The horizontal axis takes in the breadth of the ETF’s geographical remit, from Algeria to Albania and Kyrgyzstan to Kosovo1, along with EU member states and those from further afield – Australia, for example.

Such was the scene at the European Parliament as 150 delegates, speakers and facilitators gathered to discuss ways to improve the value and relevance of vocational qualifications, and to examine best practice from within and beyond the EU. The scope of the event was too broad to be covered in this brief discussion. However, some notable themes and trends emerged.

For one, the tendency that many countries seem to have, in the words of one workshop participant, for “endlessly reinventing the system”. She prefaced her brief presentation with these words: “What I’m about to show you won’t last long!” Sometimes the reinvention is little more than the recycling of previously unfashionable projects, said a delegate from the United Kingdom.

There was also a focus on language. Jan Truszczyński, Director General for Education and Culture at the European Commission, opened the proceedings. He commented on the apparent disparity between the complex language of educational instruction and assessment on the one hand, and, on the other, the simpler style of actual occupational requirements. His point was reinforced by ETF Director Madlen Serban. She noted that it is unrealistic to expect citizens to “learn all the different ‘languages’ of competing providers and systems”.

Christian Lettmayr, Deputy Director of Cedefop, made the striking observation that the ‘currency’ of qualifications is based on confidence and trust. A qualification which is not trusted and cannot be exchanged is a waste of time. In other words, a qualification certificate is a piece of paper backed up by a ‘promise’, just like money. And, just like money, it relies on a common acceptance of that promise, and the institution making it, for its value. Lettmayr added piquancy to his remarks with a quote from Franz Kafka; “What is a man without papers? Less than papers without a man!”

At the workshop entitled ‘Ensuring Trust’ these matters were explored in some detail. The analogy with money became even clearer. Qualifications have ‘currency’ in the sense of both time and

money. Time, because they need to be up-to-date, relevant and convenient for employers and graduates; and money, because they depend on the confidence of all actors to grant the givers and holders of qualifications the necessary security. This in turn allows employers, and other institutions such as universities, the comfort they require to accept the qualified as, indeed, being qualified. That acceptance is a necessary if not sufficient condition for the transparent, transferable and transnational system to which those present at the conference collectively aspired. ■

Text: Ezri Carlebach

09

Since 2009 the ETF has held three major international events on qualifications.

Photos: ETF/Juha Roininen – EUP&Images

Who took part in the conference

� 27 ETF Partner Countries � 19 EU Member States � European Commission, Cedefop, Unesco

� Other countries, i.e. Hong Kong, Australia

“ThE CUrrENCY OF qUAlIFICATIONS IS

SIMIlAr TO ThE CUrrENCY IN OUr WAllETS”

1 This designation is without prejudice to position on status, and is in line with UNSCR 1244 and the ICJ Opinion on the Kosovo Declaration of Independence – hereinafter ‘Kosovo’.

Page 10: Live&Learn Issue 31

10 Live&Learn July 2014

SOCIAl INClUSION AND vOCATIONAl SChOOlS

In 2013 the ETF and the experts from the London School of Economics and Political Science completed a research project that mapped vocational education and training policies and practices for social inclusion and social cohesion in South Eastern Europe, Turkey and Israel. Recently the ETF published the cross-country report with findings and recommendations.

The project was unique in scope and methodology. It took the approach of participatory action research: seeking to understand the problems by trying to solve them, collaboratively and following an evidence-based reflection.

Seven hundred and forty-two teachers and 2,862 students were surveyed. More than 300 people, key to the issues of education and training, took part in the interviews, and more in the focus group discussions.

Findings

VET policy: Modernisation of VET systems has been taking place at differing speeds across the region. Policy orientations in the countries vary: some emphasise social inclusion, others focus on adjustment to the labour market. Yet implementation in general has been slow.

School choice: Education systems are a powerful source for the transmission of social exclusion. Working-class and disadvantaged youth are more likely to be channelled into vocational schools compared with children of middle-class parents.

Experience at school: Vocational schools often appear to reinforce the social exclusion of students. They suffer from underinvestement and often have outdated curricula and poor teaching methods. Disabled students are significantly less happy with their school

experience. Students become less happy with their schools as they move from their first to their final year. Few participate in extracurricular activities. Practical training is insufficient to provide many students with a sound basis of vocational knowledge and experience.

Dropping out: Young people most at risk of dropout in South Eastern Europe, Turkey and Israel, come predominantly from lower-level socio-economic groups. School dropout should be regarded as a process, rather than a single occurrence, resulting from a complex interaction of different factors.

Transition from education to work: The lack of adequate career guidance and counselling at vocational schools was a common theme in the country case studies as was the prominent role played by the presence or absence of family contacts in the search for a job, highlighting the importance of social networks and connections.

Overall conclusion: There is strong evidence of the cascading effect of exclusion as students progress through school and beyond. Initial gaps in school performance widen, increasing inequality in educational outcomes and leading to adverse effects on social cohesion. ■

Text: Marcin Monko, ETF

Facts and figures Social inclusion

Page 11: Live&Learn Issue 31

11Live&LearnJuly 2014

POLICY MAKERS Should make vocational education more attractive for both high and low achievers.

VOCATIONAL SCHOOLS Should provideeducation andtraining that isoptimised for

the labourmarket.

TEACHER EDUCATORS Should train teachers and school directors to be more socially inclusive.

SCHOOLDIRECTORSShould work

closely with localbusinesses and

employers, primary schools,municipalities, employment

offices, families and guardians.

BREAKING THE CYCLE: WHO CAN DO WHAT?

Family background can restrict

opportunities

More students drop out of vocational education

Disadvantaged students tend

towards vocational education, e.g. as a

second chance

Vocational education

can reinforce disadvantage

REINFORCING THE EXCLUSION

CYCLE

SOCIAL INCLUSION AND VOCATIONAL EDUCATION IN SOUTH EASTERN EUROPE,

TURKEY AND ISRAEL: KEY FINDINGS

Page 12: Live&Learn Issue 31

12 Live&Learn July 2014

FASTEr, bETTEr, DIFFErENT SEEkINg NOvEl WAYS TO CArrY OUT prOjECTS

New at the ETF Innovation and knowledge management

As the ETF moves into the new mid-term perspective covering 2014-2017, its management has been increasingly aware of the need to actively stimulate project innovation. This concerns especially, but not exclusively, the way the ETF uses technology, and most specifically those emerging opportunities that the new technologies, such as web 2.0, social media, simulation and immersive technologies and methods, may provide. For this reason the ETF adopted a new policy to

address new knowledge and innovation needs.

Why innovate?

Innovation of course is not new, but the need to understand it better and cultivate environments where it can thrive has become vital in achieving ETF objectives in a world of ever more limited resources and shrinking time horizons for change. Creativity and hence innovation in training and labour market activities with partner

countries is part of the ETF’s strategic outlook. We need to be faster, better, different.

The ETF has always been a knowledge broker for change and a disseminator of good examples of policy development and practice. Innovation strategies have been integral to the implementation of projects in entrepreneurial learning, entrepreneurial communities, work-based leaning, etc. The ETF now aims to mainstream this “cultural” pre-disposition

Photo: Flickr Creative Commons

Page 13: Live&Learn Issue 31

13Live&LearnJuly 2014

to innovation across all of its operational projects. The challenge will be to maintain a balance within the agency’s work, to continue to produce high quality, efficient, effective and compliant actions at the same time as striving to find innovative means of achieving its objectives.

Why technology?

Complex social problems, globalising markets and societies, multiple stakeholders, long-term and system perspectives and environment and climate threats, economy and demographics – these are the problems we face nowadays. With new and emerging technologies we will be able to develop deeper insights, new ways-of-knowing and novel working solutions to these intractable problems. Technology can provide solutions to problems, and at the same time, give an organisational driver for pursuing sustainable and inclusive processes, fostering ownership and delivering tailored interventions.

What are the difficulties?

As a public administration the ETF maintains, for good reasons, strong risk and innovation resistant responses to change. The need to produce greater impact with the same or less resources (more with less), raises the urgency of stimulating the most positive innovations possible in the workplace. These solutions need to be delivered faster and more resiliently compared to the past. So for the ETF, the policy will need to understand

well the context, and work from that point to stimulate creativity in its staff and partners. New ideas need to be integrated into planning and programming cycles, and change can be brought about in a natural and self-sustaining manner.

Technology innovation exists in the two domains of ETF work. Firstly, in the way we do our work. For example, how the Torino Process data is gathered, discussed, opinions debated, evidence presented and so on. Secondly, in the way our partner countries make policy. This latter domain is strongly linked to the ETF core projects of the Turin Process and the capacity building. The concepts, awareness, evaluation and practice of innovation have to be incorporated into both projects and policy learning.

Innovation is not just about technology. Public policy innovation is a clear goal for partner countries. Within this framework a wealth of innovation types are visible: open innovation, innovation communities, crowdsourcing, eco-innovation models, social innovation, to name just a few. Innovation is also about vision and strategy. Innovation is a product from and an input to vision and strategy building and to embed well in ETF operations it needs to be treated in its own right.

Where are we now and what’s next for 2014 and beyond?

An ETF community for innovation has been recently established after expert consultations, internal debate on

innovation, including a comparative study of similar organisations, and engagement with EU-funded Crossover 2.0 project on next generation of policy making. It brings together useful and relevant information and knowledge, discussion, ideas and experience of project innovations. ■

Text: Ian Cumming, ETF

Some examples of current and projected innovations at the ETF

� Visual analytics of key indicators for the Torino Process

� Making the most of qualitative data, including narrative in studies

� Collaborative tools at the disposal of all projects to harness in-house and external expertise

� Serious games in policy learning and capacity building

� Online tools for virtual peer reviews in entrepreneurial learning project

� Webinars and online meetings

Innovation is not just about technology. Public policy innovation is a clear goal for partner countries.

Photo: ETF/Alberto Ramella

Page 14: Live&Learn Issue 31

14 Live&Learn July 2014

TACklINg UNEMplOYMENT ThE ACTIvE WAYTraining programmes tend to be the most effective in tackling unemployment, especially in the long run, because they help people accumulate human capital.

Active labour market policies support the unemployed in looking for and finding jobs. Their aim is long-term employment, rather than a safety net for the jobless.

Photo: ETF/Ard Jongsma

Opinion Active labour market policies

Page 15: Live&Learn Issue 31

15Live&LearnJuly 2014

Jochen Kluve is Professor of empirical labour economics at the Humboldt University of Berlin.

For policy makers trying to tackle unemployment or ease youth’s transition from school to work, active labour market policies, or ALMPs, are essential tools. In a recent working paper prepared for the ETF, we take stock of the available evidence and analyse the usage and effectiveness of ALMPs in the partner countries and at the international level.

Active labour market policies, as their name suggest, are policies that actively help the unemployed find employment or increase their earnings. They are called active because they usually complement passive policies – that is, income replacement policies such as unemployment benefits.

There are essentially four types of ALMPs:

� Job search assistance, such as job search training, counselling, and monitoring.

� Training programmes. These can be classroom or on-the-job training, basic skills or life skills training. Often several components are combined.

� Financial incentives in the private sector to alter companies’ or workers’ behaviour. Typically these are either wage subsidies to the employer for hiring the unemployed, or grants for the jobless to start their own business.

� Public employment, i.e. public works or direct employment in the public sector.

Are ALMPs better then passive policies?

Passive and active policies have different objectives. The goal of a passive policy is the so-called consumption smoothing. Here the policy maker tries to make sure that people who lose their job are not left suddenly without any income. It is part of the social safety net. With ALMPs you want to actively support jobseekers in looking for and finding a job that will possibly employ them in the long run.

Training programmes tend to be the most effective among ALMPs, especially in the long run, because they help people

accumulate human capital. This was the original idea of an active policy: to train people, to increase their skills. Sometimes such programmes do not have an immediate effect – after all, training takes time – but in the long term, even years later, the additional human capital helps people attain better jobs and higher wages.

Other active policies have a mixed record when it comes to their effectiveness. Public works or public employment usually do not help the unemployed find a long-lasting employment; if anything they serve as an alternative social safety net. Job search assistance can quickly help people look more effectively for a job and create new job-matches – but the long-term effects are not clear, and likely cannot be expected from this programme type. Wage subsidy programmes do create strong incentives to employ new workers; in many contexts, however, it is likely that these subsidised workplaces substitute for jobs that would have been created also in the absence of the policy.

The bottom line is, therefore, that training programmes are crucial. If we teach people vocational skills, practical skills and life skills, then this is something that will be useful for them, often even in the long run.

Are youth-focused ALMPs effective?

In the OECD we often find youth programmes in general not to be too effective. Part of the explanation is that these programmes typically target youths who are left behind in a system in which

most young people manage to get a good education, good skills, and a good start into the labour market. If you are left behind at the age of 25, for instance, if you are a long-term unemployed with no secondary degree, then it is very difficult for an active programme to help you. On top of this, OECD countries often have rather restrictive labour markets in which employment protection rules and minimum wages effectively create barriers to labour market entry for the young, unemployed, and low-skilled workers. ALMPs then simply are not strong enough to bring youth across these barriers.

At the same time, in low and middle income countries the results are much more encouraging. There ALMPs also usually target the most disadvantaged, and they provide skills that are comparatively rare and demanded in a more flexible market. This suggests that youth-focused ALMPs show considerable promise for ETF partner countries. ■

Text: Jochen Kluve

ETF research on active labour market policies

The ETF’s community of practice on employment has recently completed a research project on active labour market policies (ALMPs). Innovative design and expanded coverage of activation policies are key points of concern in most transition and developing countries. Confronted with sluggish economic development and staggering levels of precarious work, unemployment and inactivity, ETF partner countries strive for better designed and funded ALMPs. The working paper written by Professor Kluve, describes the role and typology of ALMPs in ETF partner countries, analyses the effectiveness of the programmes under different conditions, suggests how countries can engage in a more systematic use and evaluation of ALMPs, and how the ETF can further support them.

Page 16: Live&Learn Issue 31

16 Live&Learn July 2014

“Governance for Employability in the Mediterranean” (GEMM) is a regional project carried out by the ETF. It includes pilot actions in nine countries. Here we present what these activities have achieved so far in Jordan, Lebanon and Tunisia.

“Reforming vocational education and training [VET] systems cannot be done by governments alone,” says Abdelaziz Jaouani, team leader of the GEMM project, “it must be done in cooperation with the users of the competences.”

These end users – stakeholders in the jargon – are employers, workers, civil society, students and their families. Engaging such a diverse group, some of whom are based in structured organisations, others private individuals, in social dialogue with governments in countries where power is often very centralised calls for a totally new approach.

In order to make a difference, these new, more participative and transparent forms of governance must take place at several levels – the national, the regional, all the way down to the profoundly local such as the way in which an individual vocational school interacts with nearby businesses and the community, says Jaouani.

From this conviction was born Governance for Employability in the Mediterranean or GEMM. This three-year project focuses on three policy areas – planning, quality assurance and financing – as levers for change, and has two components. The first involves analysing and building capacity on multilevel governance with national authorities and social partner organisations using a policy learning approach. For the second, each country has chosen a small but innovative project which will in some way help boost young people’s, and particularly women’s, chances of finding work. The learning will then be shared.

Together with those of Morocco and Algeria, the pilot projects of Jordan, Lebanon and Tunisia seen here have made the fastest progress so far, followed by those of Israel, Libya and Palestine. Egypt, which officially joined GEMM in March this year, is currently designing its own pilot.

Lebanon – careers guidance can reduce skills mismatch

The Lebanese project aims to reduce the skills mismatch by setting up a careers guidance office in a public vocational school in a Beirut suburb. “Companies need qualified Lebanese technicians… but students are not aware of the kind of jobs they could aspire to,” says Tina Comaty, project manager at the IECD (European Institute for Cooperation and Development).

Working with ministries and social partners, the project will integrate the new unit into the IECD’s existing network of guidance offices at four private vocational schools. “If the Ministry of Education sees it works, it may go on to implement this in other schools,” says Comaty.

The ICED’s experience shows it can make a difference. In 2012 a sample of 28 electrotechnics students at Cortbawi Institute vocational school in Adma were given guidance and work experience. A second group of 28 from other industrial courses were not. One year later, 60% of both groups had found work but of these, 82% of electrotechnics graduates were working in their field while only 30% of the others were doing so. “Good professional orientation can have an impact on the choice of job,” says Comaty.

The new GEMM unit will benefit around 200 students over the two years; “if by the end we have one person in the school formally recognised as the reference point for students and companies, that would be an achievement,” says Comaty.

David al-Chabab, IECD coordinator of vocational training projects, goes a step further: “I would like to have feedback from a bunch of graduates who have received help in getting a job from our office.”

rEFOrMINg gOvErNANCE OF vOCATIONAl EDUCATION AND TrAININg IN ThE MEDITErrANEAN

Project update GEMM

Photo: ETF/Marcin Monko

Page 17: Live&Learn Issue 31

17Live&LearnJuly 2014

Jordan – apprenticeships sparking social partnership

An apprenticeship scheme in Jordan hopes to create more opportunities for young people, especially women, in a country with one of the lowest female activity rates in the world and pioneer a new way for national and local partners to work together.

Only 14.1% of Jordanian women are active in the labour market compared to 61.3% of men, according to the Jordanian Department of Statistics (2013). The reasons for this are complex but social customs play a big role.

“Women in Jordan are very protected by their families, unless a job offers good conditions, they are not very likely to take it,” says Nadera al-Bakheet, director of the TVET Council Secretariat.

But the growing popularity of shopping malls since the first one opened in Amman in 2001 has created new, socially acceptable job opportunities for women.

The GEMM pilot will build on this by setting up an apprenticeship scheme for retail sales, comprising six-month workplace and school-based training, for 30 young people in Jordan’s second city Zarqa. There will be a second scheme, piloting apprenticeships for hybrid vehicle maintenance, run along similar lines.

Equally significant for Al-Bakheet is the chance to try out new forms of governance. A local committee, involving vocational schools, local employers, trade unions and NGOs, will work with national organisations to define needs, design curricula and generally run the project.

In a country with traditions of strong central government and weak social dialogue this is significant. “We know employers have a need for training, but it would be good to see people not waiting for the government to solve all their problems but getting involved themselves,” says al-Bakheet.

Tunisia – power to the regions

Tunisia is looking to use employment office counsellors and other regional actors to update VET, boost young people’s chances of finding work and, in the process, make governance by the regions a reality rather than an empty promise.

Focusing on the south-eastern governorate of the same name, during its first year the Gabès project will carry out capacity building activities to track where current VET graduates end up, identify employers’ needs for skills and suggest how the education and training on offer in the region could be updated to meet them.

By sitting all the social partners at the same table, it aims not only to make VET more relevant but also to put the abstract principle of regional decision making into practice.

“Before the revolution, central government decided everything,” says Jabrane

Bouraoui, deputy secretary general of the General Union of VET, Employment and Immigration, “now we realise that we, the social partners, and young people have a big role to play if we want to see them prosper in their working lives.”

That the need exists is clear. Youth unemployment was one of the drivers of Tunisia’s 2011 revolution but the situation has not improved. While unemployment at the end of 2013 stood at 15.3% according to Tunisia’s National Statistics Institute, for university graduates it was almost 32%. “There is a great mismatch between the real needs of the labour market and the training on offer which has not evolved in years,” says Bouraoui. ■

Text: Rebecca Warden, ICE

GEMM PILOTS AT A GLANCE

Country (location) Theme

Algeria (Blida) Improve matching through decentralisation & local capacity building

Egypt Decision pending

Israel (South) Improve matching through local capacity building

Jordan (Zarqa) Improve matching through apprenticeships for retail sales & hybrid vehicle maintenance

Lebanon (Dewaneh) Careers guidance for technical students

Libya (Region of Tripoli) Improve matching through training needs analysis for tourism & hospitality

Morocco (Tanger and Tetuan) Improving matching through training needs analysis for logistics & car maintenance

Palestine (West Bank) Tracking system for VET graduates to improve guidance, especially for women

Tunisia (Gabès governorate) Regional approach to upgrade VET

Page 18: Live&Learn Issue 31

18 Live&Learn July 2014

For societies to be able to deliver workers with the right skills requires a joined-up approach from policy makers to ensure that education and training policies respond to existing and future needs of the labour market.

Human resources development is at the heart of FRAME – a project funded by the European Commission and managed by the ETF. FRAME aims to assist countries to develop the right skills needed for sustainable economic development in the medium to long term, by 2020. The project covers Albania, the former Yugoslav Republic of Macedonia, Kosovo, Montenegro, Serbia and Turkey. It is made up of four inter-related components which together will assist each country to create a vision of their future economy and also a roadmap for an action plan that will achieve the vision.

‘FRAME is about aligning education and training policies to meet the needs of the

economy in 2020’, says Simona Rinaldi, team leader of the FRAME project at the ETF. ‘Within this project, countries look first at the skills that need to be developed and then the changes that need to be made so that institutions can provide the right training enabling people to acquire the skills.’ The two other components of the project cover monitoring and regional cooperation.

Creating a shared vision

Foresight is a driving force behind the project. The ETF describes foresight as ‘a future-oriented, participatory, systematic process.’ It is a four-step process that combines both qualitative

and quantitative tools and methodology that delivers a shared vision for skills and corresponding action plan. The second component is the review of institutional arrangements which identifies the capacity of institutions individually and as a whole to achieve the vision outlined in foresight.

‘If foresight asks “which skills should we develop between now and 2020?”, then the review stage asks “what should we change in our institutions so that we can give people the right skills?” ’ says Rinaldi. ‘Asking both questions and creating a shared understanding of the answers gives countries a greater chance of success in achieving their goals.’

FrAMINg ThE FUTUrE WITh FOrESIghT

Project update FRAME

The study visit to Malta was an example of regional cooperation.

Photo: ETF/Daniela Clara

Page 19: Live&Learn Issue 31

19Live&LearnJuly 2014

Armed with a vision and action plan, countries are then encouraged to monitor their progress. Again, participation of national and regional stakeholders reinforces commitment to the shared vision and action plan.

Sharing knowledge across the region

Knowledge sharing is central to the regional component of the project. And labelled as a separate component of FRAME, regional cooperation occurs throughout the project. From the outset, foresight workshops feature participants from across the region. FRAME enhances the regional dialogue and cooperation between countries through

mutual learning and exchanging best practice.

Developing and implementing a vision can be a lengthy, complex process involving many stakeholders. However, FRAME has shown that two things are clear. First, having a vision for future skills is good. Second, following through with actions to ensure a workforce with those skills is even better. ■

Text: Claire Bose, ETF

THE FOUR COMPONENTS OF FRAME

Component 1: Foresight

Develop and adapt a foresight methodology to build a vision and a roadmap for HRD (which skills should be developed by 2020 and how can they be generated by the education and training system?)

Component 2: Review

Produce and implement a methodology to identify the capacity development needs of institutions in charge of implementing the 2020 vision for skills (what are the capacity needs of institutions to achieve the 2020 vision for skills?)

Component 3: Monitoring

Develop a performance-monitoring system based on indicators to monitor progress in HRD reform in line with national strategic objectives and the SEE Strategy 2020 and EU 2020 targets (what indicators are needed to monitor progress and targets for the 2020 vision for skills?)

Component 4: Regional cooperation

Facilitate mutual learning among the countries in the region that will allow results to be exchanged and will pave the way for future joint activities (how can countries create synergy in HRD and learn from each other?)

Life after FRAME

As FRAME is set to end in the second half of 2014, the main stakeholders in the project gathered at the ETF on 26 and 27 June 2014 to discuss the follow-up actions. They took stock of the results achieved and challenges that emerged during implementation of FRAME initiative in each partner country. The FRAME validation event will be held in Tirana on 6 October 2014 with high-level representatives from South Eastern Europe and Turkey.

The bookshelf of a foresightful policy maker

One of the aims of FRAME was to give policy makers in EU enlargement countries practical guidance on reforming their human resources development (HRD) system. That is why the ETF, together with international experts, produced two guides: the Foresight Guide and the Guide for the Review of Institutional Arrangements.

The Foresight Guide has three broad functions. It provides a general overview of the role of foresight for developing and implementing HRD policies. It also serves as a toolkit for experts and other people responsible for implementing

foresight activities. Finally it promotes a wider awareness of foresight approaches, processes and results among educators, young people, employers, careers advisers and academics.

The Guide for the Review of Institutional Arrangements helps primarily those who take part in FRAME carry out reviews of HRD institutions. The reviews identify the necessary institutional arrangements for archiving a shared ‘2020 Skills Vision’. Ultimately, the information collected in these reviews will help the European Commission to effectively plan EU pre-accession assistance, or so-called IPA II, until 2020.

Page 20: Live&Learn Issue 31

20 Live&Learn July 2014

The story of Abdelghani El Haoudi shows how difficult it is to decide on one’s education and career path. Getting a practical vocational qualification, after a less labour-market-demanded academic degree, may be a good solution. But if you choose to migrate to another country, prepare for a string of challenges.

Abdelghani El Haoudi, a 28 year-old graduate of literature turned assistant chef from Morocco, came to Turin three years ago.

“In Morocco it’s difficult to find work with a literature degree, so if you want to get a job you need to choose a clear career path and pursue a profession.”

Cooking qualification after literature degree

This is why he decided to take a sous-chef course in his hometown of Casablanca. “I thought this course could offer me many opportunities for work even abroad,

especially in Italy, the country where I was planning to move to after my qualification.”

The course lasted one year. Abdelghani was happy with the teachers who were professional chefs themselves and had plenty of work experience within the industry. Students spent three days in the classroom and one day in the kitchen.

The school had one more advantage: good relations with several hotels and restaurants in the area. So when Abdelghani completed the course he was referred for an internship to a restaurant in Ajiad Hotel in Casablanca city centre.

“At school you are allowed to make mistakes,” he said recalling the internship, “but at work there is more pressure to deliver meals to real customers and you have to take responsibility for every plate that goes out of the kitchen.”

When he arrived in Turin he immediately understood that to find a job he needed to speak proper Italian. “Working in a kitchen means also becoming part of a team, and in order to be a good team member you need to be able to communicate with your colleagues efficiently.”

bETWEEN lITErATUrE, COUSCOUS AND pIzzA

My vocation

Abdelghani had to retake a year-long cooking course in Italy to have his skills certified. Here pictured with an employee of Immaginazione e Lavoro, a training centre in Turin, where he collected his Italian diploma in April 2014.

Photos: Cristian Afker

Page 21: Live&Learn Issue 31

21Live&LearnJuly 2014

“AT SChOOl YOU ArE AllOWED TO MAkE

MISTAkES, bUT AT WOrk ThErE IS MOrE prESSUrE TO DElIvEr… YOU hAvE TO TAkE rESpONSIbIlITY.”

Studying again in Italy

After having looked for a job for one year without success, his friend suggested an assistant chef course in Turin. After all he knew how to prepare great couscous, but had little experience with pizza. When he started studying again in Italy he found that some topics were the same as the ones he had already studied in Morocco, health and safety for example. But as his diploma wasn’t recognised, he had to follow the whole course. “I was part of a class with other students who had never studied this topic.”

After the course he got an internship in a bar in Turin. “What I learnt at school is very valuable, but what I learnt during the internship is just as important. Theory without practice is worthless.”

Abdelghani is now again looking for a job in Italy, but he sees his future back in Morocco. “My dream is that one day I’lll have a steady job and be able to buy a house in Morocco, find a wife, have kids, become a chef and perhaps open my own restaurant.” ■

Text: Marcin Monko, ETF

Abdelghani El Haoudi, a 28 year-old graduate of literature turned assistant chef from Morocco, came to Turin three years ago.

At a job interview in one of Turin’s restaurants. Abdelghani is looking for a job in Italy, but he sees his future back in Morocco.

Page 22: Live&Learn Issue 31

22 Live&Learn July 2014

NEW pUblICATIONSWOrk-bASED lEArNINg: A hANDbOOk FOr pOlICY MAkErS AND SOCIAl pArTNErS

All of us learn when we work. Much of this learning is not systematic, and it is not formally organised. It occurs randomly or by chance, and so some workers benefit while others do not. This new handbook has been written to help policy makers and organisations that represent employers and employees to understand some of the ways in which learning in the

workplace can be encouraged and how its quality can be improved. Prepared for the ETF by Professor Richard Sweet. http://goo.gl/ioMv32

Off the press and on the web

ETF hOSTS MEETINg ON pEOplE AND SkIllS IN EU DANUbE STrATEgY

The 7th Meeting of the Steering Group of Priority Area ‘People and Skills’ of the EU Danube Strategy was hosted by the ETF in Turin on 15 and 16 May 2014. Representatives from the countries and regions involved in the EU Strategy for the Danube Region as well as the European Commission (Directorate General for Regional and Urban Policy) discussed the latest developments in education and labour market reforms and possibilities of cooperation under the new funding instruments. http://goo.gl/kXPymO

DIgITAl UpDATE

SOUTh EASTErN EUrOpE, ISrAEl AND TUrkEY – TrENDS, pErSpECTIvES AND ChAllENgES IN STrENgThENINg vOCATIONAl EDUCATION FOr SOCIAl INClUSION AND SOCIAl COhESION

Seven hundred and forty-two teachers and 2,862 students were surveyed. More than 300 key stakeholders were interviewed in schools and local communities in South Eastern Europe, Turkey and Israel. The purpose? To find out how to make

vocational schools more inclusive places and to use their strengths for more cohesive societies. http://goo.gl/q5zEj2

MAkINg bETTEr vOCATIONAl qUAlIFICATIONS

This new study on the reform of qualifications systems in the partner countries to the east and south of the EU was carried out by the ETF in the run up of the international conference on the subject in April 2014. It gives an empirical analysis of reform processes, formulates lessons learned and makes recommendations. It shows

a changing landscape of vocational qualifications: it is a work in progress and the experiences that countries have faced in carrying out reforms are worth sharing. http://goo.gl/F6SDiR

Ph

oto

: ETF

/Ard

Jon

gsm

a

Ph

oto

: ETF

Page 23: Live&Learn Issue 31

23Live&LearnJuly 2014

Digital update

ETF AWArDED OSCArS OF COMMUNICATION

The ETF won two Gold Quill Awards of Excellence in the 2014 annual competition of the International Association of Business Communicators (IABC).

The awards recognised the ETF’s Communication Week, a series of training sessions on communication, organised jointly by the human resources and communication departments in September 2013. The ETF was awarded a Gold Quill in 2011 at the lower merit level for its project work using social media for social inclusion. ■

gUESTS FrOM TUrkEY lEArN AbOUT INTEgrATED ApprOAChES TO hUMAN rESOUrCES DEvElOpMENT

Thirty-five Turkish officials, employment and education experts and academics, as well as representatives of employers’ organisations and civil society, came to Turin for a two-day seminar to learn about integrated approaches in human resources development and employment policies. As part of the seminar, a group of delegates visited an industrial robot factory owned by COMAU in Turin. The company served as a showcase for a successful cooperation between business, schools and regional authorities to develop human resources and jobs. http://goo.gl/M6hmYB

SkIllS FOr bUSINESS IN ThE MEDITErrANEANThere’s a wealth of good practice in entrepreneurial learning, but progress in policy making in this area is limited in the countries of the EU southern neighbourhood. While more training is available for SMEs, the data on their needs and cooperation with businesses must be improved. These are two main conclusions of the ETF assessment carried out as part of a broader review of the region’s business climate under the Euro-Mediterranean Charter for Enterprise. http://goo.gl/Fqjnwa

gOvErNINg bOArD ADOpTS rEpOrT ON 2013 ACTIvITIES AND bIDS FArEWEll TO ChAIrMAN

On 6 June 2014 the ETF Governing Board unanimously adopted the agency’s Annual Activity Report for 2013. The Governing Board – which comprises representatives of EU Member States, the European Commission, the European Parliament as well as observers from the partner countries – welcomed the high level of ETF activities and quality of results. This was the last meeting for the Governing Board Chairman Jan Truszczyński, Director General for Education, Training, Culture and Youth at the European Commission, who is retiring from his function. http://goo.gl/7fBWpV

Ph

oto

: ETF

/Mar

cin

Mon

ko

Ph

oto

: ETF

/Juh

a R

oini

nen

– E

UP

&Im

ages

Ph

oto

: ETF

/Alb

erto

Ram

ella

– E

UP

&Im

ages

Page 24: Live&Learn Issue 31

TA-A

F-14

-031

-EN

-CHOW TO CONTACT US

For information on our activities, job and tendering possibilities please visit our website: www.etf.europa.eu

For other enquiries please contact:

Communication Department European Training Foundation Villa Gualino Viale Settimio Severo, 65 I – 10133 Torino

T +39 011 630 2222 F +39 011 630 2200 E [email protected]