Rome, 20/09/2012

FP7 Space Research

Proposal evaluation and role of the REA

Christine BernotHead of UnitEuropean CommissionREA S2 Space Research

Overview

1. REA missionREA activities & Space Research Role of REA & ECGMES - SSF - Cross-cutting budget

2. Understanding the evaluation processEvaluation roadmap for the call FP7-SPACE-2013-1Evaluation process ESA input

3. Key highlights on proposal preparationEligibility Evaluation criteriaAdditional explanations and hints

Where to find relevant information

1. REA mission

REA = Research Executive AgencyCreated by EC decision to externalise programme management- Separate legal entity with its own administrative budget- Implementing part of FP7 using EC budget lines- Controlled by the European Commission (steering committee) -Director and HoU seconded from the EC

The REA implements some actions:o Space and Security actions (Cooperation)o SME-specific actions (Capacities)o Marie Curie actions (People)

The REA provides also horizontal services across all programmes (evaluation facility, validation of beneficiaries, FP7 enquiry service)

Space Researchat the REA

The REA Space Research unit is responsible for implementing the part of the Space work programme dedicated to co-financing research projects

In practice it means:

Organising and being responsible for the evaluation

Negotiating projects, preparing and signing financial and legal commitments

Monitoring projects progress, dealing with contractual issues

Checking on the use of resources and making payments

Implementing audit results

REA & ECThe European Commission’s role in FP7 Space research

Policy work remains within the Commission - DG Enterprise (ENTR)

> this includes the definition of the Space work programme

DG ENTR finances directly specific topics identified in the work programme (e.g. development of the Space component of GMES)

Space Research budget: 1.43 billionaround 640 externalised to REA

Interaction REA.S2 - ENTR.H2

DG ENTRWork programme definitionDealing with Programme Committee (PC)Presentation of the WP andsupport to applicants

Presentation to PCInter-Services ConsultationCommission selection DecisionDevelopment of WP, publication on projects, organisation of conferences

REAGuides for applicants, Call publication Organising the evaluationSupport to applicantsEvaluation of proposalsReporting on evaluation outcomeNegotiations processEnd of negotiationsFinancial commitment and signature of grant agreementInformation on projects, success stories, contributions to conferences & publications

Smooth cooperation on evaluation, negotiation, project implementation, dissemination...

EU funding per activity (and call)

€0

€20.000.000

€40.000.000

€60.000.000

€80.000.000

€100.000.000

€120.000.000

FP7-SPACE-2007-1

FP7-SPACE-2009-1

FP7-SPACE-2010-1

FP7-SPACE-2011-1

FP7-SPACE-2012-1

GMES

SSF

CC

M€141,9

M€54,6

M€121,8M€107,3

M€89,1

M€ 285,6M€ 193,7

M€ 35,5

Total: M€ 514,8204 projects

SSF 38%

CC 7%

GMES 55%

FP7-SPACE-2012-1 evaluation results

49

109

28

18 20

11

0

20

40

60

80

100

120

GMES SSF CC

evaluated proposalssuccessful proposalsEvaluated proposalsSelected proposals

2. Understanding the evaluation process

Evaluation roadmap for the call FP7-SPACE-2013-1

Evaluation process

ESA input

2. Understanding the evaluation processEvaluation roadmap forthe call FP7-SPACE-2013-1

Call publication: 10 July 2012

Call closure / submission deadline: 21 November 2012

at 17:00:00

Selection of experts: June- Sept 2012

Eligibility checks andallocation of experts to proposals: December 2012

Evaluation: December 2012 –February 2013

Information to coordinators: March 2013

Start of negotiations: April 2013

Experts: evaluate proposal in an individual manner and then agree on marks and comments during consensus meetings

Moderators: REA/EC staff moderating discussions during consensus meetings

Rapporteurs: read but do not evaluate proposals, draft consensus reports, support the harmonisation process (consistency between comments and marks)

Panel of experts: examine and compare consensus reports in a given area, check consistency, recommend priority order of proposals if necessary

Independent observers: assess the evaluation and report on the conduct & fairness of the evaluation

Selection procedure -Main actors

2. Understanding the evaluation process

2. Understanding the evaluation processOverview FP7 selectionprocedure

Full Proposal

Proposalforms

Evaluators

Eligibility

Evaluators Evaluators Final ranking list

PanelSubmission ConsensusIndividual reading

Proposals insuggestedpriority order

Rejection list

Finalisation

Criteria

Criteria

Criteria

REA EC

“remote

”

using

IT to

ol

Brusse

ls

External experts

Brusse

ls

Moderated by REA

Moderated by REARapporteur

2. Understanding the evaluation processEvaluation procedurefor each proposal

Proposal Xcopy 1

Proposal Xcopy 2

Proposal Xcopy 3

IERexpert 1

IER expert 2

IER expert 3

Consensus meeting CR

(+EIR?) 3 experts

Note: There may be more than 3 expert evaluatorsIER=Individual evaluation reportCR=Consensus ReportEIR=Ethics Issues Report

Individual reading

Objectivityo Each proposal is evaluated as it is written

Transparencyo Evaluation experts make their judgment against the

official evaluation criteria, and nothing else

Consistencyo The same standard of judgment is applied to each

proposal (including resubmitted proposals)

Guiding principles

2. Understanding the evaluation process

Minimum 3 experts per proposal

Initial selection of experts based on topics of the work programme (June to September 2012) - taking into account geographical and gender balance, types of organisation, broad coverage of areas of expertise

Final selection of experts based on proposal received (end November) - taking into account expertise and avoiding conflict of interest (e.g. remove experts if their organisation participate to a proposal)

Experts mainly from EU MS and associated countries, but can be any other country (eg USA). Each evaluation 40 to 50% of new experts

If you want to become an evaluation expert yourselfregister through the participant portal https://ec.europa.eu/research/participants/portal/page/experts

When describing your areas of expertise, use the keywords of the work programme and keep your profile up-to-date!

Selection of experts

2. Understanding the evaluation process

ESA input

• DG ENTR works closely with ESA on the development of the Space policy and its implementation through FP7

• ESA does not participate in FP7 Space projects

• ESA is involved in the evaluation process by providing comments on potential duplication with their programmes.

• Finally, ESA may provide experts with the role of reviewers for running projects

3. Key highlights on proposapreparation

Eligibility

Evaluation criteria

Additional explanations and hints

3. Key highlights on proposal preparation

A proposal is eligible if:

1. Received before the deadline (seconds count!)

2. Minimum conditions for participation meto At least 3 independent legal entities from 3 Member states or

Associated countries o Other conditions in the work programme met

(e.g. maximum requested EU contribution respected, presence of SMEs)

3. Complete o Requested administrative forms

+ proposal description (readable, accessible and printable)

4. (At least partially) in scope of topic(s) and funding schemeIneligible proposals are not evaluatedIneligibility can also be discovered later during evaluation

Eligibility

EU Member States:Austria, Belgium, Bulgaria, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, United Kingdom.

FP7 Associated Countries:Albania, Bosnia and Herzegovina, Croatia, Faroe Islands, FYR Macedonia, Iceland, Israel, Liechtenstein, Montenegro, Norway, Serbia, Switzerland and Turkey.

EU Member States and FP7 Associated Countries

3. Key highlights on proposal preparation

Third countries may participate in FP7 activities If the minimum conditions in FP7 ‘Rules for Participation’ & the WorkProgramme are met.

1) International Cooperation Partner Countries (ICPCs) (complete list: ftp://ftp.cordis.europa.eu/pub/fp7/docs/icpc-list.pdf).- eligible to receive EU funding. - they can be the coordinator

2) High-income countries- all the countries not included in the ICPC list and not associated to FP7.- e.g. USA, Canada, Japan, Korea, Singapore, Australia and New Zealand. - eligible to receive EU financial contribution if certain conditions are met:1. if funding is provided for in a bilateral scientific agreement or2. if clear provisions are included in the work programme/call for proposalsNB: NASA has signed a specific agreement with EC for FP7 Space Programme

International cooperation Third country participants

3. Key highlights on proposal preparation

Evaluation criteria

3. Key highlights on proposal preparation

1. Scientific and/or technological Excellence (relevant to the topics addressed by the call)

2. Quality and efficiency of the implementation and the management

3. The potential impact through the development, dissemination and use of project results

Collaborative projects

(NO CSA in 2013 Call)

Soundness of concept, and quality of objectives

Progress beyond the state-of-the-art

Quality and effectiveness of the S/T methodology and associated work plan

Appropriateness of the management structure and procedures

Quality and relevant experience of the individual participants

Quality of the consortium as a whole (including complementarity, balance)

Appropriateness of the allocation and justification of the resources to be committed (staff, equipment,…)

Contribution, at the European and/or international level, to the expected impacts listed in the work programme under relevant topic/activity

Appropriateness of measures for the dissemination and/or exploitation of project results, and management of intellectual property.

The evaluation experts

use assessment forms

with these questions

3. Key highlights on proposal preparation

Or in other words ….

Evaluation criteria

Scientific and/or technological Excellence

Objectives should be focused (what do you want to achieve in this project?) and clearly linked to the call topic.

Progress beyond the State of the Art should be up-to-date! (be careful if resubmission...)

Methodology and work plan: include risk analysis & contingency plans

What are the experts looking for

3. Key highlights on proposal preparation

Implementation & Management

Management structure & procedures:Include decision making & conflict resolution

Quality & relevant experience of individual participants

Quality of the consortium: complementarities & balanceAllocation of resources: should be explained and justifiedo How is the totality of the necessary resources mobilisedo Will the resources be integrated in a coherent wayo Is the overall financial plan adequate for the project

Subcontracting: no core tasks - no predefined subcontractorsMust be described and an estimated budget provided

If third parties involved described their role !

What are the experts looking for

3. Key highlights on proposal preparation

Impact, Dissemination and IPR

Contribution to the expected impact listed in the work programme under the relevant topic/activity

Dissemination and exploitation of project results How this will increase the impact? What are the targeted audiences of the activities?

Management of intellectual property: Which existing knowledge is needed for the project? How this existing knowledge will be available to the partners?

What are the experts looking for

3. Key highlights on proposal preparation

0 - The proposal fails to address the criterion under examination or cannot be judged due to missing or incomplete information1 - Poor. The criterion is addressed in an inadequate manner, or there are serious inherent weaknesses.2 - Fair. While the proposal broadly addresses the criterion, there are significant weaknesses.3 - Good. The proposal addresses the criterion well, although improvements would be necessary.4 - Very good. The proposal addresses the criterion very well, although certain improvements are still possible.5 - Excellent. The proposal successfully addresses all relevant aspects of the criterion in question. Any shortcomings are minor.

Scores and meaning

Scores with half points

are possible Individual criteria threshold: 3

Total score threshold: 10

3. Key highlights on proposal preparation

The expertise of the SMEs should be clearly highlighted and their work well linked to the activities: part of the quality of the consortium

Can be an added value for the exploitation of results and thus the impact

Potential criterion taken into account if prioritisation is needed between proposals

Rule: no special evaluation criterion

Exception: Topic 3.1-1: mandatory SME participation!

Presence of SMEs

3. Key highlights on proposal preparation

Collaborative projects (CP)o RTD (Research and Technology Development)

Including Scientific and Technical coordination

o Demonstration: key aspects for projects demonstrating that they are close to market/users

o Management: Overall legal, ethical, financial and administrative managementScientific technical coordination under RTD

o Other activities e.g. IPR issues, training and dissemination (including publications)

Type of activities

3. Key highlights on proposal preparation

In the consortium, promote mixed teams at all level (Work Packages, Work Package leaders, steering committees, advisory boards...)

Check if gender may be a factor to be taken into account in your research (e.g. users)

Proactive activities to promote gender equality in your project or field of research (e.g. specific events)

Gender “toolkit” available at http://www.yellowwindow.be/genderinresearch/

Gender aspects

3. Key highlights on proposal preparation

Highlight in the proposal any ethics issues, e.g.

- data protection issues (privacy)

- Dual use: research having direct military use or the potential for terrorist abuse

New enhanced ethics issues procedure

Rules for submission, evaluation, selection, award

Pre-screening by evaluators

“Flagged” proposals go through Ethical Review(body of independent ethics experts)

= > implementation of requirements at negotiation stage

Ethics issues

3. Key highlights on proposal preparation

Understand the work programme topic and address its essential spirito A proposal fully out of scope is ineligibleo A proposal partially out of scope will score low

Respect requirements and instructions in the work programme (e.g. maximum amount of requested EU funding, SME participation)

and check the Guide for Applicants (e.g. page limits).

Proposers need to convince the evaluation expertso They are under time pressure during both the individual evaluation

and the consensus meetingo Make it easy for them to find the answers in your proposals to the

questions to assess the proposal against the evaluation criteriao They must justify their marks, give them evidence.o Avoid inconsistencies in the proposal to make a good impression.o Evaluators are instructed to look at the substance, not the

presentation, but a careful presentation helps.

Put yourself in the shoesof the experts

3. Key highlights on proposal preparation

Where to find relevant informationRelevant reference documents

FP7 Legal basis documents generally applicableDecision on the Framework ProgrammeRules for Participation

Legal documents for implementationRules for submission, evaluation, selection, awardREA grant agreementRules on verification of existence, legal status,operational and financial capacity

Guidance documentsNegotiation Guidance NotesGuide to Financial IssuesGuide to IPRChecklist for the Consortium AgreementTemplates for Description of Work

Call documents: Participant portal

http://cordis.europa.eu/fp7/find-doc_en.html

Ethics issues:

Ethics check listSupporting documents

Where to find relevant information

Relevant information sources

Participant Portal: http://ec.europa.eu/research/participants/portal/CORDIS: http://cordis.europa.eu/fp7/participate_en.html

More background documents on http://ec.europa.eu/enterprise/policies/space/research

Call-specific questions (up to 2 weeks before deadline):rea-space-calls@ec.europa.euGeneral FP7 related questions: http://ec.europa.eu/research/enquiries

Specialised and technical assistanceeFP7 Service Desk: http://ec.europa.eu/research/participants/portal/page/contactusEPSS Help desk: support@epss-fp7.orgIPR help desk: http://www.ipr-helpdesk.orgEthics help desk: http://cordis.europa.eu/fp7/get-support_en.htmlNational Contact Points: http://cordis.europa.eu/fp7/ncp.htm

REA – Research Executive Agencyhttp://ec.europa.eu/research/rea

COV2 ▪ 1049 Brussels ▪ Belgiumsite 16 Place Rogier ▪ 1210 Brussels

+32-2-299 11 11+32-2-297 96 09

Copyright © 2007-2009 Close Comfort

Thank you for your attention!