the evaluation of sps applications - nato - homepage · the evaluation of sps applications ... tips...

October 20th, 2014 Lisbon, Portugal

The Evaluation of SPS Applications

ISEG - Independent Scientific Evaluation Group

Lisbon, Portugal 20 October, 2014

Horsehead Nebula


ISEG Primary Functions 1. Provide scientific and technical advice on applications for

bottom-up SPS activities;

2. Evaluate (peer-review) and rank bottom-up applications against SPS Key Priorities as set by the SPS Political and Partnership Committee (PPC);

3. Review bottom-up and top-down Science for Peace (SfP) project plans submitted by applicants;

4. Review Nationally-Funded Activities;

5. Have an horizon scanning role at the multi-disciplinary level;

6. Prepare and present reports to the PPC on the progress of the SPS activities;

7. Initiate applications or develop plans for new top-down projects that respond to identified key priorities.


ISEG Membership ISEG members are technical experts on

different fields, usually from NATO countries.

NATO Countries • Albania, Belgium, Bulgaria, Canada, Croatia,

Czech Republic, Denmark, Estonia, France, Germany, Greece, Hungary, Iceland, Italy, Latvia, Lithuania, Luxembourg, Netherlands, Norway, Poland, Portugal, Romania, Slovak Republic, Slovenia, Spain, Turkey, United Kingdom, United States


How ISEG works Submissions are sent electronically to all ISEG members!

For each proposal, several reviewers are selected by SPS staff • Based on proposal subject matter and reviewers expertise

Reviewers submit their evaluations electronically to SPS • All members can contribute to evaluation process;

• Submit evaluations

• Participate in discussion at meeting

Decisions taken at ISEG meetings with all members • Typically two or three ISEG meetings per year

• Decision on which projects to recommend for financial support


How ISEG works Panel is multidisciplinary and acts in a multidisciplinary

way!

Applications can be submitted anytime • For 2015 there are 3 deadlines (1 March, 2015; 1 July 2015; and 1

October 2015) to prepare for specific upcoming ISEG meetings.

• Note; a decision on funding may take up to 9 months

After a project is recommended for financial support • A “godfather/godmother” is selected from the ISEG members to follow

and assist project development.

Submission and evaluation process is fast, has little bureaucracy, is flexible and supportive


Support Mechanisms in SPS

• Projects • Multi-year Grants to collaborate on applied R&D projects

• Training • Advanced Study Institutes (ASI): grants to organize high-level

tutorial courses to convey the latest developments in a subject of relevance to NATO to an advanced-level audience.

• Advanced Training Courses (ATC): designed to enable specialists in NATO countries to share their security-related expertise with trainees from NATO's partner countries.

• Workshops • Advanced research Workshops (ARW): grants to organize expert

workshops where advanced-level, intensive discussions aim at finding solutions for today's security challenges.



• Projects • Multi-year Grants to collaborate on applied R&D projects

• Training • ASI: grants to organize high-level tutorial courses to convey

the latest developments in a subject of relevance to NATO to an advanced-level audience.

• ATC: designed to enable specialists in NATO countries to share their security-related expertise with trainees from NATO's partner countries.

• Workshops • ARW: grants to organize expert workshops where

advanced-level, intensive discussions aim at finding solutions for today's security challenges.


How ISEG evaluates proposals


Proposal Evaluation Criteria

Must address Key Priorities!!

Impact Scientific/Technical: important problem

Defense and Security

Partnership

Public Diplomacy

Young Scientists

Potential for Success Proposers

Schedule & Work Plan

Risks

Project value

End-users or stake-holders


Must address SPS Key Priorities

Mutually beneficial cooperation on issues of common interest, to meet emerging security challenges

• Counter-Terrorism • Energy Security • Cyber Defense • Defense against CBRN (Chemical, Biological, Radiological, and Nuclear

Defense ) Agents • Environmental Security • Enhance support for NATO-led operations and missions • Enhance awareness on security developments including through early

warning, with a view to preventing crises. Security-related Advanced Technology

• Border and Port Security • Mine and Unexploded Ordnance Detection and Clearance • Human and Social Aspects of Security related to NATO’s strategic

objectives


• Scientific/Technical • Important problem?

• Original idea?

• Defense & Security • Important issue?

• Impact on security?

• Impact on society: local or global?

• Partnership

• Public diplomacy

• Involvement of Young Scientists?

Evaluation Criteria: Impact


Estimate capability for Execution Expertise, Infrastructure

Resources (human, institutional support) are sufficient?

Schedule and work plan Proposed work to accomplish the goals?

Schedule realistic?

Division of labor and budget appropriate?

Expected outputs

Risk identification Technical, programmatic, political or other

Risk mitigation strategies (alternatives)

Partnership quality and management aspects

Evaluation Criteria: Success


Tips for proposal writing (1)

• Ensure you address key priorities! • One or multiple

• Select proposal topic carefully • Clear idea, clear problem, clear solution

• Relevance of topic should be very clear

• Select a team appropriate for the topic • Relevant expertise, experience

• Complementary expertise

• Appropriate end-users and stake-holders

• Must be committed, involved

Good team + relevant topic = good project



• Program activities in timely manner • Describe context and relevance of problem

• Describe solution and plan, goals, how achieve them

• Clear set of tasks, clear schedule

• Task clearly linked to research objectives

• Identify who does what, when and what is the result

• Budget must be realistic, well justified • Reason for expenses should be clear

• Avoid unclear, undocumented expenses

• Budget should be in line with activity, schedule



• Project should foster collaboration • Regular exchanges and meetings

• Joint work, joint publications

• Encourage involvement of young researchers • Teamed with experienced researchers

• Project results have to be disseminated • Present a dissemination plan: where to publish, when

• ISI Journals, conferences, book

• Highlight practical aspects, impact on partners, efficiency

• Consider budget for young researchers involvement



• Identify risks, potential impact, alternatives • Relevant risks should be pointed out as well as impact

• Describe mitigating strategies, alternatives

• Do not hide potential risks, better to be up front

• Strong strategy to mitigate high risk better than minimize or ignore risk

• Project results should generate new ideas • Should have a positive, evident and durable societal impact

• Should generate further research: imagine the next step

• Must be achieved within the duration of the Project • Clear in schedule, in line with budget

• Clear end-users, stake-holders


• Be clear!!

• Revise proposal before submitting

• Check for all items

• Be convinced

• If you are not convinced, neither will the reviewers

• Be convincing

• Reviewers should feel compelled to fund you




• Projects • Multi-year grants to collaborate on applied R&D projects

• Training • ASI: grants to organize high-level tutorial courses to convey

the latest developments in a subject of relevance to NATO to an advanced-level audience.

• ATC: designed to enable specialists in NATO countries to share their security-related expertise with trainees from NATO's partner countries.

• Workshops • ARW: grants to organize expert workshops where

advanced-level, intensive discussions aim at finding solutions for today's security challenges.


Proposal Evaluation Criteria

Must address Key Priorities!!

Impact Event likely to have impact?

Will impact extend beyond participants?

Does it overlap other efforts?

Attendees Is it the right mix?

Depends on type of event

Organization Title appropriate? Comprehensible?

Well organized? Budget realistic?

Program focused, appropriate?



• Ensure you address key priorities

• One or multiple

• Topic and Program appropriate

• Relevance of topic should be very clear

• Clear on focus, impact

• Duration appropriate

• Attendees and topic match type of event

• ASI: tutorial, advanced-level audience

• ATC: training, trainees

• ARW: experts, share advanced knowledge



• Background and expertise of speakers

• Relevant expertise, experience, on topic

• Strong speakers, recognized leaders, practitioners

• Appropriate budget

• Realistic, efficient

• Location

• Appropriate?

• Local members involved?

• Output of Event

• Book, notes, course

• Course reusable/repeatable?


Take advantage of the program, submit!

Thank you

the evaluation of sps applications - nato - homepage · the evaluation of sps applications ... tips...

Documents