nsf merit review criteria intellectual merit broader impacts additional considerations integration...

Robert Noyce Teacher Scholarship Program:

Strengths and Weaknesses of

Submitted Proposals

NSF Review Criteria NSF Merit Review Criteria• Intellectual Merit• Broader Impacts

Additional Considerations• Integration of Research & Education• Integrating Diversity into NSF Programs

Additional review criteria specific to Noyce Program, dependent on proposal type

Review Criteria: Phase I Scholarship Proposals Capacity /ability of institution to effectively conduct

program Number /quality of students to be served by program Justification for

◦ number of students◦ amount of stipend ◦ scholarship support

Quality/feasibility of recruitment/marketing strategies

Strong: Provides data to justify need and realistic expectations; indicates number of participants

Weak: Projections not supported by data

Review Criteria: Phase I Scholarship Proposals

Ability of program to recruit STEM majors who would not otherwise pursue a teaching career

Strong: Indicates they will recruit beyond those who are already in the program

Weak: Not expanding beyond current pool

Review Criteria: Phase I Scholarship Proposals

Quality of the preservice educational program

Strong: Provides details about program Provides evidence that graduates are

successful Research based

Weak: Little information provided

Review Criteria: Phase I Scholarship Proposals Extent to which STEM/education faculty are

collaborating in developing/ implementing the program

Strong: Good representation of STEM and

education facultyDefined roles in management planShared responsibility

Weak: No evidence of collaboration (“in name

only”)

Review Criteria: Phase I Scholarship Proposals

Quality of preservice student support and new teacher support infrastructure

Strong: Clear plan for supporting students and

new teachers to ensure successStrong partnership with school district

Weak: No support beyond the financial support

Review Criteria: Phase I Scholarship Proposals

Extent to which proposed strategies reflect effective practices based on research

Strong: Based on educational literature and

evidence from research findings

Weak: No references or not clear how the project

is based on research

Review Criteria: Phase I Scholarship Proposals Degree to which proposed programming will

enable scholarship/ stipend recipients to become successful math/ science teachers

Strong: Program designed to address specific

needs of Noyce Scholars

Weak: Program does not appear to be designed

to support needs of Noyce Scholars

Review Criteria: Phase I Scholarship Proposals

Feasibility/ completeness of evaluation plan measuring effectiveness of proposed strategies

Strong: Independent evaluator Clear objectives and measures Describes data collection and analysis

aligned with evaluation questions

Weak: No objective evaluator Evaluation not aligned with project objectives

Review Criteria: Phase I Scholarship Proposals Institutional support for program and extent to

which institution commits to making program a central organizational focus

Strong: Evidence of support from departments and

administrators Likely to be sustained Integrated with other STEM initiatives

Weak: Lack of supporting letters from administrators Little involvement beyond the PI

Proposal does not follow Noyce guidelines ◦ Students must complete STEM major◦ Little information about teacher preparation

program◦ Unrealistic enrollment projections◦ Recruitment/selection strategies not well described◦ Lack of

support for new teachers involvement of STEM faculty (or education

faculty) plans for monitoring compliance with teaching

requirement◦ Weak evaluation or lack of objective evaluator◦ Lessons learned from prior work lacks details

Summary of Common Weaknesses

Review Criteria: TF/MTF Proposals

Capacity/ ability of institution to effectively conduct program

Number/ quality of Fellows the program will serve Justification for

◦ number of Fellows served ◦ amount of stipend ◦ salary supplements

Quality/ feasibility of recruitment/ marketing strategies

Review Criteria: TF/MTF Proposals

Extent to which proposed strategies reflect effective practices based on evidence from research

Degree to which proposed programming enables participants to become successful math/ science teachers or Master Teachers

Extent to which STEM/ education faculty collaborate in developing/ implementing a program with the specialized pedagogy needed to ◦ enable teachers to effectively teach

math/science◦ assume leadership roles in their schools.

Review Criteria: TF/MTF Proposals Feasibility/ completeness of an objective

evaluation plan measuring effectiveness of proposed strategies

Institutional support for program and the extent to which it is committed to making the program a central organizational focus

Evidence of cost sharing commitments

Plans for sustainability beyond NSF funding

Review Criteria: TF/MTF ProposalsNSF Teaching Fellows only: Ability of program to recruit

◦ Individuals not otherwise pursing teaching career

◦ Members of underrepresented groups Quality of Master’s degree program leading to

teacher certification Quality of preservice student support and new

teacher support infrastructure

NSF Master Teaching Fellows only: Quality of professional development that will be

provided

Strong partnership with participating school district

Required matching funds identified Clear description of program elements for• preservice for Teaching Fellows • professional development for Master Teaching

Fellows Detailed recruitment and selection plans Clear vision of Master Teacher roles/

responsibilities, including preservice involvement Attention to content and pedagogy Detailed evaluation plans

Strong TF/MTF Proposals include:

Insufficient details for • preservice and induction program for Teaching

Fellows• professional development program for Master

Teaching Fellows Vague recruitment plans Selection plans do not follow guidelines Master Teacher roles and responsibilities not discussed Matching funds not identified Role of non-profit organization not clear School district partnership not strong Evaluation weak

Weaknesses of TF/MTF Proposals

Demonstrating a Strong Partnership

Individuals from all participating institutions have clear roles and communication structures

Management plan includes a description of communication, meetings, roles, division of responsibilities, and reporting

Distribution of resources is appropriate to the scope of the work

All partners contribute to the work and benefit from it

Letters of commitment are provided

What Makes a Proposal Competitive?

Original ideas Succinct, focused project plan Realistic amount of work Sufficient detail provided Cost effective High impact Knowledge and experience of PIs Contribution to the field Rationale and evidence of potential

effectiveness Likelihood the project will be sustained Solid evaluation plan

Tips for Success Consult the program solicitation (NSF 11-517) and

NSF Proposal & Award Policies & Procedures Guide (NSF 11-1)

Test drive FastLane Alert your Sponsored Research Office and observe

internal deadlines for signatures Follow page and font size limits Be aware of current literature in the field and cite

it Provide details for key areas of your project Discuss prior results Include evaluation plan with timelines and

benchmarks

Tips for Success (cont.)

Put yourself in the reviewers’ place Consider previous reviewers’ comments if

resubmitting a proposal Have someone else read the proposal Spell check; grammar check Meet deadlines Follow NSF requirements for proposals

involving Human Subjects Call or email NSF Program Officers

Return Without Review

Submitted after deadline Fail to separately and explicitly address

intellectual merit and broader impacts in the Project Summary

Fail to follow requirements for formatting (e. g. page limitation, font size, and margin limits)

Fail to describe mentoring activities for postdoctoral researchers, if any included in proposed budget

Fail to provide a data management plan

Not ready to submit a proposal this year?

Consider serving as a reviewer!

Send a letter of interest and a CV to one of the program officers.

Contact us:

Joan Privaljprival@nsf.gov

Richard Alóralo@nsf.gov

Mary Lee Ledbettermsledbet@nsf.gov

Other resources: www.nsf.govwww.nsfnoyce.org

Questions?