summary document june 2011 investing in innovation (i3) fund pre-application meeting note: these...

Summary Document

June 2011

Investing in Innovation (i3) Fund Pre-Application Meeting

Note: These slides are intended as guidance only. Pleaserefer to the official documents published in the Federal Register.

Note for Webinar Participants

2

The slides that will be presented during today’s pre-application meeting are available for download on the Resources page of the i3 website at:

http://www2.ed.gov/programs/innovation/resources.html

A Few Notes on Q&A

We have budgeted time during the meeting for Q&A– Live participants should submit their questions via the

computers around the room that are set up to receive questions

– Webinar participants should submit their questions via the webinar chat function

Due to time constraints, we will not be able to answer all questions received

If you have additional questions, please send them to the i3 mailbox: [email protected]

3

A Few Notes on Q&A (cont’d)

What WeCan Address

What WeCan Address

What WeCannot Address

What WeCannot Address

• Questions about the eligibility of a specific entity • Questions about the competitiveness of a

specific entity or research focus• Questions about 2010 applications• Substantive explanation of the rationale behind

inclusion or exclusion of specific items in i3 beyond what is in the Federal Register

• Content of the Investing in Innovation (i3) Notices and Application

• Timeline of the i3 program• Application process

4

Schedule for Today

5

Investing in Innovation (i3) Fund Summary

PurposePurpose

FundingFunding

ApplicantsApplicants

To provide competitive grants to applicants with a record of improving student achievement, attainment or retention in order to expand the implementation of, and investment in, innovative practices that are demonstrated to have an impact on:•Improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates•Increasing college enrollment and completion rates

$148.2 million (est.) to be obligated by December 31, 2011

Eligible applicants are: (1)Local educational agencies (LEAs) (2)Nonprofit organizations in partnership with (a) one or more LEAs or (b) a consortium of schools

6

i3i3

Types of Awards Available Under i3

Funding Available

Up to $3MM/award Up to $15MM/award Up to $25MM/award

Estimated Awards

Up to 15 Up to 5 Up to 2

Evidence Required

Reasonable - research findings or hypotheses, including related research or theories in education and other sectors

Moderate – either high internal validity and moderate external validity, or vice versa

Strong – both high internal validity and high external validity

Scaling Required

Able to further develop and scale

Able to be scaled to the regional or state level

Able to be scaled to the national, regional, or state level

7

What Makes i3 Different

• Builds portfolio of solutions to some of America’s most persistent educational challenges

• Aligns amount of funding with level of evidence

• Aims explicitly to scale effective programs and create a pipeline of promising innovations

• Provides funding for required independent evaluation in order to build understanding of “what works”

8

How We Think About Innovation

Innovationproduct, process,

strategy, or approach that improves

significantly upon the status quo and reaches scale

Innovationproduct, process,

strategy, or approach that improves

significantly upon the status quo and reaches scale

Invention

BaselineBaseline

Scale

Gre

ater

Im

pact

TrendTrend

9 Note: The definition of innovation on this slide is presented as an overview of the concept, not as a specific definition in the i3 program

Vibrant Competition in 2010

• Nearly 1700 applications across all three grant categories

• 49 grantees - 4 Scale-up, 15 Validation, 30 Development grants – aiming to collectively serve millions of students

• All 49 grantees secured private-sector matching

• Multiple unfunded i3 applicants subsequently have identified organizations to fund at least part of their proposal

10

Warnings from 2010

• Applicants failed to submit applications on time and so were not reviewed (this year, submit no later than 4:30:00PM DC time on August 2, 2011)

• Some highly-rated applicants were declared ineligible for funding because they did not meet the evidence eligibility requirements for the type of grant they requested

READ THE NOTICES and FAQs

UNDERSTAND THE REQUIREMENTS

PLAN AHEAD11

Schedule for Today

12

Major Changes from 2010

Funding by Grant Type

The maximum award for each grant type has changed:

• Scale-up: Up to $25 million

• Validation: Up to $15 million

• Development: Up to $3 million

Absolute Priorities

The competition now includes five APs, with the changes noted below:

• Retaining: Teachers and Principals Standards and Assessments Low-Performing Schools

• Adding: Promoting STEM Education Improving Rural Achievement

13

Major Changes from 2010

Competitive Preference Priorities

• Applicants may identify no more than two competitive preference priorities that they wish scored.

• Applicants may address as many of the competitive preference priorities as they wish for the purpose of comprehensiveness. However, the Department will review and award points only for the maximum of two CPPs the applicant identifies.

• The competition now includes five CPPs, with the changes noted below:

• Retaining: Early Learning College Access and Success Students With Disabilities and Limited English Proficiency

• Adding: Productivity Technology

14

Major Changes from 2010Selection Criteria

• The number of selection criteria has been reduced to 4

• Specifically, selection criteria that were addressed elsewhere last year – Strength of Research, Significance of Effect, and Magnitude of Effect and Experience of the Eligible Applicant – are no longer selection criteria (but remain important parts of the competition and should be addressed by applicants)

• Allocation of points by selection criterion varies by competition

Matching Requirements

• The percentage of required private sector match now differs by competition:

• Scale-up: 5% of the total award requested

• Validation: 10% of the total award requested

• Development: 15% of the total award requested

• Applicants may still request a reduction of the required match percentage

15

Major Changes from 2010Limits on Grant Awards

• The limits on grant awards have been revised such that no grantee may receive more than one new Scale-up or Validation grant in any two-year period

• This is in addition to clarifying the existing limits of two new grant awards and no more than $55MM in funding in a single year

• This does not affect current Scale-up or Validation grantees’ opportunity to receive new Development grants or to partner on other applications

16

Schedule for Today

17

All Eligible Applicants Must Implement Practices, Strategies, or Programs for

High-need Students

18

MUST

MUST

High-need student means a student at risk of educational failure, or otherwise in need of special assistance and support, such as students who are living in poverty, who attend high-minority schools, who are far below grade level, who are over-age and under-credited, who have left school before receiving a regular high school diploma, who are at risk of not graduating with a regular high school diploma on time, who are homeless, who are in foster care, who have been incarcerated, who have disabilities, or who are limited English proficient.

High-need student means a student at risk of educational failure, or otherwise in need of special assistance and support, such as students who are living in poverty, who attend high-minority schools, who are far below grade level, who are over-age and under-credited, who have left school before receiving a regular high school diploma, who are at risk of not graduating with a regular high school diploma on time, who are homeless, who are in foster care, who have been incarcerated, who have disabilities, or who are limited English proficient.

Applicants

19

Eligible Applicants can be:

1)A local educational agency (LEA)

2)A nonprofit organization in partnership with (a) one or more LEAs or (b) a consortium of schools

Eligibility Requirements: LEA

20

MUST

MUST

Eligible Applicant:

LEA

Eligible Applicant:

LEA

MUST, TO RECEIVE A GRANT

•Address one absolute priority•Demonstrate that it:

(a) significantly closed achievement gaps between groups of students or demonstrated success in significantly increasing academic achievement for all groups of students, and(b) made significant improvement in other areas

Establish partnerships with private sectorSecure commitment for required private sector matchMeet the evidence requirement for the type of grant for which it has applied

•Address one absolute priority•Demonstrate that it:

(a) significantly closed achievement gaps between groups of students or demonstrated success in significantly increasing academic achievement for all groups of students, and(b) made significant improvement in other areas

Establish partnerships with private sectorSecure commitment for required private sector matchMeet the evidence requirement for the type of grant for which it has applied

TO RECEIVE A GRANT,MUST

Eligibility Requirements:Non-Profit with LEA/Consortium

21

MUST

Eligible Applicant:

Non-profits, in partnership with LEA(s) or a consortium of schools

Eligible Applicant:

Non-profits, in partnership with LEA(s) or a consortium of schools

•Address one absolute priorityDemonstrate that the non-profit organization has a record of significantly improving student achievement, attainment, or retention through its record of work with an LEA or schools

Secure commitment for required private sector matchMeet the evidence requirement for the type of grant for which they have applied

•Address one absolute priorityDemonstrate that the non-profit organization has a record of significantly improving student achievement, attainment, or retention through its record of work with an LEA or schools

Secure commitment for required private sector matchMeet the evidence requirement for the type of grant for which they have applied

TO RECEIVE A GRANT,MUST

Key Definition: Partners

22

Official partner means any of the entities required to be part of a partnership under section 14007(a)(1)(B) of the ARRA (i.e., a non-profit organization, an LEA, or a consortium of schools).

Official partner means any of the entities required to be part of a partnership under section 14007(a)(1)(B) of the ARRA (i.e., a non-profit organization, an LEA, or a consortium of schools).

Other partner means any entity, other than the applicant and any official partner, that may be involved in a proposed project.

Other partner means any entity, other than the applicant and any official partner, that may be involved in a proposed project.

In the case of an eligible applicant that is a partnership between a nonprofit organization and(1) one or more LEAs or (2) a consortium of schools, the partner that was the applicant, and became the grantee when the partnership was selected to receive an award, may make subgrants to one or more of the official partners

Why It Is Important

23

Q&A

Schedule for Today

24

Improve Achievementfor High-Need

Students


Students

Teacher and Principal Effectiveness


Promoting STEM Education


College- and Career-ready Standards and Assessments


Improving Rural Achievement


Early LearningEarly Learning

College Access and Success


Serving Students with Disabilities and Limited

English Proficient Students



ProductivityProductivity

i3 Priorities

Required forall applications

Must address oneAbsolute Priority

May address up to twoCompetitive Preferences

(0 or 1 point each)

Improving Achievement in Persistently Low-

performing Schools


performing Schools

TechnologyTechnology

25

26

Notes on Absolute Priority 1:Teacher and Principal Effectiveness

• “…increase the number or percentages of teachers or principals who are highly effective teachers or principals or reduce the number or percentages of teachers or principals who are ineffective, especially for teachers of high-need students…”

• “…by identifying, recruiting, developing, placing, rewarding, and retaining highly effective teachers or principals (or removing ineffective teachers or principals).”

• “…teacher or principal effectiveness should be determined through an evaluation system that is rigorous, transparent, and fair; performance should be differentiated using multiple rating categories of effectiveness; multiple measures of effectiveness should be taken into account, with data on student growth as a significant factor, and the measures should be designed and developed with teacher and principal involvement.”

Two Possible Routes for

Teachers or Principals

Two Possible Routes for

Teachers or Principals

Multiple Measures of Effectiveness

Multiple Measures of Effectiveness

Multiple MethodsMultiple Methods

27

Notes on Absolute Priority 2Promoting STEM Education

“(a) Providing students with increased access to rigorous and engaging coursework in STEM. (b) Increasing the number and proportion of students prepared for postsecondary or graduate study and careers in STEM. (c) Increasing the opportunities for high-quality preparation of, or professional development for, teachers or other educators of STEM subjects.(d) Increasing the number of individuals from groups traditionally underrepresented in STEM, including minorities, individuals with disabilities, and women, who are provided with access to rigorous and engaging coursework in STEM or who are prepared for postsecondary or graduate study and careers in STEM.(e) Increasing the number of individuals from groups traditionally underrepresented in STEM, including minorities, individuals with disabilities, and women, who are teachers or educators of STEM subjects and have increased opportunities for high-quality preparation or professional development.”

Multiple Areas of Focus

Multiple Areas of Focus

Focus on Teachers or

Students


Students

Focus on High-Need

Populations

Focus on High-Need

Populations

28

Notes on Absolute Priority 3:Standards and Assessments

• “…standards and assessments that measure students’ progress toward college and career-readiness…”

• “…may include, but are not limited to, … (a)increase the success of underrepresented student

populations in academically rigorous courses and programs…;

(b)increase the development and use of formative assessments or interim assessments, or other performance-based tools and “metrics” that are aligned with high student content and academic achievement standards; or

(c)translate the standards and information from assessments into classroom practices that meet the needs of all students, including high-need students.”

• “…eligible applicant must propose a project that is based on standards that are at least as rigorous as its State’s standards…”

Focus on College & Career Readiness

Focus on College & Career Readiness

Range of Allowable Projects

Range of Allowable Projects

RigorousStandardsRigorousStandards

29

Notes on Absolute Priority 4:Persistently Low-Performing Schools

• “Whole-school reform, including, but not limited to, comprehensive interventions to assist, augment, or replace Investing in Innovation Fund Absolute Priority 4 schools, including the school turnaround, restart, closure, and transformation models of intervention … or …”

• “Targeted approaches to reform, including, but not limited to:

(1) Providing more time for students to learn core academic content by expanding or augmenting the school day, school week, or school year, or by increasing instructional time for core academic subjects

(2) integrating ‘‘student supports’’ into the school model to address non-academic barriers to student achievement

(3) creating multiple pathways for students to earn regular high school diplomas”

Projects May Choose Either

Approach

Projects May Choose Either

Approach

30

i3 Priority 4 SchoolsElementary Schools

• Title 1 schools in corrective action or restructuring

• Title 1 schools in improvement and in the lowest-achieving five percent (or lowest-achieving five schools)

Middle Schools

• Elementary school categories +• Title 1 eligible schools that would be in

corrective action or restructuring

High Schools• Elementary and middle school categories +• Title 1 eligible schools with a graduation

rate lower than 60% over a number of years

31

Notes on Absolute Priority 5:Improving Rural Achievement

• “…designed to address accelerating learning and helping to improve high school graduation rates (as defined in this notice) and college enrollment rates”

• “…designed to address accelerating learning and helping to improve high school graduation rates (as defined in this notice) and college enrollment rates”

• “…for students in rural local educational agencies”

Outcomes Focused

Outcomes Focused

Rural StudentsRural Students

Rural local educational agency means a local educational agency (LEA) that is eligible under the Small Rural School Achievement (SRSA) program or the Rural and Low-Income School (RLIS) program authorized under Title VI, Part B of the ESEA. Eligible applicants may determine whether a particular LEA is eligible for these programs by referring to information on the Department’s Web site at http://www2.ed.gov/nclb/freedom/local/reap.html.

Rural local educational agency means a local educational agency (LEA) that is eligible under the Small Rural School Achievement (SRSA) program or the Rural and Low-Income School (RLIS) program authorized under Title VI, Part B of the ESEA. Eligible applicants may determine whether a particular LEA is eligible for these programs by referring to information on the Department’s Web site at http://www2.ed.gov/nclb/freedom/local/reap.html.

Multiple MetricsMultiple Metrics

Guidance to Applicants and Peer Reviewerson Absolute Priority 5

• “Secretary encourages applicants that choose to respond to Absolute Priority 5 to also address how their applications meet one of the other Absolute Priorities”

• “…applicants that choose to respond to Absolute Priority 5 should identify in the application and the i3 Applicant Information Sheet all rural LEAs where the project will be implemented, or identify in the application how the applicant will choose any rural LEAs where the project will be implemented”

Focus on Key Reforms

Focus on Key Reforms

Identify Rural Locations

Identify Rural Locations

32

Guidance to Applicants and Peer Reviewerson Absolute Priority 5

• “…explain how the proposed innovative practices, strategies, or programs address the unique challenges of high-need students in schools within a rural LEA, resulting in accelerated learning and improved high school graduation and college enrollment rates”

• “Applicants may also provide information on the applicant’s experience and skills, or the experience and skills of their partners, in serving high-need students in rural LEAs in responding to Selection Criterion D. Quality of the Management Plan and Personnel”

Focus on Rural NeedsFocus on

Rural Needs

Demonstrate Past Experience

Demonstrate Past Experience

33

Note on Removing Use of Data as an Absolute Priority

“For this year’s competition, the Secretary chooses not to use the priority Innovations That Improve the Use of Data (Absolute Priority 2 in the 2010 i3 NFP). This action is not intended to discourage applicants from proposing projects that improve the use of data, so long as the proposal addresses one of the absolute priorities in this notice. Specifically, proposed projects that address Absolute Priority 1--Innovations That Support Effective Teachers and Principals, Absolute Priority 3--Innovations That Complement the Implementation of High Standards and High-Quality Assessments, and Absolute Priority 4--Innovations That Turn Around Persistently Low-Performing Schools may also include using data in innovative ways to support the broader aims of the absolute priorities”

Effective Data Use Allowable in Other Priorities

Effective Data Use Allowable in Other Priorities

34


Students


Students









Early LearningEarly Learning







ProductivityProductivity

i3 Priorities

Required forall applications

Must address oneAbsolute Priority

May address up to twoCompetitive Preferences

(0 or 1 point each)


performing Schools


performing Schools

TechnologyTechnology

35

36

Explanation of Competitive Preference Priorities

• Applicants for all types of grants may, but are not required to, identify up to two competitive preference priorities to earn extra points

• Points will be awarded depending on how well the applicant addresses a particular competitive preference priority, based on the judgment of the peer reviewers

• All competitive preference priorities are worth up to one point, and the point is awarded all or nothing by peer reviewers

Important Note on CPPs

The Department will not review or award points under any competitive preference priority for an application that:

(1) fails to clearly identify the competitive preference priorities it wishes the Department to consider for purposes of earning the

competitive preference priority points, or (2) identifies more than two competitive preference priorities

37

Notes on Competitive Preference Priority 6:Early Learning

• “…improve educational outcomes for high-need students who are young children (birth through 3rd grade) by enhancing the quality of early learning programs”

• “…(a) improving young children’s school readiness (including social, emotional, and cognitive readiness) so that children are prepared for success in core academic subjects (as defined in section 9101(11) of the ESEA); (b) improving developmental milestones and standards and aligning them with appropriate outcome measures; and (c) improving alignment, collaboration, and transitions between early learning programs that serve children from birth to age three, in preschools, and in kindergarten through third grade…”

Focus on High-need Children

Focus on High-need Children

Projects Must Address All 3Projects Must Address All 3

38

Notes on Competitive Preference Priority 7:College Access and Success

• “… enable kindergarten through grade 12 (K–12) students, particularly high school students, to successfully prepare for, enter, and graduate from a two- or four-year college…”

• “…(a) address students’ preparedness and expectations related to college; (b) help students understand issues of college affordability and the financial aid and college application processes; and (c) provide support to students from peers and knowledgeable adults.”

Focus on College

Graduation

Focus on College

Graduation

Projects Must Address All 3Projects Must Address All 3

39

Notes on Competitive Preference Priority 8:Students with Disabilities & Limited English Proficiency

• “…address the unique learning needs of students with disabilities, including those who are assessed based on alternate academic achievement standards, or the linguistic and academic needs of limited English proficient students.”

• “…must provide for the implementation of

particular practices, strategies, or programs that are designed to improve academic outcomes, close achievement gaps, and increase college- and career-readiness, including increasing high school graduation rates (as defined in this notice), for students with disabilities or limited English proficient students.”

Focus on Either Student

Population

Focus on Either Student

Population

Projects That Improve Specific

Outcomes

Projects That Improve Specific

Outcomes

40

Notes on Competitive Preference Priority 9:Improving Productivity

• “…applications for projects that are designed to significantly increase efficiency”

• “…use of time, staff, money, or other resources while improving student learning or other educational outcomes (i.e., outcome per unit of resource)”

• “Such projects may include innovative and sustainable uses of technology, modification of school schedules and teacher compensation systems, use of open educational resources (as defined in this notice), or other strategies.”

Make Significant Improvement

Make Significant Improvement

Must Improve Outcomes

Must Improve Outcomes

Multiple Possible Approaches

Multiple Possible Approaches

41

Notes on Competitive Preference Priority 10:Technology

• “…projects that are designed to improve student achievement or teacher effectiveness”

• “…use of high-quality digital tools or materials”

• “…which may include preparing teachers to use the technology to improve instruction, as well as developing, implementing, or evaluating digital tools or materials”


Students


Students

Two Types of Products

Two Types of Products

Multiple Possible Projects

Multiple Possible Projects

42

Q&A

Schedule for Today

43

Evidence in i3

44

• We will provide an overview of the role of evidence in the i3 competition today

• The Institute of Education Sciences will be hosting a webinar(s) in coming weeks to provide greater detail on evidence and evaluation

• June 30 – further details will be available soon

Grant Types and Evidence

45

• All applications must meet the evidence requirement for the type of grant they are seeking

• Applications that do not meet the evidence requirement will not be eligible for a grant award, regardless of scores on the selection criteria

• If an application is judged not to meet the evidence requirement of the grant type to which it applied, it will not be considered for a different type of i3 grant

Strong Evidence: Scale-up

46

Internal Validity and External Validity

Evidence from previous studies whose designs can support causal conclusions (i.e., studies with high internal validity), and studies that in total include enough of the range of participants and settings to support scaling up to the State, regional, or national level (i.e., studies with high external validity)

Practice, Strategy, or Program in Prior Research

The same as that proposed for support under the Scale-up grant

Participants and Settings in Prior Research

Included the kinds of participants and settings proposed to receive the treatment under the Scale-up grant

Significance of Effect

Effect in prior research was statistically significant, and would be likely to be statistically significant in a sample of the size proposed for the Scale-up grant

Magnitude of Effect Based on prior research, substantial and important for the target population for the Scale-up project

Note: Italicized items are addressed as part of selection criterion A

Strong Evidence: Scale-up (cont’d)

47

Moderate Evidence: Validation

48


Evidence from previous studies whose designs can support causal conclusions (i.e., studies with high internal validity) but have limited generalizability (i.e., moderate external validity), or studies with high external validity but moderate internal validity


The same as, or very similar to, that proposed for support under the Validation grant


Participants or settings may have been more limited than those proposed to receive the treatment under the Validation grant

Significance of Effect

Effect in prior research would be likely to be statistically significant in a sample of the size proposed for the Validation grant

Magnitude of Effect Based on prior research, substantial and important, with the potential of the same for the target population for the Validation project


Moderate Evidence: Validation (cont’d)

49

Examples of Moderate Evidence

(1) At least one well-designed and well-implemented experimental or quasi-experimental study, with small sample sizes or other conditions of implementation or analysis that limit generalizability;

(2) At least one well-designed and well-implemented experimental or quasi-experimental study that does not demonstrate equivalence between the intervention and comparison groups at program entry but that has no other major flaws related to internal validity; or

(3) Correlational research with strong statistical controls for selection bias and for discerning the influence of internal factors

Reasonable Hypothesis: Development

50


Theory and reported practice suggest the potential for efficacy for at least some participants and settings


The same as, or similar to, that proposed for support under the Development grant


Participants or settings may have been more limited than those proposed to receive the treatment under the Development grant

Significance of Effect Practice, strategy, or program warrants further study to investigate efficacy

Magnitude of Effect Based on prior implementation, promising for the target population for the Development project


Reasonable Hypothesis: Development (cont’d)

51

Example of Reasonable Hypothesis

(1) Evidence that the proposed practice, strategy, or program, or one similar to it, has been attempted previously, albeit on a limited scale or in a limited setting, and yielded promising results that suggest that more formal and systematic study is warranted; and

(2) A rationale for the proposed practice, strategy, or program that is based on research findings or reasonable hypotheses, including related research or theories in education and other sectors

52

Q&A

Schedule for Today

53

Notes on i3 Selection Criteria and Points

54

• The selection criteria are the criteria against which the peer reviewers score each application

• The selection criteria for each grant type have different factors

• The points assigned to each selection criterion vary by grant type

• Detailed wording for each selection criterion may be found in the Notices at the i3 website: http://www.ed.gov/programs/innovation/index.html

i3 Selection Criteria and Points

* Depending on the number of applications received, Validation and Development grants may be judged in two tiers: all eligible applications will be scored on Criteria A, B, and D and the competitive preference priorities; then high-scoring applications will be scored on Criterion C by a different panel of reviewers.

Selection Criteria Development Validation

Scale-Up

A. Need for the Project 35 25 30

B. Quality of the Project Design

25 25 30

C. Quality of the Project Evaluation

20 25 20

D. Quality of the Management Plan and Personnel

20 25 20

Total Points 100 100 100

55

Scoring Guidance for Peer Reviewers

56

Max

Point

Value

Quality of Applicant’s ResponseCriterion is

not

addressed

Criterion is

poorly

developed

(major

weaknesses)

Criterion is

adequately

developed

(some

weaknesses)

Criterion is

well

developed

(minor

weaknesses)

Criterion is

fully

developed

with no

weaknesses

35 0 1-12 13-23 24-34 35

30 0 1-10 11-20 21-29 30

25 0 1-8 9-16 17-24 25

20 0 1-7 8-13 14-19 20

Notes on Selection CriterionA. Need for the Project

57

Extent Projects Are Exceptional

Proposals for the Priority

Extent Projects Are Exceptional

Proposals for the Priority

• “The extent to which the proposed project represents an exceptional approach to the priority or priorities established for the competition.” Scale-up, Validation, and Development


58

• “The magnitude of the need for the services to be provided or the activities to be carried out by the proposed project.” Scale-Up and Validation

• “The importance or magnitude of the effect expected to be obtained by the proposed project, including the extent to which the project will substantially and measurably improve student achievement or student growth, close achievement gaps, decrease dropout rates, increase high school graduation rates, or increase college enrollment and completion rates. The evidence in support of the importance and magnitude of the effect would be the research-based evidence provided by the eligible applicant to support the proposed project.” Scale-up and Validation

Extent of the Need for the

Project

Extent of the Need for the

Project

Substantial Expected

Improvement (e.g., prior evidence, statistical

significance)

Substantial Expected

Improvement (e.g., prior evidence, statistical

significance)

Guidance for Scale-up and Validation Grants

59

Applicants may correct known shortcomings from prior evidence, but …

Note Linking Magnitude of Effect to Presented Evidence: The Secretary notes that the extent to which the proposed project is consistent with the research evidence provided by the eligible applicant to support the proposed project is relevant to addressing the third factor of Selection Criterion A and, therefore, will be considered by the Secretary in evaluating the importance and/or magnitude of the impact expected to be obtained by the proposed project.


60

• “The extent to which specific gaps or weaknesses in services, infrastructure, or opportunities have been identified and will be addressed by the proposed project, including the nature and magnitude of those gaps or weaknesses.” Development

• “The extent to which the eligible applicant demonstrates that, if funded, the proposed project likely will have a positive impact, as measured by the importance or magnitude of the effect, on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates.”Development

Extent Project Develops

Solutions to Specific Gaps

Extent Project Develops

Solutions to Specific Gaps

Extent and Likelihood of Impact (e.g.,

prior evidence, statistical

significance of research)

Extent and Likelihood of Impact (e.g.,

prior evidence, statistical

significance of research)

• “The extent to which the proposed project has a clear set of goals and an explicit strategy, with actions that are (a) aligned with the priorities the eligible applicant is seeking to meet, and (b) expected to result in achieving the goals, objectives, and outcomes of the proposed project.” Scale-Up, Validation, and Development

• “The extent to which the costs are reasonable in relation to the objectives, design, and potential significance of the proposed project.”Scale-up, Validation, and Development

• “The potential and planning for the incorporation of project purposes, activities, or benefits into the ongoing work of the eligible applicant and any other partners at the end of the … grant.”Scale-up, Validation, and Development

Notes on Selection CriterionB. Quality of the Project Design

61

Clarity of Project Goals

and Strategy to Achieve Them

Clarity of Project Goals

and Strategy to Achieve Them

Sustainability Designed into Project Plan

Sustainability Designed into Project Plan

Balance of Costs with

Outcomes of Project

Balance of Costs with

Outcomes of Project

Notes on Selection CriterionB. Quality of the Project

Design• “…estimate of the cost of the proposed project,

which includes the start up and operating costs per student per year (including indirect costs) for reaching the total number of students proposed to be served by the project…an estimate of the costs for the eligible applicant or others (including other partners)…” Scale-up: “…to reach 100,000, 500,000, and 1,000,000 students.”Validation and Development: “…to reach 100,000, 250,000, and 500,000 students.”

• “The extent to which the services to be provided by the proposed project reflect up-to-date knowledge from research and effective practice.”Scale-up and Validation

Cost Effectiveness of Scaling Project

to Larger Populations

Cost Effectiveness of Scaling Project

to Larger Populations

Balance of Proposal and Best Evidence

Balance of Proposal and Best Evidence

62

Guidance for All Grant Types

Note: The Secretary considers cost estimates both (a) to assess the reasonableness of the costs relative to the objectives, design, and potential significance for the total number of students to be served by the proposed project, which is determined by the eligible applicant, and (b) to understand the possible costs for the eligible applicant or others (including other partners) to reach the scaling targets of [the grant type requested]. An eligible applicant is free to propose how many students it will serve under its project, and is expected to reach that number of students by the end of the grant period. The scaling targets, in contrast, are theoretical and allow peer reviewers to assess the cost-effectiveness generally of proposed projects, particularly in cases where initial investment may be required to support projects that operate at reduced cost in the future, whether implemented by the eligible applicant or any other entity. Grantees are not required to reach these numbers during the grant period.

Scaling targets help assess cost-effectiveness and ARE NOT number all applicants are expected to reach

63

Notes on Selection CriterionC. Quality of Project Evaluation

• “The extent to which the methods of evaluation will provide high-quality implementation data and performance feedback, and permit periodic assessment of progress toward achieving intended outcomes.”Scale-up, Validation, and Development

• “The extent to which the proposed project plan includes sufficient resources to carry out the project evaluation effectively.”Scale-up, Validation, and Development

Sufficient Funding to Carry Out Evaluation

Sufficient Funding to Carry Out Evaluation

Understanding of Implementation

and Intermediate Outcomes of

Success

Understanding of Implementation

and Intermediate Outcomes of

Success

64

Notes on Selection CriterionC. Quality of Project

Evaluation

• “The extent to which the evaluation will provide sufficient information about the key elements and approach of the project so as to facilitate replication or testing in other settings.”Scale-up and Validation

• “The extent to which the proposed evaluation will provide sufficient information about the key elements and approach of the project to facilitate further development, replication, or testing in other settings.”Development

Evaluation Provides

Information to Support Follow-

on Scaling or Other Activities

Evaluation Provides

Information to Support Follow-

on Scaling or Other Activities

65

Notes on Selection CriterionC. Quality of Project

Evaluation

• “The extent to which the methods of evaluation will include a well-designed experimental study, or if a well-designed experimental study of the project is not possible, the extent to which the methods of evaluation will include a well-designed quasi-experimental design.”Scale-up

• “The extent to which the methods of evaluation will include a well-designed experimental study or a well-designed quasi-experimental study.”Validation

Methodologies of the

Evaluation

Methodologies of the

Evaluation

66

Guidance on Evaluation Goals

All grantees are encouraged to:

Respond to i3 performance measures

Increase strength of evidence available on the impact or promise of i3-supported interventions

Scale-up and Validation grantees are especially encouraged to:

Produce evidence on the impact of the i3-supported intervention (reflecting any changes to the intervention or delivery model) as implemented at scale (reflecting the additional sites served and changes in types of participants and settings served)

67

Guidance on Evaluation PlansApplicants should present clear, detailed evaluation plans

High-quality evaluation plans are encouraged to include:

Key questions and proposed methods for addressing them

Logic model connecting inputs with intermediate and final outcomes

Sampling plan and how it will represent implementation at proposed scale

Summary of data collection measures and methods

Justification of budget

Qualifications of proposed independent evaluation staff

For experimental and quasi-experimental studies: how treatment and control/comparison groups will be formed and plan for measuring treatment/control contrast on key implementation and outcome variables68

Notes on Selection CriterionD. Quality of the Management Plan

• “The adequacy of the management plan to achieve the objectives of the proposed project on time and within budget, including clearly defined responsibilities, timelines, and milestones for accomplishing the project tasks, as well as tasks related to the sustainability and scalability of the proposed project.”Scale-up, Validation, and Development

Whether There Is a Viable Plan to Carry Out the

Project

Whether There Is a Viable Plan to Carry Out the

Project

69


• “The qualifications, including relevant training and experience, of the project director and key project personnel, especially in managing large, complex, and rapidly growing projects.”Scale-up

• “The qualifications, including relevant training and experience, of the project director and key project personnel, especially in managing complex projects.”Validation

• “The qualifications, including relevant training and experience, of the project director and key project personnel, especially in managing projects of the size and scope of the proposed project.”Development

Team’s Experience

Leading Projects Like

the One Proposed

Team’s Experience

Leading Projects Like

the One Proposed

70


• “…capacity (e.g., in terms of qualified personnel, financial resources, or management capacity) to bring the proposed project to scale on a national, regional , or State level working directly, or through partners, either during or following the end of the grant period.” Scale-up

• “…capacity (e.g., in terms of qualified personnel, financial resources, or management capacity) to bring the proposed project to scale on a State or regional level (as appropriate, based on the results of the proposed project) working directly, or through partners, either during or following the end of the grant period.”Validation

Capacity of the Resources to

Scale the Project Further

Capacity of the Resources to

Scale the Project Further

71

i3 Peer Review Process

72

For all three grant types…For all three

grant types…

• The Department will use independent peer reviewers from various backgrounds and professions who have been thoroughly screened for conflicts of interest

• Evaluation experts will score the selection criterion (C) focused on evidence and evaluation

• Peer reviewers will determine whether any competitive preference priority points should be added

Validation & DevelopmentValidation & Development

Scale-Up Only…

Scale-Up Only…

• Peer review will take place in a single tier, with three reviewers reading Selection Criteria A, B, and D, and two reviewers reading Selection Criterion C

• Validation & Development applications may be reviewed in a two tier process (if large numbers of applications are received)

– In Tier 1, all complete applications will be reviewed and scored against Selection Criteria A, B, and D. Competitive Preference Points will also be added as appropriate by peer reviewers.

– Only those highest rated in Tier 1 will advance to Tier 2, where selection criterion C will be scored

73

Q&A

Schedule for Today

74

Key Requirements That Must Be Met Before an Award Is Made

75

The Department, before awarding i3 grants, will confirm that all eligibility requirements have been met by potential grantees, including:

•Requirements related to evidence

– Scale-up applications must be supported by strong evidence

– Validation applications must be supported by moderate evidence

– Development applications must be supported by a reasonable hypothesis

• Evidence of an adequate match has been provided

Details About Evidence Eligibility Review

Performed based on the What Works Clearinghouse standards for determining internal validity

Experts trained in the What Works Clearinghouse standards will review the evidence submitted in support of the applicant’s claim to Strong or Moderate evidence

76

i3 Matching Requirement

77

• Eligible applicants must obtain matching funds or in-kind donations before receiving a grant award equal to:

o at least 5 percent of grant award for Scale-upo at least 10 percent of grant award for Validationo at least 15 percent of grant award for Development

• Only contributions from non-governmental (private) sources count towards matching requirement

• Contributions may be cash or in-kind

• Eligible applicants may count existing private sector support towards the required match so long as these funds are reallocated in support of the proposed project, and the applicant submits appropriate evidence of this commitment

i3 Matching Requirement

78

Pre-submission

Pre-submission

• Applicants may begin to secure matching funds• Applicants may indicate in their application whether they have already secured matching funds

• A pre-secured match will have no competitive impact

• As in last year’s competition, applicants that believe they will be unable to secure the private match may request a reduced matching amount by submitting a waiver request at the time of application

Post - Peer Review

Post - Peer Review

• Eligible applicants will be notified if they are “highest-rated”

• Highest-rated applicants will have several weeks to secure the required match amount and provide evidence of that match

Explanation of Limits on Grant Awards

79

Award CapNo grantee may receive more than two grant awards or more than $55 million in grant awards under this program in a single year. Additionally, no grantee may receive more than one Scale-up or Validation grant in any two-year period.

Award CapNo grantee may receive more than two grant awards or more than $55 million in grant awards under this program in a single year. Additionally, no grantee may receive more than one Scale-up or Validation grant in any two-year period.

Allowable Examples

•Scale-up ($25M) + Development ($3M)•Validation ($15M) + Development ($3M)•2 Development ($3M each)•Scale-up in 2010 + Development in 2011

Allowable Examples

•Scale-up ($25M) + Development ($3M)•Validation ($15M) + Development ($3M)•2 Development ($3M each)•Scale-up in 2010 + Development in 2011

Unallowable Examples

•2 Scale-up or Validation•Scale-up + Validation•Scale-up in 2010 + Validation in 2011•3 Development ($3M each)

Unallowable Examples

•2 Scale-up or Validation•Scale-up + Validation•Scale-up in 2010 + Validation in 2011•3 Development ($3M each)

Notes:•Applicants with more than 2 highest-rated applications may select which 2 applications receive awards•The Award Cap applies to the applicant; official partners and other partners may participate in more than 2 successful applications•2010 Scale-up or Validation grantees may receive up to 2 Development grants in 2011

Notes:•Applicants with more than 2 highest-rated applications may select which 2 applications receive awards•The Award Cap applies to the applicant; official partners and other partners may participate in more than 2 successful applications•2010 Scale-up or Validation grantees may receive up to 2 Development grants in 2011

Schedule for Today

80

Post Award Requirements

81

MUST

MUST

All i3 Grantees All i3 Grantees

• Evaluation– Conduct an independent project evaluation*– Cooperate with technical assistance provided by the

Department or its contractors– Share broadly the results of any evaluation (and data sets for

Validation and Scale-up)• Participate in, organize, or facilitate, as appropriate, communities

of practice for the i3 program

• Evaluation– Conduct an independent project evaluation*– Cooperate with technical assistance provided by the

Department or its contractors– Share broadly the results of any evaluation (and data sets for

Validation and Scale-up)• Participate in, organize, or facilitate, as appropriate, communities

of practice for the i3 program

MUST

* Note: The quality of an applicant’s project evaluation is also a selection criterion.

Schedule for Today

82

Registration for Grants.gov

The Grants.gov registration process involves five (5) basic steps:

1. Obtain a DUNS number2. Register with the Central Contractor Registry (CCR)3. Set up your Authorized Organization Representative (AOR)

profile4. Get authorized as an AOR by your organization’s e-Biz POC5. Track your AOR status

The Grants.gov registration process takes 3-14 business days to complete.

You do not have to register with Grants.gov to view opportunities or to download application packages. You must register to submit.

83

Use the following steps to find the i3 application:

•Log onto www.Grants.gov

•Find Grant Opportunities (on the left)

•Basic Search

•In the basic search, type 84.411 and select the applicable i3 application

Applicants should be careful to select the correct application from the three grant types

How to Find the Application

84 Note: Applicant must download the correct version of Adobe in order to read any Grants.gov application packages

• Click Application (across the top of the page)

• Click Download (towards the bottom of the page)

• At the bottom of the page, you will find the Application Package AND the Application Instructions. These are two separate folders and you should download both. The Application Package is a complete PDF file of all required forms. The Application Instructions contain information on submission procedures and instructions on how to complete all of the mandatory forms.

• You may download both folders to your desktop or some place on your computer. You may then work offline and save and submit the application when you are finished.

How to Find the Application (cont.)

85

Important Information

• Once you download the application, multiple people can work on it and you can work offline

• Save often

• Once the application is complete, the “save and submit” button becomes active

• Submit documents as .PDF files

86

Forms, Assurances, and CertificationsApplicants must complete all forms included in the application package. Please download and review the Application Instructions available on Grants.gov.

•Application for Federal Assistance (SF-424)

•Department of Education Supplemental Information for SF 424

•Department of Education Budget Summary Form (ED 524) Sections A & B

•Disclosure of Lobbying Activities (SF-LLL)

•General Education Provisions Act (GEPA) Requirements - Section 427

•Survey on Ensuring Equal Opportunity for Applicants

•Assurances, Non-Construction Programs (SF 424B)

•Grants.gov Lobbying form (formerly ED 80-0013 form)87

Attach the abstract to the “ED Abstract Attachment Form” in the application package in Grants.gov. The one-page abstract, limited to 2000 characters, should include the following items:

ED Abstract Form

• Project Title, if applicable

• Type of Grant Requested

• Absolute Priority

• Up to two Competitive Preference Priorities

• Brief project description including project activities

• Summary of project objectives and expected outcomes

• Target number of students to be served in the project

• Any special project features

• List of official and other partners

88

Project Narrative File(s)

Project Narrative Addresses the Project Selection Criteria and the Application Content Requirements

•Before preparing the project narrative, interested applicants should thoroughly review the Notice Inviting Applications (NIA) for FY 2011 published in the Federal Register on June 3, 2011.

•The Project Narrative should include, in detail, the eligible applicant’s response to the Selection Criteria and, if applicable, the Competitive Preference Priorities. Eligible applicants should address each of the Selection Criteria since the application will be evaluated and scored against these criteria. The maximum possible score for each criterion is indicated in the NIA.

•Follow the formatting and page limit guidance provided in the application instructions.

•Please upload the project narrative as one document.

89

Budget Form and Related Information

•The ED-524 will be uploaded to the Budget Form (Listed as Department of Education Budget Information for Non-Construction Programs ED-524)

•Section A – Budget Summary U.S. Department of Education Funds – must include a list of the Federal funds the applicant is requesting for the proposed project by budget category and by project year

•Section B – Budget Summary Non-Federal Funds – should include matching funds and other non-Federal resources contributed to the project

•Applicants are also required to fill out Section C and provide explanations and comments necessary to interpret the information you provided on the ED-524

•Sections A and B are already uploaded in Grants.gov under Mandatory Documents; however, an applicant must upload Section C under Budget Narrative Form

90

Budget Narrative Form

•Section C – Budget Narrative – must provide an itemized breakdown and justification by project year for each budget category listed in Section A for the Federal funds requested, including any procurements or subgrants. An applicant should provide a description of the sources and uses of the matching and non-Federal funds.

•Applicant must upload Section C – Budget Narrative. To do this, the applicant will select Budget Narrative Form under Mandatory Documents and receive a prompt to browse and upload its budget narrative.

91

Other Attachment File(s)

Eligible applicants should attach all appendices to the Other Attachments Form. For each appendix eligible applicants are asked to save files as a .PDF, label each file with the Appendix name (e.g., Appendix A – Eligibility Requirement Checklist) and upload the file to the Other Attachments Form.

•Appendix A: Eligibility Requirement Checklist (LEA or Nonprofit Partnership)

•Appendix B: Nonprofit 501C3 status verification or Charter School status verification

•Appendix C: Response to Statutory Eligibility Requirements

•Appendix D: Response to Evidence Standards

92

Other Attachment File(s) (cont.)

• Appendix E: Waiver Request of Private Sector Match Requirement (if applicable)

• Appendix F: Resumes of Key Personnel

• Appendix G: Letters of Support and Memoranda of Understanding (if applicable)

• Appendix H: i3 Applicant Information Sheet

• Appendix I: Eligible Applicant’s list of proprietary information found in the application

(if applicable)

• Appendix J: Other (if applicable)

93

i3 Program Formsi3 Eligibility Checklist

• An applicant applying as a local educational agency (LEA) must use the i3 LEA Eligibility Checklist

• An applicant applying as a partnership between a nonprofit organization and 1) one or more LEAs or 2) a consortium of schools must use the i3 Partnership Eligibility Checklist

Note: An eligible applicant that is a partnership between a non-profit organization and (a) one or more LEAs or (b) a consortium of schools should provide evidence of its partnership in Appendix A following its Eligibility Checklist

• Download the applicable Eligibility Checklist from the i3 Web site: http://www2.ed.gov/programs/innovation/applicant.html

• Complete, save, and upload as Appendix A – Eligibility Requirement Checklist

94

i3 Program Forms

i3 Applicant Information Sheet

• Eligible applicants must complete and submit this information sheet with each application to assist ED staff in assessing the needs of the i3 competition and provide staff with a better sense of the applicant pool

• Applicants must fill out this form electronically. Complete, “Save As” a .PDF, and upload the completed .PDF as Appendix H – i3 Applicant Information Sheet

• To download the i3 Applicant Information Sheet go to the i3 Web site: http://www2.ed.gov/programs/innovation/applicant.html

95

Submission of Proprietary Information

• Applicants should identify any propriety information, confidential commercial information, and financial information

• This will assist the Department in making any future determination regarding public release of the application

• List in Appendix I the page numbers on which this information can be found

96

Submission of Application

Successful Submission Applicants should receive a time

and date stamped confirmation and an assigned tracking number from Grants.gov

Applicants should receive a validation email from Grants.gov within two days business days. This means the application is ready for Department pickup

Applicant should receive an email with their ED assigned PR Award # (U411….)

Unsuccessful Submission Applicants should receive a

confirmation email with a time and date stamp and an assigned tracking number from Grants.gov

If the application is received after 4:30:00 PM (DC Time) on August 2, 2011 or validation is not successful, applicant should receive an error email

Email may list the error, or applicant can use their tracking number to find the submission error

97

Additional Requirements

• Limit the entire application submission to no more than 8MB

• Use 8.5" x 11“, white paper

• Front side only

• 1" margins at the top, bottom, and both sides

• Double space

• 12 point font or larger

• Use Times New Roman, Courier, Courier New, or Arial font

• Include a table of contents that specifies where each required part of the application is located

• Submit as PDF files

98

Problems with Grants.gov

• If you experience technical difficulties within Grants.gov, please contact the Grants.gov Help Desk

• Please keep all tracking numbers, emails, and complaints filed with Grants.gov

• If the problem is really a Grants.gov problem, your application may be accepted, but you must provide proof

99

Grants.gov contact center:http://www07.grants.gov/contactus/contactus.jsp

For phone assistance call:1-800-518-4726

www.Grants.gov

100

101

Q&A

Schedule for Today

102

103

i3 Key DatesNotices Published in Federal Register: June 3, 2011

Technical Assistance:Informational Conference Calls: June 2011Pre-Application Workshops: June 17 (Washington, DC)

June 24 (San Francisco)June 28 (Houston)

IES Evidence & Evaluation June 30 (Time TBD)Webinar(s):

Applications:Intent to Apply Due: June 23, 2011Applications Due: August 2, 2011Applications Reviewed: Late Summer/Fall 2011All Grant Awards Announced: December 2011

Other Important ResourcesInvesting in Innovation Fund Website: (http://www2.ed.gov/programs/innovation/index.html)

Notices of Final Revisions to Priorities, Requirements, and Selection Criteria

Application Packages for each competition (includes the respective Notice Inviting Applications) Eligibility Checklists

Frequently Asked Questions Evidence Summary Table Selection Criteria Summary Table

i3 At-A-Glance (quick reference) Call for Peer Reviewers:

http://www2.ed.gov/programs/innovation/peerreviewers.html

All questions about i3 may be sent to [email protected]

104

Schedule for Today

105

Key Things to Remember

106

• i3 will likely be extremely competitive – 1700 applications for 49 awards last year and this year we are estimating about half as many awards

• If you applied last year, make sure you understand the changes to the program – and be sure to address evidence and prior experience eligibility requirements

• Write clearly to the selection criteria: they are what the peer reviewers will use to judge your application, so explain why your project is important, what outcomes you expect, how you will achieve them, what the nation will learn

• Discuss how you will do what you claim you will do – do not just state that you will do it

Key Things to Remember

107

• Evidence is critical to i3: make sure you (or your expert advisor) understand it, and if you are not sure if you meet the evidence threshold for a grant type, you may want to consider dropping down a level

• Check the i3 Web site often for updates to FAQs (others probably have the same question as you and we may have already answered it) and other important competition information

• Register for grants.gov early, make sure you understand how to use it, and leave yourself plenty of time to submit your application on time (the deadline of 4:30:00PM DC Time applies to the completion of the submission, not the beginning)

summary document june 2011 investing in innovation (i3) fund pre-application meeting note: these...

Documents

i3 website

i3 mailbox

innovation i3 notices

i3 programapplication

i3 programvibrant competition

specific entity questions

definition of innovation

high internal validity