SRAE Performance Measures: Updates to

Participant Entry and Exit SurveysJu ly 8 , 2020

3 :00–4 :30 p .m. EDT

O l i v i a A s h l e y Public Strategies

J e n n i f e r G a r d R e a d Public Strategies

J e s s i c a J o h n s o n Family & Youth Services Bureau

2

QUESTIONS AND ANSWERS• Use the chat box to submit a

question to the presenters

• Your questions will only be visible to our team, and not other attendees

• If you do not see the chat box on your screen, click on the chat icon at the bottom of your WebEx window

3

CHANGING THE WEBINAR WINDOW• To expand your view, click on the

full-screen icon in the top right corner of your window

• To exit full-screen mode, hover your mouse at the top of your screen and select the return button from the drop-down menu that appears

• To change the size of the text on the screen, hover your mouse at the left of your screen and use the + and - buttons

4

WEBINAR MATERIALS AND RECORDING

• The webinar recording and materials will be emailed to grantees and webinar attendees about one week after the webcast

• Handouts and slides for this presentation will be available for download at the end of this webcast

5

WEBINAR TECHNICAL ASSISTANCE

• If you have technical difficulties, please call the WebEx customer service number at 1-866-229-3239, Option 1

• You can also click on the chat box to submit technical questions

WELCOME

6

Welcome

7

IMPORTANT ANNOUNCEMENTS• The data collection start date for participant entry and exit surveys

has been changed from July 1 to September 1

• The participant entry and exit surveys are OMB approved and will be released within the next two weeks

• Once the participant entry and exit surveys are released, grantees will be able to contact their federal project officer to request a waiver for sensitive items, if needed

AGENDA• Welcome and introductions

• Review of performance measures data collection and submission timelines

• Overview of changes to survey-based performance measures

• Guidance about requesting waivers

8

POLLDoes your grant serve middle school age youth?

• Yes

• No

9

POLLDoes your grant serve high school age youth?

• Yes

• No

10

THREE CATEGORIES OF SRAE PERFORMANCE MEASURES

11

1. Structure, cost, and support for program implementation

2. Attendance, reach, and dosage

3. Participant characteristics, perceptions of program effects, and program experiences

PERFORMANCE MEASURES DATA SUBMISSION SUMMARY

12

Measures Frequency of submission Period covered by dataFirst submission

window

Structure, cost, and support for program implementation

AnnuallyFederal grant year (e.g., October 2019 through September 2020)

Mid-July through August 2020

Attendance, reach, and dosage BiannuallySix months (e.g., January through June 2020)

Mid-July through August 2020

Participant characteristics, perceptions of program effects, and program experiences (entry and exit survey data)

Biannually

Six months (except first round, which will be September through December 2020)

January/February 2020

GUIDANCE FOR CSRAE GRANTEES WHOSE FUNDING ENDS SEPTEMBER 2020

13

Title V Competitive SRAE grantees whose grant period ends September 29, 2020 will be required to collect the implementation performance measures: • Structure, cost, support (for October 2019 to September 2020)• Attendance, reach, and dosage (for January to June 2020)

These grantees are not required to administer or collect entry and exit surveys.

OVERVIEW OF THE SURVEY-BASED YOUTH PARTICIPANT PERFORMANCE MEASURES

14

OVERVIEW OF THE SURVEY-BASEDPERFORMANCE MEASURES

15

• Unless indicated in orange, performance measures will remain the same for both middle school and high school participants

• Orange text indicates recent changes to the performance measures

• The middle school version of each survey is a subset of the questions in the high school version o All SRAE participants of middle or high school age should

complete performance measures surveys o This includes participants who receive programming at non-

school sites and participants at K-8 schools

MIDDLE SCHOOL AND HIGH SCHOOL PARTICIPANT ENTRY AND EXIT SURVEYS

• Age (different response options in middle school and high school versions)• Grade (different response options in middle school and high school versions)• Language(s) spoken at home or with family• Hispanic/Latino ethnicity• Race• Sex• Living situation (revised)

16

Purpose: To obtain data on the characteristics of the youth who receive SRAE programming

MIDDLE SCHOOL AND HIGH SCHOOL PARTICIPANT ENTRY AND EXIT SURVEYSAGE (QUESTION 1)

17

Version Response categories

Middle School age 10 - 16

High School age 10 - 20 or older

MIDDLE SCHOOL AND HIGH SCHOOLPARTICIPANT ENTRY AND EXIT SURVEYS GRADE (QUESTION 2)

What grade are you in? (If you are currently on vacation or in summer school, indicate the grade you will be in when you go back to school.)MARK ONLY ONE ANSWER

MIDDLE SCHOOL: HIGH SCHOOL:• 5th, 6th, 7th, 8th, 9th ● 9th, 10th, 11th, 12th• My school does not assign grade levels ● My school does not assign grade levels• I am not currently enrolled in school ● I dropped out of school, and I am not working on

getting a high school diploma or GED

I am working towards a GED●● I have a high school diploma/GED but I am not

currently enrolled in college/technical school ● I have a high school diploma/GED and I am

currently enrolled in college/technical school 18

MIDDLE SCHOOL AND HIGH SCHOOL PARTICIPANT ENTRY AND EXIT SURVEYS LIVING SITUATION (QUESTION 7)Are you currently …? MARK ALL THAT APPLY□ Living with family [parent(s), guardian, grandparents, or other relatives]□ In foster care, living with a family□ In foster care, living in a group home□ Couch surfing or moving from home to home□ Living in a place not meant to be a residence, such as outside, in a tent city or

homeless camp, in a car, in an abandoned vehicle or in an abandoned building □ Staying in an emergency shelter or transitional living program□ Staying in a hotel or motel□ In juvenile detention, jail, prison, or other correctional facility, or under the supervision

of a probation officer □ None of the above

19

MIDDLE SCHOOL AND HIGH SCHOOL PARTICIPANT ENTRY SURVEYBEHAVIORS BEFORE SRAE PROGRAMMING

20

Question Numbers Constructs

8 Alcohol, tobacco, or other substance use

9 Healthy decision-making, self-regulation, healthy relationships

10 Goal setting, focus on the future, self-sufficiency, healthy relationships

11 Healthy relationships; how to resist and avoid, and receive help

regarding, dating violence and sexual coercion

12 Success sequence

Purpose : To obtain data on participant behaviors related to topics in the legislative definition of SRAE

HIGH SCHOOL PARTICIPANT ENTRY SURVEY BEHAVIORS BEFORE SRAE PROGRAMMING

21

Question Numbers Constructs

13 Sexual intercourse

14 Pregnancy

15 Sexually transmitted infection

Purpose: To obtain data on high school participant behaviors related to topics in the legislative definition of SRAE

MIDDLE SCHOOOL AND HIGH SCHOOL PARTICIPANT EXIT SURVEY PERCEPTIONS OF PROGRAM EFFECTS

22

Question Numbers Constructs

8-12Similar to Entry Survey“Has being in the program made you more likely, about the same, or less likely to …”

Purpose: To obtain data on participant behaviors related to topics in the legislative definition of SRAE

HIGH SCHOOL PARTICIPANT EXIT SURVEY PERCEPTIONS OF PROGRAM EFFECTS

23

Question Numbers Constructs

13-14 Participant plan and reasons to abstain

Purpose: To obtain data on high school participant behaviors related to topics in the legislative definition of SRAE

MIDDLE SCHOOL AND HIGH SCHOOL

PARTICIPANT EXIT SURVEY

PROGRAM EXPERIENCES

Even if you didn’t attend all of the sessions or classes in this program, how often in this program…

24

MARK ONLY ONE ANSWER PER ROWAll of the

TimeMost of the Time

Some of the Time

None of the Time

a. did you feel interested in program sessions and classes?…………………

b. did you feel the material presented was clear?………………………………

c. did discussions or activities help you to learn program lessons?……………

d. did you have a chance to ask questions about topics or issues that cameup in the program?……………………………………………………

e. did you feel respected as a person?………………………………

f. were you picked on, teased, or bullied in this program?…………………

Purpose: To obtain data on youth program experiences

MIDDLE SCHOOL AND HIGH SCHOOL PARTICIPANT EXIT SURVEY PROGRAM EXPERIENCES

25

Purpose: To obtain data on youth program experiences Now thinking about all youth in this program, how often…

MARK ONLY ONE ANSWERAll of the

TimeMost of the Time

Some of the Time

None of the Time

a. were any youth in this program picked on, teased, or bullied?

INFORMATION ABOUT REQUESTING WAIVERS

26

AFTER SURVEYS ARE RELEASED, SHARE WITH SUBAWARDEES

27

FYSB recommends that grantees share the surveys with their subawardees first to identify any items that do not align with the policies and practices of the local implementation sites.

• Local implementation sites may have policies and practices related to data collection or sensitive questions already in place.

• FYSB strongly recommends meeting with stakeholders, including school boards and sites, to gather information about

data collection practices.

REQUEST TO WAIVE SPECIFIC ITEMS, IF NEEDED

28

• Based on information related to the policies, practices, and processes, the grantee should request to waive items that do not align.o Please note that documentation of the policies and practices

is required to be submitted with the waiver request.

• After waiver approval, FYSB will share with you a PDF of the FYSB-approved version(s) of the survey(s). The grantee should share FYSB-approved revised survey(s) with school boards and implementation sites.

REQUEST A WAIVER FORM FROM YOUR PROJECT OFFICER, IF NEEDED

The waiver form will be available when the final OMB-approved surveys are released.

Please talk with your Project Officer about your need for a waiver. Your Project Officer will send the waiver form to you upon request.

29

TYPES OF ITEMS FOR WHICH A WAIVER CAN BE REQUESTED

30

The waiver form is not restricted to specific items.

Grantees may request a waiver for any item.

Grantees will also be able to request waivers for specific:

• Ages• Grades• Program providers• Implementation sites• Implementation settings• Populations

LENGTH OF WAIVER REVIEW PROCESS

31

FYSB has established a review process that should take no longer than 2 weeks after submission to your Project Officer. During the review process, you may be requested to revise or submit additional information. If this occurs, the approval process may take longer.

WAIVER APPROVAL

32

FYSB will send a PDF of the approved revised survey to you along with the email notification that states your waiver request is approved.

33

UPCOMING TECHNICAL ASSISTANCE EVENTS

Event Timeline ContentWebinar: FYSB Guidance on Obtaining Approval from Partners

July 22, 2020

(3–4:30 p.m. EDT)

FYSB guidance on obtaining IRB and partner approvals for participant entry and exit survey performance measures

Cluster call: Online Data Collection

August 6, 2020

(3–4:30 p.m. EDT)

Small group of SRAE grantees discussion of strategies to collect participant entry and exit survey performance measures data online

Webinar: Survey Administration Guidelines

August 19, 2020

(3:30–5 p.m. EDT)

Best practices for administering participant entry and exit survey performance measures

34

QUESTIONS AND COMMENTS

?

FOR ADDITIONAL INFORMATION AND QUESTIONS

ABOUT SRAE PERFORMANCE MEASURES

• Additional information about the SRAE performance measures is

available at www.sraepas.com.

• For further support, contact Public Strategies at

SRAEperformancemeasures@publicstrategies.com or call toll-free

833-797-0166.

35

http://www.sraepas.commailto:SRAEperformancemeasures@publicstrategies.com

SRAE Performance Measures: Updates to Participant Entry and Exit SurveysQUESTIONS AND ANSWERSCHANGING THE WEBINAR WINDOWWEBINAR MATERIALS AND RECORDINGWEBINAR TECHNICAL ASSISTANCEWelcomeIMPORTANT ANNOUNCEMENTSAGENDAPOLLTHREE CATEGORIES OF SRAE PERFORMANCE MEASURESPERFORMANCE MEASURES DATA SUBMISSION SUMMARYGUIDANCE FOR CSRAE GRANTEES WHOSE FUNDING ENDS SEPTEMBER 2020OVERVIEW OF THE SURVEY-BASED YOUTH PARTICIPANT PERFORMANCE MEASURESMIDDLE SCHOOL AND HIGH SCHOOL PARTICIPANT ENTRY AND EXIT SURVEYSAGE (QUESTION 1)GRADE (QUESTION 2)LIVING SITUATION (QUESTION 7)BEHAVIORS BEFORE SRAE PROGRAMMINGHIGH SCHOOL PARTICIPANT ENTRY SURVEY BEHAVIORS BEFORE SRAE PROGRAMMINGPERCEPTIONS OF PROGRAM EFFECTSPROGRAM EXPERIENCES

INFORMATION ABOUT REQUESTING WAIVERSAFTER SURVEYS ARE RELEASED, SHARE WITH SUBAWARDEESREQUEST TO WAIVE SPECIFIC ITEMS, IF NEEDEDREQUEST A WAIVER FORM FROM YOUR PROJECT OFFICER, IF NEEDEDTYPES OF ITEMS FOR WHICH A WAIVER CAN BE REQUESTED

LENGTH OF WAIVER REVIEW PROCESSWAIVER APPROVALUPCOMING TECHNICAL ASSISTANCE EVENTSQUESTIONS AND COMMENTSFOR ADDITIONAL INFORMATION AND QUESTIONS ABOUT SRAE PERFORMANCE MEASURES