Community-Level Instrument—Revised (CLI-R)

Categorizing/Classifying Interventions

PEP-C PFS Cross-Site EvaluationTraining and Technical Assistance Team

April 2017Partnerships for Success Grantees

Welcome3

Training Logistics

• Adobe Connect Features

• A training recording and questions & answers document will be posted on the PEP-C Knowledge Base

Click phone/microphone icon next to your name to mute and unmute

Agenda/Topics

Grantee Questions

PEP-C Technical Assistance

Community-Level Instrument—Revised (CLI-R) Reporting Interventions Content Guidance

CLI-R Schedule

• The CLI-R is submitted twice per year

• Subrecipients must submit the CLI-R to grantees first, and then grantees must finalize submission

• Grantees should set internal deadlines for subrecipients so that there is sufficient time for grantees to review and accept by the CLI-R submission deadline

6

Reporting Period CLI-R Submission DeadlineOctober 1 – March 31 June 1April 1 – September 30 December 1

Content Guidance for Reporting Interventions on the CLI-R

7

CLI-R Items

• Items to be completed vary by reporting period• The item key can be found in the CLI-R hard-copy version

– Baseline (B)– Annual (A)– Every time (E; semiannual) – Final (F)

• !! Be careful about checking “This is the final reporting period” !!

Q33: Reporting Interventions

• Two steps in reporting intervention implementation:1. In Q33, list the interventions and

intervention-service types implemented

2. Complete a sub-form for each intervention-service type listed in Q33

9

Q33a–c: Intervention & Service Type

• Report Intervention and related Service types– Select Intervention

from drop-down list– Select Service type

from drop-down list– Type in Service type

name

10

Q33a–c: Selecting Other

• Select Other for Intervention and Service type only if it does not appear in the drop-down list

• If you do select Other, please include a name and description in the service type name to help us clearly understand exactly what you implemented

11

Q33a–c: Selecting Other:Media campaigns

• An issue that often comes up:– If you only have one media campaign, the

Service Type should not be labeled as Other– If you have more than one media campaign,

select Other for Service Type and clearly describe the interventions as “[NAME OF CAMPAIGN] media campaign”

12

Q33a–c: Example 1

Intervention name and service type name will often be the same. For example:

Intervention Name Service Type Service Type NameLife skills training Classroom educational

servicesLife skills training

Social norms marketing campaign

Media campaign Social norms marketing campaign

Retailer compliance checks

Compliance checks Retailer compliance checks

13

Q33a–c: Example 2

Some interventions consist of multiple service types that must each be reported in a separate sub-form. For example:

Intervention Name Service Type Service Type NameCommunity trials intervention

Classroom educational services

Life skills training

Community trials intervention

Media campaign Social norms marketing campaign

Community trials intervention

Compliance checks Retailer compliance checks

14

Service Type Name Error

To reduce reporting burden, list only the overall service type name, rather than separate activities.

Correct Names Incorrect NamesLife skills training Life Skills Grade 8 Elm Middle School

Life Skills Grade 8 Maple Middle School Life Skills Grade 8 Oak Middle School

Social norms marketing campaign

Radio Ads Billboard Posters

Social host liability ordinance

Meet with Legislators Lobby Day Community Presentation15

Activities That Are Not Interventions

• Coalition meetings• Needs assessment

surveys• Capacity building• Advertising for an

intervention• Logic modeling• Evaluations• Pre/post surveys

• There are some exceptions to this rule– For example, Communities That

Care (CTC) may include some of these activities as part of the intervention, but you should not list every activity separately (e.g., repeating CTC meetings should be under one service type, not listed separately)

16

Activities That Are Part of 1 Intervention

• Report activities for an intervention as 1 intervention and service type– For example, meeting with government officials

is not by itself an intervention, but it may be part of a larger policy change intervention

• Do not list information dissemination activities separately if the same information or information to support the same purpose is provided

• Information about when and where to find prescription drug drop boxes should not be listed separately from the drop box intervention itself

17

Impact of Listing Specific Activities

Why is more not better?• Subrecipients will have to complete a

separate sub-form for every intervention-service type they list– Resulting in increased subrecipient burden and

unhappiness

• The Data Cleaning Team will ask the grantee/PE to help the subrecipient redo service type reporting– Resulting in increased grantee burden and

unhappiness18

How Do Subrecipients Get Credit for All Activities Within an Intervention-Service Type?

• Targeting multiple locations? Use Q39• Reaching multiple groups or

implementing multiple times? Use relevant items in sub-forms (for example, Q67a–c)

• Conducting a variety of activities? Use relevant items in sub-forms (for example, Q149–Q161)

19

Grantee Questions

• Questions• Comments • Concerns

20

Q33d: Service Type Start Date

• Record the date that the intervention-service type was funded as part of your PFS initiative– If the intervention-service type existed in the

community before PFS, record the date funded as part of your PFS initiative

• The Date Started field is needed to calculate the time from the date the subrecipient was funded to the date intervention implementation began

21

Q33e: CSAP Strategy Type

• Selecting the right CSAP strategy type is critical—the strategy type determines which questions are asked in sub-forms– Community-based processes– Prevention education– Alternative drug-free activities– Problem identification and referral– Environmental strategies– Information dissemination and communication

Note that for PFS, social marketing/media campaigns are reported in the Information Dissemination sub-form

22

Q33e: CSAP Strategy Type (cont.)

• To determine the correct CSAP strategy type, refer to the Intervention-Service Type List on the PEP-C Knowledge Base– The Intervention-Service Type List matches each

intervention-service type to the correct CSAP strategy type

• If you have any questions about which CSAP strategy type is appropriate, please contact the PEP-C TA Team

– Email: PFS-PEPC@ccs.rti.org– Phone: (866) 558-0724

23

• Problems persist with subrecipients selecting wrong CSAP strategy OR reporting activities as separate interventions

• Solutions:– Some grantees have subrecipients complete the

Q33 worksheet and have it approved before the subrecipient can continue with reporting

– Some grantees enter subrecipients’ Q33 data for them before the subrecipient can continue

Preventing Intervention Reporting Problems

24

Q33h: Service Type Status

• Q33h is a “Status” field to report whether an intervention-service type was active during the period – If a service type was active at any point during the reporting

period, it must be listed as Active during this period– Submission of the CLI-R requires the completion of a sub-

form for all service types listed as Active during this period• If a service type was not active for the entire reporting

period, mark it as either…– Inactive during this period– No implementation during this period: Completed– No implementation during this period: Discontinued

25

Q39a: Locations

• Report the locations where you are implementing the intervention-service type. For example:– A prevention program in 5 schools:

List each school as a location

– An environmental strategy such as compliance checks or policy change: List targeted geographic areas—a county, one or more towns, or neighborhoods within a city

26

Q39g: Estimated Target Population Size

• Focus on the target population for the service type, not the overall community – For example, if the community has 10,000 people, but

only 3,000 of those are in the target 12- to 20-year-old age group, then the estimated target population is 3,000

• Provide an estimate for each location listed• Estimates may come from school district

information, partner agencies, or Census data

27

Reporting Dosage

• Depending on the intervention-service type selected, you will report intervention dosage in every reporting period or during the annual reporting period

• Dosage questions vary depending on the sub-form, so be sure to read those items carefully

• For example, in the Prevention Education sub-form, you will be asked to report number of sessions and average length of sessions annually

• Be consistent with how you report dosage

28

Reporting Numbers Reached

• For each intervention-service type, you will report numbers reached in the 6-month reporting period– The 6-month reporting periods are October 1 –

March 31 and April 1 – September 30 • In the second half of the Federal fiscal year (April –

September), you will also report how many were new to this period– Allows the PEP-C team to calculate an unduplicated

number of individuals who received the intervention-service type during the Federal fiscal year

29

Numbers Reached for Population-Based Strategies

• Population-based strategies* influence the environment, so numbers reached are estimates– Census information may be helpful:

http://factfinder.census.gov/faces/nav/jsf/pages/index.xhtml

*Population-based strategies: community-based processes, environmental strategies, information dissemination

30

Numbers Reached for Population-Based Strategies: Example

• For example, a compliance check effort to reduce alcohol access among youth age 12 – 20:– During the October – March reporting period, the

community used Census data to estimate reaching 3,000 (youth age 12 – 20)

– In the April – September period, the effort continued and the community again reported reaching 3,000

• The number of new individuals reached was 0– But if the campaign expanded in April – September to

reach an additional town (5,000 youth): • The number reached in April – September is 8,000 (3,000 in Town

1 + 5,000 in Town 2)• The number of new individuals is 5,000

31

Demographic Items

• For individual direct service strategies, the best source of demographic information is usually participants themselves

• For population-based strategies, estimates may be derived from…– Census data – Media companies’ audience information– Local military- and veteran-serving institutions (for

information about military status in your community)

32

Grantee Questions

• Questions• Comments • Concerns

33

Technical Assistance

34

Request Technical Assistance

• If you have questions about the cross-site evaluation– Click the Technical Assistance link in the

PEP-C MRT to access the FAQ document, Knowledge Base, and TA request form• TA providers will respond by email or phone

– Contact PEP-C by email or phone:• Email: PFS-PEPC@ccs.rti.org• Phone: (866) 558-0724

35

1

Community-Level Instrument—Revised (CLI–R) Categorizing/Classifying Interventions:

Slide Notes

PEP-C PFS Cross-Site Evaluation Training and Technical Assistance Team

April 2017 Partnerships for Success Grantees

Slide 1 - Cover [Slide 1 has no notes.]

Slide 2 - Community-Level Instrument—Revised (CLI-R) Categorizing/Classifying Interventions Hello, and welcome to the training on categorizing and classifying interventions for the Community-Level Instrument—Revised (CLI-R). My name is [name] and I am a [role on PEP-C] for the PEP-C team. I am joined today by my colleagues, [list other PEP-C team members on call].

As a reminder, the CLI-R requirement can be completed and submitted by grantee- level or community-level staff, so those responsible for entering, submitting, and approving CLI-R data should attend or review the questions and answers document developed based on this discussion today. If you are not sure about your responsibilities related to the CLI-R, please check in with your grantee-level Project Director.

Slide 3 - Welcome As a reminder, PEP-C stands for Program Evaluation for Prevention Contract. We are the team that is conducting a cross-site evaluation of the PFS efforts across the country.

Slide 4 - Training Logistics Today we will be using Adobe Connect. This platform offers a variety of features. There are several tools that you can use to communicate with us. For instance, you can raise your hand, but the most commonly used feature is the chat box located in the lower right-hand corner of your screen. You can type your questions in that box and [question moderator] will be sure to raise your questions after each section. If you prefer to ask your questions verbally and you have called into the training, you can press *6 to unmute your line and ask your question. Once you are done asking your question, we ask that you press *6 again to re-mute your line to minimize any background noise.

The PowerPoint slides as well as a Q&A document will be posted to the Knowledge Base for your reference.

Slide 5 - Agenda/Topics [Slide 5 has no notes.]

Slide 6 - CLI-R Schedule REMINDER: The CLI-R is submitted twice per year. Subrecipients and single- community grantees need to start completing CLI-Rs for the first reporting period they receive PFS funding. Subrecipients must submit the CLI-R to grantees first, and then grantees must finalize submission. Grantees should set internal deadlines for subrecipients so that there is sufficient time for grantees to review and accept by the CLI-R submission deadline.

2

The two CLI-R reporting periods follow the Federal fiscal year, which begins October 1st.

The CLI-R covering the first half of the fiscal year (October 1 – March 31) is due June 1.

The CLI-R covering the second half of the fiscal year (April 1 – September 30) is due December 1.

Grantees have 2 months to complete their CLI-Rs after the end of a reporting period.

Slide 7 - Content Guidance for Reporting Interventions on the CLI-R [Slide 7 has no notes.]

Slide 8 - CLI-R Items CLI-R items to be completed vary by reporting period. The item key can be found in the CLI-R hard-copy version available on the PFS Knowledge Base. B signifies baseline items, A signifies annual items (which appear only in the reporting period for the second half of the federal fiscal year—the April to September report due December 1), E signifies every time or semiannual items, and F signifies final reporting period items. Grantees and subrecipients need to be careful to check the “This is the final reporting period” box only when it is appropriate. It’s very important that this box is not checked until the very last period of subrecipient funding, usually at the end of the grant or no-cost extension year. Annual questions are asked when you check the “This is the final reporting period” box, regardless of reporting period. If you accidentally check “This is the final reporting period,” you may be asked final and annual questions that you don’t need to answer yet. You may still be required to answer those final and annual questions even if you realize your mistake and uncheck the box (it depends on how many items you answer before the box is unchecked). If this happens to you, please let us know so that we can alert the Data Cleaning Team to this issue. You can enter erroneous data for the annual questions that should not be appearing. Those will be cleaned out by the Data Cleaning Team. Also, if this box is checked, no follow-up reporting periods will appear until the box is unchecked because the system assumes you are done reporting. It’s also very important that grantees remind subrecipients who ARE in their final reporting period to check the final reporting period box BEFORE they start entering data so that the final and annual questions show up.

Slide 9 - Q33: Reporting Interventions In Question 33, subrecipients need to report the interventions they have implemented during the reporting period. There are 2 steps in reporting interventions that have been implemented. First, in Question 33, subrecipients will need to list the interventions and intervention-service types that have been implemented. Then subrecipients will need to complete a corresponding sub-form for each intervention- service type that they listed. We recommend that grantee evaluators use the Question 33 worksheet to check their subrecipients’ responses before subrecipients enter data in the MRT.

[PULL UP QUESTION 33 WORKSHEET]

Slide 10 - Q33a–c: Intervention & Service Type In Question 33, you will first select the intervention from the intervention name drop- down menu in 33a. The intervention name drop-down list is lengthy, but it will help enormously with data quality: In SPF SIG, this was an open text field and the data were error-filled and hard to use in analyses. Note that you can begin typing an intervention name to narrow the list (instead of scrolling through the lengthy list). In Question 33b, you will select the service type from the drop-down list. Remember, the service type you select determines the sub-form you will need to complete related to this intervention. Finally, in 33c, you

3

will need to enter the service type name. In many cases, the service type name will be the same as the intervention name.

Slide 11 - Q33a–c: Selecting Other –Select Other for Intervention Name and Service Type only if it does not appear in the drop-down list. Like I mentioned, the intervention drop-down list is lengthy, but it will help with data quality. The Intervention Name drop-down and Service Type drop- down lists are available on the Knowledge Base. That should help with making the right selection before you enter data in the MRT. Please look through the list to see if your intervention is on there under some slightly different configuration before selecting Other. But if your intervention name is not on there, please list it after selecting “Other.”

[PULL UP INTERVENTION NAME LIST]

–The interventions list is not exhaustive. It includes the names of some commonly used interventions and some interventions from evidence-based program lists (such as SAMHSA’s NREPP). But many interventions that subrecipients may choose to implement are not included on the list. If that is the case, please select Other and provide the name of your intervention. If you do select Other for intervention name or service type, please include a name and description in the service type to provide us with a clear understanding of what exactly you implemented.

Here are some examples of “Other” labels that are not very helpful—

• Awareness Campaign, Community Education Campaign, Community Presentations, Information Education, Media Advocacy, Media Campaign, Prevention-Focused Education, Social Media, Translation Services

• Little additions to these would help in a big way—for example, “Awareness Campaign” could be “Campaign for Parental Awareness of Underage Drinking” OR “Campaign to Educate Community on Not Sharing Prescription Drugs”

• It is very important to describe what is going on in the intervention so the data team can understand your data.

Slide 12 - Q33a–c: Selecting Other: Media campaigns • If you only have one media campaign, the Service Type should not be labeled as Other. You should

select “Media Campaign” from the Service Type drop-down list. • If you have more than one media campaign (for example, serving different target audiences, such

as one for youth and one for parents), select Other for Service Type and clearly describe the interventions as “[NAME OF CAMPAIGN] media campaign”

Slide 13 - Q33a–c: Example 1 The intervention name and service type name will often be the same. Here are some examples. Life skills training is the intervention name and service type name and the service type is classroom educational services. A social norms marketing campaign is the next intervention listed—social norms marketing campaign is listed as the intervention name and the service type name. The service type is media campaign. And, finally, retailer compliance checks is listed for intervention name and service type name and the service type is compliance checks.

Slide 14 - Q33a–c: Example 2 Some interventions consist of multiple service types that must each be reported in separate sub-forms. For example, the community trials intervention in this example, consists of 3 service types—classroom

4

educational services, media campaign, and compliance checks. You notice that the related service type name is the last column. The classroom educational service consists of a life skills training, the media campaign is a social norms marketing campaign, and the compliance checks are retailer compliance checks.

Note: These are the same service types and service type names used in Example 1 on the previous slide [flip back to previous slide]. The difference between how they are listed is in Example 2: these service types are all part of a broader intervention, community trials intervention. In Example 1, they are all implemented as separate interventions. This can be a little confusing, so please be sure to ask questions now or check in with the PEP-C TA Team before you enter your intervention information into the PEP-C MRT.

Slide 15 - Service Type Name Error We continue to notice errors in listing service type names, which adds extra burden to subrecipients because they need to complete a sub-form for each service type listed. To reduce reporting burden, list only the overall service type name, rather than separate activities. For example, do not list the same service type repeatedly if it was implemented in different locations—life skills training should be listed as the service type name, not the 3 locations in which it was implemented (Elm, Maple, and Oak middle schools in this example). Also, the media types used to advertise for a social norms marketing campaign should not be listed separately in the service type name field. Simply list “social norms marketing campaign” as the service type name. Finally, activities you completed as part of an overall intervention—like meeting with legislators or other policymakers, participating in a lobby day, or giving a community presentation—should not be listed separately. You should just list the overall intervention—social host liability ordinance in this example. There are other places in the CLI-R where you will note the locations, means, and activities you engaged in as part of your intervention implementation efforts. I will explain that in a couple slides.

Slide 16 - Activities That Are Not Interventions We have noticed that some subrecipients are listing organizational, systems, and evaluation activities as interventions. Coalition meetings, needs assessments surveys, capacity building, advertising for an intervention (like prescription drug drop boxes), logic modeling, evaluations, and pre/post surveys are not interventions.

As always with the CLI-R and the complex interventions you all are implementing, there are some exceptions to the rule that these activities are not interventions. For example, Communities That Care may include some of these activities as part of the intervention, but you should not list every activity separately. (For example, repeating coalition meetings should be under one service type, not listed separately.)

If you have any questions about whether an activity is an intervention or not, please reach out to the PEP-C TA Team. We would be happy to discuss this topic with you. Having these discussions beforehand can save a lot of time for you because you will be asked to change data inaccurately entered after the Data Cleaning Team processes your data.

Slide 17 - Activities That Are Part of 1 Intervention Many interventions consist of several activities. Be sure to report activities for an intervention as 1 intervention and service type. For example, meeting with government officials is not by itself an intervention, but it may be part of a larger policy change intervention. Do not list information dissemination activities separately if the same information or information to support the same purpose is provided.

5

Finally, information that was passed out to community members about when and where to find prescription drug drop boxes should not be listed separately from the drop box intervention itself.

Slide 18 - Impact of Listing Specific Activities Subrecipients may be trying to impress grantees with their quantity of activities, but doing so will end up creating more work for the grantee and subrecipient. Why is more not better? Because subrecipients will have to complete a separate sub-form for every intervention-service type they list in the CLI-R, resulting in increased subrecipient burden and unhappiness. If activities are reported as separate interventions when they are all part of one intervention, the Data Cleaning Team will need to collapse all the activities into one intervention, which will mean the grantee or subrecipient will need to re-enter all the data entered after 23 for that intervention.

The Data Cleaning Team will ask the grantee/evaluator to help the subrecipient redo service type reporting and reporting separate activities, resulting in increased grantee burden and unhappiness.

We ask grantees to specifically instruct subrecipients about these issues, as they continue to occur for every CLI-R reporting period.

Slide 19 - How Do Subrecipients Get Credit for All Activities Within an Intervention-Service Type? We know subrecipients are doing a lot of work, and we want to make sure they can report all the work they are doing. So, how do subrecipients “get credit” for all the activities within an intervention-service type? First, subrecipients can say in Question 39 if an intervention targeted multiple locations. If an intervention is reaching multiple groups or implemented multiple times, subrecipients will be asked about that in the relevant sub-forms (e.g., Question 67a–c in the Prevention Education sub- form). Finally, subrecipients that are conducting a variety of activities related to an intervention will report that in the relevant sub-forms as well (e.g., Questions 149– 161 in the Information Dissemination sub-form).

Slide 20 - Grantee Questions OK, let’s pause. Are there any questions related to what we’ve talked about so far?

Slide 21 - Q33d: Service Type Start Date Question 33d asks about the service type start date. Please record the date that the intervention-service type was funded as part of your PFS initiative. If the intervention- service type existed in the community before PFS, record the date funded as part of your PFS initiative. That is, even if the intervention was implemented before PFS funding, please report on the date that the intervention started being funded by PFS funds. We can tell whether or not an intervention is new by looking at Question 37.

The date started field is needed to calculate the time from the date the subrecipient was funded to the date intervention implementation began, so it is an important field for analysis.

Slide 22 - Q33e: CSAP Strategy Type Selecting the right CSAP strategy type is critical—the strategy type determines which questions are asked in sub-forms. The 6 CSAP strategy types are (1) community-based processes, (2) prevention education, (3) alternative drug-free activities, (4) problem identification and referral, (5) environmental strategies, and (6) information dissemination and communication.

6

**Note that for PFS, information dissemination and communication should be selected as the CSAP strategy type for social marketing/media campaigns. Media campaigns should not be reported as environmental strategies OR as prevention education strategy types. To link to the correct sub-form to report media campaign data, you must select information dissemination and communication as the CSAP strategy type.

• Please consult the CLI-R guidance manual with any questions about selecting the correct CSAP strategy types. Definitions are listed for each strategy type, along with examples of interventions related to each strategy.

• Grantee evaluators can and should assist subrecipients to select the correct CSAP strategy type; grantee evaluators or staff should contact the PEP-C team if unsure.

Slide 23 - Q33e: CSAP Strategy Type (cont.) The PEP-C team developed the Intervention-Service Type List to help grantees and subrecipients select the correct CSAP strategy type. The Intervention-Service Type List, which is available on the PEP-C Knowledge Base, matches each intervention- service type to the PEP-C suggested CSAP strategy type.

[PULL UP INTERVENTION-SERVICE TYPE LIST]

We are currently developing a list to assist PFS grantees and subrecipients to select the most appropriate service type and CSAP strategy type for each intervention that they list in their CLI-R. We will send a notice via the Listserv when that has been posted to the PEP-C Knowledge Base. These lists are only suggestions. Subrecipients implement interventions in a variety of ways, so you may feel that a different service type or CSAP strategy type is a better match for the intervention you implemented than what is suggested on these lists. If you have any questions about which CSAP strategy type or service type is appropriate, please contact the PEP-C TA Team.

Slide 24 - Preventing Intervention Reporting Problems Each reporting period, the PEP-C team works with many grantees and subrecipients with these problems who then have to redo their data entry for these interventions. Prevention is a big time-saver!

Slide 25 - Q33h: Service Type Status Q33h is a “Status” field to report whether an intervention-service type was active during the period. If a service type was active at any point during the reporting period (even if it was discontinued or completed halfway through the reporting period), it must be listed as Active during this period to report sub-form data. To submit the CLI-R, you must complete a sub-form for all service types listed as Active during this period. If you mark a service type that was active at any point during a reporting period as inactive, completed, or discontinued, the sub-form will not open and you will not be prompted to enter all the necessary data needed for the service type.

If a service type was not active at any point during the period, mark it as either 1) Inactive during this period, 2) No implementation during this period: Completed, or 3) No implementation during this period: Discontinued.

The Guidance Manual has detailed definitions of the service type statuses:

You should mark a service type Inactive if you have no CLI-R data to report for an intervention-service type because it was not implemented during the period but it will be implemented in future reporting periods.

7

For example, a student prevention education program may be offered in November of each school year. The Status field should be set to “Active” when submitting the October–March CLI-R reporting period (due June 1) and set to “Inactive” when submitting the April–September CLI-R reporting period (due December 1). You should mark a service type Completed if you have no CLI- R data to report for an intervention-service type because the implementation of the service type for your initiative was completed before the start of the reporting period, and you will no longer be implementing the service type. You should mark a service type Discontinued if you have no CLI-R data to report for an intervention-service type because the implementation occurred before the reporting period and the intervention-service type has been discarded as part of your PFS initiative.

**Recall that if a PFS intervention began with PFS support but is now being implemented through funding from a different source, it should still be considered active for PFS.

Slide 26 - Q39a: Locations As mentioned before, in Question 39 you will report the location or locations in which you are implementing the intervention-service type. For example, if you are implementing a prevention program in 5 schools, list each school as a location. If you are implementing an environmental strategy such as compliance checks or policy change, list targeted geographic areas. These targeted areas could be a county, one or more towns, or a neighborhood within a city.

Slide 27 - Q39g: Estimated Target Population Size In Question 39g, you will provide the estimated target population size for each location for the service type. You should focus on the target population for the service type, not the population of the community overall. For example, if community is 10,000 people, but only 3,000 of those are in the target 12- to 20-year-old age group, then the estimated target population is 3,000. You will need to provide an estimate for each location listed for the service type. Estimates may come from school district information, partner agencies, or Census data. For help in determining estimates, see the CLI-R Guidance Manual. It particularly helps with deriving estimates from Census data.

Question 39g is an important field for the cross-site evaluation because it tells us how many people you would like to reach (or could possibly reach) with the intervention- service type, and later in the CLI-R you will tell us how many of those you actually did reach.

Slide 28 - Reporting Dosage Depending on the intervention-service type selected, you will report intervention dosage every reporting period or during the annual reporting period.

Dosage questions vary depending on the sub-form, so be sure to read those items carefully.

For example, in the Prevention Education sub-form, you will be asked to report number of sessions and average length of sessions annually. If your program, for example, meets weekly for 12 weeks, 1 hour each time, you would report it as 12 sessions, 1 hour each. If you implemented this program with 3 different groups of people, the number of sessions and length stays the same—you shouldn’t multiply by 3. Each individual still participated in 12, one-hour sessions.

Be consistent with how you report dosage. The Data Cleaning Team has noticed that sometimes the intervention description does not match the dosage information. For example, if you have implemented a

8

media campaign, be sure to mention everything you have done for the campaign—brochures, ads, website, etc. Don’t list one or two activities in the description and then another in the dosage information; include everything in the description of the intervention. Also, as a reminder, if you are handing out brochures/flyers to advertise for the website, then don’t include the act of handing out the brochure/flyers in dosage—just the actual intervention. We have been catching a lot of data that might be inflating dosage estimates.

Slide 29 - Reporting Numbers Reached For each intervention-service type listed in Question 33, you will report the numbers reached in the 6-month reporting period. Again, the 6-month reporting periods are October 1 – March 31 and April 1 – September 30. In the second half of the Federal fiscal year (April – September), you will also report how many were newly reached in that period, which allows the PEP-C team to calculate an unduplicated number of individuals who received the intervention-service type during the entire Federal fiscal year.

• The numbers reached or served for each intervention-service type should be equal to or lower than the total target population size number reported in 39g.

Slide 30 - Numbers Reached for Population-Based Strategies For population-based strategies, such as community-based processes, environmental strategies, and information dissemination and communication, numbers reached are estimates, not the exact counts that you’d expect in individual direct services such as prevention education programs. As I mentioned before, the CLI-R Guidance Manual provides additional information on deriving estimates from Census data. Grantee evaluators should help subrecipients with these estimates.

Slide 31 - Numbers Reached for Population-Based Strategies: Example Let’s walk through an example to estimate the numbers reached for a population- based strategy. The service type is a compliance check in an effort to reduce alcohol access among youth age 12 – 20. During the October–March reporting period, the community used Census data to estimate reaching 3,000 such youth. In the April- September period, the effort continued and the community again reported reaching 3,000, so the number of new individuals reached in the second half of the Federal fiscal year was 0. But let’s say the campaign extended its reach during the second half of the federal fiscal year—April to September—to reach an additional town that had 5,000 youth age 12–20. Then the number reached in April – September would be 8,000 (the 3,000 reached in the original town plus the additional 5,000 youth from Town 2). The number of NEW individuals would be the 5,000 from Town 2.

Reporting number of new people reached is particularly important with population- based service types, as these service types may reach the same target population in both periods, and we need to be able to report an unduplicated count for the fiscal year.

Slide 32 - Demographic Items In the CLI-R sub-forms you will be asked to report participant demographic information. For individual direct service strategies, the best source of demographic information is usually participants themselves. For population-based strategies, demographic estimates may be derived from Census data and media companies’ audience information. Information about military status in your community may be available from local military- and veteran-serving institutions.

9

Slide 33 - Grantee Questions OK, let’s pause. Are there any questions related to what we’ve talked about?

Slide 34 - Technical Assistance [Slide 34 has no notes.]

Slide 35 - Request Technical Assistance If you have any questions about the cross-site evaluation, please contact the PEP-C Training and Technical Assistance Team. You can contact the TA team directly by clicking on the red technical assistance link in the top right corner of any page in the MRT and completing a TA form or by using the email and phone number listed here. The phone number goes right to a voicemail and we will return your call as soon as we are available. We strive to respond to all TA requests submitted within 1 business day or at the very least to acknowledge receipt and say that we are looking into the issue or tracking down a response. During high-volume times, our response time may take a little longer.