SPF SIG Evaluation Closeout

Amy Mason, LMSWEvaluation CoordinatorIowa Consortium for Substance Abuse Research and Evaluation

Evaluation Closeout

• County Evaluation Plan Summary of Process Taken by Strategy

• Evaluation Guidance Document Update

• ODSS Indicators

• Summary of County Accomplishments to date

• Summary of LEW Survey Results

County Evaluation Plan Summary of Process Taken by Strategy

• Summary on each strategy your county implemented, approximately 1 page

• Pull from the County Strategic Plan that was developed at the beginning of the grant (October 2012)

• Template and example provide for reference in the Project Closeout Folder on the SPF SIG Website

• Email completed summary to Amy

County Evaluation Plan Summary of Process Taken by Strategy

For Each Strategy:

• What was addressed?• What was the proposed outcome?• State main activities implemented.• State where, who and/or how the strategy was

implemented.

Evaluation Guidance Document: Final Update

• Update data from June 24, 2014 thru December 31, 2014

• Add column to your previous evaluation spreadsheet

• Enter in data for the strategies, the plan for collection

• Template and examples found in the Project Closeout Folder on SPF SIG Website

• Attach to County Evaluation Plan Summary

Evaluation Guidance Document

Question Data Source Approaches Type Measure(s) - SampleProject End

Strategies

What modifications or adaptations were applied to address unique cultural groups?

ODSS

What efforts were undertaken to sustain this strategy?

Quarterly Reports

What media advocacy was used with this strategy?

ODSS - M Number of ads, posters, direct mailings, letters to the editor, press releases, etc.

Provide a brief summary of the process taken on each component of the strategy.

ODSS - Description

ODSS Closeout Procedures

• Captures all data

• Importance for evaluation and reporting

• Enter in data for services provided

• Needs continual updating until December 31st, 2014.

ODSS Entry Flags

• All entries throughout the project period should have a green flag.

• Goal is to have no red flags.

• No flag may mean the entry should not be in ODSS or was not initially reviewed.

• Questions about no flags contact Julie Hibben.

ODSS: Indicators

• Enter and update new data as it comes available

• Note: Some indicators may not be available due to frequency of collection.

• Review and enter in Start and Stop Dates up to November 2014

ODSS: IndicatorsOptions if there is 0.00 in the value column with any of your indicators:

1. Have the indicator delete because you decide not use it OR

2. Enter in the value that belongs to that indicator or update that number OR

3. You want to keep that indicator but there was nothing captured for that period of time so it is real 0.00, if this fits your situation please make a note or attach a note to that indicator stating that you are still using the indicator.

ODSS: Indicator Reviews

• Reviews begin November 17th and will last 2 weeks.

• Feedback will be provided via email starting the December 1st.

• Review process will be repeated again before project end.

• Questions call or email Amy Mason

County Accomplishments: All Activities

• Over 9,600 activities have been recorded in ODSS by 23 counties between October 1, 2012 and November 4, 2014.

• Over 80% were recorded as Services Provided or Developmental Activities.

County Accomplishments: All Activities

LEW Survey Results

• Of the 23 Local Epidemiological Workgroup Chairpersons, 20 provided responses to the survey.

• Survey was administered in February, 2014.

• The purpose of this survey was to get the LEWs response on their process in the SPF-SIG project.

Tenure of Chair

Less than 3 months

3 months or more

More than 1 year

Entire project

0.0% 10.0% 20.0% 30.0% 40.0% 50.0%

How long have you been a LEW chair?

• 45% reported being a LEW chairperson for the entire grant.

Coalition/Collaboration Council Chair

Yes

No

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

Are you also the Coalition/Collaboration Council Chair?

• 80% reported not being a Coalition/Collaboration Chair.

LEW Member Changes

Increased

Remained stable

Decreased

0.0% 20.0% 40.0% 60.0% 80.0%

Has the number of active LEW members changed during the course of the project?

Function of LEW Group

Well

Not well

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

Currently, how well does your LEW group func-tion?

Group Function Overtime

Improved

No change

Worsened

0.0% 10.0%20.0%30.0%40.0%50.0%60.0%

How has the LEW group functionality changed over time?

How Well Coordinator Works With LEW?

Well

OK

Not well

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

Currently, how well does your SPF-SIG County Coordinator work with you and the LEW?

• Almost 89% reported their SPF-SIG Coordinator worked well with the LEW.

Relationship With Coordinator Overtime

No change

Improved

Worsened

0.0% 20.0% 40.0% 60.0%

Has your work relationship with the SPF-SIG county coordinator changed over

time?

• Half reported improvement of the relationship overtime.• Half reported no change of the relationship overtime.

LEW Meetings Lead to Changes

Yes

No

0.0% 20.0% 40.0% 60.0%

Did the LEW chair meeting(s) you at-tended lead to changes in how you

conducted LEW business?

• Over 50% reported the LEW chair meetings did not change how LEW business was conducted.

Multiple Roles

Str

ate

gic

Pla

n...

An

nu

al A

ctio

n ..

.

Eva

lua

tion

Pla

n

Su

sta

ina

bili

ty ..

.

Co

mm

un

ity A

s...

Tri

-Eth

nic

Re

a...

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

What SPF-SIG products do you have a role in completing for your county? (Check all that apply)

• On average 60% or more had a role in completing other SPF-SIG products.

Other Substances

Yes

No

0.0% 10.0%20.0%30.0%40.0%50.0%60.0%

Has your LEW looked at other substance use areas besides alcohol?

• 50% reported looking at other substance use.

Assistance to County Groups

Yes

No

0.0% 20.0% 40.0% 60.0% 80.0%

Has your LEW provided assistance to other ini-tiates, groups, or projects in your county?

• Over 70% provided assistance to other groups.

SPF-SIG Project Understanding

Yes

No

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

Do you feel like you have a concrete understand-ing of the SPF-SIG project?

• Over 80% felt they have a concrete understanding of the project.

SPF-SIG Community Presentations

No

Yes

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

Have you discussed or presented on the SPF-SIG project to others in your community or

workplace?

• Over 77% have discussed or presented to their communities.

Assessment Data Presentations

No

Yes

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0%

Have you discussed or presented on the LEW or the county assessment data to others in your commu-

nity or workplace?

• Over 66% have presented on the LEW or county assessment data in their community.

“Have you discussed or presented on the LEW or the county assessment data to

others in your community or workplace?”

• A total of six indicated the reasons shared about presenting data included:

Change norms

Provide data updates

Recruit members

Increase buy in

Lessons Learned

• Five out of thirteen indicated value and importance of data.

• Additional lessons learned included:

Allow for LEW to set own priorities and/or selecting own resources

Need for good information and leadership

Need for positive change

It is a long and difficult process

Data has great importance

Communities lack knowledge of alcohol abuse

Continue Post Project

Yes

No

0.0% 20.0% 40.0% 60.0% 80.0%

Will your LEW continue meeting and working af-ter this project ends?

“Do you have any suggestions on how the LEW process could be improved?”

• Choosing own county goals.• Increase structure of telephone conferences.• All meetings to have a specific goal or function.• Clear and concise reports for the community.• Establish needs and goals prior to the project starting.• An outline for subsequent project years.• Increase integration with Iowa Department of Public Health for

sustainability.

Questions

Email: amy-mason@uiowa.edu

Phone: 319-335-4001