the georgia apex program annual evaluation reportapex year-end survey ... utilizing both...

The Georgia Apex Program Annual Evaluation Report

July 1, 2016–June 30, 2017

Prepared by The Center of Excellence for Children’s Behavioral Health

1

CENTER OF EXCELLENCE FOR CHILDREN’S BEHAVIORAL HEALTH

Table of Contents Executive Summary.................................................................................................................2Introduction..............................................................................................................................4Measures and Methods..........................................................................................................7

Monthly Progress Reports..........................................................................................................7

Mental Health Planning and Evaluation Template.............................................................7

Apex Year-End Survey..................................................................................................................7

Parent Survey..................................................................................................................................8

Findings Overview Apex Program Year 2.............................................................................8Students Served and Program Characteristics.....................................................................8

Number of School Partners and Enrollment.........................................................................9

SBMH Programs and Title I Status..........................................................................................11

Goal 1: Increased Access to Mental Health Services........................................................12

Goal 2: Early Detection of Mental Health Needs..............................................................13

Goal 3: Increased Coordination Between Mental Health Providers and Schools..16

Year-End Survey............................................................................................................................20

Parent Survey and Student Outcomes..................................................................................26

Additional Findings from Year 1 to Year 2...........................................................................28

Summary and Conclusions..................................................................................................31Facilitators to Program Success...............................................................................................31

Barriers to Program Success.....................................................................................................31

Conclusion......................................................................................................................................32

References..............................................................................................................................34

2


Executive Summary InpartnershipwiththeGeorgiaDepartmentofBehavioralHealthandDevelopmentalDisabilities (DBHDD), the Center of Excellence for Children’s Behavioral Health(COE), housed in the Georgia Health Policy Center at Georgia State University,conducted an evaluation of the second year of the Georgia Apex Program andprovided technical assistance to each of the community-based mental healthproviders, hereafter referred to as fundees. The program created partnershipsbetweenlocalschoolsandthefundeestoprovideschool-basedmentalhealth(SBMH)services in an effort to increase access tomental health services for children andyouth,provide forearlydetectionofchildrenandadolescentmentalhealthneeds,andincreasecoordinationbetweenfundeesandthelocalschoolsandschooldistrictstheyserve.Utilizingbothquantitativeandqualitativedatacollectionandanalyses,theevaluationmeasuredprogramperformance,identifiedtechnicalassistanceneeds,andinvestigatedfacilitatorsandbarrierstoprogrammaticsuccessandsustainability.Thefollowingreportdetailsthemeasuresandmethodsutilizedfortheevaluationandprovidesasummaryoftheresultsforthesecondprogramyear.

Monthlyprogressreports(MPRs)wereutilizedtogatheraggregatedataforreferrals,utilization, and billing of SBMH services, as well as information on activities tosupportongoingcoordinationandcollaborationbetweenthefundeesandtheschoolpartners.Theseschool-levelreportsprovidedinformationabouteachoftheprogramgoals. In the second year, the 29 fundees reached 2,822 students who had notpreviouslyreceivedmentalhealthservices.Theprogramservedanaverageof235first-timestudentseachmonth.Studentswere largelyserved intheschoolsetting,withover40,044servicesprovidedinschoolsoverthesecondyear.Providersbeganserving147schools inAugust2016and increased steadily to servea totalof210schoolsbyMay2017.

Ofthe203schoolsservedbyprovidersduringthesecondyear,99wereelementary,55weremiddle,39werehigh,and10werealternativeschools.Toincreaseaccesstoservices, fundeescontinue tobeencouraged toserveTitle I schools,and92.1%ofschoolswere,infact,designatedTitleIschools.Further,48.1%ofschoolshadPositiveBehavioral InterventionsandSupports (PBIS)orothermentalhealthprograms inplace before or during the second year of Apex. Of the 213 schools reportinggeographicdata,76%ofschoolswerelocatedinruralareas,15.5%insuburbanareas,and8.4%inurbanareas.

TheCOEusedtheMentalHealthPlanningandEvaluationTemplate(MHPET)toevaluatepartnershipsandcollaborations.Fundeesprovidedaself-reportoftheireffortsinconnectingwithschoolpartnersinSeptember2016andMay2017.Overall,fundeesreportonaveragethatpoliciesandproceduresthatsupportsustainableSBMHpartnershipsare“somewhatinplace.”Meanscoresarehigheratfollow-up(May2017)acrossall11questions,orpartnershipattributes.Fundees

3


mostfrequentlyreportimprovementsintheStakeholderInvolvement(36%)andIdentification,Referral,andAssessment(33%)dimensions.

TheApexYear-EndSurveyhelpstocompilecriticalcontextualinformationabouttheschoolsserved,implementationoftheprograms,andsuccessesandchallengesrelatedtobilling,sustainability,andcollaborativerelationships.Qualitativedatafromsurveyresponseswereanalyzedforcommon

themesthatcontributedtoprogrammaticsuccessesandchallenges.Commonchallengesincludedcaremanagementorganizationrequirements(billing,credentialingrequirementsforstaff,timelyauthorizationforservices),difficultiesinschedulingtimewithstudentsduringtheschoolday,andlackofsupportandinvestmentfromleaderswithinthefundeesandschoolpartners.Commonsuccesseswerealsonoted,includingintegrationintopartneringschools,improvedcoordinationandcommunicationbetweenprovidersandschoolstaff,andincreasedaccesstoservicesforstudentsotherwisenotreceivingneededhelp.Activitiesthatwerehelpfultomovefundeestowardsustainabilitywithinschoolswereparticipatinginschoolevents,servingasaresourcetoschoolstaff(training,crisisintervention,lunch-and-learns),providingqualitycaretothestudents,andusingdatatodemonstratethebenefitsofApexprogramming.

Thefollowingreportdescribesthemethodsandmeasuresusedandtheactivitiesincludedinthesecondyear,anddiscussessomeofthegrowthandincreasedreachofApexprogrammingfromyear1to2.Theresultsmayservetoinformfutureprogramguidanceandimplementation,identifyfacilitatorstosustainabilityatthelocallevel,andaidinassessmentofrelatedpoliciesandfundingopportunities.Additionally,evaluationresultsfromyear2canserveasevidencetohelpfundeestelltheirprogrammaticstory.

Year 2 by the Numbers

• 29fundees• 203schoolsserved• 92% retention rate of schools from year 1 to

year2

• 2,822 students who had not previouslyreceivedmentalhealthservices

• 235 average number of first-time studentsservedeachmonth

• 21 average number of students served perschool

• 40,044servicesprovidedinschools11,377

4


Introduction Mentalhealthneedsarecommonamongtoday’sschool-agedchildrenandyouth.Anestimatedoneinfivechildrenundertheageof18hasadiagnosablementalhealthdisorder.1,2Theseconcerns,ifleftunaddressed,cancausechildrenandyouthtostruggletosucceedbyimpairingacademicperformance,inducingmaladaptivebehavioralpatternsthatleadtotruancyanddisciplinaryaction,andcreatingcumulative,long-termdetrimentalimpacts.Despiteagreatneedformentalhealthservicesforchildrenandyouth,anestimated75%-80%ofthosewhorequireservicesdonotreceivethem.3Barrierstoaccessingneededservicesincludelackofhealthcoverage,stigmaassociatedwithmentalhealth,limitedavailabilityofservices,andtransportationissues.School-basedmentalhealth(SBMH)servicesareagrowingavenuetoincreaseaccesstomentalhealthcare,providepreventivecare,andprovidefortheearlydetectionofmentalhealthneeds.TheGeorgiaApexProgramalignswithothertypesofSBMHsupportprograms,likePositiveBehavioralInterventionsandSupports(PBIS).ThisframeworkisrepresentedinFigure1.Moststudents’needscanbemetonTierI,universalsupportsandservices,likePBIS.About7%-10%ofstudentsrequiremoretargetedservicesandcanhavetheirneedsmetonTierIIwithearlyinterventionservices(suchasat-riskyouthorothertargetedpreventionservices).However,about3%-5%ofstudents,representedonTierIII,haveahigherlevelofneedthatrequiresclinicalintervention.TheGeorgiaApexProgramwasdesignedtomeettheneedsofthesestudentsrequiringintensiveinterventionbyfacilitatingplacementofmentalhealthprovidersinschools.

Figure 1: The Apex Triangle

• TierIII:IntensiveIntervention(3-5%)MH

Providers

• TierII:EarlyIntervention(7-10%)

Counselors,SocialWorkers,MHProviders

• TierI:UniversalPrevention(85-90%)

AllSchoolStaff

5


StudentsrequiringmoreintensivementalhealthinterventionscanalsofindsupportthroughtheSystemofCare,aspectrumofeffective,community-basedservicesandsupportstohelpchildrenatriskforbehavioralhealthchallengesfunctionathome,inschool,andthroughoutlife.4ThestateofGeorgiahasmadesignificantinvestmentsintheSystemofCareforchildrenwithsevereemotionaldisordersoverthepastfewyears.ThegoaloftheSystemofCareistoprovidecommunity-based,culturallycompetent,family-andyouth-drivencareforyouthandchildreninneedofbehavioralhealthservices.InordertocreateaSystemofCarethatmeetstheneedsofGeorgia’sfamilies,thestatehasdedicatedfundingandhasappliedforfederalgrantfundingtosupportcommunitymentalhealthprovidersinservingthispopulation.SBMHprogramsareanimportantpartoftheSystemofCare,astheypromoteaccesstomentalhealthservices,increaseearlyidentificationofmentalhealthneeds,andprovideinterventionsforchildreninneedofbehavioralhealthservices.Basedintheschoolsetting,theseprogramsprovideacontinuumofbehavioralhealthcaretostudentsandtheirfamilies,includingcrisissupportandongoingtherapyformoresevereconditionsandneeds.Furthermore,suchprogramsfostercollaborationbetweenandacrosssystems,animportantcomponentoftheSystemofCare.TheSystemofCarewrapsservicesaroundthechildandfamilytoprovidecareinthemostappropriate,community-based,andculturallycompetentsetting.InadditiontotheGeorgiaApexProgram,ProjectAWARE(AdvancingWellnessandResilienceinEducation)andProjectLAUNCH(LinkingActionsforUnmetNeedsinChildren’sHealth)arealsohelpingtoaddressthementalhealthneedsofstudentswithinschoolsinGeorgia.ProjectAWAREisafive-yearSubstanceAbuseandMentalHealthServicesAdministration(SAMHSA)grantthatwasawardedtotheGeorgiaDepartmentofEducation(DOE)inSeptember2014.Currentlyimplementedinthreeschooldistricts,ProjectAWAREprovidestraininginYouthMentalHealthFirstAidandfocusesondevelopingprocessesandproceduresforconnectingyouthandfamiliestocommunity-basedmentalhealthservices.Additionally,theDepartmentofPublicHealth(DPH)hasreceivedfundingforProjectLAUNCH,whichpromotesandsupportsearlymentalhealthscreeningandassessmentforchildrenages0-8.Inrecentyears,GeorgiaDOEhasworkedtoexpandimplementationofPBISprogramstodistrictsandschoolsthroughoutthestate.TheseprogramsareintegralpartsoftheGeorgiaSystemofCareandhaveopenedthedoorformentalhealthprovidersandschoolstocometogethertoaddresschildren’sbehavioralhealthneedsbyincreasingtheaccessibilityofcare.ThecollaborativeinvestmentmadeintotheSystemofCarebychild-servingagenciesinGeorgiahasultimatelyincreasedtheavailabilityandreachofbehavioralhealthservicesinschoolsinthestate.InthepilotyearoftheGeorgiaApexProgram,the2015-2016schoolyear,theGeorgiaDepartmentofBehavioralHealthandDevelopmentalDisabilities(DBHDD),

6


DivisionofBehavioralHealth,OfficeofChildren,YoungAdults,andFamilies,providedfundingsupportto29fundeestocoordinatewiththeirlocalschoolsandrespectiveschooldistrictstoprovideSBMHservices.ThegoalsoftheApexprogramareto:

• Increaseaccesstomentalhealthservicesforchildrenandyouth;

• Provideforearlydetectionofchildrenandadolescentmentalhealthneeds;and

• Increasecoordinationbetweencommunitymentalhealthprovidersandtheir

localschoolsandschooldistricts.InpartnershipwithDBHDD,theCenterofExcellenceforChildren’sBehavioralHealth(COE),housedintheGeorgiaHealthPolicyCenteratGeorgiaStateUniversity,providedtechnicalassistancetoeachoftheApexfundeesandconductedaprogramevaluation.Overthecourseofthefirstyear,fundeeswereabletoreach2,419studentswhohadnotpreviouslyreceivedmentalhealthservices.Theprogramservedanaverageof944studentseachmonth.Studentswerelargelyservedintheschoolsetting,withover20,000servicesprovided.Bytheendofthefirstyear,thefundeesmaintainedactivepartnershipswith136schoolsthroughoutthestate.Challengestoprogramimplementationascitedbyfundeeswererelatedtofamilyengagementandunclearreferralprocesses,whileprogramsuccesscenteredonincreasedcoordinationandcommunicationbetweenfundeesandschools.Dataanalysesfromyear1provideevidencethattheprogrammadesignificantprogresstowardthethreegoalsofincreasedaccess,earlydetection,andincreasedcoordinationbetweenfundeesandtheirlocalschools.Giventhepositiveresultsandcommunityfeedbackfromthepilotyear,DBHDD,partneringagainwiththeCOE,whowastaskedtoprovidetechnicalassistanceandprogramevaluation,continuedfundingthesameprovidersforasecondyear,withtheexpectationofsustainedprogressinreachingtheprogramgoalsandfurtherexpansionofSBMHservicesthroughoutthestate.

Evaluationgoalsoutlinedforyear2includeto:

• Demonstratetheprogram’sabilitytomeetintendedgoals/outcomes;

• Identifyfacilitatorsandchallengestosustainabilityandreplication;and

• Serveasfoundationalinformationforprogramstotelltheirstories.Thefollowingreportdetailsthemeasuresandmethodsutilizedfortheevaluationofyear2andprovidesasummaryoftheresults.

7


Measures and Methods TheCOEappliedamixed-methodsapproachtotheevaluationofthesecondyearoftheGeorgiaApexProgram.TheevaluationincorporatesbothprimaryandsecondaryqualitativeandquantitativedatatoassessfundeeeffortstowardachievingthethreeApexprogramgoalsandtodemonstrateexpansionofserviceprovisionwithintheirregions. Monthly Progress Reports Themonthlyprogressreport(MPR)wascompletedbyallfundeesparticipatingintheApexprogram.TheMPRisdesignedtomeasureprogramdevelopmentandtohelpfundeesandDBHDDidentifyopportunitiesfortechnicalassistancethatstrengthentheprogram’sabilitytofacilitateimprovedoutcomesforthestudentsandfamiliesserved.InpartnershipwithDBHDD,theCOEcreatedanelectronicdatacollectionsurveyusinganonlinesurveyadministrationtool.AlinktotheMPRwasemailedtoallApexfundeecontactsonthefirstdayofeachmonthduringtheprogramperiod.Fundeesweregiven15daystocompleteandsubmitthissurveyreportdetailinginformationfromthepreviousmonth.TheApexevaluationteamattheCOEprovidedsupportforquestionsorconcernsrelatedtothetechnicalinputoffundeedataandtheinterpretationofthedataitemsrequested.Quantitativedatasubmittedforthisreportwereanalyzedusingstatisticalsoftware.Qualitativedatawereanalyzedforcommonthemesandexploredindepthacrossfundees.AcopyofthetoolisavailableinAppendixA.Mental Health Planning and Evaluation Template TheCOEutilizedselecteditemsfromtheMentalHealthPlanningandEvaluationTemplate(MHPET;AppendixB)toassessfundeeeffortstosustainandincreasecoordinationbetweenthemselvesandtheirpartnerschool(s)astheyenhancedanexistingordevelopedanewSBMHprogram.TheMHPETisafreeonlinetooldevelopedbytheNationalAssemblyofSchool-BasedMentalHealthCenters,inconjunctionwiththeCentersforSchool-BasedMentalHealth.FurtherdiscussionoftheMHPETcanbefoundunderGoal3,page17.

Apex Year-End Survey FundeeswereaskedtocompletetheApexYear-EndSurveyinJune2017.Thissurveycollectedinformationabouttheschoolsserved,servicesprovided,billingpractices,programcharacteristics,providerpresence,andintegrationwithinschools.Open-endedquestionshelpedtoidentifysuccessesandchallengesof

8


programimplementationandsustainability.Responseswerereviewedforcommonthemes.Additionally,fundeeswereaskedtoreportonthepartnershipstheycreatedandmaintainedforyear2ofApex.Acopyoftheyear-endsurveyisprovidedinAppendixC.

Parent Survey TheGeorgiaApexProgramParent/GuardianSurveyisa14-iteminstrumentdesignedtoexploretheparents’perspectivesonhowreceivingservicesthroughApexhasimpactedthebehavioralhealthoftheirchildren.Duringfamilymeetings,fundeesadministerthissurveytotheparentsofstudentsreceivingSBMHservicesthroughApex.Parentsareaskedavarietyofquestionsrelatedtotheirchildren’sfunctioningsincejoiningtheApexprogram,includingsatisfactionwiththeservicesthechildhasreceived,thechild’sabilitytohandledailylife,andimprovementinworkand/orschool.Itemsarescoredusingafive-pointscalerangingfrom1(stronglydisagree)to5(stronglyagree).Thescoresfromtheparentsurveyweresummedtocreateacompositescoreofthechild’sleveloffunctioningasperceivedbytheparent.ThesescoreswerethenanalyzedalongwiththenumberofreporteddaysinApexservicestoseewhethertheprogramhadanimpactonchildfunctioning.AcopyoftheparentsurveyisprovidedinAppendixD.

Findings Overview Apex Program Year 2

Students Served and Program Characteristics Thefindingspresentedinthisreportarereflectiveofdatafromavarietyofsources.Therewereatotalof293schoolsreportedviatheMPRssubmittedinyear2.Ofthese,79schoolswerenotreportedonbythefundeesintheyear-endsurvey.Thesereportingdiscrepanciescouldbeduetothestrengthsornatureofthepartnerships,andwhethertheyweresustainedthroughouttheschoolyear.Asaresultofthevariedreportingofschools,therewereatotalof214schoolsrepresentedontheyear-endsurvey.InordertoprovidedescriptiveinformationabouttheschoolsservedbyApexfundees,publiclyavailabledatafromtheGeorgiaDOEandtheGovernor’sOfficeofStudentAchievement(GOSA)werealignedwithschoolsreportedonintheyear-endsurvey.Informationaboutenrollment,TitleIstatus,andschooltypewasobtainedforatotalof203schoolsfromtheyear-endsurvey.Thenumberofschoolsforwhichthefollowinginformationispresentedisrepresentedinthetitleforclarity.Additionally,thetableinAppendixEfurtherdetailsthenumberofschoolsservedbydatasource.

9


Number of School Partners and Enrollment Overthecourseofthesecondyear,fundeeswereencouragedbythefundertocontinueservinginschoolswheretheyhadformedstrongcollaborationsandwerealsochallengedtoforgenewpartnerships(seeFigure2).Thenumberofschoolsservedduringyear2steadilyincreasedandultimatelyrangedbetweentwoand36schoolsforeachfundee.Theaveragenumberofschoolsservedbyfundeeforyear2was10,howeverseveralpartnershipsdidnotresultinsustainedservicedelivery.SchoolpartnershipswerelowestduringthesummerbeforethesecondschoolyearinJuly2016(113schools)andhighestinMarch2017(214schools).

Figure 2: Number of Schools Served Monthly a

aDataSource:Monthlyprogressreports(MPRs)

By the end of year 2, fundees reported active partnerships in 203 schools on the year-end survey. These schools represent a total

enrollment of 141,355 students for the 2016-2017 school year, which means that over 140,000 students potentially had access to SBMH

services through the Apex program (see Figure 8, page 15).

113

147 150

175 174167

207

185

214

197210

201

0

50

100

150

200

250

Jul-16 Aug-16 Sep-16 Oct-16 Nov-16 Dec-16 Jan-17 Feb-17 Mar-17 Apr-17 May-17 Jun-17

10


AlmosthalfoftheschoolsservedbyApexfundeesinyear2wereelementaryschools(48.8%).Additionally,5%ofschoolswereconsidered“Alternative”schools.AccordingtotheGeorgiaDOE,alternativeschoolsandprogramsareanoptionforstudentswhomayrequireinnovativeorcreativestructuredalternativestoatraditionaleducationsetting.5Theseschoolsandprogramsincludecombinedschooltypes,specialeducationcenters,andalternativeschools(seeFigure3).

Figure 3: Type of School Served by the Apex Program (N=203)b

Additionally,fundeesreportedonthegeographyandsurroundinglandscapeoftheirpartnerschools.Themajority(76.1%)ofschoolsservedbyApexfundeeswerelocatedinaruralsetting(seeFigure4).

Figure 4: Geography of Apex Schools (N=213)c

bDataSource:ApexYear-EndSurveymergedwithGOSAandDOEdatacDataSource:ApexYear-EndSurvey(missingvalues=1)

99(48.8%)

55(27.1%) 39

(19.2%)10

(4.9%)

0102030405060708090100

Elementary Middle High Alternative

Rural162

(76.1%)

Suburban33

(15.5%)

Urban18

(8.4%)

11


SBMH Programs and Title I Status Tomeettheprogramgoalforearlydetection,fundeeshavecontinuedtobeencouragedbythefundertoconsiderelementaryandmiddleschoolpartners,andtoprioritizeschoolswithunmetneeds.Inyear1,fundeeswereencouragedtopartnerwithschoolsthathadpreviouslyimplementedPBISprogramstoeaseintegrationintotheschoolsetting.IntegrationwithschoolsthatalreadyhavesomeformofSBMHprogramscanalsofacilitateprogramsustainability,asschoolleadershipmaybemorereceptivetothedemonstratedvalueofofferingtheseservicesintheirdistricts.Althoughthiswasnotarequirementinyear2,103outofthe214schoolsreportedonintheyear-endsurveyhadeitherPBISand/orotherSBMHprogrammingbeforeApex(seeFigure5).Additionally,fundeeswereurgedtoconsiderschoolpartnerswithTitleIstatusinanattempttohaveagreaterimpactonstudentswithlowerresourcesandgreaterpotentialforhavingunmetneeds.Inyear2,92.1%ofpartnerschoolswereTitleIschools(seeFigure6).Theseschoolsrepresentedatotalenrollmentof126,000studentsfromlow-incomeorlow-resourcebackgroundswhohadaccesstoSBMHservices.

Figure 5: Schools With SBMH or PBIS Prior to Apex Program Participation (N=214)d

dDataSource:ApexYear-EndSurvey

103(48.1%)

111(51.9%)

0

20

40

60

80

100

120

Yes No

12


Figure 6: Schools and Enrollment With Title I Status (N=203)e

Goal 1: Increased Access to Mental Health Services OneofthegoalsoftheApexprogramistoincreaseaccesstomentalhealthservicesforchildrenandyouth,particularlyforthosewhoareinneedofservicesandnotreceivingthem.Overthecourseofthesecondyear,theprogramservedanaverageof1,937studentseachmonth,morethandoublethemonthlyaveragefromyear1(926students).

Thevastmajorityofserviceswereprovidedintheschoolsetting,consistentwiththepurposeoftheApexprogram.

eDataSource:ApexYear-EndSurveymergedwithGOSAandDOEdata

187(92.1%)126,454Students

16(7.9%)14,901Students

0

20

40

60

80

100

120

140

160

180

200

Yes No

The number of services provided in schools and the number of referrals made to public providers almost doubled from year 1 to year 2. In total, 40,044 services were provided in schools (compared to 22,640 from year 1) and 4,785 referrals were made to public providers (compared to 2,468 in year 1).

13


Referralsmadetoprivateprovidersdecreasedfrom270inthefirstprogramyearto111inthesecond.Apotentialreasonforthisdecreasecouldbethedevelopmentofsummerprogrammingorcontinueduseofschoolspaceduringbreaksensuringcontinuityofserviceswithinschools.Anotherplausiblereasoncouldbelackofinsurancecoveragebyprivateproviders—thereforefundeesaremorelikelytorefertopublicproviders.Additionalreasonscanbefurtherexploredinfutureanalyses.Servicesandreferralsforallstudentsservedinyear2arerepresentedinFigure7.

Figure 7: Total Services and Referrals for All Studentsf

Goal 2: Early Detection of Mental Health Needs Anotherprogramgoalistoincreaseearlydetectionofmentalhealthneeds.Fundeeswereencouragedtopartnerwithelementaryandmiddleschoolstoidentifyneedsandmakeservicesavailabletostudentsatayoungerage.Asnotedearlier,the29fundeespartneredwithatotalof99elementaryschools,55middleschools,39highschools,and10alternativeschoolsoverthecourseofthe2016-2017schoolyear(seeFigure3).

fDataSource:Monthlyprogressreports

1375

2565

1553

3604 36333308

4497

4003

5115

4384 4264

1743

110 258 150 125 152 133498 565 608 600 775 811

45 0 0 0 4 0 6 40 8 3 5 00

1000

2000

3000

4000

5000

6000


ServicesinSchools PublicProviderReferrals PrivateProviderReferrals

14


Duringthecourseoftheprogram,themaximumnumberofschoolsfundeesattemptedtopartnerwithinaone-monthperiodwas214(seeFigure2).

Figure 8: Type of School Served by the Apex Program and Enrollment (N=203)g

Thenumberoffirst-timestudentsserveddecreased,overall,forJanuarythroughJune.Thisdecreasemayindicatethattheprovidersencounteredfewerstudentswithfirst-timeneedduringthesemonths,whileconcurrentlyservingagreaternumberofstudentsoverall(seeFigure9).Additionally,first-timereferralsweremadetoApexprovidersforatotalof2,822studentswhohadnotpreviouslyreceivedmentalhealthservicesthroughApexforamonthlyaverageof235first-timestudentsserved.Fivethousandninehundredforty-two(5,942)totalserviceswereprovidedinschooland828referralsweremadetopublicprovidersforfirst-timestudents.Thenumberofservicesandreferralsprovidedtofirst-timestudents

gDataSource:ApexYear-EndSurveymergedwithGOSAandDOEdata

Alternative3,945

Students(2.8%)

Elementary65,914Students(46.6%)Middle

38,754Students(27.4%)

High32,742Students(23.2%)

The Apex program provided access to services for 65,914 students in elementary schools, 38,754 students in middle schools, and 32,742 in high schools, for a total of over 140,000 students (see Figure 8).

15


onlyfollowsasimilarseasonalpatternasthenumberofstudentsserved,withademonstrateddecreasefromthemiddle(February)towardtheendoftheschoolyear(June)(seeFigure10)

Figure 9: Number of Students Served, Total and for the First Time by Monthh

Figure 10: Services and Referrals for First-Time Students by Monthi

hDataSource:MonthlyProgressReportsiDataSource:Monthlyprogressreports

728

14061631

19302035 2048

2343 2293

2684

2347 2369

1426

62

360 347 331 337156

277 260 267 207 143 75

0

500

1000

1500

2000

2500

3000


NumberofChildrenServed NumberofFirstTimeStudentServed

157

708

827

651

739

314

708

539597

548

311

181

5

173

56 32 34 2264

118 89120

7837

2 3 0 0 0 0 2 5 3 0 0 00

100

200

300

400

500

600

700

800

900


ServicesinSchools PublicProviderReferrals PrivateProviderReferrals

16


Goal 3: Increased Coordination Between Mental Health Providers and Schools StrongpartnershipsbetweenschoolsandmentalhealthprovidersareessentialtopromotingthesustainabilityofSBMHprograms.Assuch,fundeeshavecontinuedtomakeintentionaleffortstofostercloserelationshipsinthesecondyearoftheApexprogram.ThethirdgoaloftheApexprogramistoincreasecollaborationandcoordinationbetweencommunitymentalhealthprovidersandthelocalschoolsanddistrictsinwhichtheyserve.Strongpartnershipsprovidethefoundationforinfrastructurethatcanbemaintainedshouldtherebeanegativeshiftinfunding.AnadaptedversionoftheMHPETwasemployedtoexaminefundeeandschoolpartnereffortsatcreatingandfosteringpartnershipsandinfrastructuretosupportSBMHservices.Qualitativequestionswerealsoadministeredtofundeesintheyear-endsurveytoassessthekeyelementsthatcontributetosustainablepartnerships.

Mental Health Planning and Evaluation Template TheMHPETassessesschoolmentalhealthprogramsacrosseightdimensionsbasedonitem-levelresponses:

• Operations;• Servicedelivery;• Stakeholderinvolvement;• Schoolcoordinationandcooperation;• Staffandtraining;• Communitycoordinationandcooperation;and• Identification,assessment,andreferral.

AnabbreviatedversionoftheMHPETincorporatingonetotwoquestionsperdimensionwasaddedtotheSeptemberandMayMPRs.Surveyscoreswerecomparedacrossthetwosurveyadministrationperiods.ForboththeSeptember2016(baseline)andMay2017(follow-up)surveys,anaveragewascalculatedforeachMHPETquestionbyschool.Question-levelresponseswereanalyzedtoassesschangesinfundees’perceptionsoftheleveltowhichpoliciesandproceduresareinplacetosupportthesustainedrelationshipbetweenthefundeeagenciesandtheirschoolpartners.Baselineandfollow-upsurveymeanswerecomparedbetweenlevelsofschoolattributesusingpairedt-tests.TheMHPETscalerangesfrom1to6,with1meaningtheitemwasnotatallinplace,2through5meaningtheitemwassomewhatinplace,and6meaningtheitemwasfullyinplace.Twenty-fourfundeescompletedbaselineMHPETsurveysfor148schoolsinSeptember2016.Follow-upMHPETsurveyswerecompletedfor210schoolsinMay2017.Matchedsurveyresultswereanalyzedfor132schoolpartnerships.Overall,fundeesreportonaveragethatpoliciesandproceduresthatsupportsustainableSBMHpartnershipsare“somewhatinplace.”Meanscoresare

17


higheratfollow-upacrossall11questionsorpartnershipattributes.Pairedt-testsindicatethatthechangesinmeanscoresasawholearesignificantatp<.001.

Table 1: MHPET Overall Mean Survey Scores (N=132)j

Dimension Question September

Average (T1)

May Average

(T2)

Operations

1.Thereareclearprotocolsandsupervisionforhandlingstudents’severeproblemsandcrisis(e.g.,suicidalideation,psychosis)

4.83 5.25

2.Mentalhealthservicesadheretoclearpoliciesandprocedurestoshareinformationappropriatelywithinandoutsideoftheschoolandtoprotectstudentandfamilyconfidentiality.

5.41 5.65

Stakeholder Involvement

3.Familiesarepartnersindevelopingandimplementingservices. 4.55 5.08

4.Teachers,administrators,andschoolstaffunderstandtherationaleformentalhealthserviceswithintheirschoolandareeducatedaboutwhichspecificbarrierstolearningtheseservicescanaddress.

4.42 4.87

Staff & Training

5.Mentalhealthstaffreceivestrainingandongoingsupportandsupervisioninimplementingevidence-basedpreventionandinterventioninschools.

4.83 4.88

6.Mentalhealthstaffreceivestraining,support,andsupervisioninprovidingstrengths-basedanddevelopmentallyandculturallycompetentservices.

4.92 5.11

Identification, Referral & Assessment

7.Mentalhealthserviceprovidersandtheschoolhaveadoptedasharedprotocolthatclearlydefineswhenandhowtoreferstudents.

5.08 5.43

Service Delivery 8.Arangeofactivitiesandservices,includingschoolwidementalhealthpromotion,prevention,earlyintervention,andtreatmentservices,areprovidedforyouthingeneralandspecialeducation.

3.92 4.60

School Coordination & Collaboration

9.Mentalhealthstaffdevelopsandmaintainsrelationshipsandparticipatesintrainingandmeetingswitheducatorsandschool-employedmentalhealthstaff(ifapplicable)

4.66 4.83

Community Coordination & Collaboration

10.Servicersarecoordinatedwithcommunity-basedmentalhealthandsubstanceabuseorganizationstoenhanceresourcesandtoservestudentswhoseneedsextendbeyondscopeorcapacity.

4.42 4.63

Quality Assessment & Improvement

11.Astakeholder-informedmentalhealthqualityassessmentandimprovement(QAI)planisimplementedthatincludesmeasuresofconsumersatisfaction,individualstudentoutcomes(e.g.,measuresofbehavioraloremotionalhealth),andschool-relatedoutcomes(e.g.,attendance,behavior,academicperformance).

3.64 4.00

Overall Mean 4.61 4.94

jDataSource:MonthlyprogressreportsforSeptember2016andMay2017

18


Changesinthelevelofimplementationofpartnershipdimensionswereassessed.Themajorityofprovidersreportnochangeinthelevelofdimensionimplementationonthefollow-upsurveycomparedtothebaselinesurvey.Thepercentageofproviderswhoratethedimensionlevelofimplementationhigheronthefollow-upsurveyrangedbetween20%and36%.FundeesmostfrequentlyreportimprovementsintheStakeholderInvolvement(36%)andIdentification,Referral,andAssessment(33%)dimensions.Conversely,thepercentageofproviderswhoratedimensionlevelofimplementationloweronthefollow-upMHPETsurveythanatbaselinerangedbetween7%and20%.TwentypercentoffundeesreportalowerlevelofimplementationintheCommunityCoordinationandCollaboration(19.7%)andSchoolCoordinationandCollaboration(16.7%)dimensions(seeTable2,below).

Table 2: Changes in Level of Implementation of Partnership Attributes (N=132)k

Dimension/Question Percent

Improved (a)

Percent Declined (b)

No Change

Operations Q1 29.6% 10.6% 59.8%

Q2 28.7% 16.7% 54.5%

Stakeholder Involvement Q3 36.4% 12.9% 50.8%

Q4 23.5% 6.8% 69.7%

Staffing & Training Q5 23.5% 14.4% 62.1%

Q6 25.0% 11.3% 63.6%

Identification, Referral & Assessment Q7 33.3% 11.4% 55.3%

Service Delivery Q8 28.0% 12.1% 59.8%

School Coordination & Collaboration Q9 23.5% 16.7% 59.8%

Community Coordination & Collaboration Q10 23.5% 19.7% 56.8%

Quality Assessment & Improvement Q11 20.5% 15.2% 64.4%

(a)Changeinscorefrom“Notatallinplace”to“Somewhatinplace”or“Somewhatinplace”to“Fullyinplace”(b)Changeinscorefrom“Fullyinplace”to“Somewhatinplace”or“Somewhatinplace”to“Notatallinplace”

kDataSource:MonthlyprogressreportsforSeptember2016andMay2017

19


PubliclyavailabledatafromtheGeorgiaDOEandtheGOSAwereutilizedtoexaminetherelationshipbetweenseveralschool-levelcharacteristics(schooltype,geography,presenceofSBMHservicespriortoApeximplementation,andpresenceofPBISpriortoApeximplementation)andMHPETtotalbaselineandfollow-upmeanscores.Ofthe132schoolsthathadcompletedboththebaselineandfollow-upMHPETsurveys,15schoolsdidnothaveDOEorGOSApubliclyavailabledata.Therefore,thetotalnumberofschoolsassessedinthefollowinganalysiswas117.Table3resultsdemonstratedifferentialimprovementsinMHPETmeanscoresbyschooltype(e.g.,elementary,middle,high,andalternative),geography,andwhetherornotschoolshadSBMHorPBISprogramsinplacepriortoApex.Specifically,follow-upscoressignificantlyimproved,comparedtobaseline,forpartnershipswithbothelementaryandhighschools,schoolslocatedinruralsettings,andschoolsthatdidnothaveSBMHorPBISprogramsinplacepriortoApex.Improvementswithinmiddleschoolswerealsoobserved,thoughnotatstatisticallysignificantlevels.Follow-upsurveymeanscoresdecreasedfrombaselinesurveymeanscoresamongpartnershipswithschoolslocatedinurbanandsuburbanareas.

WhileschoolsthatdidhaveSBMHorPBISpriortoApexdidnotproducestatisticallysignificantresults,follow-upsurveymeanscoresdidincreasefrombaseline.ThelackofsignificancecouldbeattributedtothesamplesizeortothepossibilitythatschoolswithnoSBMHorPBISpriortoApexhadopportunitytomakegreatergain.

Schools without SBMH and PBIS in place prior to Apex improved on most dimensions of the MHPET and yielded statistically significant results.

20


Table 3: Mean Scores for MHPET Overall Survey by School-Level Attributes (N=117)l

School-Level Attributes Baseline Mean Follow-Up Mean Difference in Means

School Type Elementary 4.69 5.01 0.32*Middle 4.60 4.77 0.17High 4.30 4.92 0.62**Alternative 5.30 4.34 -0.95

Geography Rural 4.61 5.09 0.48**Urban 4.66 3.80 -0.86Suburban 4.61 4.49 -0.12

SBMH Prior to Apex Yes 4.30 4.72 0.42No 4.71 4.95 0.24*

PBIS Prior to Apex Yes 4.71 4.94 0.23No 4.53 4.86 0.33**p<.05**p<.01

Year-End Survey Fundeeswereaskedtocompleteayear-endsurveyinJune2017.Thissurveycollectedinformationabouttheschoolsserved,servicesprovided,billingpractices,programcharacteristics,andsuccessesandchallengesofprogramimplementation.Thesurveysyieldedquantitativeandqualitativedataaboutprogramcharacteristics,implementation,andsustainability.Additionally,fundeesreportedonthepartnershipstheycreatedandmaintainedforyear2.Responseswerereviewedforcommonthemes.Fundeescompletedyear-endsurveysfor214schoolstotal,althoughthereweremissingvaluesforsomeofthemeasurespresentedbelow.School Partnerships EachApexfundeedeterminedhowbesttoimplementitsprogrambasedonitslocalcommunity.Implementationstrategiesvariedbetweenfundees,butallinvolvedcreatingorstrengtheningpartnershipswithlocalschools,principals,superintendents,orschoolboards.SomefundeesformedpartnershipswithlocalschooldistrictstodetermineatwhichschoolstoimplementApexprogramming,and

lDataSource:MonthlyprogressreportsforSeptember2016andMay2017mergedwithGOSA/DOEandApexYear-EndSurveyData

21


othersutilizeddatafromtheGeorgiaDOEtoidentifypotentialschoolpartnersandapproachedtheindividualschool.

Accordingtotheyear-endsurvey,4,372uniquestudentswereservedacross201schoolsinyear2.mThisisanaverageofabout21studentsservedperschoolinyear2.Fundeesreportedtheyattemptedtopartnerwithanaverageof12schoolsacrossyear2,withabout10ofthoseattemptsonaveragereportedasturningintosuccessfulpartnerships.Theaveragenumberofschoolseachfundeeservedbytheendofyear2accordingtotheyear-endsurveywas14.Includedinthisaverageareschoolsthatthefundeeshadbeenservingsinceyear1oftheprogram.Intotal,58%oftheschoolsreportedintheyear-endsurveywereservedbyApexinyear1(n=125).Additionally,fundeesindicatedthattheyhadapreviousrelationshipwith72%(n=153)oftheschoolsreportedontheyear-endsurvey,suggestingthatthefundeesareintentionallyestablishingrelationshipsanddevelopingpartnershipswithschoolsbeforeprovidingdirectservicestostudents,whichhelpsbuildastrongfoundationforprogramsustainability.Data Tracking and Sharing Fundeesreportedutilizingavarietyofstrategiestogatherandsharedatarelatedtostudentimprovementandacademicperformance.ThemajorityoffundeesarenotcollectingdataindependentofCOE’sevaluation.However,fundeesarecollectingdatainpartnershipwithschoolsandschoolsystems,predominantlyschooloutcomesdatarelatedtogrades,attendance,andbehavior.Fundeesreportedtrackingattendancein55%ofschoolsandtrackinganymeasureofacademicperformancein59%ofschools.Notably,fundeesaretrackinginformationrelatedtobehaviordisruptionsfortheirstudentsfor70%oftheschoolsreported.Thisissignificant,asbehaviordisruptionsareonewayforbehavioralhealthproviderstounderstandiftheirservicesarehavinganimpactonchildfunctioning.Ofthe129schoolsforwhichfundeestrackacademicperformance,almost100%(n=127)ofthoseschoolsreportdataongrades.Fundeesonlycollectinformationrelatedtogradepointaverage(GPA)for14%ofschoolsandtrackscoresontheGeorgiaMilestoneAssessment(GMA)for11%ofschools.Othersourcesofacademicdatareportedbyfundeesincludeteacherreportsandlocaldatafromanindividualschool’sreportingsystems.Inordertocollectdataassociatedwiththefrequencyandimpactofservicesaswellasserviceprovision,onefundeeaddedaschoolfieldontheadmissionform,whichallowedfurtherdataqueries.Fundeeswereaskedwithwhomtheysharethisdatatodemonstratetheimpactoftheirprogramstogarnercommunitysupport.Fundeesreportedtheyshareddatawithmentalhealthagencyleadershipfor76%oftheirschoolsandwithindividualschoolleadershipin61%ofschools.Fundeessharedatawitheithercommunityleadershiporschooldistrictleadershipinlessthan25%ofschools.Ofthefundees

mDataSource:ApexYear-EndSurveywith13missingvalues

22


whodointeractwithcommunityandschooldistrictleadership,thesharingofbothdataandsuccessstorieshasbeenthemostcommonstrategytodemonstratethebenefitsofApexprogramming.Fundeespresentatvariouscommunitypartnermeetingsandschoolboardmeetingsaswellasresourcefairsandoutreachevents.Additionally,theyaresharingtheirdatawithlocalclubssuchastheLionsorRotary.Broadeningtheaudiencewithwhichfundeesaredemonstratingtheimpactoftheirprogramtoincludebusinessleaderscanhelptodiversifyfunding.Havingschooldistrictandcommunitybuy-inisanimportantaspectofplanningforsustainabilityoftheApexprogram.Onefundeereportedthatsharingdataatacommunityserviceboardmeetinginspiredaboardmembertoprovidegrantfundingtosupportsummercampactivities. School Integration and Provider Presence Integrationofservicesandproviderpresencevariedacrossfundeesandwithinschools.Inthemajorityofschools(70%),providersregularlyattendedschoolmeetings.Providershadaschoolemailaddressin25%ofschools,andaschoolbadgein29%ofschools.One-thirdoftheprovidersreported“other”waysofintegratingintotheschools,suchashostinglunch-and-learnsforschoolstaff,participatinginPTAmeetings,andattendingresourcefairsandfielddays.Additionally,in75%oftheschoolsserved,providersreportedtheyhadaprivateofficespaceforprovidingservices,with14%reportinghavingasharedspace.Officespaceiscrucialforproviderstobeabletoservestudentsinaconfidentialmanner.Providerslackingadesignated(privateorshared)spaceidentifiedalternativeoptionssuchasconferencerooms,copyrooms,andemptyclassrooms.Fundeesalsoreportedonthetypesofprovidersservingineachschool.Morethanhalfofyear2schoolswerestaffedbyproviderswithaBSW(bachelor-levelsocialworker)andjustoverone-thirdofschoolswerestaffedbyproviderswithanLPC(licensedprofessionalcounselor)certification.ComparativelyfewerschoolswerestaffedbyaproviderwithLAPCn(28%),LMSW(18%),LCSW(17%),and/orMSW(11%)certifications.Lessthan10%ofschoolswerestaffedwithanRNo(8%),psychologist(7%),orLMFT(5%).Acrossallschoolswithcompletedatafortheyear-endsurvey(202schools),justunderone-third(30%)werestaffedbyonlythoseproviderswhowerenotlicensed,meaningthatin70%ofyear2Apexschools,atleastonelicensedproviderwasonstaffprovidingApexservices.Providerpresenceintheschooliscategorizedbydaysperweekandbyhoursperday.However,itisdifficulttoestimatetheaveragenumberofhoursperweektheproviderswerepresentinanindividualschoolduetothephrasingofthequestiononthesurvey.In96%ofschools,fundeesreportedthatproviderswerepresentat

nLicensedassociateprofessionalcounselor,licensedmastersocialworker,licensedclinicalsocialworker,mastersocialworkeroRegisterednurse,licensedmarriageandfamilytherapist

23


leastonetoatmostfivedaysaweek.Onlythreeschoolshadaproviderservinglessthanonehourperday,perweek.Onlyeightschoolshadprovidersservinglessthanonedayperweek(seeTable4).

Table 4: Provider Presence in Schools by Days per Week and Hours per Day (N=214)

Provider Days per Week

Provider Hours per Day <1hour 1-4hours 5-8hours Total

<1 day 3 5 0 8

1-2 days 0 41 65 106

3-5 days 0 9 91 100

Total 3 55 156 214

Fundeeswerealsoaskedtoreportonthestaffingdetailsforeachschooltheyservedinincludinghowmanyfull-andpart-timestaffmemberswerededicatedtoeachindividualschool(seeTable5).

Table 5: Provider Staffing in Schools by Type (Full-Time and Part-Time) (N=213)p

Staff Type Frequency of Number of Staff per School <1(including0) 1 2-5 >5

Full-Time 54 80 73 15

Part-Time 139 53 8 14

Therewereavarietyofinconsistenciesinthedatareportedbyfundees,therebymakingitsomewhatdifficulttoaccuratelydeterminetheactualnumberofstaffallocatedtotheschools.Futureanalyseswillreflectmoreaccuratedataasthequestionsrelatedtostaffinghavebeenrevisedtorepresentamoreaccurateunderstandingofthetherapist’sallocationoftime.Excludingoutliers,thefollowingpatternswereobserved:Themostfrequentresponsetothequestionabouthowmanyfulltimeproviderswerededicatedtotheindividualschoolwasone(n=80,38%).Additionally,70%oftheschoolsreportedhavingonetothreeprovidersservingstudents(n=152).Ofthe75totalschoolsreportedashavinganypart-timestaff,61hadbetweenoneandfiveproviders(81%).Challenges to Billing and Financial Sustainability Thechallengesrelatedtobillingasreportedbyfundeesontheyear-endsurveymaysubsequentlyimpactfinancialsustainabilityforprograms.Fundeesfrequentlycitedthecaremanagementorganization’s(CMO’s)processesandprocedures,lackofcredentialedstaff,andinsurance-relatedissuesasprimarybillingchallenges.TheCMO’srequirementsandcriteriarelatedtoobtainingauthorizationandsecuring

pDataSource:ApexYear-EndSurveywithonemissingvalue

24


reimbursementwerenotedasmostchallengingacrossallfundees.Furthermore,fundeesreportthetimeandprocesstoreceiveauthorizationforservicesascumbersomeandlaborious.Duetoincreasedcaseloadsandcrisissituationsthatrequireimmediateattention,fundeesreportedoftenbeingdelayedinsubmittingdocumentation.Somefundeesdesignateadministrativestafftohelpsupporttimelysubmissions.Sincemanyinsurancecarriersonlyreimburseifservicesareprovidedbyfullylicensedcliniciansandmanyfundeesemployassociatelicensedclinicians,billingprivateinsuranceisanaddedchallenge.Fundeesalsoreporttheinabilityofprivateinsurancetocollectcopayswithintheschoolsettingasabarrier.Thislimitationpresentsadditionalchallengestofundees,asparentsdonotalwayscommunicateinsuranceinformationornotifyfundeesifthereisalapseincoverage,resultinginmorereimbursementdelays.ThesechallengesmaynecessitatetheneedforfundeestotakeproactivestepsinparticipatingonprivateandCMOinsurancepanelsinordertoadvocatefortheoptionofbillingforSBMHservicesregardlessofclinicianstatusandtoworktowardaddingschoolsasappropriatevenuesfordeliveringsupportservices.Fundeesreportinstructionaltimeandtestingschedulesasadditionalchallengesinbillingandlong-termfinancialsustainability,asprovidershavelimitedhoursinthedaytomeetwithstudents,whichresultsinlessbillabletime.

Challenges to the Sustainability of the School Partner Relationship Overall,fundeesreportpositiverelationshipswithschoolpartners,especiallywhenfundeestakethetimetoembedthemselveswithinschoolsandparticipatein,aswellascontributeto,schoolactivities.However,somefundeesreportedalackofsupportfromschooladministratorsasachallengeforsustainingtheApexpartnership.Fundeesalsostatethatturnoverwithinschoolsmakesitdifficulttoimplementprogrammingandtomaintainconsistentinvestmentintheprogramonbehalfoftheschoolpartners.Additionally,stablestaffingamongtheprovidersisimportantaswell.Feedbackfromtheyear-endsurveyconveyshowdifficultitistomaintainrapportandtrustwiththeschoolpartnersandstudentsifprovidersareconstantly

The time spent on activities critical for school buy-in such as school engagement, integrating into the school culture, and supporting staff and administrators are nonbillable. Fundees report that Apex funding has been crucial in offsetting the cost associated with nonbillable activities. In the absence of Apex funding, these significant activities that are crucial for onboarding new schools and continuing to support existing school partners would make sustainability difficult.

25


changing.Fundeesreportactivitiessuchasprovidingtrainingstoschoolstaffonmentalhealthawareness,hostinglunch-and-learnsforparentstounderstandmentalhealthstigma,andparticipatinginnumerousschooleventshelptopromotesuccessfulcollaboration.However,theseactivitiesarenonbillableservices.Facilitators in Creating Financial Sustainability Thegreaternumberofreferralsintotheprogrammeansincreasedproductivityandgreateropportunitytobillforservices.Fundeesreporttheyhaveemployedcreativestrategiesandsolutionstoincreasingreferrals.Manyfundeesaddedsummercampsandactivitiesthroughoutthesummertoincreasetheiravailabilityandaccessibilitytostudentsinwhathavehistoricallybeenlessactivemonths.Manyfundeeshaveinitiatedofferingtherapeuticgroupservicesasanotherwaytoprovideoutreach.Fundeesreportthatbeingpresentatallschoolactivitiesincreasesthevisibilityoftheprogramandservices.Whilethispresenceishelpfulincreatingfinancialsustainability,itisalsoachallenge,giventhatparticipatinginschoolactivitiesisnonbillabletime.FundeesalsoreflectedontheimportanceofseekingsupportfromcommunitypartnersandbusinessesasastrategytowardfinancialsustainabilitysinceprovidingSBMHservicestostudentsbenefitstheentirecommunity.Facilitators in Sustaining Relationship with Schools Thequalitativeresponsestotheyear-endsurveyrevealedmoredetailaboutthenatureofimplementingandimprovingschoolpartnershipsthanthepreviouslyreviewedquantitativedata.Theclearmajorityoffundeessupportthenotionof“embeddingthemselveswithintheschool’sculture.”ParticipatinginactivitiessuchasdevelopingPBIScurriculumandindividualizededucationplanmeetings;attendingfamilynights,parentorientations,andopenhouses;hostinglunch-and-learnevents;andprovidingtrainingopportunitiesforadministratorsandstaffaresomeoftheexamplesofhowfundeesareembeddingthemselves.Fundeesalsoreportitisimportanttohaveaconsistentpresenceatschoolsinordertoprovidequalitycare.Maintainingadependablescheduleofthedaysandhoursspentinschoolscontributestoperceptionsofreliability,whichfundeeshaveindicatedasbeingimportanttoschoolstaff.Additionally,frequentandeffectivecommunicationwithstaffandadministratorshelpssustaintherelationshipwiththeschoolpartners.Usingdatatodemonstratethevalueofservicesandpositiveoutcomesforstudentsbolstersrelationshipswithschools.Facilitators in Sustaining Relationships With Community Partners ParticipatingincommunitycollaborativemeetingssuchasLocalInteragencyPlanningTeams(LIPT),FamilyConnection,andsimilarendeavorshasbeenimportantforfundeestoincreaseawarenessandknowledgeoftheprogram.Asfundeesreport,visibilitywithinthecommunityiscriticaltosustainingrelationshipswithcommunitypartners.Sharingdatawithcommunitypartnershasbeenasuccessfulstrategyformanyfundeestoobtainbuy-in,demonstratethevalueofApexprogramming,andsolicitinvestment.

26


Marketing Activities Printedmaterialintheformofbrochures,postcards,andflyers,aswellasproductmarketing(pens,stressballs,waterbottles),aresomeexamplesofhowfundeesaremarketingtheirprograms.Manyarealsocreatingvideos,participatinginradiointerviews,writingarticlesforthelocalnewspaper,andmakingappearancesonlocalTVprograms.FundeesarerepresentingApexatschoolandatcommunityeventsinadditiontohostingeventsdedicatedtoeducatingthelargercommunityaboutmentalhealthawareness. Apex Program Goals for Year 3 ThegoalsoftheApexprovidersforyear3alignedwithfivemainthemes(seeTable6).ThemostnotablegoalwastoexpandApexschoolswithintheirregions.

Table 6: Year 3 Goals for Apex Program as Identified by Fundees

• ExpandApexservicestomoreschools• Increaseparentengagement• Ensuresustainabilityofproviders• Collect/sharemoredata• Increaseengagementwithcommunitypartners

Thestrongesttheme,whichcutsacrossallthequestionsrelatedtosustainabilityontheyear-endsurveyreportedbyfundees,isthesignificanceoftheactivitiesthatarenonbillable.Fundeesreportthatactivitiessuchasprovidingtrainingstoschoolstaffonmentalhealthawareness,hostinglunch-and-learnsforparentstounderstandmentalhealthstigma,andparticipatinginnumerousschooleventssuchasopenhousesorcurriculumnightscontributetobuildinglastingrapportwithschoolpartners.Equallyimportantforfundeesisbeingvisibleinthecommunitytomarkettheprograminanefforttoincreasecommunitycollaborationanddiversifyfunding.Thesenonbillableactivitiesarecrucialtothesustainabilityoftherelationshipsandprogramming.IntheabsenceofApexfunding,fundeesareextremelyconcernedwithhowtheywouldcoverthecostoftheseactivitiesandtheimplicationsofnotengaginginthem.

Parent Survey and Student Outcomes Throughouttheyear,fundeesadministeredsurveystotheparentsofchildreninApexservices.Thesurveyaskedquestionsaboutthechild’sfunctioningsincejoiningtheApexprogram,including“OverallIamsatisfiedwiththeservicesthatmychildhasreceived,”“Mychildisbetterathandlingdailylife,”and“Mychildisdoingbetterinschooland/orwork.”Thesurveyconsistsof14items,scoredfrom1

27


(stronglydisagree)to5(stronglyagree).ThesurveyisdesignedtoshowhowreceivingservicesthroughApeximpactsthebehavioralhealthofthechildasreportedbytheparent.Theparentsurveyisadministeredtoparentsinthefundeeclinic,atthepointwhentheirchildrenarebeingassessedwiththeChildandAdolescentNeedsandStrengths(CANS)questionnaire.TheCANSassessesexposuretotrauma,needs,andstrengthsthroughthefollowingdomains:LifeFunctioning,ChildRiskBehaviors,Acculturation,ChildBehavioral/EmotionalNeeds,TraumaticStressSymptoms,Traumatic/AdverseChildhoodExperiences,ChildStrengths,SubstanceUse,Caregiver(s)NeedsandStrengths.Withineachdomainisasetofitemswhichusesafour-levelratingsystemandisdesignedtotranslateassessmentintoactionbasedonvaryinglevelsofstrengthsandneedspresentedwithinanindividualchild.TheCANSassessmentisintendedtobeadministeredevery90daysasatooltotrackprogressovertime.TheparentsurveyalsoasksabouthowlongthechildhasbeeninApexservices.TheMPRsincludeinformationabouthowmanychildrenreceivedaCANSassessmenteachmonth,aswellashowmanywereeligibleforareassessment,howmanywerereassessed,andhowmanyhadascorethatindicatedimprovedfunctioninginthatreassessment.ResultsoftheparentsurveycanhelpinformwhetherApexservicesarepositivelyimpactingchildren’sfunctioning.Acrosstheyear,only92parentsurveyswerecompleted(includinginformationaboutthelengthoftimeinApexservices).Theaveragescoreacrossall14itemsontheparentsurveywas4.07,indicatingrelativelyhighagreementwiththestatementsregardingthechild’simprovedfunctioning(asperceivedbytheparent).Additionally,whenthescoresaresummedacrossalloftheitems,thereisaweakbutstatisticallysignificantrelationshipbetweenimprovedchildfunctioningandlengthoftimeinApexservices(R=.188,p<.1).ThissuggeststhattheApexservicesarehavinganimpactonchildren’sbehaviorandarehelpingprovidersmeettheneedsforthesechildren.TheCANSdataalsosupportthefindingsoftheparentsurveyaroundimprovedfunctioningforApexstudents.Intotal,2,798studentscompletedaCANSassessmentacrossallprovidersforyear2.Additionally,2,185studentswereeligibleforaCANSreassessmentthroughouttheyear.OfthestudentsacrosstheentireyearwhowereeligibleforaCANSreassessmentandreceivedone(n=1,095),74.6%showedanimprovedCANSscore(n=817).qWhileonly1,095studentsreceivedaCANSreassessment,whichaccountsforaboutone-fourthofallreporteduniquestudentsserved,thisisstillanimpressiveproportionofstudentsshowingimprovement.Collectively,thisinformationsuggeststhattheApexprogramiseffectiveatimprovingbehavioralhealthforchildren.Thelongerstudentsremaininservices,themorelikelytheyaretodemonstrateimprovementbothintheirbehaviorsathomeasinterpretedbytheparentsandasreportedbytheirprovidersthroughtheCANSassessment.

qDataSource:Monthlyprogressreports

28


Additional Findings from Year 1 to Year 2 School Retention DuringthefirstyearofApex,thefundeesservedatotalof155schools,whileinyear2ofApex,thefundeesservedatotalof283schools,asindicatedbythesubmissionofatleastoneMPRduringeitherevaluationyear.

Statedalternatively,13ofthe155schoolsservedinyear1werenotservedinyear2.Duringyear2,141newschoolswereservedthatwerenotservedduringyear1.Acrossbothyears,atotalof296uniqueschoolswereserved:

• 142schoolswereservedinbothyears;• 13schoolswereservedinyear1butnotyear2;and• 141schoolswereservedduringyear2butnotyear1.

Diversification of Referrals OneimportantindicatoroftheimplementationofSBMHservicesisthenetworkofreferralsthatplacestudentsintocare.Adiversityofreferralspotentiallyindicatesahighintegrationofcommunitypartnersintotheschools.ThereferralnetworkforthefirsttwoyearsofApexisdisplayedinFigure11below.Duringyear1ofApex,atotalof6,114referralsforSBMHservicesweremadeacross155schools.Duringyear2,atotalof11,377referralsweremadetoApexfundeesacross283schools.Inbothyears,counselorsrepresentedthemajorityofreferralstoApex(55%inyear1and64%inyear2).Allothersourcesaccountedforlessthanhalfofthetotalpercentageofreferralsduringbothyears.Thiscouldbepartiallyduetointernalschool-levelprotocolsandprocessesaroundreferringtoApexproviders.

One hundred forty-two (142) of the schools served in year 1 were also served in year 2, representing a 92% school retention rate across the first two years of Apex implementation.

29


Figure 11: Apex Program Referral Sources From Year 1 to Year 2r Increased Access to Service Across Years AsApexfundeesreceivedcontinuedfundingacrossimplementationyears,oneimportantconsiderationistheexpansionofserviceswithinschoolsovertime.Animportantquestionforconsiderationis,Withinschoolsservedduringbothyears,didthenumberofstudentsservedincrease?Weansweredthisquestionbycomparingtheoverallnumberofstudentsserved,aswellasthenumberoffirst-timestudentsservedacrossbothyears.Wespecificallycomparedthesetwomeasuresduringthebeginningoftheschoolyears(September)andtheendoftheschoolyears(May).Thatis,school-leveldatafromSeptemberofyear1werematchedwithSeptemberofyear2,whileMayofyear1wasmatchedwithMayofyear2.Seventy-seven(77)schoolswerematchedforSeptemberofyear1andyear2,and105schoolswerematchedforMayofyear1andMayofyear2.AsdisplayedinTable7,duringSeptemberofyear1,302studentswereservedacrossthe77matchedschools.Bythebeginningofyear2,thenumberofstudentsservedacrossthesameschoolsroseto977.Onaverage,aboutninemorestudentswereservedineachofthematchedschoolsinyear2.Thedifferenceinthenumberofstudentsservedwasstatisticallysignificantandrepresentsalargeeffectsize.Duringtheendoftheschoolyearforyear1,atotalof1,136studentswerebeingservedacrossthe107matchedschools.Inyear2,1,365studentswerebeingservedacrossthesameschools.Accordingly,betweentheendofyears1and2,abouttwomorestudentsonaveragewerebeingservedineachofthematchedschools.Thisresultwasalsostatisticallysignificantbutrepresentsasmalleffectsize.Theresultsoftheanalysisindicatethatinschoolpartnershipsthatwerecontinuedbetweenbothyears,the

rDataSource:Monthlyprogressreports

55%

13%9%

7% 6% 5% 5%

64%

6%

12%7%

2%5% 4%

0%

10%

20%

30%

40%

50%

60%

70%

Counselor Principal Parent SocialWorker SupportStaff Teacher Other

Year1 Year2

30


gaininthenumberofstudentsservedwasmuchhigheratthebeginningoftheschoolyearcomparedtotheendoftheschoolyear.

Table 7: Differences in Number of Students Served Across Yearss

Time

Number of

Matched Schools

Number of

Students Year 1

Number of

Students Year 2

Raw Difference

Mean Difference

Effect Size

Beginning of School (September) 77 302 977 +675 +8.64*** Large

End of School (May) 107 1136 1365 +227 +2.14* Small

Analysisconsistedoftwo-taileddependentt-test.EffectsizesderivedfromCohen’sd.Location of Services During the Summer Months ThelocationsofservicesprovidedduringJuneofbothprogramyearsareoutlinedbelowinTable8.DuringJuneofyear1,atotalof876studentswereserved.Anyonestudentcouldreceivemultipleservicesduringareportingmonth,sothetotalnumberofservicesprovidedisgreaterthanthenumberofstudentsserved.DuringJuneofyear1,despitethedropinthetotalnumberofstudentsserved,almostallservicesreceivedoccurredintheschoolsetting(87%).Inthesummerofyear2,althoughmorestudentsoverallwereserved(n=1,426),comparativelyfewerserviceswerereceivedintheschoolsetting(54%).Thedropinthepercentageofsummerservicesfromyear1toyear2appearstobedrivenbyashiftinthepercentageofservicesprovidedbyapubliccommunityprovider.DuringJuneofyear1,13%ofserviceswereprovidedbyapubliccommunityprovider,whileinJuneofyear2,thispercentageroseto25%.

Table 8: Location of Services in June of Year 1 and 2t

Total Number of Students Served

School Setting Home Other

Referred to Public

Community Provider

Referred to Private

Community Provider

Total Number

of Services

June 2016 876 1905 N/A N/A 276 13 2194

June 2017 1426 1743 288 371 811 0 3213

sDataSource:MonthlyprogressreportstDataSource:Monthlyprogressreports

31


Distribution of Billing in Year 2 ThedistributionofbillingsourcesamongstudentsservedinSeptemberandMayofyear2isdisplayedinTable9.Atbothtimepoints,thedistributionremainslargelythesame.Almostone-thirdofstudentsateachtimepointreceivedservicesthatwerebilledtoWellCare.PeachState,Amerigroup,andprivateinsurancealsoaccountedforalargeshareofthebillingsourcesforstudentsreceivingApexservices.RelativelyfewservicesreceivedbystudentswerebilledtoDBHDDoran“othergovernment”source.InSeptember2016,59students(3.64%oftotalstudentsservedthatmonth)receivedservicesthatwerenotbillable,whileinMay2017,117students(4.94%oftotalstudentsservedthatmonth)receivedservicesthatwerenotbillable.

Table 9: Billing Sources at Beginning and End of Apex Year 2

Summary and Conclusions

Facilitators to Program Success Fundeesciteschoolintegrationandproviderpresenceinschoolsaskeycomponentstoprogramsuccess.Themoreaprovidercanembedhimselfintothecultureoftheschoolandparticipateinaswellascontributetoschoolevents,themoreitresultsinadditionalbuy-infromtheschools.Increasingvisibilitywithintheschoolsbyhostinglearning/trainingopportunitiesandservingasanadvocatefortheimportanceofmentalwell-beinginthelargercommunityalsocontributestoprogramsuccess.

Barriers to Program Success WhilefundeescontinuedtomeettheprogramgoalsofApexinyear2,theydidexperiencesomebarrierstoprogramsuccess.Mostnotably,fundeesreportedonbarriersrelatedtoalackofsupportfromschooladministrators,challengeswithseeingyouthduetoinstructionaltimeandtestingschedules,andfactorsrelatedtobillinginsurance.Lastly,perhapsthebarriermostfrequentlycitedbyfundeesis

Total Private

Insurance Amerigroup Medicaid DBHDD

Other Govt

Amerigroup 360

WellCare Peach State

Sep 2016

162197

(5.98%)285

(17.58%)175

(10.80%)61

(3.76%)25

(1.54%)25

(1.54%)509

(31.40%)371

(22.89%)

May 2017

2369108

(4.56%)461

(19.46%)256

(10.81%)101

(4.26%)66

(2.79%)23

(0.97%)747

(31.53%)470

(19.84%)

32


relatedtotheessentialconsultationservicesfundeesengageandthelackofbillability.

Conclusion GeorgiaDBHDD’sApexprogramprovidedfundingto29community-basedmentalhealthproviderorganizationstointegratementalhealthprovidersintolocalschoolsettings.Bytheendofthesecondyear,theproviderscoordinatedserviceswithatleast214schoolsthroughoutthestateandprovidedover40,000servicesintheschoolsetting.Ofthechildrenserved,morethan2,800studentswerereceivingservicesforthefirsttimefromApexfundees.Theprogramcontinuestomakesignificantprogresstowardthethreegoalsofincreasedaccess,earlydetection,andincreasedcoordinationbetweencommunitymentalhealthprovidersandtheirlocalschools.

FindingsfromtheevaluationindicatethatinGeorgia,fundingtobuildinfrastructureforSBMHprogramscontributestoincreasedcoordinationbetweencommunitymentalhealthprovidersandlocalschools,specificallyintheareasofstakeholderinvolvementandstafftraining.Thisincreaseincollaborationandcoordinationalsosupportsearlyidentificationandaccessto,anddeliveryofservicesto,studentsandfamilieswhoneedthemmost.SchoolswithoutSBMHinplacepriortoApeximprovedonmostdimensionsoftheMHPET,whileschoolsthatdidhaveSBMHpriortoApexalsoshowedimprovement,althoughnotstatisticallysignificant.Lessonslearnedfromtwoyearsofimplementation,specificallyengagingwithfundeesthroughconsultationwithtechnicalassistanceproviders,peer-to-peerlearningopportunitiessuchastheSystemofCareAcademyorthePeerLearningSeminar,andcollectingbothquantitativeandqualitativedatahaveresultedinamoreinformedunderstandingaboutbestpracticesofanSBMHmodelinGeorgiaandcriticalcomponentstosustainability.Datacollectionforyear3willnotonlyenablemonitoringofprogresstowardreachingApexgoals,butitcanalsoincludetheschoolvoiceinassessingthepartnershipandcollaborationeffortsinimplementingApexinschoolsthroughfocusgroups.Technicalassistancewillfocusonhelpingtofacilitateconversationsandimplementationoffactorsnecessarytopositionforsustainabilitybothindividuallyandcollectivelyacrossallfundees.Additionally,moredatawillbecollectedandanalyzedtoassesshowwelltheprogramisachievingtheprogramgoalsoutlinedinthisreportandthedifferencemadebytheprogramonstudent-levelbehavioralhealthoutcomes.Georgiahasmadeaconsiderableinvestmentinsupportingthementalwell-beingofitsyouthbyplacingSBMHprogramminginschoolsthroughprogramslikeApex,ProjectAWARE,andProjectLAUNCH.Withtheinclusionofchildren’smentalhealthinthefederalgovernment’sEveryStudentSucceedsAct(ESSA),andtheIndividuals

33


withDisabilitiesEducationAct(IDEA)legislationandregulations,thenationaldialoguesurroundingSBMHserviceshasbeenelevated.Suchdialoguewillhelptofacilitatefurthermultistakeholderengagementandserveasanopportunitytostrategizefundingopportunities.TheGeorgiaSystemofCareplandraftedbytheInteragencyDirector’sTeamalsosupportsSBMHandexpansionofservicesthroughoutthestate.EvaluationfindingsfromtheApexprogramwillcontinuetoinformprogramplanning,implementation,andfunding.

34


References 1. NewFreedomCommissiononMentalHealth,AchievingthePromise:

TransformingMentalHealthCareinAmerica,SAMHSA.U.S.DepartmentofHealthandHumanServices,Editor.2003:Rockville,MD.

2. Kessler,R.,etal.,LifetimePrevalenceandtheAge-of-onsetDistributionsofDSM-

IVDisordersintheNationalComorbiditySurveyReplication.ArchivesofGeneralPsychiatry,2005.62(6):p.593-602.

3. Kataoka,S.,L.Zhang,andK.Wells,UnmetNeedforMentalHealthCareamongU.S.Children:VariationbyEthnicityandInsuranceStatus.AmericanJournalofPsychiatry,2002.159(9):p.1548-1555.

4. Stroul,B.,Blau,G.,&Friedman,R.(2010).Updatingthesystemofcareconceptandphilosophy.Washington,DC:GeorgetownUniversityCenterforChildandHumanDevelopment,NationalTechnicalAssistanceCenterforChildren’sMentalHealth.

5. GeorgiaDepartmentofEducation,(n.d.).AlternativeEducationProgram.Retrievedfrom:http://www.gadoe.org/School-Improvement/School-Improvement-Services/Pages/Alternative-Education-Program-and-Magnet-Schools.aspx.

AppendixA:MonthlyProgressReportPage:1

1


GeorgiaAPEX—School-BasedMentalHealthMonthlyMonitoringReport

NameofOrganization:(dropdownbox)NameofPersonSubmittingReport:______________________________Phone:________________________DateReportSubmitted:____________________________SchoolName:(dropdownbox;OTHERoption)TargetedGradestoReceiveSBMHServicesatSchool:__________________Information gathered through this report aligns with Georgia Apex Program objectives. It is understood that you may not have activities in all areas every month. Please enter N/A for measures that are not applicable within a given reporting period.Objective1:Providegreateraccesstomentalhealthservicesforstudents.

a. Indicatethetotalnumberofstudentsreferredforschool-basedmentalhealth(SBMH)servicesbyreferralsourceinthecurrentreportingperiod:

Note:totalsinthetablebelowmayexceedthetotalnumberofuniquestudentsreportedinObjective1b.above,asastudentmayreceiveorbereferred

formorethanoneserviceinagivenreportingperiod.

ReferralSource No.ofStudents

Teacher Counselor SocialWorker Principal Nurse SchoolSupportStaff Parent Other Total

b. IndicatethetotalnumberofuniquestudentswhoreceivedservicesfromtheSBMHproviderduringthecurrentreportingperiod:_____________

c. Indicatethenumberofstudentswhoreceivedservices(includingscreening,evaluationortreatment)fromtheSBMHproviderbyservicetypeinthe

currentreportingperiod.(TotalsinthetablebelowmayexceedthetotalnumberofuniquestudentsreportedinObjective1b.above,asastudentmay

receiveorbereferredformorethanoneserviceinagivenreportingperiod.):


2


ServiceType

ProvidedinSchoolSetting

ProvidedinHome

ProvidedOtherSetting

ReferredtoCommunityProvider TotalStudentsPublicProvider PrivateProvider

BehavioralHealthAssessment DiagnosticAssessment CrisisIntervention PsychiatricTreatment CommunitySupportIndividualServices IndividualOutpatientServices(i.e.,therapysession) GroupOutpatientServices FamilyOutpatientServices OtherServices Total

d. Ofthe__uniquestudentswhoreceivedservicesfromtheApexprovider(s)duringthecurrentreportingperiod,forhowmanystudentswereyouable

tobillthefollowingsources?PLEASENOTE:Thetotalnumberofstudentsforwhomyouhavebilledshouldnotexceedthenumberofuniquestudents

served.

Payer No.ofStudents

Privateinsurance Amerigroup Amerigroup–GeorgiaFamilies360 WellCare PeachState Medicaid/PeachCareFee-for-Service(i.e.,DCH) DBHDDFee-for-Service Othergovernment(e.g.,Medicare,Tricare,StateHealthBenefitPlan,etc.) Other Total


3


e. PleaseindicatethenumberofstudentsinthefollowingcategoriesastheyrelatetoCANSassessmentsduringthisreportingperiod.

1. NumberofstudentswhoreceivedabaselineCANSassessment:

2. NumberofstudentseligibleforaCANSreassessment:

3. NumberofstudentseligibleforaCANSreassessmentwhoreceivedone:

4. NumberofstudentswhoreceivedaCANSreassessmentandhadanimprovedCANSscore(ascomparedtobaseline/intake):

Objective2:Provideforearlydetectionofstudents’mentalhealthneeds.

a. Ofthetotalnumberofstudentsreferred(asreportedinObjective1a.),indicatethenumberofstudentsreceivingafirst-timereferralformentalhealth

servicesinthecurrentreportingperiod.(First-timereferralinformationmaybegatheredduringtheintakeinterviewwiththestudent’sparent/legal

guardian.):

ReferralSource No.ofStudents

Teacher Counselor SocialWorker Principal Nurse SchoolSupportStaff Parent Other Total


4


b. Ofthetotalnumberoffirst-timereferredstudentsreportedinObjective2a.,indicatethetotalnumberofuniquestudentswhoreceivedservicesfromtheSBMHproviderduringthecurrentreportingperiod:_____________

c. Indicatethenumberoffirst-timereferredstudentswhoreceivedservices(includingscreening,evaluation,ortreatment)fromtheSBMHproviderby

servicestypeinthecurrentreportingperiod.(TotalsinthetablebelowmayexceedthetotalnumberofuniquestudentsreportedinObjective2b.

above,asastudentmayreceiveorbereferredformorethanoneserviceinagivenreportingperiod):

ServiceType

ProvidedinSchoolSetting

ProvidedinHome

ProvidedOtherSetting

ReferredtoCommunityProvider TotalStudentsPublicProvider PrivateProvider

BehavioralHealthAssessment DiagnosticAssessment CrisisIntervention PsychiatricTreatment CommunitySupportIndividualServices IndividualOutpatientServices(i.e.,therapysession) GroupOutpatientServices FamilyOutpatientServices OtherServices Total

d. Isthereanyadditionalinformationthatwasnotcapturedinthepreviousquestionsthatyouwouldliketoshare?Ifso,pleaserecorditinthespace

below.


5


AppendixB:MentalHealthPlanningandEvaluationTemplate(MHPET)

6


AdaptedMentalHealthPlanningandEvaluationTemplate(MHPET)fortheGeorgiaApexProgram

Forthefollowingquestions,pleaseselectthenumberthatbestreflectsthedegreetowhichtheitemisimplementedattheschoolusingthe1to6scale:• 1indicatestheitemdescribedisnotatallinplace.Forthoseitemsthathavemultiplecomponents,meetingnoneofthecomponentswouldmerita1.• 6indicatestheitemdescribedisfullyinplace.Forthoseitemsthathavemultiplecomponents,meetingallofthecomponentswouldmeritthisrating.

Yourratingshouldhonestlyreflectthepresentstatus.Avoidthepositivebiascommoninself-ratingmethods(i.e.,ratingserviceshigherthanactuallyexist).

Domain Statement 1 2 3 4 5 6

Operations

1.Thereareclearprotocolsandsupervisionforhandlingstudents’severeproblemsandcrises(e.g.,suicidalideation,psychosis).

2.Mentalhealthservicesadheretoclearpoliciesandprocedurestoshareinformationappropriatelywithinandoutsideoftheschoolandtoprotectstudentandfamilyconfidentiality.

StakeholderInvolvement3. Familiesarepartnersindevelopingandimplementingservices. 4.Teachers,administrators,andschoolstaffunderstandtherationaleformentalhealthserviceswithintheir

schoolandareeducatedaboutwhichspecificbarrierstolearningtheseservicescanaddress.

Staff&Training

5.Mentalhealthstaffreceivestrainingandongoingsupportandsupervisioninimplementingevidence-basedpreventionandinterventioninschools.

6.Mentalhealthstaffreceivestraining,support,andsupervisioninprovidingstrengths-basedanddevelopmentallyandculturallycompetentservices.

Identification,Referral&Assessment

7.Mentalhealthserviceprovidersandtheschoolhaveadoptedasharedprotocolthatclearlydefineswhenandhowtoreferstudents.

ServiceDelivery 8.Arangeofactivitiesandservices,includingschoolwidementalhealthpromotion,prevention,earlyintervention,andtreatmentservices,areprovidedforyouthingeneralandspecialeducation.

SchoolCoordination&Collaboration

9.Mentalhealthstaffdevelopsandmaintainsrelationshipsandparticipatesintrainingandmeetingswitheducatorsandschool-employedmentalhealthstaff.

CommunityCoordination&Collaboration

10. Servicersarecoordinatedwithcommunity-basedmentalhealthandsubstanceabuseorganizationstoenhanceresourcesandtoservestudentswhoseneedsextendbeyondscopeorcapacity.

QualityAssessment&Improvement

11. Astakeholder-informedmentalhealthqualityassessmentandimprovement(QAI)planisimplementedthatincludesmeasuresofconsumersatisfaction,individualstudentoutcomes(e.g.,measuresofbehavioraloremotionalhealth),andschool-relatedoutcomes(e.g.,attendance,behavior,academicperformance).

AppendixC:Year-EndSurveyPage:1

1


ApexYear-EndSurvey2017

Nameofyourorganization:{dropdownbox}Overthecourseofthe2016-2017schoolyear,howmanyschoolshasyourorganizationattemptedtopartnerwith?{numericresponseonly}Howmanyofthosepartnershipsresultedinprovidingschool-basedmentalhealthservicesintheschool?{numericresponseonly}Howmanyschoolsdoesyourorganizationcurrentlyserve?{numericresponseonly}Note:“Serving”inaschoolisdefinedasprovidingoutpatientbehavioralhealthservicesattheschoolsite.Providingonlycrisisservicestoaschoolshouldnotbeconsidered“serving”aschool.IndividualSchoolsandProgramDescriptionsThefollowingquestionsareabouttheindividualschoolsyouarecurrentlyservingthroughtheApexProgram.Thesequestionswillberepeatedtoallowforentryofresponsesforeachindividualschoolpartner.Pleasecompleteallquestionsforeachindividualschoolserved.1. Informationabouttheschool:

a. NameofSchool(enteroneonly—dropdown)b. DidyouragencyservethisschoolduringApexyear1,2015-2016?c. PriortoApex,wereschool-basedmentalhealthservices(SBMH)availableinthisschool?d. PriortoApex,didtheschoolimplementPositiveBehaviorInterventionandSupports(PBIS)?e. PriortoApex,didtheschoolprovideanyotherbehavioralhealthservices?

i. Ifyes,pleasename/describef. Inwhattypeofareaistheschoollocated?

i. Ruralii. Urbaniii. Suburban

g. Pleaselistanyotherschool-basedhealthservicesofferedatthisschool:h. OutsideoftheSBMHprogramyouimplemented,howaccessiblearechildren’sbehavioralhealth

servicesinthecommunity?i. Veryaccessibleii. Somewhataccessibleiii. Somewhatinaccessibleiv. Veryinaccessible

i. Doyoucollectthefollowingdataforthisschool?Checkallthatapply:i. Attendanceii. Behaviordisruptioniii. Academicperformance

1. IfYes:Whatdatadoyouusetoassessacademicimprovement?Checkallthatapply:__Grades,__GPA,__GeorgiaMilestonesAssessmentSystem,__Other(textentry)


2


j. Pleasedescribeanybarriersyouhaveexperiencedincollectingschooldata.{textbox}k. Pleasedescribeanysuccessfulstrategiesyouhaveusedtoaccessandcollectschooldata.{text

box}l. Pleaseidentifykeystaffwhohelpedtoaccessthisdatawithintheschool.m. Pleaseindicatebelowwhoyousharethisdatawith.Checkallthatapply:

__Schoolpartner/leadership__Schooldistrict__Communityleadership__Mentalhealthleadership__Other(textentry)

n. Pleasedescribeanybarriersyouhaveexperiencedinsharingthisdata.{textbox}o. Pleasetellusaboutanyotherspecialcharacteristicsoftheschool:

2. StudentsServed

a. Whatgradesdidtheprogramserve?i. Allii. Somegrades(list):

b. Howmanyuniquestudentswereservedatthisschoolduringyear2ofApex,2016-2017?{numericresponseonly}

ServiceCharacteristics1. Numberoffull-timeemployeesdedicatedtothisschoolandtheircredentials{numericresponse

only}

2. Numberofpart-timeemployeesdedicatedtothisschool{numericresponseonly}

3. Credentialsofprovider(s)atthisschool.Checkallthatapply:__BSW__MSW__LMSW__LCSW__LAPC__LPC__Psychologist__LMFT__RN

__Other(text):

4. Presenceattheschool:a. Howmanydaysperweekwastheproviderpresentatthisschool?

i. Lessthanonedayperweekii. Onetotwodaysperweekiii. Threetofivedaysperweek

b. Ofthedayspresent,howmanyhoursperdaywastheproviderpresent?i. Lessthanoneii. OnetoFouriii. FivetoEight


3


c. Where,intheschool,wastheproviderlocated?

i. Privateofficeii. Sharedspaceiii. Theproviderwasnotlocatedintheschool(other,textbox).

d. Pleasedescribeanywaysinwhichthementalhealthproviderwasintegratedwiththeschools.• Providerhadschoolemailaddress• ProviderhadschoolbadgeorIDcard• Providerattendedstaffand/orcommitteemeetings• Other:(textbox)

ImplementationLessons1. Priortothe2016-2017schoolyear,wasthereapreviousrelationshipbetweenyourorganization

andthisschool?i. Ifyes,skiptoSustainabilityLessons.ii. Ifno,pleasedescribehowthisrelationshipwasformed.

b. Pleasedescribetheprocessyouusedforformingcommunitypartnershipsandanychallengesexperiencedinthisarea.

c. Pleasedescribehowhelpfulthefollowinglocalstakeholdersweretotheimplementationprocess.i. Localschoolsuperintendent(s)ii. Schoolprincipal(s)iii. Schoolsocialworker(s)iv. Schoolcounselor(s)v. Schoolresourceofficer(s)vi. Other(textbox)

d. Werethereanychallengesassociatedwiththeintegrationofthementalhealthprovidersandtheschool?Ifyes:

i. Whatwerethechallenges?ii. Howweretheyaddressed?iii. Weretheyresolved?

2. Pleasedescribeanysuccessstoriesorthingsthatworkedwellduringprogramimplementationin

thisschool.

3. Pleasedescribeanychallengesassociatedwithprogramimplementationinthisschool.


4


SustainabilityLessonsThefollowingquestionsareaboutbilling,sustainability,andstrategicplanningmovingforwardwiththeApexprogram.1. Didyoubillforanyoftheschool-basedservicesprovidedduringtheyear?

a. Forwhatpercentageofservices(approximately)areyouabletobillathird-partypayer(Medicaid,CMOs,orprivateinsurance)?

i. Noneii. 0-25%iii. 26-50%iv. 50-75%v. 75-90%vi. 90-100%

b. PleasedescribeanychallengestobillingforApexservices:{textbox}

2. Whatchallengescanyouidentifytothe:a. Financialsustainabilityforthisprogram?{textbox}b. Sustainabilityoftherelationshipwiththeschoolpartners?{textbox}

3. Arethereanythingsthatyouhavefoundtobehelpfulincreating:

a. Financialsustainabilityforthisprogram?{textbox}b. Sustainabilityoftherelationshipswiththeschools?{textbox}c. Sustainabilityofrelationshipswithcommunitypartners?{textbox}

4. Didyouapplyforand/orreceiveanyothersupplementalfunding(federal,state,DOE,foundations)?

a. Ifyes:Pleasedescribe:

5. HowcouldyouorhaveyoudemonstratedthebenefitsofSBMHservicestoyourcommunityandtootherpotentialfunders?

6. PleasedescribeanymarketingactivitiesthatyouhaveundertakenfortheApexprogram.

7. Otherthantheinformationrequiredinyourmonthlyprogressreportsandreportedinthissurvey,didyoucollectanydataonyourown,orinpartnershipwiththeschoolsorschoolsystems,todetermineprogramoutcomes?

a. Ifyes:pleasedescribe.

8. IsthereanythingelsethatyouwouldliketoshareabouttheimplementationofApexatnewschoolsthathasnotbeencovered?

9. PleasedescribethreegoalsthatyouhavefortheApexprograminyourserviceareaforyear3.

Youhavenowfinishedthesurvey.Thankyouforyourtimeandeffortincompletingthissurvey.Ifyouhaveanyquestionsorconcerns,pleasecontactDimpleDesaibyemailatddesai@gsu.eduorbyphoneat404-727-0346.

AppendixD:ParentSurvey

1


GeorgiaApexProgramParent/GuardianSurvey

OrganizationName:____________________________________________________________SchoolName:_________________________________________________________________NumberofDaysthatStudenthasbeeninApexServices:_______________________________PleaserespondtothequestionsaboutyourchildandfamilysincereceivingSchool-BasedMentalHealthServices.

Question StronglyAgree Agree Neutral Disagree Strongly

DisagreeDon’tKnow

1. OverallIamsatisfiedwiththeservicesthatmychildhasreceived.

2. Mychildisbetterabletodothingsheorshewantstodo.

3. Mychildisbetterathandlingdailylife. 4. Mychildgetsalongbetterwithfamily

members.

5. Mychildgetsalongbetterwithfriendsandotherpeople.

6. Mychildisdoingbetterinschooland/orwork.

7. Mychildisbetterabletocopewhenthingsgowrong.

8. Iamsatisfiedwithourfamilyliferightnow. 9. Mychild’ssymptomsarenotbothering

him/herasmuch.

10. Mychildisabletoseethesamebehavioralhealthproviderwhenschoolisnotinsession.

11. Icanspendmoretimeatworksincemychildcanreceivebehavioralhealthserviceswhileatschool.

12. Iammoreequippedtorespondtomychild’ssymptoms.

13. Ihaveimprovedinmyabilitytoadvocatefortheneedsofmychild.

14. Iknowhowtoaccessappropriateresourcesformychild.

Pleasecompleteandmailbackto:GeorgiaApexEvaluationTeam

55ParkPlace8thFloor

Atlanta,GA30303

AppendixE:NumberofSchoolsServedbyDataSource Page:1

1


Data Source Number of Schools Referenced in Report Notes

Monthly Progress

Reports

293

School Characteristics

Additional Findings from Year

1 to Year 2

Duplicative reporting for

10 schools served by two

providers at least one

time throughout year 2;

283 individual schools

reported

283 Additional Findings from Year

1 to Year 2

Duplicative reporting for

10 schools served by two

providers at least one

time throughout year 2;

283 individual schools

reported

261 School Characteristics

32 schools without a

monthly report since

March 2017,

partnerships not

sustained through the

end of the year

241 School Characteristics

20 schools with either

one report only in May or

June 2017 (new

partnerships) or between

two and four reports

with a recent report in

April, May, or June (new

partnership and/or

developing relationship)

Year-End Survey 214

Figure 5: Schools with SBMH

or PBIS Prior to Apex

Participation, Year 2

Year-End Survey: School

Partnerships

Year-End Survey: Data

Tracking and Sharing

Year-End Survey: School

Integration and Provider

Presence

Total schools reported by

fundees on year-end

survey. Three of the

schools reported on the

year-end survey had two

fundees serving in them,

for a total of 211

individual schools.

AppendixE:NumberofSchoolsServedbyDataSource Page:2

2


Year-End Survey +

Public Data 203

Figure 3. Type of School

Served by Apex Program, Year

2

Figure 4: Type of School

Served by the Apex Program

by Enrollment, Year 2

Figure 6: Apex Year 2 Schools

and Enrollment With Title I

Status, Year 2

Figure 8: Geography of Apex

Schools, Year 2 (1 missing

value)

School Characteristics

11 schools from the year-

end survey do not have

publicly available data

MHPET 132

Table 1. Overall Mean Survey

Scores

Table 2. Changes in Level

Implementation of Partnership

Attributes

Twenty-four fundees

completed baseline

MHPET surveys for 148

schools in September

2016. Follow-up MHPET

surveys were completed

for 210 schools in May

2017. Matched survey

results were analyzed for

132 school partnerships.

MHPET + Public Data 117

Table 3: Mean Scores for

Overall Survey by School-Level

Attributes

Of the 132 schools that

had completed both the

baseline and follow-up

MHPET surveys, 15

schools did not have DOE

or GOSA publicly

available data.

the georgia apex program annual evaluation reportapex year-end survey ... utilizing both...

Documents