mhhsc program constructing a cross-site evaluation of ethnic minority hiv mental health services the...
TRANSCRIPT
MHHSC PROGRAM
Constructing a Cross-Site Evaluation of Ethnic Minority
HIV Mental Health Services
The Mental Health HIV Services
Collaborative (MHHSC) Program
Michael Costa, AbtBarbara J. Silver, CMHSMaria Madison, AbtTandiwe Njobe, AbtGabriela Garcia, Abt
MHHSC PROGRAM
Presentation Goals
Describe the MHHSC Program Components Context Relevance
Convey: The process of this collaborative. utilization-
focused evaluation Accomplishments to date (products) Uses of the outputs of products
MHHSC PROGRAM
Substance Abuse Mental Health Services Administration
(SAMHSA)
Center for Mental Health Services (CMHS)
Center for Substance Abuse Prevention (CSAP)
Center for Substance Abuse Treatment (CSAT)
MHHSC PROGRAM
SAMHSA HIV/AIDS HISTORY
Prior to 2001: CSAT funds targeted capacity/HIV/AIDS Substance Abuse treatment programs for African American, Hispanic/Latino, and other racial/ethnic minorities.
2001: CMHS funds a similar targeted/expanded capacity program for community based organizations (CBOs) serving African American, Hispanic/Latino, and other racial/ethnic minorities.
MHHSC PROGRAM
HIV Infection among People with Severe Mental Illness
Across all published studies, the rate of HIV infection among psychiatric patients is 10%, 25 times higher than that of the general population.
Cournos & McKinnon, 1997;Krakow et al., 1998;Rosenberg et al., 2001
HIV Prevalence in Psychiatric and General Populations
10
0.4
0
2
4
6
8
10
12
Psychiatric General
MHHSC PROGRAM
Number of partners
Number of risky or anonymous partners
Frequency of sex trading
Rates of coerced sexMcKinnon et.al., 1996, 1999
COMPARED TO GENERAL POPULATION, PATIENTS HAVE FEWER COMPARED TO GENERAL POPULATION, PATIENTS HAVE FEWER EPISODES OF SEX WITH A PARTNER, BUT THEY HAVE:EPISODES OF SEX WITH A PARTNER, BUT THEY HAVE:
Sexual Risk Behavior Among People With Severe Mental Illness:
MHHSC PROGRAM
Elevated risk for HIV infection in psychiatric
Patients
Risk factors: Alcohol and other drug use Unsafe sex Environmental circumstances (poverty, institutionalization, etc.)
Substance use is associated with both psychiatric symptoms and HIV risk
Psychiatric Disorders and Risk for HIV Infection
MHHSC PROGRAM
MHHSC Program
21 Mental Health Service Sites – CBOs at least 2 years experience in behavioral
health care services MH Centers, Substance Abuse facilities, Primary Health Care &/or HIV/AIDS clinics
Abt Associates, Inc. = Coordinating Center
MHHSC PROGRAM
MHHSC Program
Congressional requirement (CBC & CHC): provide these new HIV/AIDS-related mental health services in both traditional and non-traditional settings.
Funding for mental health treatment services and related case management only.
However, grantees are required to develop comprehensive integrated individual treatment plans and monitor primary and substance use treatment.
MHHSC PROGRAM
WHO ARE THE SERVICE SITES?
New HIV/AIDS-Related Services New services (no prior HIV/AIDS-related MH services)– 5 sites Expanded services – 16 sites
Service Delivery Settings Traditional (primarily clinic-based) – 13 sites Non-traditional (e.g., mobile treatment, ) – 1 site Both settings – 8 sites
Target Populations African American -19 sites Hispanic/Latino – 14 sites Haitian – 1 site Native American – 1 site
MHHSC PROGRAM
Demographics – Gender
61.90%
37.10%
0.90%0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
Male Female Other
MHHSC PROGRAM
Hispanic Latino/a = 30.65%
Demographics – Race / Ethnicity
Native American3%
Asian0%Pacific Islander
0%
White17%
Other Race30%
African American/ Black50%
MHHSC PROGRAM
Demographics - Age
Age Category PercentLess than 20 years 1.72%
20 – 24 years 4.68%
25 – 29 years 7.41%
30 – 34 years 10.76%
35 – 40 years 20.83%
40 – 44 years 22.78%
45 – 50 years 14.51%
50 – 54 years 8.74%
55 years and older 6.01%
MHHSC PROGRAM
DSM IV Diagnoses
27.21%
5.39%
50.51%
16.89%
0%
10%
20%
30%
40%
50%
60%
Single Dx Dual Dx Triple Dx More than 3 Dx
MHHSC PROGRAM
DSM IV Diagnoses Categories
9.44%
13.88%
6.63%
45.71%
24.33%
6%
12%
18%
24%
30%
36%
42%
48%
AdjustmentDisorder
AnxietyDisorder
BipolarDisorder
DepressiveDisorder
OtherIncluding HIV
Dementia
MHHSC PROGRAM
MHHSC Program Goals
Expand
Effective
Culturally Competent
Mental Health Services
For PLWHIV
In Minority Communities
MHHSC PROGRAM
MHHSC Capacity Building
The Coordinating Center provides technical assistance to grantees:Local and regional trainings (e.g., ethics, neuropsychology, cultural competence)Expert speakers at national meetings on topics of mental health and psychiatry, gender issues, consumer involvement, etc.Assistance with data collection and management
MHHSC PROGRAM
CULTURAL COMPETENCE
Strategies to Promote Culturally-Competence Service Delivery Demographically similar staffing as target population(s) Treatment facilities reflect cultural interests of target
group(s) (e.g., artwork/decor, waiting room music, etc.) Diversity/cultural competence training Consumer feedback
This is being addressed in great detail by the MHHSC Cultural Competence Subcommittee
MHHSC PROGRAM
MHHSC Utilization-Focused Cross-site Evaluation
Clinically/Programmatically Relevant Evaluation meet the needs of the clinical and other program staff in
their efforts to better serve their clients
The MHHSC cross-site evaluation is voluntary, except for submission of required aggregate data critical that the cross-site evaluation be clinically and
programmatically relevant and valuable, otherwise no site would agree to participate
MHHSC PROGRAM
Evaluation as a Pain in the NeckMental health/social service providers’ historical experience with evaluation Intrusive reporting requirement Necessary to satisfy Local, State and Federal funding
requirements.
Often data are never reported back to programs
Concerns that data will used to make them look bad or draw inappropriate comparisons between sites and providers
MHHSC PROGRAM
MHHSC X-Site Evaluation Collaborative Process
Engaging the key stakeholdersLocal site-specific evaluatorsLocal site cliniciansProgram AdministratorsConsumer Advisory Boards (CABS)
MHHSC PROGRAM
MHHSC X-Site Evaluation Collaborative Process
EvaluationSubcommittee
(ESC)
Logic ModelWorkgroup
Eval/ClinicianCommunication
Workgroup
Site ProgramLogic Models
Site-SpecificEvaluation Summaries
Determination ofEvaluation Foci
Across Sites
MHHSC PROGRAM
MHHSC X-Site Evaluation Collaborative Process
Face-to-FaceMeeting w/MHHSC
Stakeholders
X-Site CandidateEvaluation
Foci
MHHSC PROGRAM
X-Site Candidate Evaluation Foci
Foci by Quantitative and Qualitative Methods
0
5
10
15
20
25
Fre
q o
f Met
ho
ds
Acr
oss
Site
Pla
ns
Quantitative Methods
Qualitative Methods
MHHSC PROGRAM
Quantitative Methods Used Across Sites
Quantitative Methods
0
5
10
15
20
25
Clie
nt
Sat
isfa
ctio
nS
urv
ey
Ad
min
istr
ativ
eC
lien
t C
on
tact
Dat
aA
bst
ract
ion
CM
HS
GP
RA
(wit
h a
nd
wit
ho
ut
aug
men
tati
on
)
Su
rvey
Dev
elo
ped
by
Eva
luat
or
Clin
ical
Rec
ord
sA
bst
ract
ion
Inta
ke a
nd
Ass
essm
ent
To
ol
Fre
q A
cro
ss S
ite
Pla
ns Quantitative Methods
MHHSC PROGRAM
Qualitative Methods Used Across SitesQualitative Methods
0
5
10
15
20
25
Focus Groups Interviews Observation ClinicalRecordReview
Fre
q A
cro
ss
Sit
e P
lan
s
MHHSC PROGRAM
MHHSC X-Site Evaluation Collaborative Process
Breakout Groups(BOGs)
EvaluationDesign
Workgroup
MHHSCNationalMeeting
ConsumerNetwork
Committee
X-Site CandidateEvaluation
Foci
Face-to-FaceMeeting w/MHHSC
Stakeholders
MHHSC PROGRAM
MHHSC X-Site Evaluation Collaborative Process
Next Steps
Pilot Test
Instruments/Protocols
Final DraftField X-SiteEvaluation
Pray Everything Goes As Planned
Instruments/ProtocolsBeta Version
X-Site DesignSite Review
MHHSC PROGRAM
Final Cross Site Foci
Four main domains* of interest across sites:Mental HealthClient SatisfactionClient Retention and Service UtilizationQuality of Life & Medical Health
*Cultural Competence appears in all domains.
MHHSC PROGRAM
Common Questions of Interest
Who is being served? What are the barriers to care and to services for the target population? How do programs overcome these barriers? What services are being used by the target population? Are the services being provided in a culturally competent manner?
MHHSC PROGRAM
Mental Health
Given the target population being served, what are the Prevailing mental health diagnoses?Co-occurring disorders? (e.g., substance use)Changes in mental health symptoms?Disorders that may be more prevalent with HIV
positive status?Physiological HIV disease factors that
contribute to mental health symptoms?
MHHSC PROGRAM
Client Satisfaction
Questions relating to care received by clients.What care is being received?What is the frequency of care?What is the level of client involvement in care?What is the setting in which care is received? Is the client satisfied with the care?
MHHSC PROGRAM
Client Retention & Service Utilization
Client RetentionHow many cases become active/inactive in a
given time period?What is the site definition for active/ inactive
clients?What are the site policies for case closing? What
reasons are given for case closings?What retention strategies have been effective
with the target population?
MHHSC PROGRAM
Client Retention & Service Utilization - continued.
Service UtilizationWhat is the type, frequency and duration of
services used?How are services provided?
Referrals in and out of programAgency linkages - in-house and external - to other
providersService setting – traditional/ non-traditional
MHHSC PROGRAM
Quality of Life
How does the quality of life status and general medical health of a client impact uponTreatment compliance (medications, ITP)?Response to treatment?Client retention? Service utilization?
How does quality of life status and general medical health change over time with treatment?
MHHSC PROGRAM
Methods & Indicators1. CLIENT LEVEL INSTRUMENT
confidential – unique client id
Demographics
Social supports
Acculturation
Substance use and risk behavior
Trauma
Medical health
Medication adherence
Clinicians report
MHHSC PROGRAM
Methods & Indicators2. CLIENT SATISFACTION SURVEY
anonymous
Client characteristics
Service utilization
CareClient involvement in care Access/ barriers to care Cultural competence in care Overall satisfaction with care
MHHSC PROGRAM
Methods & Indicators3. FOCUS GROUPS
with clients on site
Focus groups will provide qualitative backdrop to analyze quantitative data from client satisfaction survey and client instrument
Types of services used Satisfaction with services Barriers to care Cultural competency
MHHSC PROGRAM
Methods & Indicators4. SITE VISIT DATA COLLECTION
MHHSC Coordinating Center Staff
Continuum of services
Location of site Geographic setting Site community
Service setting (traditional / non traditional)
Client retention strategies
Site activities to overcome identified barriers to mental health care
MHHSC PROGRAM
Utility of Cross-Site Analyses From the Sites’ Perspective
Site buy-in has been an on-going process
Clinicians and local evaluators participate in the evaluation design work group
They took ownership of the evaluation design
Made decisions on utility of collecting certain types of data across sites
MHHSC PROGRAM
Client Focused Domain
Client Characteristics
Clients’ Presenting Diagnosis
Barriers/Access to Care (enhance/maintain Client’s Quality of Life/Health)
MHHSC PROGRAM
Program Context Domains
Program StructureCBO VS Large System of care Staffing patterns (FT, PT)
Sustainability EffortsPlan in place
MHHSC PROGRAM
Client Focused DomainsClient Characteristics Site Uses
Systematic documentation type of clinical services offered to what target population across the sites
Tailor services to specific populations
Identify (ID) staff training & resource needs
Determine success or not in reaching target population
ID changing client populations
Ability to pool data across all sites by client characteristics
Identify types of clients (positively) impacted by type of treatment
Learn from sites’ experiences
ID promising practices
MHHSC PROGRAM
Client Focused Domains (cont.)Mental Health Site UsesExamine patterns/changes in symptomatology. Are symptoms unique to a given population
Better assess clients
Determine client case-mix
Document increased client needs
Gain insight on co-occurring disorders
Better understand client needs
Tailor services offered
Inform client retention strategies
Examine patterns of mental health disorders across sites
Understand impact of services
Inform site staff on cultural competence issues
MHHSC PROGRAM
Client Focused Domains (cont.)
Barriers / Access to Care Site Uses
Identify barriers to care specific to target population
Improve engagement
Improve retention
ID methods for overcoming stigma
Determine strategies for inter-agency collaboration
Gain insight into cultural competence issues
Gain better understanding of ease of access to mental health services
MHHSC PROGRAM
Program Context DomainsProgram Structure Site Uses
Understand contexts in which other sites operate
Determine relevance/applicability of findings across sites
Identify replicable program components
Understand uniqueness of site challenges
Learn from other sites' experiences about ways to overcome barriers to care
Identify promising practices
Create a menu of tools to address barriers
MHHSC PROGRAM
Program Context Domains (cont.)
Site Program Structure (cont’d)
Site Uses
Understand differences in the range of program services
Identify gaps in services
Identify staffing/resource needs
Inform program expansion and sustainability
Identify promising practices in client referrals
Improve client treatment fidelity
Improve client retention
Create a menu of tools to address barriers to care
MHHSC PROGRAM
Program Context Domains (cont.)
Sustainability Site Uses
Examine data on services provided to support funding opportunities
Develop sustainability plans at individual site or cross-site level if similarity in client needs
Develop plan to target specific funding sources
MHHSC PROGRAM
Purposes/Uses of Data
Cross-site analysis Attention to differences across sites, client
characteristics Careful & appropriate Nuanced, not reductionist Program context critical Use of qualitative and program-level data (e.g.,
differences in resources/capacity available)
MHHSC PROGRAM
Purposes/Uses of Data (Recap)UTILIZATION-FOCUSED EVALUATIONData/analysis to be provided to sites User friendly feedback A timely manner For program improvement /development of
better intervention strategiesNOT REPORT CARDS Individual clinicians Individual sites