measuring the impact of early intervention programs for first episode psychosis: experiences and...

Measuring the Impact of Early Intervention Programs for First Episode Psychosis:

Experiences and Lessons Learned from Oregon and Maryland

December 16, 2014

Oregon Early Assessment and Support Alliance

Tamara G. Sale, MA


• Introduction to EASA

• How we have approached data and evaluation• Uses• Data categories and elements

• Considerations in data collection & evaluation

• National & international collaboration opportunities


• Created 2001 by five-county mental health authority (Mid-Valley Behavioral Care Network or MVBCN) • Data system created 2002

• Statewide implementation began 2007• 23 counties with over 90% of population; • 9 rural counties implementing; 4 remaining

• Portland State University EASA Center for Excellence in 2013- reconstructing data infrastructure

• New state data system (Measures & Outcome Tracking System or MOTS) • Goal to replace and enhance data items

Early psychosis intervention concepts and goals

• Engaging individuals in early stages of psychosis• Reducing delays in treatment • Preventing unnecessary hospitalization & disability

• Evidence-supported treatment

• Improved individual functioning and symptoms

• “Triple Aim”: Better quality, better population health, better use of dollars (Institute for Health Care Improvement, http://www.ihi.org/Engage/Initiatives/TripleAim/Pages/default.aspx)

EASA Eligibility

• Goal: change trajectory for all teens and young adults who begin to develop schizophrenia or schizoaffective disorder in Oregon

• Schizophreniform and bipolar spectrum first episode psychosis going back to 12 months

• Core age range 15-25; some below and some over

• Added psychosis risk syndrome 2010

• All payors

EASA Services

• Community education

• Outreach and proactive engagement, rapid access

• Assessment and consultation• Diagnostic, holistic strengths & needs/developmental goals,

comprehensive risk & safety planning

• Transdisciplinary team (credentialing process)• Individualized planning, intensity• Vocational support (IPS model)- work & school• Family education and support (multi-family groups and single family)• Occupational therapy- cognitive, sensory, functional needs• Medical- prescribing, psychoeducation, nursing, wellness• Counseling & psychoeducation including alcohol & drug treatment

(motivational interviewing, cbt)• Participatory decision making & transition

How EASA Data is Used

• Accountability:• Transparency around community education efforts , referrals, utilization, fidelity

• Quality improvement:• Cross-county and intracounty comparisons of referral patterns, intakes, fidelity,

outcomes, discharge info over time• Benchmarking of key indicators for improvement efforts• Convene routinely by phone and periodically in conference where data is shared

and goals are discussed• Review of data by program participants and graduates to help design improved

interventions and policies (Young Adult Leadership Council)

• Sustainability efforts & program development• Changes in insurance and utilization; costs vs. resources, testing assumptions

around financial modeling; recognizing variability in resources based on population • State, Association of County Mental Health Programs, program directors

How Data Is Used

• Legislative communication• Hospitalizations, school/work• Utilization by community• Narrative stories through direct testimony & anecdotes

• Department of Justice Olmstead• Number served

• Informing research/improvement questions & design• Data pulls focused on specific questions

• Examples: relationship between substance abuse, legal involvement and hospitalizations; impact of clinician turnover on outcomes; profiles of who disengages early

• Informing articles and networking with other sites (learning health care)

Example of Data Uses (legislative reports)

Percent in School or Working, Individuals Discharged from EASA at 12 months or Longer (n=522)

EASA Measurement Methods

• Staff credentialing database- training, supervision, review of documents• Program fidelity review process measuring consistency with practice

guidelines: direct interviews, chart reviews, focus groups• Clinician-completed forms:

• Entered remotely; able to access historical data; repeated data fields fill in automatically (i.e. admit date)

• Community education• Referrals (source, demographics, eligibility criteria)• Intakes (demographic, functional data)• Quarterly outcome reviews (diagnosis, service, demographic & functional data;

reason for discharge)• State data system: specific services

• Periodic surveys, focus groups, targeted research• Administrative data (hospital databases, MOTS,

agency billing, etc.)

EASA Core Measures

• Engaging individuals• Number referred and intakes by county • Penetration: Number entering with demographics• Accuracy: Diagnosis• Reason for discharge (drop-outs)• Would like to look at people who don’t get referred

• Preventing delays• Source of referral (non-crisis/outside mh)• Timing of referral to intake• Difficulty finding: Pathway to care (qualitative)• Delay in care: Duration of untreated psychosis

Core Measures

• Evidence-supported treatment methods• Fidelity review and credentialing process• Continuity of care (clinician turnover, hospitalization

outreach, transition planning)• Sustained or improved short-term and long-term

functioning• School, work, living situation, substance use, legal

involvement, hospitalizations, insurance, disability status• “Triple Aim”: Better quality, better population health,

better use of dollars• Reduced hospitalizations• Cost/utilization study

International Consensus Measures

• Acceptability• Accessibility• Appropriateness• Continuity• Effectiveness• Competence• Efficiency• Safety

Addington et al. (2005). Performance measures for early psychosis treatment services. Psychiatric Services, vol 56 no 12, p. 1570-1582.


• Acceptability- satisfaction, complaints, involvement in decision making

• Accessibility- waits, reach, access by homeless, psychiatry, primary care, open referral, reducing barriers, perceptions of access, duration of untreated psychosis, point of first contact; (EASA- insurance status)

Addington et al. Performance Measures for Early Psychosis Treatment Services. Psychiatric Services, December 2005, vol 56 no 12, p. 1570=1582.


• Appropriateness• Family & individual psychoeducation• Gatekeeper education• Acute care: average length, readmissions (EASA- %

involuntary)• Medication measures • Clinical: Assertive treatment, vocational &

psychological



• Continuity• Case management• Change of therapist• Post-hospital follow-up• Drop-out rates• ER visits



• Effectiveness• Global functioning• Positive, negative, depressive symptoms & symptom remission• Work & occupational functioning• Educational functioning• Community tenure• Improvement in quality of life• Improvement in family burden• Housing support• Financial status disability)• Assessment of substance abuse• Perception of recovery• Mortality rates• (EASA): arrests & legal involvement

Addington et al.


• Competence• Knowledge & application of evidence-based practice• Formal and continuing education

• Efficiency• Spending per capita on early psychosis services• Administration costs• Cost per patient

Addington et al.


• Safety• Side effects• Suicide rate & attempts• Homicides & attempts• Medication errors

Addington et al.

Considerations in Evaluation Start-Up

• State or region-level coordination functions needed• Data management, follow-up for accuracy & completeness, analysis

• Administrative data can be difficult to access • Disconnect between academic and community environments

• HIPAA sharing arrangements & capacity- where does research end and quality improvement begin?

• Clinician buy-in & concerns (workload, duplication & relevance), accuracy

• Logistical ability to access & recruit participants directly• Engagement vs. consent requirements

• Forums for sharing and use• Early stages- “selling” or justifying investment; may create disincentive

for acknowledging lower performance areas or multiple interpretations

Learning Healthcare System

• National conversation among early psychosis network with NIMH leadership

• Based on Institute of Medicine recommendations

• Shared data and analysis to identify and target improvements• Individual characteristics/ demographics• Services• Outcomes

Conversations with Dr. Robert Heinssen; Institute of Medicine Committee on the Learning Health Care System in America. (2013). Best care at lower cost: the path to continuously learning health care in America. Washington DC: National Academies Press.

Continuously Learning Health Care System

• Access to best practice knowledge

• Data from care experience

• Patient and caregiver inclusion

• Incentive alignment & transparency

• Leadership and competency development around continuous learning culture

Institute of Medicine Committee on the Learning Health Care System in America. (2013). Best care at lower cost: the path to continuously learning health care in America. Washington DC: National Academies Press.

National Dialogue: PEPPNET

• Prodrome and Early Psychosis Program Network

• Forum for national collaboration and coordination

• Workgroups

International Resource

• International Early Psychosis Association• Biennial conference (next one in Milan, Italy)

University of Maryland RA1SE Connection Program

Ann Hackman, M.D.

Baltimore’s RA1SE Connection Program

• Began in 2010 • Part of RA1SE research• Fully developed program utilizing evidence

based practices• Research ended in 2013. Program continued

through the University of Maryland’s Division of Community Psychiatry.

• Part of a larger early intervention initiative

Admission criteria

• Age: 15-35• Diagnosis: Schizophrenia, schizoaffective disorder,

schizophreniform disorder, brief psychotic disorder, psychosis not otherwise specified, or delusional disorder

• Psychopathology: At least one symptom of psychosis at any time during the current episode (or the recent episode if the individual is seen as he/she is recovering)

• Duration of Illness: ≤ 2 years since the first onset of psychotic symptoms

• Exclusion criteria including intellectual disability

Team staffing

• Team Leader: overall coordination of services individual therapy, case management, crisis intervention, information gathering, safety planning

• Recovery Coach: Social skills training, weekly participant group, monthly family group, school coordination

• Employment/Education specialist – job development, addressing work and school related goals and problems

• Psychiatrist: prescribing, shared decision making• Administrative support and occasional nursing

available from clinic where program is embedded

Some components of service

• Regular prescribed meetings with participant and team (e.g. weekly with psychiatrist in the 1st four weeks)

• Family involvement whenever possible• Safety planning• Shared decision making• Psychoeducation• Recovery focus• Flexibility• 24-hour on call availability• Participant recovery and social skills group

More components

• Family psychoeducation group• Individual sessions• Facilitating interface between participants• Preferred antipsychotic list (LAIs, aripiprazole,

risperidone, perphenazine, loxapine)• Discipline specific expert consultation• Shared meetings with the New York team• Access to unique materials (e.g. from the National

Empowerment Center)• Regular evaluations including with standardized

instruments

Transition beyond research

• Research ended in June 2013• Now funded through SAMHSA/State of Maryland and

part of a larger plan to proliferate evidence based early intervention programs in the state

• Program continues as part of a still developing Maryland Early Intervention Program (MEIP) (a collaborative program including University of Maryland (UM) School of Medicine, UM Medical System UM Baltimore County and the Maryland Division of Health and Mental Hygiene). MEIP Includes:• Outreach and Education Services• Clinical Services• Consultation Services• Training and Implementation Support Services

Data

• Initially most of the data collected was directed by the research (most collected by the team, some by research staff) with a small portion required by the UMD Division of Community Psychiatry

• Currently most standardized measures are in the public domain (except for violence assessments) and are collected in conjunction with clinical sessions

Data

• Data are reviewed quarterly by treatment team and by Maryland Early Intervention Program

Data Collected

• All outreach educational efforts to community• Referrals: when received, when contacted by team,

disposition• Recording of all client/family contacts, attention to

frequency and location of visits, use of on- call service

• Client hospitalizations• Client arrests • Individuals who leave treatment or are lost to follow-

up

Data collected

• Height, weight, waist circumference (quarterly)

• Labs CBC, Comprehensive metabilic panel, glucose, lipids, liver function test, HgbA1C, TSH, insulin level (semi-annual)

• Other labs as indicated (e.g. lithium level during titration and every six months)

• Consumer satisfaction questionnaire

Scales and measures

• On Intake• UCLA-PTSD Index (ages 12-18) or PTSD Checklist (PCL-C) (over 18)• Historical Clinical Risk Assessment-20 (HCR-

20)• Structured Assessment of Violence Risk in

Youth (SAVRY)• Suicide Risk Assessment (currently a clinic

instrument; may be repeated as needed)

Scales and measures

• On intake then annually• Cornblatt Social/Role Functioning Scale• Abnormal Involuntary Movement Scale

(AIMS)

• On intake then semi-annually• Individualized safety plan• Maryland Assessment of Recovery in People

with Serious Mental Illness (MARS)

Scales and measures

• On intake and then quarterly• Four Item Positive Symptom Rating Scale

(quarterly)• Four Item Negative Symptom Rating Scale

(quarterly)• Four Item Depressive Symptom Scale

(quarterly)

Data collection discontinued following research

• Quarterly Symptom Side Effect Checklist• Quarterly Psychiatric Medication side effect

checklist• Simpson Angus EPS rating scale• Wellness plan• Psychiatric visit log• Current experience with and preferences with

medications form• Serious Adverse Event form• Check off of Connection program handout for meds

Lessons learned

• Participants in the program were generally cooperative with data collection

• Importance of collecting information, using standardized instruments which can be incorporated into regular clinical care • With relative ease• With a minimum of redundancy

Thoughts for the future

• More computer based data standardized assessment instruments (including for self report)

Some web sites

• Four Item positive, negative and depression checklists (quarterly) http://www.sccp.sc.edu/sites/default/files/45107%20padforproofing.pdf • Cornblatt

http://schizophreniabulletin.oxfordjournals.org/content/33/3/688.full.pdf+html• UCLA-PTSD Index http://www.ptsd.va.gov• PCL-C http://www.mirecc.va.gov/docs/visn6/3_PTSD_CheckList_and_Scoring.pdf • HCR-20 http://hcr-20.com/ • SAVRY http://www4.parinc.com/Products/Product.aspx?ProductID=SAVRY

(available for purchase)• AIMS http://www.atlantapsychiatry.com/forms/AIMS.pdf • MARS http://www.nyc.gov/html/doh/downloads/pdf/mh/measuring-recovery-toolkit.pdf

Maryland Early Intervention Program http://marylandeip.com

Some references

• Borum, R., Bartel, P., & Forth, A. (2008). Structured assessment of violence risk in youth (SAVRY). In B. Cutler (Ed.), Encyclopedia of psychology and law. (pp. 771-773).

• Douglas, K. S., Hart, S. D., Webster, C. D., & Belfrage, H. (2013). HCR-20V3: Assessing risk of violence – User guide. Burnaby, Canada: Mental Health, Law, and Policy Institute, Simon Fraser University.

• Steinberg AM, Brymer MJ, Kim S, Ghosh C, Ostrowski SA, Gulley K, , Briggs, EC, Pynoos, RS (2013). Psychometric properties of the UCLA PTSD Reaction Index: Part 1, Journal of Traumatic Stress, 26, 1-9.

• Steinberg, A. M., Brymer, M., Decker, K., & Pynoos, R. S. (2004). The UCLA PTSD Reaction Index. Current Psychiatry Reports, 6, 96-100.

• Weathers, F.W., Litz, B.T., Keane, T.M., Palmieri, P.A., Marx, B.P., & Schnurr, P.P. (2013).The PTSD Checklist for DSM-5 (PCL-5). Scale available from the National Center for PTSD at www.ptsd.va.gov.

• Also New York Connections Program http://nyspi.org/nyspi/patients-and-families/ontrack-ny-connections-program

http://www.nyc.gov/html/doh/downloads/pdf/mh/measuring-recovery-toolkit.pdf

measuring the impact of early intervention programs for first episode psychosis: experiences and...

Documents

easa data

data categories

data items

review of data

oregon early assessment

ma slide

transition slide

mvbcn data system