Implementation of an Electronic Information System to Enhance Practice at an Opioid Treatment Program (R01 DA022030)

Steven Kritz, MD; Carlota John-Hull, MD; Ben Louie, BA; Melissa Lin, MS; Roberto Zavala, MD; Charles Madray, RPA-C, MBA; Lawrence S. Brown, Jr., MD, MPH, FASAM

Division of Medical Services, Research and Information Technology, Addiction Research and Treatment Corp, Brooklyn, NY 11201

NIDA RFA-DA-06-001 (R01): Enhancing Practice Improvement in Community-Based Care for Prevention and Treatment of Drug Abuse

DISCLOSURES

• There are no financial interests or other disclosures to report for any of the authors involved in this project

• Grant support provided by NIDA: R01 DA022030

BACKGROUND Electronic information systems rarely

utilized or evaluated in substance abuse treatment settings

ARTC serves a racially, ethnically and economically disenfranchised population

ARTC serves a population that experiences significant disparities in access and quality of healthcare

PURPOSE• Evaluate the implementation of an

electronic information system using the following domains: – Quality – Risks– Satisfaction– Productivity– Finance

STUDY DESIGN Prospective, comparative study Pre-post implementation evaluation 3-year timeline

AIMS & HYPOTHESESSpecific AIM 1: Quality

Hypothesis - Improved capture or

timeliness of: Medical Assessments Multidiscipline

Assessments HCV Viral Load

Specific AIM 2: RisksHypothesis - Rates will

decrease for: Patient Complaints Patient Incidents Medication Errors

Specific AIM 3: SatisfactionHypothesis - Overall satisfaction will increase for:

Patients Clinicians Managers

AIMS & HYPOTHESES

Specific AIM 4: ProductivityHypothesis: Visits will increase for:

Counseling Primary Care HIV Case

Management

Specific AIM 5: Financial Performance

Hypotheses: Revenue per capita

staff will increase Cost per visit will

decrease

Aim Measure Data Source(Pre)

Data Source(Post)

Quality

% Medical assessments

performed on-time

Paper chart Electronic record

% Multidiscipline assessments

performed on-time

Paper chart Electronic record

% HCV Viral Loadobtained

Paper chart Electronic record

Risks# of Complaint /

Incidents / Medication Error

Reports

Reports to CQI Manager

Reports to CQI Manager

AIMS, MEASURES & DATA SOURCES

Aim Measure Data Source (Pre)

Data Source (Post)

Patient Satisfaction Survey SurveySatisfaction

Productivity

Clinician/Manager Satisfaction

Survey Survey

# Counseling visits Clinician logs Electronic logs# Primary care visits Clinician logs Electronic logs

# HIV case management visits

Clinician logs Electronic logs

FinancialRevenue per capita Finance/

HR DeptsFinance/

HR DeptsPerformance

Cost per visit Finance Dept Finance Dept

AIMS, MEASURES & DATA SOURCES

PRE-POST DATA ANALYSIS

Quality Chi-square exact test

Risks Cochran-ArmitageSatisfaction Chi-square testProductivity t-testFinance t-test

QUALITY: ANNUAL MEDICAL & MULTIDISCIPLINE ASSESSMENTS

Hypothesis: Implementation of the electronic health record will result in a higher percent of patients having annual medical assessments performed within 30 days of admission anniversary; and annual multidiscipline assessments performed on or before the admission anniversary

QUALITY: ANNUAL MEDICAL & MULTIDISCIPLINE ASSESSMENTS

Pre-implementation Eligibility: Admission date: 7/1/06 to 6/30/07Length of stay >365 days# of Eligible Patients: 420Post-implementation Eligibility: Admission date: 11/1/08 to 10/31/09Length of stay >365 days# of Eligible Patients: 423

QUALITY: STUDY POPULATION

PRE POST

Admission Date: 7/1/06 – 6/30/07 11/1/08 – 10/31/09

N 420 423 Mean Age: 45+/- 8.8 yrs 47+/-9.9

yrs (19-66) (20-78) Percent Female: 31% 30% Race/Ethnicity:

% Hispanic 52% 47% % African American 36% 41%

Measure Study Period# (%)

On Time# (%) Late + Not

Completed

P-value

AnnualMedical

Assessments*

Pre-implementati

on

350 (83%)

70 (17%)

<0.001Post- implementati

on

411 (97%)

12(3%)

Annual Multidisciplin

e Assessments

**

Pre-implementati

on

294(70%)

126 (30%)

<0.0001Post-

implementation

407 (96%)

16(4%)

QUALITY: ASSESSMENT RESULTS

* + 30 days of 1-year anniversary ** < 365 days after admission

QUALITY: 30-DAY & 90-DAY MULTIDISCIPLINE ASSESSMENTS

Hypothesis: Implementation of the electronic health record will result in a higher percent of patients having 30-day and 90-day multidiscipline assessments performed on or before the due date

QUALITY: 30-DAY MULTIDISCIPLINE ASSESSMENTS Pre-implementation Eligibility: Admission date: 7/1/06 to 6/30/07Length of stay >30 days # of Eligible Patients: 613Post-implementation Eligibility: Admission date: 10/1/09 to 9/30/10Length of stay >30 days # of Eligible Patients: 704

QUALITY: 90-DAY MULTIDISCIPLINE ASSESSMENTS Pre-implementation Eligibility: Admission date: 7/1/06 to 6/30/07Length of stay >90 days# of Eligible Patients: 576Post-implementation Eligibility: Admission date: 10/1/09 to 9/30/10Length of stay >90 days# of Eligible Patients: 608

Measure Study Period# (%)

On Time# (%) Late + Not

Completed

P-value

30-DayMultidisciplin

e Assessments

*

Pre-implementati

on

441 (72%)

172 (28%)

<0.001Post- implementati

on

614 (87%)

90 (13%)

90-Day

Multidiscipline

Assessments**

Pre-implementati

on

242 (42%)

334 (58%)

<0.001 Post-

implementation

423 (70%)

185 (30%)

QUALITY: ASSESSMENT RESULTS

* < 30 days after admission ** < 90 days after admission

QUALITY: HEPATITIS C VIRAL LOAD

Hypothesis: Implementation of the electronic health record will result in a higher percent of Hepatitis C antibody positive patients tested for hepatitis C viral load

Pre-implementation Eligibility: Admission date: 7/1/06 to 6/30/07 Length of stay: >60 days # of Eligible Patients: 670

Post-Implementation Eligibility: Admission date: 10/1/09 to 9/30/10 Length of stay: >60 days # of Eligible Patients: 653

Study PeriodHCV

Antibody Positive

(%)/Negative

(%)

Appropriately Referred +

Refused

HCV VL Done / HCV VL Done + Not Done

(%)

P-value

Pre-implementati

on

342 (51%) /328 (49%)

160 + 4 = 164

151/178 (85%)

NS

Post-implementati

on

296 (45%) /357 (55%)

212 + 5 = 217

64/79(81%)

QUALITY: HCV RESULTS

* HCV VL Not Done: 27 (pre); 15 (post)

RISK: MEDICATION ERRORS, PATIENT COMPLAINTS & PATIENT

INCIDENTS

Hypothesis: Implementation of the electronic information system will result in a lower number of Medication Errors, Patient Complaints & Patient Incidents

RISK: MEDICATION ERRORS, PATIENT COMPLAINTS & PATIENT

INCIDENTS Pre-implementation Period: 7/1/06 to 6/30/07 # of Medication Errors: 8 # of Patient Complaints: 15 # of Patient Incidents: 64

This domain was not included in the post-implementation data collection: - N relatively small - Processes did not change post-

implementation

SATISFACTION: PATIENT SURVEYS (6 QUESTIONS)Hypothesis: Implementation of the electronic health record will result in increased patient satisfaction

# of Surveys Administered (Pre & Post): 1,000 - Apportioned by Clinic Census

- Process: First come/first served with 2-ride MetroCard

Patient Survey:How long have you been with ARTC?

< 90 days 90 days-1 yr > 1 yr0

102030405060708090

100

8%

22%

70%

8%17%

75%

%

SATISFACTION: PATIENT SURVEY RESULTS

• Length of time with ARTC not different pre-post

• 35% completed pre & post-implementation surveys

• Mean Score & Standard Deviation: Q1-6 - Pre: Mean Score: 3.78* Std Deviation: 0.750 - Post: Mean Score: 3.74* Std Deviation: 0.775

*P=NS for each question & overall

Patient Survey: Q6: How satisfied are you with the overall quality of care you receive?

Not sat

isfied

Sligh

tly Sa

tisfie

d

Somew

hat S

atisfie

d

Satis

fied

Very

satisfi

ed0

20406080

100

3% 8% 15%

48%26%

3% 9% 16%

49%

23%%

PRE: Mean: 3.86 Std Deviation: 0.991 Range: 1-5POST: Mean: 3.80 Std Deviation: 0.999 Range: 1-5

SATISFACTION: STAFF SURVEYS (17 QUESTIONS)

Hypothesis: Implementation of the electronic health record will result in increased staff satisfaction

Eligible employees: Clinicians and Managers Pre Post

# of eligible employees: 148 155

# of surveys returned: 99 (67%) 92 (59%)

SATISFACTION: STAFF SURVEY RESULTS

• 54% completed pre & post-implementation surveys

• Mean Score & Standard Deviation: Q1-17 - Pre: Mean Score: 3.11* Std Deviation: 0.819 - Post: Mean Score: 3.32* Std Deviation: 0.728

*P=NS overall; P<0.03 for Q1, 7, 8, 11, 13Pre Mean Score > Post Mean Score for Q2 & 16 only (P=NS)

SATISFACTION: STAFF SURVEYS

Questions with significant post-survey satisfaction:

Q1: How satisfied are you with the ability to access needed reports or obtain information for needed reports? (P=0.008)Q7: How satisfied are you with the ability of the system to track your productivity and/or your staff? (P=0.005)Q8: How satisfied are you with the organization of the patient records and/or reports? (P=0.03)Q11: How satisfied are you that the patient record and/or management report format helps to prevent you from overlooking information? (P=0.03)Q13: How satisfied are you that you can communicate patient and/or administrative information to and from administrative staff? (P=0.03)

Staff Survey:Q6: How satisfied are you with the system overall?

Not sat

isfied

Sligh

tly Sa

tisfie

d

Somew

hat S

atisfie

d

Satis

fied

Very

satisfi

ed0

20406080

100

5% 14%

48%28%

5%5% 7%

46% 40%

2%%

PRE: Mean: 3.14 Std Deviation: 0.904 Range: 1-5POST: Mean: 3.26 Std Deviation: 0.841 Range: 1-5

PRODUCTIVITYHypothesis: Implementation of the electronic health record will result in increased visits per clinician for addiction counseling, primary medical care, and HIV-related case managementEligible staff: • Human Services Counselors• Medical Staff• Case Managers

Pre Evaluation Period: 7/1/06 to 6/30/07Post Evaluation Period: 11/1/09 to 10/31/10

PRODUCTIVITY Human Services Staff Addiction related counseling services (Pre):

64,345 Addiction related counseling services (Post):

52,652 Medical Services Staff Primary medical care services (Pre): 5,221 Primary medical care services (Post): 4,028 Case Managers HIV counseling services (Pre): 2,680 HIV counseling services (Post): 3,058

HUMAN SERVICES

CLINIC PRE POST CHANGE T-VALUE P-VALUESIGN

TEST P VALUE

11 10791 8652 -2139 -7.41 0.0003 0.01613 9984 8440 -1544 14 12298 11012 -128621 8682 5926 -275622 6707 5668 -103923 8401 6722 -167924 7482 6232 -1250

PRODUCTIVITY: RESULTS

MEDICAL SERVICES

CLINIC PRE POST CHANGE T-VALUE P-VALUESIGN

TEST P VALUE

11 833 748 -85 -234 0.057 0.1113 921 443 -478 14 507 369 -13821 809 547 -26222 820 548 -27223 599 737 13824 732 636 -96

PRODUCTIVITY: RESULTS

CASE MANAGER SERVICES

CLINIC PRE POST CHANGE T-VALUE P-VALUESIGN

TEST P VALUE

11 429 447 18 0.40 0.72 1

13 188 981 793

14 852 533 -319

21 844 690 -154

22 367 407 40

PRODUCTIVITY: RESULTS

FINANCIAL PERFORMANCE

Measures PRE (2007) POST (2010)Revenue per capita staff: $66,900 $67,280

Cost per patient visit: $28.09 $29.68

Hypotheses: Implementation of the electronic health record will result in: Increased revenue per capita staff Decreased cost per patient visit

STUDY FINDINGSQuality: Highly statistically significant

improvement in timely completion of Annual Medical and Annual, 30-Day & 90-Day Multidiscipline assessments

No statistically significant change in obtaining HCV Viral Load for patients positive for HCV Antibody

STUDY FINDINGSRisks: The prevalence of risk management

events was too low to detect a statistically meaningful change

Satisfaction: Patient Satisfaction unchanged pre & post

implementation of the electronic system Staff Satisfaction trended upward post

implementation of the electronic system

STUDY FINDINGSProductivity: Productivity significantly declined for

Human Services staff There was a non-significant productivity

decline for Medical Services staff There was a non-significant productivity

increase for Case Manager staff

STUDY FINDINGSProductivity Confounders: Staff capability to utilize the electronic

system varied considerably Increased turnover of staff due to

inability to adapt to new system Electronic system upgrades required

frequent retraining of staff Preparation for APGs and other billing

issues required major training

STUDY FINDINGSFinancial Performance:

Revenue per capita staff increased by 0.6%

Cost per patient visit increased by

5.7%

IN CONCLUSION …Despite results that were somewhat less robust than expected in some of the domains examined; had we

not implemented the electronic information system, the recent changes in documentation and

reimbursement for services would have paralyzed our agency

ACKNOWLEDGEMENTS PATIENTS AND

STAFF OF THE ADDICTION RESEARCH AND TREATMENT CORPORATION, A COMMUNITY-BASED SUBSTANCE ABUSE SERVICE AGENCY

STUDY TEAM Principal Investigator: Lawrence S. Brown, Jr., MD, MPH, FASAM Executive Senior Vice President Sub-investigators: Carlota John-Hull, MD, Director of Medical Services Melissa Lin, MS, Director of Evaluation and Research Steven Kritz, MD, Research Project Manager Roberto Zavala, MD, Research Assistant Ben Louie, BA, Implementation Project Manager Consultants: Crystal Fuller, PhD, Mailman School of Public Health, Columbia University John Kimberly, PhD, Wharton School of Business, University of Pennsylvania