evaluate outcomes of an obesity management program (craig nelson)

8/8/2019 Evaluate Outcomes of an Obesity Management Program (Craig Nelson)

http://slidepdf.com/reader/full/evaluate-outcomes-of-an-obesity-management-program-craig-nelson 1/37

A Return-On-Investment Estimation Model toEvaluate Outcomes of an Obesity Management Program

Craig F. Nelson, MS, DC

Director Health Services Research • American Specialty Health



This Presentation

• Define the issue

• Identify challenges

• Offer one option for a solution



“What’s the ROI?”

Provide an answer to this question that is …

• Credible

• Generalizable

• Timely

• Practical



Logic of Wellness Program ROI

Wellness

Program

Wellness

Program



Obesity Costs

• The estimated obesity-related cost burden to employers is

substantial.

• Medical expenditures are estimated to be one-fourth to one-thirdhigher for overweight and obese employees compared with

normal-weight counterparts.• Obesity has been attributed to increased absenteeism, decreased

presenteeism, and increased utilization of short-term disability.



Wellness Program Impact Framework

Timeframe for ImpactWeeks/Months Months/Years Years/Decades

III. Productivity/Quality of LifeIII. Productivity/Quality of LifePresenteeism

Absenteeism

Quality of Life

II. Behavior Change/Modifiable Risk FactorsII. Behavior Change/Modifiable Risk FactorsI. ProcessMeasures- Engagement Methods- Participation Rates- Number And Types of -

Contacts- # of Interventions(Web, Communication,

Coach, Onsite)- Number of Referrals

- Etc…

I. ProcessMeasures- Engagement Methods

- Participation Rates- Number And Types of -

Contacts- # of Interventions(Web, Communication,

Coach, Onsite)- Number of Referrals

- Etc…

Self-Efficacy/ ConfidenceReadiness to Change Risky

Behaviors

Social Isolation

Stress/Anxiety

Motivation

Depression

Perception of Health

Perceived susceptibility to and

severity of illness

Self-ManagementProper Nutrition and Exercise

Tobacco Reduction/ Cessation

Medication Adherence

Sleep

Safety

Alcohol/Drug Use

Screening and Preventive Svcs.

Cancer Screening

Lipid And Glucose Screening

BP Measurement

BMI AssessmentImmunizations

Eye Exam

Dental Care

Physical Exam

Health Status

Body Mass Index (BMI )

Cholesterol (Total, HDL, LDL,

Triglycerides, Total/HDL Ratio)

Blood Glucose

Blood Pressure

Health and Clinical

Outcomes

Psychosocial

Drivers

Health

Behaviors/Risk

(*As a result of increased screening and preventive services)

IV. Appropriate Utilization and Medical CostsIV. Appropriate Utilization and Medical Costs

Short-Term*

Doctor Visits

Laboratory

Preventive Care

Long Term

Hospitalizations

Total Medical and RX Cost

STD/Workers Comp Offsets

Long Term Disability

Medium-Term

ER Visits

Outpatient

Pharmacy



The ROI Concept

• How much you (expect to) save from a weight loss program,

compared to how much you (expect to) spend or spent on it• Expressed as ratio (e.g., 2:1) or dollars--Net Present Value (NPV)

• ROI and savings are not the same:

– Savings reflect differences in $ with versus withoutintervention

– ROI ratio = savings / program costs

– ROI can be low, but savings can be high – Conversely, ROI can be very large but savings very small

– A appropriate balance must be struck between savings and

ROI



The Bottom Line

• The expectation, supported by the science (and by vendor

claims), is that workplace wellness programs will reduce morbidityand associated health care costs.

• Further, the expectation is that the cost savings will more than

offset program costs and thus generate a positive ROI.



The Challenge:

• Deliver on the promise

• Reduce risks

• Improve health

• Save money• Measure the above



The Science of Wellness Care vs.Wellness Program Evaluation



The Science

A wellness/weight management program is a treatment, an

intervention, like any other and its effects can only be fullyunderstood with an RCT.

Wellness/weight management program as the independent variable

and health status and health care costs as dependent variables.

• Such a study would cost millions (tens?) of dollars

• Would require a sample size of thousands (tens?)

• Would have a duration of >5 years of the intervention



Program Evaluation

“Default method”

• Pre-post

• Within group cohort study

• Behavioral and biometric variables

• Self-report



Challenges to Measuring ROI in Wellness Programs

• Availability of claims data

• Variance and distribution of claims data

• Scope of program and effect sizes

• Temporal profile of effects

― Duration of program

― Impatience for an answer

• Cost of evaluation versus cost of program

• Sample size



Sample Size



Claims-Based ROI for Wellness Programs

• Given claims data variance and distribution …

• And given the relatively low unit cost of wellness programs …

• And given the uncertainties of trend analysis …

• ROI results may be more a function of trending methodology andchance, than of program success or failure.



Healthyroads Program

• Telephonic coaching (maximum 48 sessions)

• Individualized counseling on …

―Diet

―Exercise―Stress management

• Written materials to support coaches

• Web-based trackers



HERO Risk Factors

• Exercise

• Diet

• Tobacco use

• Stress

• Depression

• BP

• Serum cholesterol

• Serum glucose

• Obesity

• Alcohol/drug use



HERO Risk Factors

• Exercise

• Diet

• Tobacco use

• Stress

• Depression

• BP

• Serum cholesterol

• Serum glucose

• Obesity

• Alcohol/drug use

The Healthyroads HRA is designed to measure HERO risk factors in amanner comparable to original data collection process and operationaldefinitions.



Study Design

Study participants

• Inclusion criteria: age ≥ 18, BMI ≥ 25, and presence of comorbidityassociated with obesity, or recommendation to program.

• 1,542 eligible participants, 890 employees provided baseline, andfollow-up data.

– Participants represented a convenience sample with a 42.3%attrition rate from baseline to follow-up.

• Baseline HRA data were obtained before or just after individuals’ initial

consultation with personal health coach.

• Follow-up HRA data were obtained one-year after individuals’ initialconsultation with personal health coach.



Table 1. Demographic characteristics of study population at baseline

Demographic Characteristic Average or Percentage

N 890

Age (avg.) 44.2

Female (%) 74.3

Ethnicity (%)

American Indian or Alaskan Native 0.7

Asian 3.2

Hispanic 9.2

Black 6.6

White 75.3

Pacific Islander 0.9

Multiracial or other race 0.8

Unknown 3.4

Overweight or obese (%) 76.4

Weight (avg.) 191.4

Body Mass Index (avg.) 30.6

Study Population at Baseline



Estimation of ROI

Baseline medical costs

• Using MarketScan Commercial Claims and Encounters Database for 2006, thebaseline costs were determined to be $4,804/participant, adjusted to U.S.Census Bureau 2007 Consumer Price Index values.

Productivity

• Estimates of the costs of presenteeism (lost productivity at work) were derivedfrom medical literature (Burton et al.6)

• Productivity was monetized by multiplying total hours of productivity gained in

the year by the participant’s average hourly wage ($25.93), derived from June2007 Bureau of Labor Statistics.

Costs of Healthyroads

• $300 per program participant per year incurred by employer



The Healthyroads ROI Model

Two forms of input for the Healthyroads ROI estimation model:

1. A demographic profile of target population

- The user supplies demographic data for employees or beneficiaries and the projected annual increases or decreases for each characteristic

2. A health profile of the target population

- The user provides the risk profile of the targeted population(based on HRA data) and the actual or expected annualchange in each risk factor



Model Inputs

Figure 1. ROI model inputs screen—demographics and financial measures



Model Inputs

Figure 2. ROI model input screen—time 1 and time 2 changes in weight andhealth risks for program participants



Results – Health Risk Outcomes

Over one year, participants experienced statistically significant

reductions in 7 out of 10 health risk categories (Table 2).

• Substantial reductions in poor eating habits (21.3% reduction)and inadequate physical activity (15.1% reduction) occurred.

• Overweight and obesity as measured by BMI experiencedreductions at 5.8% weight reduction, 4.5 pounds reduction, and0.9% BMI reduction.

Alcohol consumption significantly increased (from 13% to 16%),while smoking status and depression remained unchanged.



Results – Health Risk Outcomes (Cont.)

MeanLower

CIUpper

CI MeanLower

CIUpper

CI

Poor Eating 0.66 0.63 0.69 0.45 0.41 0.48 -21.30% <.0001

Poor Exercise 0.64 0.61 0.67 0.49 0.46 0.52 -15.10% <.0001

Former Smoker 0.25 0.22 0.28 0.22 0.19 0.25 -3.30% 0.0032

Current Smoker 0.07 0.05 0.09 0.06 0.05 0.08 -0.70% 0.3763

High Cholesterol 0.22 0.2 0.25 0.06 0.04 0.07 -16.40% <.0001High Glucose 0.06 0.04 0.08 0.03 0.02 0.04 -2.90% 0.0005

High Blood Pressure 0.11 0.09 0.13 0.02 0.01 0.03 -8.50% <.0001

High Stress 0.18 0.15 0.2 0.12 0.1 0.14 -6.00% <.0001

Depressed 0.06 0.04 0.08 0.05 0.03 0.06 -1.20% 0.1658

High Alcohol 0.13 0.11 0.15 0.16 0.14 0.18 2.90% 0.0132

Overweight or Obese 0.76 0.74 0.79 0.71 0.68 0.74 -5.80% <.0001Absolute

∆

P value (t-

test)

BMI 30.6 30.1 31 29.7 29.2 30.2 -0.9 <.0001Weight 191.4 188.1 194.7 186.9 183.7 190.1 -4.5 <.0001

Risk Factor

Time 2Time 1

% pt ∆

P value

(McNamar’s

Chi-Square)

Table 2. Changes in health risks factors for the cohort group



Results – Financial Outcomes

Total expenditures

• Experienced reduction of $311,755 in one year Medical savings

• 59% of reduction in total expenditures is attributable to a 4.3%

reduction in medical expendituresProductivity savings

• 41% of reduction in total expenditures is attributable toproductivity gains

Program cost

• $300 per program participant per year



Results – Financial Outcomes cont.

Net present value

• Calculated as the present (discounted) value of the projectedsavings less the program costs.

• A value of $44,755 was realized.

Break-even point

• Determined to be 3.20, meaning that all risks need to be reducedby an average of 3.20% points for the program to pay for itself.

Projected ROI

• $1.17 to $1.00, indicating employers saved a $1.17 for everydollar invested in the program.



Results – Financial Outcomes (Cont.)

Reference

Scenario*

Program

Scenario

Break-Even

Scenario**

Total Expenditure (2009-2009) $4,275,560 $4,090,978 $4,029,345

Change Between Baseline and Year 1Follow-up (%) 0.00 -0.04 -0.06

Medical Savings $184,582 $246,215

Productivity Savings $127,173 $20,782

Total Savings $311,755 $266,998

Program Cost $267,000 $267,000

Net Present Value $44,755 ($2)

Return on Investment $1.17 $1.00

Table 3. Year 1 Results

*No program

**Risks are reduced by 3.20% per year.



Strengths

• It is predicated on publically available published scientific data and

on known relationships between risk factors and costs• It allows programs to be compared to one another

• It is not sensitive to assumptions about medical cost trends

• It is inexpensive to perform

• It’s methodology is transparent



Limitations

• Absence of control group

―Regression to the mean

―Selection bias

• Reporting bias

• Self-report limitations

• Limitations of the model

“All models are wrong. Some are useful.” George Box

Statistician



Limitations

• This evaluation and publication is not intended to be a

definitive analysis of the program’s effectiveness.• Rather, it is intended to be a discussion the issue of ROI

measurement in wellness programs.



“Will You Guarantee This Result?”



“Given a wellness program’s cost, scope and duration,

number of participants, and given the availability (or non-availability) of claims data and of an appropriate

control group, what program evaluation design is

indicated such that the additional cost associated withthe additional rigor is warranted and can be justified by

the higher quality of data that results?”



Program Evaluation Rigor:



What Now?

• Precision and certainty in wellness program ROI estimations

are likely to remain elusive• Continue to advance the science of wellness care

• Promote transparency

• Do not put all your eggs in the ROI basket

― Process measures

―Member satisfaction― Behavior change

evaluate outcomes of an obesity management program (craig nelson)

Documents