towards personalizing monitoring with wearable...

Towards Personalizing Monitoring with Wearable Sensors

Bobak MortazaviSystems & Technology for Medicine & IoT

bobakm@tamu.eduhttp://stmi.engr.tamu.edu/

1

Outline

• Goal of this talk• Description of End-to-End Systems• Stage-wise learning of recovery

– Risk Models– Remote Sensing– Personal Modeling

2

Goals: Questions to Answer!

1. What improvements to the research infrastructure are needed?

2. What types of training are most important for this type of research?

3. What are the future research needs (methods, analyses and interventions, etc.)?

3

End-to-end systems

• Start with fit patient trending to unfit• Unfit patient trending to heart attack• Heart attack causing damage to the heart muscle• Heart attack needing operation• Post-operation recovery

4

End-to-end systems

• Start with fit patient trending to unfit• Unfit patient trending to heart attack• Heart attack causing damage to the heart muscle• Heart attack needing operation• Post-operation recovery• What are we calling an end-to-end system?

5

End-to-End Clinical

6

- Goals:- Model adaptability and robustness for remote health sensing- Task-oriented feature extraction, both personalization and generalization

- Proposed methods:- Context-aware deep modeling - Bayesian uncertainty quantification Gaussian Process model

- Implementations: - The alpha-beta network: a deep mixture of experts- Maximum entropy learning (ICML workshop'19) - Adaptive monitoring (published in AISTATS'19)

Task Oriented Features

Hand Designed Features

End-to-End Remote

End-to-end systems

• Start with fit patient trending to unfit• Unfit patient trending to heart attack• Heart attack causing damage to the heart muscle• Heart attack needing operation• Post-operation recovery

8

ML-based Clinical Outcomes Models

9

• Update CathPCI Major Bleeding Model

• Use ML Techniques and additional data to improve identification of bleeds and non-bleeds

Study Design

• Evaluate if Machine Learning – alone – can naively improve model performance

• Evaluate if increased fidelity in data – alone – can naively improve model performance

• Evaluate a balance of understanding the data and modeling techniques will improve models

Study Aims

Outcomes Model

Data Source

10

Existing Clinical Model

• “Bedside” Model• Variables Sub-selected from Full Post-

PCI model• Want to re-introduce all available

variables

11

Power of Additional Data

Variable Description and CathPCI field

Age Patient age 2050

Age >70 Is patient age >70? (yes/no) 2050

Age ≤70 Is patient age ≤70? (yes/no) 2050

BMI Body mass index 4055, 4060

BMI ≤30 Is BMI ≤30? (yes/no) 4055, 4060

Pre-procedure

hemoglobin

Pre-procedure hemoglobin

(continuous)7320

Pre-procedure

hemoglobin ≤13 g/dL

Pre-procedure hemoglobin ≤13 g/dL

(yes/no) 7320

Pre-procedure

hemoglobin >13 g/dL

Pre-procedure hemoglobin >13 g/dL

(yes/no) 7320

12

Continuous Representation

13(Mortazavi et al 2019, JNO)

Findings and Opportunities

Key Takeaways1. Intelligent selection of variables

is needed.2. More fluid representation of risk

factors presents higher accuracies.

Limitations1. Still a single decision point!

Confounded by future decisions?2. Clinical journal – do

engineering/CS reviewers know how to interpret these?

3. Identified an additional 168 bleeding cases and 949 non-bleeding cases per 100,000 cases.

14

Findings and Opportunities

Key Takeaways1. Intelligent selection of variables

is needed.2. More fluid representation of risk

factors presents higher accuracies.

Limitations1. Still a single decision point!

Confounded by future decisions?2. Clinical journal – do

engineering/CS reviewers know how to interpret these?

3. Identified an additional 168 bleeding cases and 949 non-bleeding cases per 100,000 cases. So What?

15

Goals: Questions to Answer!

1. What improvements to the research infrastructure are needed?Data availability – How do we continue pushing the boundaries on this?

2. What types of training are most important for this type of research?Language/Interaction – How do we encourage joint conferences or overlap in journals?

3. What are the future research needs (methods, analyses and interventions, etc.)?

Allow for analyses that work on the margin/in the special cases

16

EHR-based “Dynamic” modeling

17

• Develop models from EHR data for adverse events post-PCI

• Use ML Techniques and timely data!

Study Design

• Evaluate if models can be as accurate with EHR data

• Evaluate challenges in replicating registry-level variables in EHR data

Study Aims

Data Timeline

Dataset: EHR Extraction

18

Results

19(Mortazavi et al 2017, IEEE JBHI)

Findings and Opportunities

Key Takeaways1. Can make decisions from EHR in

timely fashions2. Registries can guide us on data

cleaning in EHRs

Limitations1. Still a single decision point!!!

Confounded by future decisions?2. Engineering Journal – Do clinical

researchers read these?

20

Goals: Questions to Answer!

1. What improvements to the research infrastructure are needed?Data availability – How do we continue pushing the boundaries on this?

2. What types of training are most important for this type of research?Language/Interaction – How do we encourage joint conferences or overlap in journals?

3. What are the future research needs (methods, analyses and interventions, etc.)?

Allow for analyses that work on the margin/in the special casesStart accounting for real-time, prospective use models and analyses that enable data processing for prospective use

21

End-to-end systems

• Start with fit patient trending to unfit• Unfit patient trending to heart attack• Heart attack causing damage to the heart muscle• Heart attack needing operation• Post-operation recovery

22

Heart Failure and Readmissions

• High Readmission rates impose a tremendous burden on patients and healthcare systems. Modeling of recovery has been challenging.

• Understanding Ambulatory BP more predictive of outcomes than clinic-based BP.

• Diet and Exercise are important factors in personal care• Cardiac rehabilitation programs have shown promise but low

adherence– Limited program availability, cost of attending, cost of transportation

all impact performance – Home-based program too hard to follow, or approximate too much

and lose individual feedback and engagement

23

Tele-HF Readmissions

24(Mortazavi et al 2016, Circ: CQO)

Wearable Health

25

Some Prior Work: Developing Required Sensors

26

(Mortazavi et al 2012, IEEE BSN)(Mortazavi et al 2013, IEEE JBHI)(Lee et al 2013, IEEE JBHI)

Personal (Cardiac) Rehabilitation Needs

• Heart Rate, V02, Exercise• Resting Heart Rate, Respiration Rate, and Blood Pressure• Diet• Must understand variety of measurements, intensities, and

context the data is gathered in

27

Activity and Context Tracking

28

Understanding Context

• Provide participants with CGMs• Provide breakfasts with known

carbs/fats/proteins after 8 hours of fasting• Limit physical activity and measure impact

of meals

Study Design

• Understand and measure post-prandial glucose response under varying carbs/fats/proteins.

• Understand subject-to-subject variability to similar meals.

• Develop machine learning model to estimate amount of carbs/fats/proteins from CGM Signals

Study Aims

Smartwatches and Smartphones

Personalized Learning: Context and Uncertainty Quantification

29

(Solis et al 2019, IoTDI)

(Ardywibowo et al 2019, AIStats)(Solis et al., under review – IEEE JBHI)(Pakbin et al., under review - AIStats)

Diet Monitoring

30

Estimating Macronutrients

• Provide participants with CGMs• Provide breakfasts with known

carbs/fats/proteins after 8 hours of fasting• Limit physical activity and measure impact

of meals

Study Design

• Understand and measure post-prandial glucose response under varying carbs/fats/proteins.

• Understand subject-to-subject variability to similar meals.

• Develop machine learning model to estimate amount of carbs/fats/proteins from CGM Signals

Study Aims

Continuous Glucose Monitors

Varying Carbohydrates and Measuring Glucose

31

The glucose response (averaged over all participants) to three meals with known fats/proteins and varied quantity of carbs shows expected larger and longer glucose response

We encode meals as CxPxFxwhere x is

• 1 = low quantity

• 2 = med quantity

• 3 = high quantity 50

100

150

1 6 11 16 21 26 31 36

Bloo

d gl

ucos

e (m

g/dL

)

Time (15 min)

C3P2F2 C2P2F2 C1P2F2

Varying Proteins and Fats

32

50

100

150

1 6 11 16 21 26 31 36

Bloo

d glu

cose

(mg/

dL)

Time (15 min)

C2P3F2 C2P2F2 C2P1F2

50

100

150

1 6 11 16 21 26 31 36

Bloo

d gl

ucos

e (m

g/dL

)

Time (15 min)

C2P2F3 C2P2F2 C2P2F1

Similarly, we find that varying quantities of proteins dulls amplitude and extends duration of recovery to baseline.

Varying quantities of fats dulls amplitudes and extends duration of recovery to baseline.

Subject to Subject Variability!

33

0

50

100

150

200

250

1 6 11 16 21 26 31 36

Guco

se le

vel (

mg/

dL)

Time (15 min intervals)

Sample of individual responses to a meal that represents the average American diet (medium amount of carbs/fats/proteins) shows high subject-to-subject variability!

Estimation

34

Capturing fasting glucose levels, subject body weight, and important features regarding the glucose response shape and duration leads to accurate carbs/fats/proteins modeling:• Correlation coefficients of 0.76, 0.57, and 0.37 respectively.• Balanced accuracy of classifying low vs. medium vs. high amounts

carbs/fats/proteins of 95.1%, 65.1%, and 65.4% respectively

(Huo et al. 2019, IEEE BHI)(Mortazavi et al. Under Review, IEEE JBHI)

Cuffless Blood Pressure

35

Personal Models

• Design Bio-Impedance Sensor• Understand/capture range of BP

measurements• Develop personal models of PTT/PWV to

SBP/DBP• Validate against ABPM

Study Design

• Understand data needs for models to estimate SBP/DBP from PTT/PWV

• Evaluate strengths of Bio-Z approach over PPG

• Develop transfer learning techniques for rapid calibration

Study Aims

Cuffless BP Monitor

BP through Bio-impedance

36(Ibrahim and Jafari 2019, IEEE TBioCas)

IP

DIA

MS

SYS

Diastolic Phase

Systolic Phase

Max. Slope

Inflection Point

Heart Beat

*Bio-Z flipped

PTT Bio-Z2*

Bio-Z1*Points

DetectionFeatures

Extraction DBP Regression

Model

Systolic BP

Diastolic BP

SBP Regression

Model

Bio-Z1Bio-Z2

Bio-Z3Bio-Z4

Radial Artery Ulnar Artery

I*Bio-Z flipped

V

I

Arterial Pulse Wave

Findings and Opportunities

Key Takeaways1. Measuring the context in which

data is captured in remote/personal environments is extremely important.

2. It is important to model subject-to-subject variability in biometric measurements.

3. Have to balance big data ML problems with small data ML problems.

Limitations1. How do we create personal

models without burdening user with massive amounts of data collection?

2. Where is the linkage back to clinical data-driven outcomes models?

37

Goals: Questions to Answer!

• 1. What improvements to the research infrastructure are needed?Data availability – How do we continue pushing the boundaries on this?Data linkage between remote/personal models and EHR/RCT modelsData sharing infrastructure!

• 2. What types of training are most important for this type of research?Language/Interaction – How do we encourage joint conferences or overlap in journals?

• 3. What are the future research needs (methods, analyses and interventions, etc.)?

Allow for analyses that work on the margin/in the special casesStart accounting for real-time, prospective use models and analyses that enable data processing for prospective useN of 1 studies – personalization and longitudinal data collection needsData sharing infrastructure!

38

Treatment?

• Start with fit patient trending to unfit• Unfit patient trending to heart attack• Heart attack causing damage to the heart muscle• Heart attack needing operation• Post-operation recovery

Treatment?

• Start with fit patient trending to unfit• Unfit patient trending to heart attack• Heart attack causing damage to the heart muscle• Heart attack needing operation• Post-operation recovery

Treatment?

• Start with fit patient trending to unfit• Unfit patient trending to heart attack• Heart attack causing damage to the heart muscle• Heart attack needing operation• Post-operation recovery

Acknowledgements

42

Sponsors:

Dr. Harlan Krumholz Dr. Roozbeh Jafari Dr. Erica Spatz

Dr. Sahand Negahban Dr. Emily Bucholz Dr. Chenxi Huang Dr. Nihar Desai Dr. Sanket Dhruva Dr. Wade Schulz

My Students and So Many more!

Goals: Questions to Answer!

• 1. What improvements to the research infrastructure are needed?Data availability – How do we continue pushing the boundaries on this?Data linkage between remote/personal models and EHR/RCT modelsData sharing infrastructure!

• 2. What types of training are most important for this type of research?Language/Interaction – How do we encourage joint conferences or overlap in journals?How do we encourage interdisciplinary needs to catch up to “state of the art” in both fields? Coursework? Postdocs?

• 3. What are the future research needs (methods, analyses and interventions, etc.)?

Allow for analyses that work on the margin/in the special casesStart accounting for real-time, prospective use models and analyses that enable data processing for prospective useN of 1 studies – personalization and longitudinal data collection needsData sharing infrastructure!Emphasis on funding opportunities to foster/enable big team collaborations.

43

Questions?

• bobakm@tamu.edu• Thank You!