internet of medical thingslu/talks/iomt.pdf · Ø knn performs better than neural network (nn) q...

Internet of Medical Things�Predicting Clinical Outcomes and Digital Phenotyping with Wearables and Machine Learning

Chenyang LuCyber-Physical Systems Laboratory

Department of Computer Science & Engineering

https://www.cse.wustl.edu/~lu/

Internet of Medical Things

Ø Wearables: wristbands, smartwatches... q  Continuous monitoring: activity, heart rate, sleep, location…

Ø Connectivity: Bluetooth, WiFi, cellular…q  Real-time monitoring and just-in-time intervention

Ø Cloud: computing and storage.q  Scalable to large cohorts

Ø Analytics: machine learning and signal processingq  Interpret data and predict outcomes

A powerful tool for clinical studiesinside and outside hospitals!

12/19/2019 Chenyang Lu 2

Smartwatch as a Healthcare Tool

12/19/2019

Two-way communication

ecological momentary assessments

Open, programmable platform

Wear OS, Research Kit, onboard analytics

Continuous, passive

measurements activity, heart

rate, sleep, location…

Chenyang Lu 3

Machine Learning for Healthcare

Ø Machine learning: data-driven tools for healthcare.q  Predicting clinical outcomes

q  Discover risk factors associated with outcome

q  Support clinical decisions to improve outcome

Ø Discover the insights behind diverse dataq  Mobile Health (mHealth)

•  Continuous monitoring outside hospitals•  Small data: mHealth studies usually have moderate population

q  Electronic Health Record (EHR)•  In-patient data•  Big data: large patient population


IoMT Use Cases

Ø  Predicting readmissionsq  Feasibility of monitoring with Fitbit – data yield and compliance

q  Predictive models – machine learning with “small data”

Ø Measuring mobilityq  Timed Up and Go with smartwatches

Ø Measuring stressq  Ecological momentary assessment (EMA)

q  Machine learning model based on physiological signals


Predict Readmissions with Fitbit Ø  Hospital readmission rate is high for heart failure patients.

q  ~25% patients readmitted within 30 days

Ø  Predict deterioration (readmission+death) after dischargeq  Fitbit provides continuous monitoring of outpatientsq  Just-in-time intervention à better outcome and lower cost

12/19/2019

Joint work with Thomas Bailey (Infectious Diseases), Marin Kollef (Critical Care), Dingwen Li (CSE) �D. Li, J. Vaidya, M. Wang, B. Bush, C. Lu, M. Kollef and T. Bailey, Feasibility Study of Monitoring Deterioration of Outpatients Using Multi-modal Data Collected by Wearables, ACM Transactions on Computing for Healthcare, accepted.

Chenyang Lu 6

Objectives Ø  Assess the feasibility of collecting multi-modal data from wearablesØ  Develop and test predictive models based on machine learning

Ø  Provide guidelines for future studies


Data

Model

Day i-2 Day i-1 Day i

Input Predict

Data Label

Deterioration Early Warning

Data Data

Day 1 Day N

Time of discharge from hospital

InputModel

Predict

N=20

Will deteriorate?

Deterioration Risk Prediction

Challenges

Ø Outpatient complianceq  Wear devices continuously and properly outside hospitals

q  Potential problems•  Forget to charge the device•  Reluctant to wear the device•  Improperly wear the device

Ø  Learning from small dataq  Overfitting can lead to models that are not generalizable

q  Imbalanced dataset can affect predictive performance


Study Protocol

Ø  25 heart failure patients were recruited.q  Each patient is given Fitbit ChargeHR wristband.

q  Continuously monitor patients after discharge.

q  Outcomes: 60-day deterioration events.

Ø Heart rate (HR), step count and sleep stages were collected.q  Sampling period: 1 min (step, heart rate); 1 day (sleep)


Yield Ø  Yield: fraction of samples successfully collected and stored in databaseØ  Participants with data yield > 80%

q  Step: 88% of participantsq  Heart rate: 60% of participantsq  Sleep: 68% of participants


Compliance Ø  Participants with data yield > 80%

q  Step: 88% of participantsq  Heart rate: 60% of participants à wore Fitbit properlyq  Sleep: 68% of participants à wore Fitbit at night

12/19/2019

Feasibility of long-term, continuous monitoring

Chenyang Lu 11

Latency Ø  Median: 8.6 min; 99% percentile: 22.5 hour

Ø  Did not cause data loss, as Fitbit can locally store data for 7 days.

Ø  Feasibility for daily interventionØ  Can be reduced if analysis performed natively in Fitbit cloud

12/19/2019

Latency

Chenyang Lu 12

Rich Features of Wearable Data Ø  Heart rate (HR), step count and sleep quality were collected.

q  Sampling period: 1 min (step, heart rate); 1 day (sleep)

Ø  Statistical features:q  First- and second- order features extracted from sliding windowq  1st order: mean, max, min, skewness, kurtosisq  2nd order: energy, entropy, correlation, inertia and local homogeneity

Ø  Detrended Fluctuation Analysisq  Determine statistical self-affinity of time seriesq  The fluctuation is then used as feature

Ø  Sedentary behavior


Important Features for Deterioration Early Warning

Predict Deterioration Risk Ø  Input: Fitbit data collected in the first N days since dischargeØ  Deterioration: composite outcome of either readmission or deathØ  Small data: 25 patients (18 with no deterioration vs. 7 deteriorated)

12/19/2019

Xdaysinthefuture

Data Data

Day1 DayN

Timeofdischargefromhospital

InputModel

Predict

N=20

Willdeteriorate?

Chenyang Lu 14

Assess Predictability Ø  Analysis of Variance (ANOVA) à predictability of outcomes

q  Test significant difference in the features between patients of different outcomes.•  deteriorated patients vs. non-deteriorated patients

q  Assess feasibility of learning a stable predictor with the set of features.

Ø  Significant differences in features à predictability

12/19/2019

Features F pHR skewness 7.9125 0.0099

HR correlation 5.4789 0.0283

HR DFA 10 5.3353 0.0302

Restless duration 5.2912 0.0308

Time in bed 5.2663 0.0312

Features with the largest F values. High F-statistic and low p-value à significant difference between group means.

Chenyang Lu 15

Feature Selection Ø  Select features using sequential forward feature selection

q  Avoid overfittingq  Improve performance

Ø  Features selected by the models have significant differences in ANOVA test

12/19/2019

Features F psleep DFA 60 0.2254 0.6395

min asleep 4.3128 0.0492daily step 4.3625 0.0480restless count 4.2324 0.0512awake count 4.0429 0.0562min awake 2.2073 0.1509

HR LH 4.0282 0.0566HR DFA 10 5.3353 0.0302

Chenyang Lu 16

Test Overfitting Ø  Impact of the number of nearest neighbors (K) on KNN performance

12/19/2019

Smaller difference between training and testing errors à less overfitting

Chenyang Lu 17

Evaluation with Small Data Ø  Leave-one-out cross validation

q  Leave a sample (or a group of samples) out for testing and train the model with rest of sample (or groups of samples).

q  Iterate through all the samples.

Ø  For imbalanced dataset, do not just look at accuracy!q  Example: for 1:9 positive/negative ratio, the predictor can achieve 0.9 accuracy

if predicting everything as negative.

q  A good predictor should perform well for sensitivity, specificity and precision.


Deterioration Risk Prediction Ø  Small data: 25 patients (18 with no deterioration vs. 7 with deterioration)Ø  KNN achieves higher specificity and precision than LACE index used in

clinical practice.Ø  KNN performs better than neural network (NN)

q  Simple models help avoid overfitting on small data set.


Model Sensitivity Specificity Precision Accuracy

NN 0.0770 0.9459 0.3333 0.720

KNN 0.5385 0.9820 0.9130 0.8667LACE 0.7647 0.6250 0.5556 0.7826

Machine Learning with Small Data Ø  Apply statistical analysis to assess predictability with the features.

Ø  Mitigate overfitting in modelsq  Adopt simple models q  Reduce the number of variables through feature selectionq  Use an ensemble of models (Gradient Boosted Trees, AdaBoost)

Ø  Handle imbalanced data setsq  Random oversampling of the minority class(es)q  Random undersampling of the majority class(es)q  Synthetic minority over-sampling technique (SMOTE)

Ø  Evaluationq  Don’t just look at accuracy, especially for imbalanced datasetq  Assess overfitting by comparing accuracy on training/testing data


Predict Surgical Complications •  Patients undergoing pancreatectomy are monitored with Fitbit before

surgery, during hospital stay, and 30 days post-discharge.

•  Machine learning models predict complications and readmissions. •  54 patients enrolled (goal: 130).

PreoperativeClinic Surgery OutpatientRecovery

Joint work with Chet Hammill (Surgery), Dingwen Li, Ruixuan Dai (CSE)


IoMT Use Cases







Timed Up and Go with Smartwatch

12/19/2019

Ø  Watch appq  Remind participants to take the assessmentq  Automatically upload the data to the cloud for analysisq  Analyze gait and motion featuresq  Feedback to physicians and participants

Joint work with Matthew Spraker (Radiation Oncology), Ruixuan Dai (CSE)

Chenyang Lu 23

https://www.cse.wustl.edu/~lu/TUG.mp4

Ø  Assess physical health and fall risk during prehabilitation.q  20 participants undergoing neoadjuvant radiotherapy followed by surgeryq  Patients will complete TUG at home with the smartwatch for 90 days.

IoMT Use Cases







Measure Stress with EMA and PPG Sensor

12/19/2019

Joint work with Thomas Kannampallil (Anesthesiology, Informatics), Michael Avidan (Anesthesiology), Eric Lenze (Psychiatry), Ruixuan Dai (CSE)

Chenyang Lu 25

EMA in Daily Life Ø  EMAs are auto fired 12 times/day with snooze option

q  Buzz again when no response in 5 minutes (max: 3 buzzes)q  32 healthy participants

Ø  95.7% response rate (86.8% responded to the first buzz)Ø  85th percentile response time is less than 30s


Do Stressors Work?

Ø Use EMA to validate stressors

12/19/2019

Efficacy of stressors vary à Test with watch-based EMA!

Chenyang Lu 27

Predict Stress with PPG Signals

12/19/2019

ML model based on PPG signals is more consistent with EMA

Chenyang Lu 28

Stress

Sensitivity is fixed at around 0.9

Model Precision Sensitivity Accuracy

SVM(rbf) 0.28 0.86 0.73

Random Forest 0.25 0.86 0.69

Gradient Boosting 0.23 0.87 0.65

AdaBoost 0.21 0.86 0.61

Logistic Regression 0.31 0.90 0.74

SVM (Linear) 0.32 0.88 0.75

Label based on Stressors Label based on EMAsModel Precision Sensitivity Accuracy

SVM(rbf) 0.32 0.86 0.83

Random Forest 0.39 0.87 0.87

Gradient Boosting 0.28 0.87 0.81

AdaBoost 0.24 0.87 0.77

Logistic Regression 0.53 0.91 0.92

SVM (Linear) 0.55 0.87 0.92

IoMT Use Cases

Ø  Predicting readmissionsq  Feasibility of monitoring with wearables – data yield and compliance


Ø Measuring mobilityq  Timed Up and Go (TUG) with smartwatches

Ø Measuring stressq  Model based on physiological signals

q  Ecological momentary assessment (EMA)



A powerful tool for clinical studiesinside and outside hospitals!







We can do a lot more when combiningwearables and electronic health records






internet of medical thingslu/talks/iomt.pdf · Ø knn performs better than neural network (nn) q...

Documents