internet of medical thingslu/talks/iomt_icts_2019.pdfØ 25 heart failure patients were recruited. q...

Internet of Medical Things�Predicting Clinical Outcomes and Digital Phenotyping with Wearables and Machine Learning

Chenyang LuCyber-Physical Systems Laboratory

Department of Computer Science & Engineering

https://www.cse.wustl.edu/~lu/

InternetofMedicalThings

Ø Wearables: wristbands, smart watches…q  Continuous monitoring: activity, heart rate, sleep, spO2…

Ø Connectivity: Bluetooth, WiFi, cellular…q  Real-time monitoring and intervention

Ø Cloud: computing and storage.q  Scalable to large cohorts

Ø Analytics: machine learning and signal processingq  Interpret data and predict outcomes

A powerful tool for clinical studiesinside and outside hospitals!

10/23/2019 ChenyangLu 2

SmartWatches

10/23/2019 ChenyangLu

Two-wayCommunicationPushecologicalmomentaryassessments

Open,programmableplatform

AndroidWear,AppleResearchKit

TailoredonboardanalyticsShorterLatency

SensorsAccelerometer,gyroscope,

magnetometer,HeartRate,

GPS…

3

IoMTUseCases

Ø  Predicting readmissionsq  Feasibility of monitoring with wearables – data yield and compliance

q  Predictive models – machine learning with “small data”

Ø Measuring mobilityq  Timed Up and Go (TUG) with smart watches

q  Fall detection – how smart watches can help

Ø Measuring stressq  Model based on physiological signals through smart watches

q  Ecological momentary assessment (EMA)


IoMTUseCases

Ø Predicting readmissionsq  Feasibility of monitoring with wearables – data yield and compliance







PredictingReadmissionsØ Hospital readmission rate is high for heart failure patients.

q  ~25% patients readmitted within 30 days

Ø  Early detection of deterioration (readmission + death)q  Just-in-time intervention à better outcomeq  Reduce health care cost

Ø  LACE index: state of practice for predicting readmissionq  Assess the risk of readmission based on inpatient dataq  Length (L) of stay; Acuity (A) of the admission; Co-morbidities (C);

number of Emergency (E) department visits within the last 6 months

Ø Wearable devices provide convenient technologies for continuous outpatient monitoring.


Objective

Ø Assess the feasibility of collecting multi-modal data from wearable device.

Ø  Explore the potential of predicting clinical deteriorationq  Develop and test predictive models based on machine learning

Ø  Provide guidelines for future studies


Joint work with Tom Bailey, Marin Kollef, Dingwen Li, Jay Vaidya, Michael Wang, Ben Bush�D. Li, J. Vaidya, M. Wang, B. Bush, C. Lu, M. Kollef and T. Bailey, Feasibility Study of Monitoring Deterioration of Outpatients Using Multi-modal Data Collected by Wearables, ACM Transactions on Computing for Healthcare, accepted.

Challenges

Ø Outpatient complianceq  Wear devices continuously and properly outside hospitals

q  Potential problems•  Forget to charge the device•  Reluctant to wear the device

•  Improperly wear the device

Ø  Learning from small sample sizeq  May cause overfitting

q  Imbalanced dataset


SystemOverview


StudyProtocol

Ø  25 heart failure patients were recruited.q  Each patient is given Fitbit ChargeHR wristband.

q  Continuously monitor patients after discharge.

q  Outcomes: 60-day deterioration events.

Ø Heart rate (HR), step count and sleep status were collected.q  Sampling period: 1 min (step, heart rate); 1 day (sleep yield)


YieldØ  Yield: fraction of samples successfully collected and stored in databaseØ  Participants with data yield > 80%

q  Step: 88% of participantsq  Heart rate: 60% of participantsq  Sleep: 68% of participants


Compliance

Ø  Participants with data yield > 80%q  Step: 88% of participants

q  Heart rate: 60% of participants à wore Fitbit properly

q  Sleep: 68% of participants à wore Fitbit at night


Feasibility of long-term, continuous monitoring

Latency

Ø Median: 8.6 min; 99% percentile: 22.5 hrØ  Did not cause data loss, as Fitbit can locally store data for 7 days.

Ø  Feasibility for daily interventionØ  Can be reduced if analysis performed natively in Fitbit cloud


Latency

PredictDeteriorationRisk

Ø Task: identifies participants at high risks of deterioration using data collected from the beginning of monitoring

Ø Dataset: 25 patients as samples, including 18 with no deterioration vs. 7 with deterioration

Ø Model: ML models for clinical readmission predictionq  Random forest, SVM, logistic regression, multi-level perceptron,

K-nearest neighbor


Data Data

Day1 DayN

Timeofdischargefromhospital

InputModel

Predict

N=20Xdaysinthefuture

Willdeteriorate?

DataPreprocessing

Ø  Raw data formatq  Raw heart rate and step count per minute

q  Derived data from Fitbit API, such as time in bed, sleep efficiency in sleep summary data

Ø Data cleansingq  Extract step-related data only in 7am-7pm

q  Fill in missing data by carry forward imputation

q  Discard a whole day’s data if missing data last longer than the day


FeatureEngineering

Ø  Statistical features:q  First- and second- order features extracted from sliding window

q  1st order: mean, max, min, skewness, kurtosis

q  2nd order: energy, entropy, correlation, inertia and local homogeneity

Ø Detrended Fluctuation Analysisq  Determine statistical self-affinity of time series

q  The fluctuation is then used as feature

Ø  Semantic featuresq  Sedentary behavior


AssessPredictability

Ø Analysis of Variance (ANOVA)q  Are there statistically significant differences between patients with

different outcomes? q  High F-statistic and low p-value à significant difference among the

group means.

Ø  Indicate the feasibility of learning a stable predictor.

Ø Assumptionsq  Independent observations

q  The variable follow a normal distribution in each group

q  The variances within all groups must be equal (needed only for very imbalanced groups)


ANOVAResults

Ø Groups: deteriorated patients vs. non-deteriorated patientsØ ANOVA test reveals significant differences in some features

à suggesting feasibility of predictive models


Features F pHR skewness 7.9125 0.0099

HR correlation 5.4789 0.0283

HR DFA 10 5.3353 0.0302

Restless duration 5.2912 0.0308

Time in bed 5.2663 0.0312

Features with the largest F values

FeatureSelectionØ  Select features using sequential forward feature selectionØ  Features selected by the models have significant differences in ANOVA test


Features F psleep DFA 60 0.2254 0.6395

min asleep 4.3128 0.0492daily step 4.3625 0.0480restless count 4.2324 0.0512awake count 4.0429 0.0562min awake 2.2073 0.1509

HR LH 4.0282 0.0566HR DFA 10 5.3353 0.0302

EvaluationwithSmallDatasetØ  K-fold cross validation: divide whole dataset into k equal-sized

subgroups. Use one subgroup for testing and the remaining k-1 subgroups for training the model.

Ø  Leave-one-out cross validation: leave a sample (or a group of samples) out for testing and train the model with rest of sample (or groups of samples).


https://en.wikipedia.org/wiki/Cross-validation_(statistics)#/media/File:K-fold_cross_validation_EN.svg

Illustrationofk-foldCV.Leave-one-outCVisaspecialcaseofk-foldCVwithk=n.

TestOverfittingØ  Impact of the number of nearest neighbors (K) on KNN performance


Smaller difference between training and testing errors à less overfitting

EvaluationMetricsØ  For imbalanced dataset, it’s insufficient to just look at accuracy!

q  Example: for 1:9 positive/negative ratio, the predictor can achieve 0.9 accuracy if predicting everything as negative.

Ø  A good predictor for imbalanced data should perform well for all ofq  sensitivity, specificity and precision.


Positive Negative

Predictedpositive TP: true positives FP: false positives

Predictednegative FN: false negatives TN: true negatives

Sensitivity =TP

TP + FNSpecificity =

TN

TN + FPPrecision =

TP

TP + FP

EvaluatePredictiveModelsØ  Use all sensing modalities (step, HR, sleep)Ø  Use first 20 days’ data as input Ø  Specificity fixed at 0.95 to ensure an acceptable false alarm rate


Model AUC-ROC AUC-PR Sensitivity Specificity Precision Accuracy

RF 0.7434 0.5551 0.3077 0.9459 0.6667 0.780

SVM 0.5943 0.4016 0.1795 0.9459 0.5385 0.7467

LR 0.7829 0.6118 0.4872 0.9009 0.6333 0.7933

NN 0.4002 0.2048 0.0770 0.9459 0.3333 0.720

KNN 0.7533 0.6680 0.5385 0.9820 0.9130 0.8667LACE 0.7647 0.6250 0.5556 0.7826AUC-ROC:areaunderReceiverOperatingCharacteristiccurveAUC-PR:areaunderprecision-recall(sensitivity)curve

KNN achieves higher specificity and precision than LACE index.

MachineLearningwithSmallData

Ø Apply statistical analysis first to assess feasibility.

Ø  Be careful with small datasets.q  Don’t just look at accuracy, especially for imbalanced dataset

q  Assess overfitting by comparing errors with training/testing data

q  Applying techniques to mitigate overfitting

Ø Clean the dataset before training the modelq  Remove invalid valuesq  Impute missing values

q  Normalize the dataset if needed


IoMTUseCases








Example:TimedUpandGo@Home

2/6/2018 ChenyangLu

Ø  Remind participants to take the assessmentØ Automatically upload the data to the cloud for analysis

Ø Analyze gait and motion featuresØ  Real-time analytics à feedback to physicians and participants

26

https://www.cse.wustl.edu/~lu/TUG.mp4JointworkwithRuixuanDai

LessonsfromaFallStudy

Ø  Fall data need to be recorded and annotated in real-time (instead of fall journals).

Ø Continuous connectivity with devices is key.q  Remote communication with sensors.

q  Visibility into the logs, and inspecting the system.

Ø Avoid limitations when selecting sensor hardware.q  ON/OFF switch, accurate wall-clock, user interface, confortness.

Ø  Plan larger studies.

ChenyangLu 27

Joint work with Susan Stark

OvercomeChallengeswithSmartwatches

Ø  Fall data need to be recorded and annotated in real-time (instead of fall journals).

Ø Continuous connectivity with devices is key.q  Remote communication with sensors.

q  Visibility into the logs, and inspecting the system.

Ø Avoid limitations when selecting sensor hardware.q  ON/OFF switch, accurate wall-clock, user interface, confortness.

Ø  Plan larger studies.

ChenyangLu 28

IoMTUseCases








MeasureStresswithEMAandPPGSensor

11/6/19 30

Joint work with Thomas Kannampallil, Ruixuan Dai

DoStressorsWork?

Ø Use EMA to validate stressors

11/6/19 31

Efficacy of stressors vary à Test with watch-based EMA!

StressbasedonPPGSensor(Preliminary) Math Cold

11/6/19 32

Model Precision

Recall

Accuracy

SVM 0.5 0.61 0.89

RandomForest 0.85 0.55 0.94

LogisticRegression

0.32 0.69 0.81

Model Precision

Recall

Accuracy

SVM 0.15 0.25 0.88

RandomForest 0.46 0.13 0.95

LogisticRegression

0.09 0.51 0.69

ML model is more accurate under effective stressor

IoMTUseCases








InternetofMedicalThings

Ø Wearables: wristbands, smart watches…q  Continuous monitoring: activity, heart rate, sleep, spO2…

Ø Connectivity: Bluetooth, WiFi, cellular…q  Real-time monitoring and intervention

Ø Cloud: computing and storage.q  Scalable to large cohorts

Ø Analytics: machine learning and signal processingq  Interpret data and predict outcomes

A powerful tool for clinical studiesinside and outside hospitals!


internet of medical thingslu/talks/iomt_icts_2019.pdfØ 25 heart failure patients were recruited. q...

Documents