internet of medical thingslu/talks/iomt_icts_2019.pdfØ 25 heart failure patients were recruited. q...
Post on 01-Oct-2020
2 Views
Preview:
TRANSCRIPT
Internet of Medical Things�Predicting Clinical Outcomes and Digital Phenotyping with Wearables and Machine Learning
Chenyang LuCyber-Physical Systems Laboratory
Department of Computer Science & Engineering
https://www.cse.wustl.edu/~lu/
InternetofMedicalThings
Ø Wearables: wristbands, smart watches…q Continuous monitoring: activity, heart rate, sleep, spO2…
Ø Connectivity: Bluetooth, WiFi, cellular…q Real-time monitoring and intervention
Ø Cloud: computing and storage.q Scalable to large cohorts
Ø Analytics: machine learning and signal processingq Interpret data and predict outcomes
A powerful tool for clinical studiesinside and outside hospitals!
10/23/2019 ChenyangLu 2
SmartWatches
10/23/2019 ChenyangLu
Two-wayCommunicationPushecologicalmomentaryassessments
Open,programmableplatform
AndroidWear,AppleResearchKit
TailoredonboardanalyticsShorterLatency
SensorsAccelerometer,gyroscope,
magnetometer,HeartRate,
GPS…
3
IoMTUseCases
Ø Predicting readmissionsq Feasibility of monitoring with wearables – data yield and compliance
q Predictive models – machine learning with “small data”
Ø Measuring mobilityq Timed Up and Go (TUG) with smart watches
q Fall detection – how smart watches can help
Ø Measuring stressq Model based on physiological signals through smart watches
q Ecological momentary assessment (EMA)
10/23/2019 ChenyangLu 4
IoMTUseCases
Ø Predicting readmissionsq Feasibility of monitoring with wearables – data yield and compliance
q Predictive models – machine learning with “small data”
Ø Measuring mobilityq Timed Up and Go (TUG) with smart watches
q Fall detection – how smart watches can help
Ø Measuring stressq Model based on physiological signals through smart watches
q Ecological momentary assessment (EMA)
10/23/2019 ChenyangLu 5
PredictingReadmissionsØ Hospital readmission rate is high for heart failure patients.
q ~25% patients readmitted within 30 days
Ø Early detection of deterioration (readmission + death)q Just-in-time intervention à better outcomeq Reduce health care cost
Ø LACE index: state of practice for predicting readmissionq Assess the risk of readmission based on inpatient dataq Length (L) of stay; Acuity (A) of the admission; Co-morbidities (C);
number of Emergency (E) department visits within the last 6 months
Ø Wearable devices provide convenient technologies for continuous outpatient monitoring.
10/23/2019 ChenyangLu 6
Objective
Ø Assess the feasibility of collecting multi-modal data from wearable device.
Ø Explore the potential of predicting clinical deteriorationq Develop and test predictive models based on machine learning
Ø Provide guidelines for future studies
10/23/2019 ChenyangLu 7
Joint work with Tom Bailey, Marin Kollef, Dingwen Li, Jay Vaidya, Michael Wang, Ben Bush�D. Li, J. Vaidya, M. Wang, B. Bush, C. Lu, M. Kollef and T. Bailey, Feasibility Study of Monitoring Deterioration of Outpatients Using Multi-modal Data Collected by Wearables, ACM Transactions on Computing for Healthcare, accepted.
Challenges
Ø Outpatient complianceq Wear devices continuously and properly outside hospitals
q Potential problems• Forget to charge the device• Reluctant to wear the device
• Improperly wear the device
Ø Learning from small sample sizeq May cause overfitting
q Imbalanced dataset
10/23/2019 ChenyangLu 8
SystemOverview
10/23/2019 ChenyangLu 9
StudyProtocol
Ø 25 heart failure patients were recruited.q Each patient is given Fitbit ChargeHR wristband.
q Continuously monitor patients after discharge.
q Outcomes: 60-day deterioration events.
Ø Heart rate (HR), step count and sleep status were collected.q Sampling period: 1 min (step, heart rate); 1 day (sleep yield)
10/23/2019 ChenyangLu 10
YieldØ Yield: fraction of samples successfully collected and stored in databaseØ Participants with data yield > 80%
q Step: 88% of participantsq Heart rate: 60% of participantsq Sleep: 68% of participants
10/23/2019 ChenyangLu 11
Compliance
Ø Participants with data yield > 80%q Step: 88% of participants
q Heart rate: 60% of participants à wore Fitbit properly
q Sleep: 68% of participants à wore Fitbit at night
10/23/2019 ChenyangLu 12
Feasibility of long-term, continuous monitoring
Latency
Ø Median: 8.6 min; 99% percentile: 22.5 hrØ Did not cause data loss, as Fitbit can locally store data for 7 days.
Ø Feasibility for daily interventionØ Can be reduced if analysis performed natively in Fitbit cloud
10/23/2019 ChenyangLu 13
Latency
PredictDeteriorationRisk
Ø Task: identifies participants at high risks of deterioration using data collected from the beginning of monitoring
Ø Dataset: 25 patients as samples, including 18 with no deterioration vs. 7 with deterioration
Ø Model: ML models for clinical readmission predictionq Random forest, SVM, logistic regression, multi-level perceptron,
K-nearest neighbor
10/23/2019 ChenyangLu 14
Data Data
Day1 DayN
Timeofdischargefromhospital
InputModel
Predict
N=20Xdaysinthefuture
Willdeteriorate?
DataPreprocessing
Ø Raw data formatq Raw heart rate and step count per minute
q Derived data from Fitbit API, such as time in bed, sleep efficiency in sleep summary data
Ø Data cleansingq Extract step-related data only in 7am-7pm
q Fill in missing data by carry forward imputation
q Discard a whole day’s data if missing data last longer than the day
10/23/2019 ChenyangLu 15
FeatureEngineering
Ø Statistical features:q First- and second- order features extracted from sliding window
q 1st order: mean, max, min, skewness, kurtosis
q 2nd order: energy, entropy, correlation, inertia and local homogeneity
Ø Detrended Fluctuation Analysisq Determine statistical self-affinity of time series
q The fluctuation is then used as feature
Ø Semantic featuresq Sedentary behavior
10/23/2019 ChenyangLu 16
AssessPredictability
Ø Analysis of Variance (ANOVA)q Are there statistically significant differences between patients with
different outcomes? q High F-statistic and low p-value à significant difference among the
group means.
Ø Indicate the feasibility of learning a stable predictor.
Ø Assumptionsq Independent observations
q The variable follow a normal distribution in each group
q The variances within all groups must be equal (needed only for very imbalanced groups)
10/23/2019 ChenyangLu 17
ANOVAResults
Ø Groups: deteriorated patients vs. non-deteriorated patientsØ ANOVA test reveals significant differences in some features
à suggesting feasibility of predictive models
10/23/2019 ChenyangLu 18
Features F pHR skewness 7.9125 0.0099
HR correlation 5.4789 0.0283
HR DFA 10 5.3353 0.0302
Restless duration 5.2912 0.0308
Time in bed 5.2663 0.0312
Features with the largest F values
FeatureSelectionØ Select features using sequential forward feature selectionØ Features selected by the models have significant differences in ANOVA test
10/23/2019 ChenyangLu 19
Features F psleep DFA 60 0.2254 0.6395
min asleep 4.3128 0.0492daily step 4.3625 0.0480restless count 4.2324 0.0512awake count 4.0429 0.0562min awake 2.2073 0.1509
HR LH 4.0282 0.0566HR DFA 10 5.3353 0.0302
EvaluationwithSmallDatasetØ K-fold cross validation: divide whole dataset into k equal-sized
subgroups. Use one subgroup for testing and the remaining k-1 subgroups for training the model.
Ø Leave-one-out cross validation: leave a sample (or a group of samples) out for testing and train the model with rest of sample (or groups of samples).
10/23/2019 ChenyangLu 20
https://en.wikipedia.org/wiki/Cross-validation_(statistics)#/media/File:K-fold_cross_validation_EN.svg
Illustrationofk-foldCV.Leave-one-outCVisaspecialcaseofk-foldCVwithk=n.
TestOverfittingØ Impact of the number of nearest neighbors (K) on KNN performance
10/23/2019 ChenyangLu 21
Smaller difference between training and testing errors à less overfitting
EvaluationMetricsØ For imbalanced dataset, it’s insufficient to just look at accuracy!
q Example: for 1:9 positive/negative ratio, the predictor can achieve 0.9 accuracy if predicting everything as negative.
Ø A good predictor for imbalanced data should perform well for all ofq sensitivity, specificity and precision.
10/23/2019 ChenyangLu 22
Positive Negative
Predictedpositive TP: true positives FP: false positives
Predictednegative FN: false negatives TN: true negatives
Sensitivity =TP
TP + FNSpecificity =
TN
TN + FPPrecision =
TP
TP + FP
EvaluatePredictiveModelsØ Use all sensing modalities (step, HR, sleep)Ø Use first 20 days’ data as input Ø Specificity fixed at 0.95 to ensure an acceptable false alarm rate
10/23/2019 ChenyangLu 23
Model AUC-ROC AUC-PR Sensitivity Specificity Precision Accuracy
RF 0.7434 0.5551 0.3077 0.9459 0.6667 0.780
SVM 0.5943 0.4016 0.1795 0.9459 0.5385 0.7467
LR 0.7829 0.6118 0.4872 0.9009 0.6333 0.7933
NN 0.4002 0.2048 0.0770 0.9459 0.3333 0.720
KNN 0.7533 0.6680 0.5385 0.9820 0.9130 0.8667LACE 0.7647 0.6250 0.5556 0.7826AUC-ROC:areaunderReceiverOperatingCharacteristiccurveAUC-PR:areaunderprecision-recall(sensitivity)curve
KNN achieves higher specificity and precision than LACE index.
MachineLearningwithSmallData
Ø Apply statistical analysis first to assess feasibility.
Ø Be careful with small datasets.q Don’t just look at accuracy, especially for imbalanced dataset
q Assess overfitting by comparing errors with training/testing data
q Applying techniques to mitigate overfitting
Ø Clean the dataset before training the modelq Remove invalid valuesq Impute missing values
q Normalize the dataset if needed
10/23/2019 ChenyangLu 24
IoMTUseCases
Ø Predicting readmissionsq Feasibility of monitoring with wearables – data yield and compliance
q Predictive models – machine learning with “small data”
Ø Measuring mobilityq Timed Up and Go (TUG) with smart watches
q Fall detection – how smart watches can help
Ø Measuring stressq Model based on physiological signals through smart watches
q Ecological momentary assessment (EMA)
10/23/2019 ChenyangLu 25
Example:TimedUpandGo@Home
2/6/2018 ChenyangLu
Ø Remind participants to take the assessmentØ Automatically upload the data to the cloud for analysis
Ø Analyze gait and motion featuresØ Real-time analytics à feedback to physicians and participants
26
https://www.cse.wustl.edu/~lu/TUG.mp4JointworkwithRuixuanDai
LessonsfromaFallStudy
Ø Fall data need to be recorded and annotated in real-time (instead of fall journals).
Ø Continuous connectivity with devices is key.q Remote communication with sensors.
q Visibility into the logs, and inspecting the system.
Ø Avoid limitations when selecting sensor hardware.q ON/OFF switch, accurate wall-clock, user interface, confortness.
Ø Plan larger studies.
ChenyangLu 27
Joint work with Susan Stark
OvercomeChallengeswithSmartwatches
Ø Fall data need to be recorded and annotated in real-time (instead of fall journals).
Ø Continuous connectivity with devices is key.q Remote communication with sensors.
q Visibility into the logs, and inspecting the system.
Ø Avoid limitations when selecting sensor hardware.q ON/OFF switch, accurate wall-clock, user interface, confortness.
Ø Plan larger studies.
ChenyangLu 28
IoMTUseCases
Ø Predicting readmissionsq Feasibility of monitoring with wearables – data yield and compliance
q Predictive models – machine learning with “small data”
Ø Measuring mobilityq Timed Up and Go (TUG) with smart watches
q Fall detection – how smart watches can help
Ø Measuring stressq Model based on physiological signals through smart watches
q Ecological momentary assessment (EMA)
10/23/2019 ChenyangLu 29
MeasureStresswithEMAandPPGSensor
11/6/19 30
Joint work with Thomas Kannampallil, Ruixuan Dai
DoStressorsWork?
Ø Use EMA to validate stressors
11/6/19 31
Efficacy of stressors vary à Test with watch-based EMA!
StressbasedonPPGSensor(Preliminary) Math Cold
11/6/19 32
Model Precision
Recall
Accuracy
SVM 0.5 0.61 0.89
RandomForest 0.85 0.55 0.94
LogisticRegression
0.32 0.69 0.81
Model Precision
Recall
Accuracy
SVM 0.15 0.25 0.88
RandomForest 0.46 0.13 0.95
LogisticRegression
0.09 0.51 0.69
ML model is more accurate under effective stressor
IoMTUseCases
Ø Predicting readmissionsq Feasibility of monitoring with wearables – data yield and compliance
q Predictive models – machine learning with “small data”
Ø Measuring mobilityq Timed Up and Go (TUG) with smart watches
q Fall detection – how smart watches can help
Ø Measuring stressq Model based on physiological signals through smart watches
q Ecological momentary assessment (EMA)
10/23/2019 ChenyangLu 33
InternetofMedicalThings
Ø Wearables: wristbands, smart watches…q Continuous monitoring: activity, heart rate, sleep, spO2…
Ø Connectivity: Bluetooth, WiFi, cellular…q Real-time monitoring and intervention
Ø Cloud: computing and storage.q Scalable to large cohorts
Ø Analytics: machine learning and signal processingq Interpret data and predict outcomes
A powerful tool for clinical studiesinside and outside hospitals!
10/23/2019 ChenyangLu 34
top related