csc 2541: machine learning for healthcare lecture 3 ... - lecture 3.pdf · project proposals (feb 6...
TRANSCRIPT
![Page 1: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/1.jpg)
CSC 2541: Machine Learning for Healthcare
Lecture 3: Clinical Time Series Modelling
Professor Marzyeh Ghassemi, PhDUniversity of Toronto, CS/MedVector Institute
![Page 2: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/2.jpg)
2
Course Reminders!
• Submit the weekly reflection questions to MarkUs!
• Sign up for a paper presentation slot!
• Homework 1 due next week!
• Think about your projects!
![Page 3: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/3.jpg)
3
Logistics
• Course website:https://cs2541-ml4h2020.github.io
• Piazza: https://piazza.com/utoronto.ca/winter2020/csc2541
• Grading:• 20% Homework (3 problem sets)• 10% Weekly reflections on Markus (5 questions)• 10% Paper presentation done in-class (sign-up after the first lecture)• 60% course project (an eight-page write up)
![Page 4: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/4.jpg)
4
Schedule
Jan 9, 2020, Lecture 1: Why is healthcare unique?Jan 16, 2020, Lecture 2: Supervised Learning for Classification, Risk Scores and SurvivalJan 23, 2020, Lecture 3: Clinical Time Series ModellingJan 30, 2020, Lecture 4: Causal inference with Health Data --- Dr. Shalmali Joshi (Vector)
Problem Set 1 (Jan 31 at 11:59pm)Feb 6, 2020, Lecture 5: Fairness, Ethics, and Healthcare
Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph Paul Cohen (MILA)
Problem Set 2 (Feb 14 at 11:59pm)Feb 20, 2020, Lecture 7: Clinical NLP and Audio -- Dr. Tristan Naumann (MSR)Feb 27, 2020, Lecture 8: Clinical Reinforcement LearningMar 5, 2020, Lecture 9: Interpretability / Humans-In-The-Loop --- Dr. Rajesh Ranganath (NYU)
Problem Set 3 (Mar 6 at 11:59pm)Mar 12, 2020, Lecture 10: Disease Progression Modelling/Transfer Learning -- Irene Chen (MIT)Mar 19, 2020, Project Sessions/LectureMar 26, 2020, Course PresentationsApril 4, 2020, Course Presentations
Project Report (Apr 3 at 11:59pm)
![Page 5: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/5.jpg)
5
Outline
1. What’s Time Got To Do With It?a. Missingnessb. Representation
2. Case Study 1: MTGPs for Mortality Prediction and TBI
3. Case Study 2: RNNs/CNNs for Intervention Onset Prediction
4. Project Discussion
![Page 6: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/6.jpg)
6
Outline
1. What’s Time Got To Do With It?a. Missingnessb. Representation
2. Case Study 1: MTGPs for Mortality Prediction and TBI
3. Case Study 2: RNNs/CNNs for Intervention Onset Prediction
4. Project Discussion
![Page 7: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/7.jpg)
7
Problem: Hospital decision-making / care planning
Observe Patient Data “Real-time” Prediction Of {Drug/Mortality/Condition}
By Gap Time
Before the Doctor Acted?
![Page 8: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/8.jpg)
8
Problem: Hospital decision-making / care planning
Observe Patient Data “Real-time” Prediction Of {Drug/Mortality/Condition}
By Gap Time
Before the Doctor Acted?
![Page 9: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/9.jpg)
9
How Do We Handle Time?
• An image gives a snapshot of an object, but a video dictates form!
• We want to model patient risks/treatments/outcomes as they live.
• Strategies:• Amortize - Make features out of mean, min, max, etc. • Stack - Inputs of fixed size, and concatenate.• Deal - Use a method that addresses dynamics.
• Focus on dealing in this lecture.
![Page 10: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/10.jpg)
10
Outline
1. What’s Time Got To Do With It?a. Missingnessb. Representation
2. Case Study 1: MTGPs for Mortality Prediction and TBI
3. Case Study 2: RNNs/CNNs for Intervention Onset Prediction
4. Project Discussion
![Page 11: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/11.jpg)
What is Missingness?
![Page 12: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/12.jpg)
Missing Data DetailsData can be missing according to several regimes:
● Missing completely at random (MCAR)● Missing at random (MAR)● Missing not at random (MNAR)
![Page 13: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/13.jpg)
Missing Data DetailsData can be missing according to several regimes:
● Missing completely at random (MCAR)○ The observed pattern of missingness is independent from the observed or missing values.
● Missing at random (MAR)● Missing not at random (MNAR)
![Page 14: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/14.jpg)
Missing Data DetailsData can be missing according to several regimes:
● Missing completely at random (MCAR)○ The observed pattern of missingness is independent from the observed or missing values.
● Missing at random (MAR)○ The observed pattern of missingness is independent from the missing values (but may depend
on the observed values).
● Missing not at random (MNAR)
![Page 15: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/15.jpg)
Missing Data DetailsData can be missing according to several regimes:
● Missing completely at random (MCAR)○ The observed pattern of missingness is independent from the observed or missing values.
● Missing at random (MAR)○ The observed pattern of missingness is independent from the missing values (but may depend
on the observed values).
● Missing not at random (MNAR)○ All bets are off.
![Page 16: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/16.jpg)
Missing Data DetailsData can be missing according to several regimes:
● Missing completely at random (MCAR)○ The observed pattern of missingness is independent from the observed or missing values.
● Missing at random (MAR)○ The observed pattern of missingness is independent from the missing values (but may depend
on the observed values).
● Missing not at random (MNAR)○ All bets are off.
Healthcare lives here.
![Page 17: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/17.jpg)
Missing Data is Confounding
![Page 18: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/18.jpg)
How do we handle missing data?
![Page 19: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/19.jpg)
Imputation1. Statistical Timeseries Forecasting: ARMA/ARIMA/ARIMAX, etc.
2. Easy Baselines: Constant infilling, Sample & Hold (+ indicators), Interpolation
3. Traditional Imputation: MICE/3D-MICE, MissForest, Matrix/Tensor Completion
4. Gaussian Processes
5. Advanced neural methods (GRU-D, GANs, etc.)
![Page 20: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/20.jpg)
Imputation
Prasad, Niranjani, et al. "A reinforcement learning approach to weaning of mechanical ventilation in intensive care units." arXiv preprint arXiv:1704.06300 (2017).
![Page 21: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/21.jpg)
GANs for Imputation
![Page 22: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/22.jpg)
GANs for Imputation
Left: Jo, Youngjoo, and Jongyoul Park. "SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color." arXiv preprint arXiv:1902.06838 (2019).Middle: Zhu, Jun-Yan, et al. "Unpaired image-to-image translation using cycle-consistent adversarial networks." Proceedings of the IEEE International Conference on Computer Vision. 2017.Right: https://thispersondoesnotexist.com/
![Page 23: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/23.jpg)
GAIN: Generative Adversarial Imputation
![Page 24: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/24.jpg)
Imputation Papers1. GAIN: https://arxiv.org/pdf/1806.02920.pdf2. GRU-D: https://www.nature.com/articles/s41598-018-24271-93. GP Imputation: https://arxiv.org/pdf/1704.06300.pdf4. Interpolation-prediction network: https://arxiv.org/pdf/1812.00531.pdf
![Page 25: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/25.jpg)
Opportunities1. Improved imputation methods. How do forecasting, GP, or adversarial
methods compare to GRU-D/interpolation prediction network? Can we incorporate uncertainty offered by GPs usefully into downstream tasks? Can we make other models offer uncertainty?
2. Can we model the decision process by which clinicians choose what to measure and what to omit? How would this be helpful in downstream tasks? Can this help account for the MNAR nature of healthcare missingness?
3. Can we control for the confounding effects of missingness? Can we learn a model on underlying physiology from retrospective, care-byproduct data?
![Page 26: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/26.jpg)
26
Outline
1. What’s Time Got To Do With It?a. Missingnessb. Representation
2. Case Study 1: MTGPs for Mortality Prediction and TBI
3. Case Study 2: RNNs/CNNs for Intervention Onset Prediction
4. Project Discussion
![Page 27: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/27.jpg)
Representation: Why do we care?
![Page 28: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/28.jpg)
Representations define a notion of “similarity”
Closer in “Conceptual Space” Closer in “Pixel Space”
![Page 29: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/29.jpg)
Representations learn a notion of similarity
Conroy, Bryan, Minnan Xu-Wilson, and Asif Rahman. "Patient Similarity Using Population Statistics and Multiple Kernel Learning." Machine Learning for Healthcare Conference. 2017.
![Page 30: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/30.jpg)
Representations can stabilize changing data
Nestor, Bret, et al. "Rethinking clinical prediction: Why machine learning must consider year of care and feature aggregation." Machine Learning for Healthcare Conference. 2019
![Page 31: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/31.jpg)
Representations can stabilize changing data
Gong, Jen J., et al. "Predicting clinical outcomes across changing electronic health record systems." Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2017.
![Page 32: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/32.jpg)
Representations can join disparate modalities
Hsu, Tzu-Ming Harry, et al. "Unsupervised multimodal representation learning across medical images and reports." arXiv preprint arXiv:1811.08615 (2018).
![Page 33: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/33.jpg)
DeepCluster: Why bother with labels?
Caron, Mathilde, et al. "Deep clustering for unsupervised learning of visual features." Proceedings of the European Conference on Computer Vision (ECCV). 2018.
![Page 34: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/34.jpg)
Representation Learning in Action: Multitask Learning
![Page 35: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/35.jpg)
Representation Learning in Action: Clustering
Choi, Edward, et al. "GRAM: graph-based attention model for healthcare representation learning." Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2017.
![Page 36: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/36.jpg)
Representation Learning in Action: Clustering
Dhamala, Jwala, et al. "Multivariate Time-Series Similarity Assessment via Unsupervised Representation Learning and Stratified Locality Sensitive Hashing: Application to Early Acute Hypotensive Episode Detection." IEEE Sensors Letters 3.1 (2019): 1-4.
![Page 37: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/37.jpg)
Representation Learning in Action: Anomaly Detection
Schlegl, Thomas, et al. "Unsupervised anomaly detection with generative adversarial networks to guide marker discovery." International Conference on Information Processing in Medical Imaging. Springer, Cham, 2017.
![Page 38: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/38.jpg)
Representation Learning in Action: Anomaly Detection
![Page 39: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/39.jpg)
Key Points for Healthcare● Representations can normalize.● Generalization to unseen tasks is critical (e.g., patient subtyping).● Representations can aid in interpretability.● Representations can span many modalities.
![Page 40: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/40.jpg)
40
Outline
1. What’s Time Got To Do With It?a. Missingnessb. Representation
2. Case Study 1: MTGPs for Mortality Prediction and TBI
3. Case Study 2: RNNs/CNNs for Intervention Onset Prediction
4. Project Discussion
![Page 41: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/41.jpg)
41
Remember This? Topics Improves Mortality Prediction
• Forward-facing ICU mortality prediction with notes.
• Latent representations add predictive power.
• Topics enable accurately assess risk from notes.
![Page 42: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/42.jpg)
42
Add Information About Evolution of Signals
• Learn a new latent representation to evaluate multi-dimensional function similarity (θ).
MTGP models capture movements within and
between signals.
Transform signals into MTGP hyperparameter representation.
Compare patient similarly in the new representation.
![Page 43: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/43.jpg)
43
Learning Single Task Gaussian Processes (STGP)
• Model each signal as a GP task with mean and covariance functions.
• GP’s commonly used to predict at new indices.
• Learn the parameters (θ) of the kernel from data.
![Page 44: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/44.jpg)
44
Single vs. Multi-task Gaussian Processes
•Assume we have m sets of: • Inputs Xi
• Temporal covariance hyperparameters θit
• Estimated functions fi
• Noise terms σi
• Outcomes yi
• We can train m single-task Gaussian process (STGP) (a) or a multi-task Gaussian process (MTGP) to relate the m tasks through all prior variables, with the tasks’ labels l and similarity matrix θc (b).
![Page 45: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/45.jpg)
45
Learning MTGPs As Representations
•Use an MTGP representation to relate m inputs through Kt and Kc.
[1] Bonilla, Edwin V., Kian M. Chai, and Christopher Williams. "Multi-task Gaussian process prediction." Advances in neural information processing systems. 2007.[2] Carl Rasmussen’s minimize.m was used for gradient-based optimization of the marginal likelihood.
Movement within a signalMovement between signals
![Page 46: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/46.jpg)
46
Estimating Signal in Traumatic Brain Injury Patients
•Intracranial pressure (ICP) and mean arterial blood pressure (ABP) are important indicators of cerebrovascular autoregulation (CA) in traumatic Brain Injury (TBI) patients.
• CA sustains adequate cerebral blood flow1 and impairment risks secondary brain damage and mortality.2
• CA is assessed using a sliding window Pearson’s correlation between the ICP and ABP – the Pressure-Reactivity Index (PRx)3.
[1] Werner, C., and K. Engelhard. "Pathophysiology of traumatic brain injury." British journal of anaesthesia 99.1 (2007): 4-9.[2] Hlatky, Roman, Alex B. Valadka, and Claudia S. Robertson. "Intracranial pressure response to induced hypertension: role of dynamic pressure autoregulation." Neurosurgery 57.5 (2005): 917-923.[3] Czosnyka, Marek, et al. "Continuous assessment of the cerebral vasomotor reactivity in head injury." Neurosurgery 41.1 (1997): 11-19.
![Page 47: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/47.jpg)
47
TBI Estimation Methodology
•PRx isn’t calculated when either signal is contaminated - evaluate STGPs/MTGPs for interpolation, and MTGPs for PRx estimation.
•Collected data from 35 TBI patients with 24+ hours of ICP and ABP recordings sampled every 10 seconds.
• Selected 30 ten-minute windows where ICP/ABP were free from artifacts and missing values from each patient recording; randomly introduced artificial gaps in both signals (x’s).
![Page 48: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/48.jpg)
48
MTGP Representations Improve Signal Forecasting and Outcome Prediction
•MTGPs outperform STGPs in signal reconstruction.
•Automatically estimate cerebrovascular autoregulation.
* Final cohort consisted of 10,202 patients, with 313,461 notes.
Performance on Signal Forecasting
Performance on Mortality Prediction
•MTGP hyperparameter representations improve short-term mortality prediction.
![Page 49: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/49.jpg)
49
Outline
1. What’s Time Got To Do With It?a. Missingnessb. Representation
2. Case Study 1: MTGPs for Mortality Prediction and TBI
3. Case Study 2: RNNs/CNNs for Intervention Onset Prediction
4. Project Discussion
![Page 50: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/50.jpg)
50
Can We Predict Interventions?
• 34,148 ICU patients from MIMIC-III• 5 static variables (gender, age, etc.)• 29 time-varying vitals and labs (oxygen saturation, lactate, etc.)• All clinical notes for each patient stay
![Page 51: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/51.jpg)
51
Raw Physiology vs “Words” Embedding
• Many values are missing!
![Page 52: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/52.jpg)
52
Raw Physiology vs “Words” Embedding
• Many values are missing!• Z-score existing variables, rounding to the nearest int.
-1
![Page 53: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/53.jpg)
53
Raw Physiology vs “Words” Embedding
• Many values are missing!• Z-score existing variables, rounding to the nearest int.• Convert each z-score into its own binary column.
-1
![Page 54: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/54.jpg)
54
Raw Physiology vs “Words” Embedding
• Many values are missing!• Z-score existing variables, rounding to the nearest int.• Convert each z-score into its own binary column.
-1 A row of all zeros indicates a missing value at that hour.
![Page 55: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/55.jpg)
55
Many Ways to Model, What Do We Learn?
55
LSTM CNN
2 Layer/512 node LSTM with sequential hourly data; at end of window, use the final hidden state to predict output.
CNN for temporal convolutions at 3/4/5 hours, max-pool, combine the outputs, and run through 2 fully connected layers for prediction.
SSAM
Learn model parameters over patients with variational EM.
Logistic regression(with label-balanced cost function)
... ... ...
...... ... ...Infer hourly distribution over hidden states with HMM DP (fwd alg.).
... ......... ...... ...
Predict onset in advance
![Page 56: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/56.jpg)
56
Many Ways to Model, What Do We Learn?
56
LSTM CNN
2 Layer/512 node LSTM with sequential hourly data; at end of window, use the final hidden state to predict output.
CNN for temporal convolutions at 3/4/5 hours, max-pool, combine the outputs, and run through 2 fully connected layers for prediction.
SSAM
Learn model parameters over patients with variational EM.
Logistic regression(with label-balanced cost function)
... ... ...
...... ... ...Infer hourly distribution over hidden states with HMM DP (fwd alg.).
... ......... ...... ...
Predict onset in advance
Covered last week!
![Page 57: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/57.jpg)
57
Many Ways to Model, What Do We Learn?
57
LSTM CNN
2 Layer/512 node LSTM with sequential hourly data; at end of window, use the final hidden state to predict output.
CNN for temporal convolutions at 3/4/5 hours, max-pool, combine the outputs, and run through 2 fully connected layers for prediction.
SSAM
Learn model parameters over patients with variational EM.
Logistic regression(with label-balanced cost function)
... ... ...
...... ... ...Infer hourly distribution over hidden states with HMM DP (fwd alg.).
... ......... ...... ...
Predict onset in advance
Covered last week!
![Page 58: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/58.jpg)
To model sequences, we need:
1. To deal with variable-length sequences2. To maintain sequence order3. To keep track of long-term dependencies4. To share parameters across the sequence
Let’s turn to recurrent neural networks.
MIT 6.S191 | Intro to Deep Learning | IAP 2018Slides courtesy of Harini Suresh +
RNNs on Sequences
![Page 59: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/59.jpg)
.
.
.
.
.
.
.
.
.
input hidden output
MIT 6.S191 | Intro to Deep Learning | IAP 2018
Example Network
![Page 60: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/60.jpg)
.
.
.
.
.
.
.
.
.
input hidden output
let’s take a look at this one hidden unit
MIT 6.S191 | Intro to Deep Learning | IAP 2018
Example Network
![Page 61: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/61.jpg)
RNNS remember their previous state:
t = 0
x0 : “it” W
U
s0
s1
MIT 6.S191 | Intro to Deep Learning | IAP 2018
![Page 62: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/62.jpg)
RNNS remember their previous state:
t = 1
x1 : “was” W
U
s1
s212
MIT 6.S191 | Intro to Deep Learning | IAP 2018
![Page 63: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/63.jpg)
“Unfolding” the RNN across time:
x0
W
s0
U
s1
U
x1
W
x2
W
s2
U. . .
time
MIT 6.S191 | Intro to Deep Learning | IAP 2018
![Page 64: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/64.jpg)
“Unfolding” the RNN across time:
x0
W
s0
U
s1
U
x1
W
x2
W
s2
U. . .
time
notice that we use the same parameters, W and U
MIT 6.S191 | Intro to Deep Learning | IAP 2018
![Page 65: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/65.jpg)
“Unfolding” the RNN across time:
x0
W
s0
U
s1
U
x1
W
x2
W
s2
U. . .
time
sn can contain information from all past timesteps
MIT 6.S191 | Intro to Deep Learning | IAP 2018
![Page 66: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/66.jpg)
Why do LSTMs help?
1. Forget gate allows information to pass through
unchanged
2. Cell state is separate from what’s outputted
3. sj depends on sj-1 through addition! → derivatives don’t expand into a long product!
MIT 6.S191 | Intro to Deep Learning | IAP 2018
![Page 67: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/67.jpg)
67
Predict Onsets of Interventions
• Delay prediction by 6-hour gap time.
• Attempt to predict onest, weaning, staying off, staying on.
![Page 68: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/68.jpg)
68
NNs Do Well; Improved Representation HelpsA
rea-
unde
r-R
OC
Representations with “physiological words” for missingness significantly increased AUC for interventions with the lowest proportion of examples.
Deep models perform well in general, but words are important for ventilation tasks.
![Page 69: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/69.jpg)
69
Feature-Level Occlusions Identify Per-Class FeaturesD
ecre
ase
in A
UC
Physiological data were more important for the more invasive interventions.
Clinical note topics were more important for less invasive tasks.
![Page 70: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/70.jpg)
70
Convolutional Filters Target Short-term Trajectories
Higher diastolic blood pressure, respiratory rate, and heart rate, and lower oxygen
saturation : Hyperventilation
Most differentiated features of 10 real patient trajectories that are highest/lowest activating for each task.
Decreased systolic blood pressure, heart rate and oxygen
saturation rate : Altered peripheral perfusion or
stress hyperglycemia
Decreased creatinine, phosphate, oxygen
saturation and blood urea nitrogen :
Neuromuscular respiratory failure
![Page 71: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/71.jpg)
71
Convolutional Filters Target Short-term Trajectories
● “Hallucinations” give insight into underlying properties of the network.
● The trajectories are made to maximize the output of the model, (do not correspond to physiologically plausible trajectories).
Blood pressure drops are maximally activating for vasopressor onset.
Respiratory rate decreasing is maximally activating for ventilation onset.
![Page 72: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/72.jpg)
72
Outline
1. What’s Time Got To Do With It?a. Missingnessb. Representation
2. Case Study 1: MTGPs for Mortality Prediction and TBI
3. Case Study 2: RNNs/CNNs for Intervention Onset Prediction
4. Project Discussion
![Page 73: CSC 2541: Machine Learning for Healthcare Lecture 3 ... - Lecture 3.pdf · Project proposals (Feb 6 at 5pm) Feb 13, 2020, Lecture 6: Deep Learning in Medical Imaging -- Dr. Joseph](https://reader034.vdocuments.net/reader034/viewer/2022050604/5fabeccbfb0e3868082dafbc/html5/thumbnails/73.jpg)
73