machine learning in modern medicine with erin ledell at stanford med

H2O.ai Machine Intelligence

Machine Learning in Modern Medicine

Erin LeDell Ph.D.


Introduction


H2O.ai

H2O Company

H2O Software

• Team: 35. Founded in 2012, Mountain View, CA• Stanford Math & Systems Engineers

• Open Source Software • Ease of Use via Web Interface• R, Python, Scala, Spark & Hadoop Interfaces• Distributed Algorithms Scale to Big Data


H2O.ai Team


Scientific Advisory CouncilDr. Trevor Hastie

Dr. Rob Tibshirani

Dr. Stephen Boyd

• John A. Overdeck Professor of Mathematics, Stanford University• PhD in Statistics, Stanford University• Co-author, The Elements of Statistical Learning: Prediction, Inference and Data Mining• Co-author with John Chambers, Statistical Models in S• Co-author, Generalized Additive Models • 108,404 citations (via Google Scholar)

• Professor of Statistics and Health Research and Policy, Stanford University• PhD in Statistics, Stanford University• COPPS Presidents’ Award recipient• Co-author, The Elements of Statistical Learning: Prediction, Inference and Data Mining• Author, Regression Shrinkage and Selection via the Lasso• Co-author, An Introduction to the Bootstrap

• Professor of Electrical Engineering and Computer Science, Stanford University• PhD in Electrical Engineering and Computer Science, UC Berkeley• Co-author, Convex Optimization• Co-author, Linear Matrix Inequalities in System and Control Theory• Co-author, Distributed Optimization and Statistical Learning via the Alternating Direction

Method of Multipliers


Agenda• Motivation• An Intro to Machine Learning• Machine Learning in Medicine• H2O Machine Learning• Live Software Demo


MotivationPart 1 of 5



Data-Driven

Preventative Care

Precision Medicine

• “The Future of Medicine is Precision Health” • President Obama’s Precision Medicine Initiative• Similar efforts at Stanford, UC Berkeley, UCSF

Personalized


Electronic Health Records


Genomic Data

Genomics pioneer, Jun Wang, is one of China’s most famous scientists.

Dr. Wang recently stepped down as head of BGI to create an AI health-monitoring system that would identify relationships between individual human genomic data, physiological traits (phenotypes) and lifestyle choices in order to provide advice on healthier living and to predict, and prevent, disease.

Source: BGI


Wearables

ABI Research has projected that by 2016, wearable wireless medical device sales will reach more than 100 million devices annually.


Open mHealth

• FitBit• HealthVault• Jawbone• RunKeeper• Withings• HealthKit• HL7 EHR

Open mHealth is a nonprofit start-up breaking down the barriers to integration and bringing clinical meaning to digital health data.


Intro to MLPart 2 of 5



What is Machine Learning?

What it is: ✤ “Field of study that gives computers the ability to learn without being explicitly programmed.” (Samuel, 1959)

✤ “Machine learning and statistics are closely related fields. The ideas of machine learning, from methodological principles to theoretical tools, have had a long pre-history in statistics.” (Jordan, 2014)

✤ M.I. Jordan also suggested the term data science as a placeholder to call the overall field.

Unlike rules-based systems which require a human expert to hard-code domain knowledge directly into the system, a machine learning algorithm learns how to make decisions from the data alone.

What it’s not:


Classification

Clustering

Machine Learning Overview

• Predict a real-valued response (viral load, weight)• Gaussian, Gamma, Poisson and Tweedie • MSE and R^2

• Multi-class or Binary classification• Ranking• Accuracy and AUC

• Unsupervised learning (no training labels)• Partition the data / identify clusters• AIC and BIC

Regression


Machine Learning Workflow

Source: NLTK

Example of a supervised machine learning workflow.


ML Model Performance

Test & Train • Partition the original data (randomly) into a training set and a test set. (e.g. 70/30)

• Train a model using the “training set” and evaluate performance on the “test set” or “validation set.”

• Train & test K models as shown.

• Average the model performance over the K test sets.

• Report cross-validated metrics.

• Regression: R^2, MSE, RMSE• Classification: Accuracy, F1, H-measure• Ranking (Binary Outcome): AUC, Partial AUC

K-foldCross-validation

Performance Metrics


What is Deep Learning?

What it is: ✤ “A branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using model architectures, composed of multiple non-linear transformations.” (Wikipedia, 2015)

✤ Deep neural networks have more than one hidden layer in their architecture. That’s what’s “deep.”

✤ Very useful for complex input data such as images, video, audio.

Deep learning architectures, specifically artificial neural networks (ANNs) have been around since 1980, so they are not new. However, there were breakthroughs in training techniques that lead to their recent resurgence (mid 2000’s). Combined with modern computing power, they are quite effective.

What it’s not:


Deep Learning Architecture

Example of a deep neural net architecture.


What is Ensemble Learning?

What it is: ✤ “Ensemble methods use multiple learning algorithms to obtain better predictive performance that could be obtained from any of the constituent learning algorithms.” (Wikipedia, 2015)

✤ Random Forests and Gradient Boosting Machines (GBM) are both ensembles of decision trees.

✤ Stacking, or Super Learning, is technique for combining various learners into a single, powerful learner using a second-level metalearning algorithm.

Ensembles typically achieve superior model performance over singular methods. However, this comes at a price — computation time.

What it’s not:


What is Data Science?

Problem Formulation

• Identify an outcome of interest and the type of task: classification / regression / clustering

• Identify the potential predictor variables• Identify the independent sampling units

• Conduct research experiment (e.g. Clinical Trial)• Collect examples / randomly sample the population• Transform, clean, impute, filter, aggregate data• Prepare the data for machine learning — X, Y

• Modeling using a machine learning algorithm (training)• Model evaluation and comparison• Sensitivity & Cost Analysis

• Translate results into action items• Feed results into research pipeline

Collect & Process Data

Machine Learning

Insights & Action


Data Science Venn Diagram

Source: Drew Conway

H2O.ai Machine Intelligence Source: marketingdistillery.com

http://marketingdistillery.com


Medical MLPart 3 of 5



Machine Learning in Medicine

• Predict Viral Failure in AIDS patients• Parkinson’s disease progression prediction from

mobile phone accelerometer data• Personalized diagnostics

• Clinical research: Identify which genes are associated with breast cancer relapse

• Prognosis: Predict probability of survival in 5 years

• Real-time predictions using data from wearables• Medication adherence monitoring

• Clinical research: MRI and PET scans & Deep Learning

• Cellular image analysis: genotype, phenotype, classification, identification, cell tracking

DiagnosticTesting

Oncology

MedicalImaging

Remote PatientMonitoring


Predict Viral Failure

Problem Formulation

• Outcome of interest: Viral failure in HIV patients (0/1)• Machine Learning task: Ranking (measured by AUC)• Sampling units: Multiple viral load tests per patient —

pooled repeated measures data


Machine Learning

Insights & Action



Problem Formulation



• MEMS data collected from electronic pill boxes• Two cohorts: Uganda (UARTO) & US (MACH14)• Feature generation: Generate time-sensitive features

that capture information about medication adherence


Machine Learning

Insights & Action



Problem Formulation







Machine Learning

Insights & Action



Problem Formulation






• Results suggest that 25-31% of viral load tests could be avoided while maintaining sensitivity for failure detection at or above 95%, for a cost savings of $16-29 per person-month.


Machine Learning

Insights & Action


H2O PlatformPart 4 of 5



H2O Software

H2O is an open source, distributed, Java machine learning library.

APIs are available for:R, Python, Scala & JSON


H2O Software Overview

Speed Matters!

No Sampling

Interactive UI

Cutting-Edge Algorithms

• Time is valuable• In-memory is faster• Distributed is faster• High speed AND accuracy

• Scale to big data• Access data links• Use all data without sampling

• Web-based modeling with H2O Flow• Model comparison

• Suite of cutting-edge machine learning algorithms• Deep Learning & Ensembles• NanoFast Scoring Engine


Current Algorithm Overview Statistical Analysis

• Linear Models (GLM) • Cox Proportional Hazards • Naïve Bayes

Ensembles

• Random Forest • Distributed Trees • Gradient Boosting Machine • R Package - Super Learner

Ensembles

Deep Neural Networks

• Deep Learning • Auto-encoder • Anomaly Detection • Deep Features • Feed-Forward Neural Network

Clustering

• K-Means

Dimension Reduction

• Principal Component Analysis

Solvers & Optimization

• Generalized ADMM Solver • L-BFGS (Quasi Newton

Method) • Ordinary Least-Square Solver • Stochastic Gradient Descent

Data Munging

• Plyr • Integrated R-Environment • Slice, Log Transform


H2O on Amazon EC2

H2O can easily be deployed on an Amazon EC2 cluster. The GitHub repository contains example scripts.


H2O R Interface

R code example: Load data


H2O R Interface

R code example: Train and Test a GBM


H2O Flow Interface


Oncology DemoPart 5 of 5



Breast Cancer DiagnosticsProblem

Data

• Goal is to accurately diagnose breast masses solely on a Fine Needle Aspiration (FNA).

• Binary outcome: Malignant vs. Benign

• Predictor variables describe characteristics of the cell nuclei present in the FNA image.

• radius, texture, perimeter, area, smoothness, compactness, concavity, concave points, symmetry, fractal dimension (10 attributes).

• Calculate mean, standard error and “worst” for each for a total of 30 features.

• Each training example is labeled as “Malignant” or “Benign.”

Source: http://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29

http://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29


Predict Malignancy

Problem Formulation

• Outcome of interest: Malignancy (0/1)• Machine Learning task: Binary Classification• Sampling units: Randomly sampled patients who

have had a fine needle aspirate (FNA) image taken.

• Each patient in the training set contributes a digitized image of a FNA of a breast mass.

• Nuclear feature extraction: In each image, calculate the mean, standard error and largest values for attributes such as “radius” and “texture.”

• Train several machine learning models (in H2O)• Model evaluation and comparison• Models are highly accurate (98-99%)

• This work was turned into software called “Xcyt”, which estimates of the probability of malignancy for each case.

• The Xcyt system has been used at University of Wisconsin Hospitals.


Machine Learning

Insights & Action


Breast Cancer Data in H2O Flow


Live H2O Demo!


Where to learn more?

• H2O Online Training (free): http://learn.h2o.ai• H2O Slidedecks: http://www.slideshare.net/0xdata• H2O Video Presentations: https://www.youtube.com/user/0xdata• H2O Community Events & Meetups: http://h2o.ai/events• Machine Learning & Data Science courses: http://coursebuffet.com

H2O.ai Machine Intelligence Customers • Community • Evangelists

35 SpeakersTraining

2-Full Days

Nov. 9 - 11

http://world.h2o.ai20% Discount code:

h2ocommunity

http://world.h2o.ai


Questions?

@ledell on Twitter, GitHub [email protected]

http://www.stat.berkeley.edu/~ledell

machine learning in modern medicine with erin ledell at stanford med

Software

machine intelligence

medicine h2o machine

stanford university

digital health data

future of medicine

big data

health research

data mining author