Machine Learning for Medical Image Analysis:

What, where and how?

Dr. Debdoot SheetAssistant Professor

Department of Electrical EngineeringIndian Institute of Technology Kharagpur

www.facweb.iitkgp.ernet.in/~debdoot/

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 2

A GREAT PIECE OF CAREER ADVICE FOR EECS GRADUATES INTERESTED IN MACHINE VISION

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 3

Market Scenario and Career

Media, surveillance, automotive, graphics,

etc.($ 6 Billion)63%

Medical Image Ana-lysis

($ 3.5 Billion)37%

Market for Machine Vision Systems in 2020

24 June 2015

Modality• X-ray• Ultrasound• Computed Tomography

(CT)• Magnetic Resonance (MRI)• Nuclear Imaging (PET &

SPECT)Clinical Indications• Radiology• Cardiology• Oncology• Neurology• Obstetrics & gynecology• Breast mammographyEnd Users• Hospitals• Diagnostic centers• Research centersReport code: HIT 1309 and SE 2701 from www.marketsandmarkets.com

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 4

Medical Image Analysis (MedIA)?

• Is not Medical Imaging• Is not Medicine• Is not Image Processing• Is not Computer Vision• Is not Machine Learning• Is not Software Engineering• Is not Project Management• Is not Creating Enterprises24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 5

Overview of MedIA

• Prediction• Example based

learning• Complex

reasoning

• Denoising• Segmentation• Feature

representation• Visualization

• Tissue-energy interaction

• Image formation• Statistics

• Modalities – CT, MRI, US, Microscopy, OCT

• Organ appearance

Medical ImagingPhysics

Machine LearningImage

Processing and

Graphics

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 6

Last 35 years of MedIA• Pre 1980 – 1984:

Era of Pattern Recognition Analysis of 2D Images

• 1985 – 1991: Knowledge based Approaches

• 1992 – 1998: 3D Images and Towards Integrated Analysis

• 1999 – 2010: Machine Learning with Shallow Reasoning

• 2010 and Beyond: Machine Learning with Complex Reasoning

24 June 2015

Duncan, J.S.; Ayache, N., "Medical image analysis: progress over two decades and the challenges ahead," IEEE Trans. Pat. Anal., Mach. Intell., vol.22, no.1, pp.85,106, Jan. 2000

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 7

Key Research (Business) Areas

• Macro - meso - micro imaging

• Medical image registration

• Organ localization

• Organ segmentation

• Medical visualization and

Augmented reality

• Virtual anatomy

• Digital angiography

• Optical and ultrasonic despeckling

• 3D optical microscopy

• Digital pathology

• Computational imaging

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 8

End use of MedIA

24 June 2015

Tong, S.; Sheet, D.; Bhuiyan, S.; Zequer Diaz, M.; Taberne, A., "BME Trends Around the World : From Baby X to frugal technologies, here's what biomedical engineers are excited about in 2015. [From the Editors]," IEEE Pulse, vol.6, no.1, pp.4-6, Jan.-Feb. 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 9

MedIA Challenges in 2015• Endoscopic vision for

Computer Assisted Intervention (CAI)

• Statistical shape modeling

• Liver ultrasound tracking

• Gland segmentation in histology

• Retinal cyst segmentation

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 10

MedIA Challenges in 2015

24 June 2015

www.gra

nd-chall

enges.o

rg

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 11

CAREER ADVICES FOR MedIA PHD ASPIRANTS

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 12

Find a (Research) Challenge• Grand Challenges in Biomedical Image Analysis

– www.grand-challenges.org

• MICCAI– www.miccai.org– www.miccai2015.org

• ISBI– www. biomedicalimaging.org

• SPIE Medical Imaging– 2015 will host a grand-challenge

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 13

Tool(boxes) of the Trade• BioDigital Human

– www.biodigital.com

• MeVisLab – Medical image processing, analysis and visualization SDK– http://www.mevislab.de/

• Python (x,y)– Python 2.7 with scientific computing library for custom building tools– https://code.google.com/p/pythonxy/

• Tortoise SVN– Version control for managing large file software projects– www.tortoisesvn.net/

• MicroDicomViewer– Standalone Viewer for Dicom files– www.microdicom.com/downloads.html

• CUDA– Library for using NVIDIA GPUs

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 14

Where to Read for MedIA?

Specialist Literature• Medical Image Analysis• IEEE Trans. Medical

Imaging• IEEE Trans.

Computational Imaging• IEEE J. Biomedical and

Health Informatics• SPIE J. Medical Imaging• Computerized Medical

Imaging and Graphics

General Reading

• IEEE Trans. Image Processing

• IEEE Trans. Biomedical Engineering

• Int. J. Computer Vision• IET Image Processing• IET Computer Vision• ACM Trans. Graphics• Proc. Int. Conf. Image.

Proc.24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 15

Where to Read for ML?

Journal• IEEE Trans. Pattern Analysis

and Machine Intelligence• Machine Learning• J. Machine Learning

Research• IEEE Trans. Knowledge and

Data Engineering• IEEE Trans. Neural Networks• IEEE Trans. Sys. Man. Cyber.

Conferences• Computer Vision and Pattern

Recognition (CVPR)• Machine Learning confs.

– International (ICML)– European (ECML)– Asian (ACML)

• Neural Information Processing System (NIPS)

• Computer Vision conf.– International (ICCV)– European (ECCV)– Asian (ACCV)

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 16

Where to Listen and Socialize?Workshops and Schools• Medical Imaging Summer

School (MISS)– MICCAI Society

• IEEE SPS Summer School on Biomedical Image Analysis– IEEE Signal Processing Soc.

• IEEE EMBS Int. Summer School on Biomedical Imaging– IEEE Engineering in Medicine

and Biology Society

• Biomedical Image Analysis Summer School

Conferences• Medical Image Computing and

Computer Assisted Intervention (MICCAI)

• Int. Symp. Biomed. Imaging (ISBI)• SPIE Med. Imaging• Information Processing in Medical

Imaging (IPMI)• Information Processing in

Computer-Assisted Interventions (IPCAI)

• Microscopy Conference (MC)• Int. Conf. Sys., Med. Biol. (ICSMB)

– In IIT Kharagpur, Jan 2016

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 17

Some of the (BIG Group) Leaders• Alejandro Frangi

(Sheffield)• Alison Noble (Oxford)• Badrinath Roysam

(Houston)• Ben Glocker (Imperial) • Bram van Ginneken

(Nijmegen)• Gabor Fichtinger

(Queens)• James Duncan (Yale) • Michael Unser (EPFL)

• Michael Liebling (UCSB) • Milan Sonka (Iowa) • Nassir Navab

(TUM+JHU) • Nicholas Ayache (INRIA

Sophia Antipolis Cedex) • Pierre Jannin (INRIA

Rennes) • Polina Golland (MIT) • Terry Peters (Robarts) • Wiro Nissen (Erasmus

MC)24 June 2015

Biomedical Imaging and Graphics

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 18

Where to Publish?Conferences (Early Track)• MICCAI• ISBI• SPIE Medical Imaging• IPCAI• IPMI• CVPR• ICCV / ECCV / ACCV• ICIP

Journal (Mature)

• Medical Image Analysis

• IEEE Trans. Med. Imaging

• IEEE J. Biomed. Health Inf.

• IEEE Trans. Biomed. Engg.

• Comp. Med. Imag. Graph.

• SPIE Med. Imaging• BMC Medical Imaging• Int. J. Biomed. Imaging

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 19

Where to Look for (Creating) Jobs?

Incubation, Startup Grants• Technology Business Incubation

(TBI) – DSIR– Promoting Innovations in

Individuals, Start-ups and MSMEs (PRISM)

• Cat I – Rs. 5 lcs (Idea pitchup)• Cat II – Rs. 5-35 lcs (Tech.

Demonstrator)

• Biotechnology Industry Research Assistance Council (BIRAC)

– Biotechnology Ignition Grant (BIG)• Rs. 50 lcs (Tech. Demonstrator)

• Indo-US Science and Technology Forum (IUSSTF)

– India+US Tech. Starter Endowment• $ 400,000 (Rs. 2.5 Cr)

Investments• Mix of Loan + Grant-in-Aid

– BIRAC + Department of Electronics and Information Technology (DEITY)

• Success of BIG• 50% Grant to University + 50%

loan to Start-up• Rs. 3 Cr. (Tech.

Commercialization)

• Angels and Investors– India Angels Network– Khosla Ventures– Tata Ventures– The Indus Entrepreneurs (TiE)– Corporate Buy-outs

24 June 2015

Intro to Machine Learning for Medical Image Analysis [Debdoot Sheet] - WMLMIA 20

Take home messageSome Texts• Toennies, Guide to medical image

analysis, 2012.• Bankman, Handbook of Medical

Image Processing and Analysis, 2012.

• Dougherty, Medical Image Processing techniques and applications, 2013.

• M. N. Gurcan, "Histopathological Image Analysis: A Review", IEEE Rev. Biomed. Engg., vol. 2, pp. 147-171, 2009.

• J. A. Noble, “Ultrasound Image Segmentation: A Survey”, IEEE Trans. Med. Imaging, vol. 25, no. 8, pp. 987-1010, 2006.

MedIA in Classroom

• IIT Kharagpur: http://www.bit.do/eemia

• IIT Bombay: http://www.cse.iitb.ac.in/~suyash/cs736/

• Carnegie Mellon: http://www.cs.cmu.edu/~galeotti/methods_course/

• TU Munich: http://campar.in.tum.de/Chair/TeachingWiSe2014CAMPONE

24 June 2015