naveen garg, md - amdisamdis.org/wp-content/uploads/2019/06/imaging-ai_garg.pdf · naveen garg, md...
TRANSCRIPT
![Page 1: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/1.jpg)
Naveen Garg, MDAssociate Professor
Abdominal Imaging Section
Department of Diagnostic Radiology
Artificial Intelligence in Radiology
![Page 2: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/2.jpg)
• Owner of Garglet LLC
• Advisor for Enlitic Inc.
Disclosures
![Page 3: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/3.jpg)
The possibility of AI doing radiology
![Page 4: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/4.jpg)
Tesla autopilot
![Page 5: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/5.jpg)
maskrcnn
![Page 6: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/6.jpg)
AI Winters
![Page 7: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/7.jpg)
Increases in computing power
![Page 8: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/8.jpg)
NVIDIA Stock price
![Page 9: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/9.jpg)
Rise of Cloud computing
![Page 10: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/10.jpg)
Venture Capital Funding of medical imaging AI companies over time
![Page 11: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/11.jpg)
Number of Medical Imaging AI Companies Founded by Year
![Page 12: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/12.jpg)
AI Investments over time
![Page 13: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/13.jpg)
Venture Capital funding of AI in radiology
• 2017 - $270 million
• 2018 - $580 million
![Page 14: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/14.jpg)
Commercial software for Radiology AI
![Page 15: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/15.jpg)
ACR AI Lab
![Page 16: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/16.jpg)
Bone age from hand x-rays
![Page 17: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/17.jpg)
Detecting Pneumonia
![Page 18: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/18.jpg)
Starting to see FDA Approvals
![Page 19: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/19.jpg)
Pneumothorax triage
![Page 20: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/20.jpg)
![Page 21: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/21.jpg)
Youtube dataset
![Page 22: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/22.jpg)
Patient data across hospitals in epic
![Page 23: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/23.jpg)
Facebook Libra stablecoin
![Page 24: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/24.jpg)
HPC: inhouse vs. cloud
![Page 25: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/25.jpg)
XSEDE national supercomputing center
![Page 26: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/26.jpg)
Fast.ai
![Page 27: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/27.jpg)
Milestones in Artificial Intelligence
• Computers beat humans at chess (1997)
– IBM Deep Blue vs. Kasparov
• Computers beat humans at jeopardy (2011)
– IBM Watson vs. Rutter and Jennings
• Computers beat humans at go ! (2016)
– Google DeepMind's AlphaGo (version: Lee)[56] defeated Lee Sedol 4–1.
• AlphaZero beats StockFish at chess (2017)
![Page 28: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/28.jpg)
Milestones in AI datasets
• MNIST: 70,000 handwritten digits (1998)
• Imagenet: 14 million hand annotated images (2009)
• DeepLesion: 40,000 recist tumor measurements (2018)
• MIMIC-CXR: 371,000 chest xrays (2019)
![Page 30: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/30.jpg)
Data
Source Since millions of reports gigabytes
Radiology 2000 6.3 9
Pathology 2000 7.8 7
Transcribed Documents 1993 16.4 121
Labs 2000 269 90
![Page 31: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/31.jpg)
Data warehouse
– Electronic health records database
• Radiology reports
• Pathology reports
• H/P’s, op notes, discharge summaries etc.
• Labs
– Dicom
• headers
• presentation states / annotations
• Images
![Page 32: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/32.jpg)
Example Data lake for Radiology AI
– 1.5 million patients
– 10 million radiology reports
– 2 million presentation states
– 5 million annotated key images
![Page 33: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/33.jpg)
Server
Servervirtual processors RAM
Terabytes of storage video ram cuda cores
Garglab 32 768 gb 10 12 gb 2500
![Page 34: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/34.jpg)
Features
• Cohort Identification
• Natural Language processing
• Sentence analysis
• Numeric range search
• Lesions size
• site of radiation
• relative time points
![Page 35: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/35.jpg)
Rad Path Cross Search
![Page 36: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/36.jpg)
Clinical Documents and Labs
![Page 37: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/37.jpg)
![Page 38: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/38.jpg)
![Page 39: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/39.jpg)
Sample Queries
• melanoma – Path proven
– treated with ipilimumab
– had an FDG PET-CT.
• Liver masses– Ct abdomen with contrast
– liver lesions 1 to 2 cm in size
• breast cancer – Recurrent
– Invasive ductal
– Triple negative
– Family history of colon cancer
• Cholangiocarcinoma– Had radiation to the liver
– Mri before and after radiation
![Page 40: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/40.jpg)
![Page 41: Naveen Garg, MD - AMDISamdis.org/wp-content/uploads/2019/06/Imaging-AI_Garg.pdf · Naveen Garg, MD Associate Professor Abdominal Imaging Section Department of Diagnostic Radiology](https://reader030.vdocuments.net/reader030/viewer/2022041105/5f0714447e708231d41b32ce/html5/thumbnails/41.jpg)