tutorial on medical image retrieval - application...
TRANSCRIPT
Tutorial on Medical Image Retrieval- application domains-
Medical Informatics Europe 2005
28.08.2005
Henning Müller, Thomas Deselaers
Service of Medical InformaticsGeneva University & Hospitals, SwitzerlandAachen Technical University, Germany
2©2005 Hôpitaux Universitaires de Genève
Overview
•• Current applicationsCurrent applications
•• Tools to manage archivesTools to manage archives•• Semi-automatic coding, DICOM header correction, Semi-automatic coding, DICOM header correction, ……
•• TeachingTeaching•• Access to teaching files for lecturersAccess to teaching files for lecturers
•• …… and for students and for students
•• ResearchResearch•• Find good examples, quality controlFind good examples, quality control
•• Include visual features into studiesInclude visual features into studies
•• Diagnostic aidDiagnostic aid•• Very focused domain, evidence-based medicine, case-basedVery focused domain, evidence-based medicine, case-based
reasoningreasoning
•• Example systems and fieldsExample systems and fields
•• OthersOthers
3©2005 Hôpitaux Universitaires de Genève
Current applications
•• This should rather be This should rather be emptyempty
•• No programs for visual information retrieval areNo programs for visual information retrieval arecurrently used in clinical routine, at least to mycurrently used in clinical routine, at least to myknowledgeknowledge•• Assert on lung image retrievalAssert on lung image retrieval
•• IRMA in image classification and semi-automatic codingIRMA in image classification and semi-automatic coding
•• Research applications Research applications and large number ofand large number ofprojectsprojects•• MelanomaMelanoma
•• Pathology slidesPathology slides
•• Mammography, lung Mammography, lung CTsCTs
•• PACS-like databasesPACS-like databases
4©2005 Hôpitaux Universitaires de Genève
Tools to manage archives
•• NavigationNavigation in large archives in large archives
•• Find lost images (without/with wrong annotations)Find lost images (without/with wrong annotations)
•• DICOM is not enoughDICOM is not enough
•• Semi-automatic Semi-automatic codingcoding
•• Propose codes of visually similar imagesPropose codes of visually similar images
•• Quality controlQuality control
•• Control the codes and find images with abnormalControl the codes and find images with abnormal
codes based on visual similaritycodes based on visual similarity
•• DICOM headers contain errors (~16% in anatomicDICOM headers contain errors (~16% in anatomic
region) that can be correctedregion) that can be corrected
5©2005 Hôpitaux Universitaires de Genève
Semi-automatic annotation (IRMA)
6©2005 Hôpitaux Universitaires de Genève
Teaching
•• Manage teaching filesManage teaching files
•• myPACSmyPACS.net, MIRC (Medical Imaging Resource Center,.net, MIRC (Medical Imaging Resource Center,
RSNA), HEAL, RSNA), HEAL, PathoPicPathoPic, , ……
•• Resource for Resource for studentsstudents to find and explore to find and explore
databases and casesdatabases and cases
•• Casimage Casimage (used for exams, teaching CDs, (used for exams, teaching CDs, ……))
•• Resource for Resource for lecturerslecturers to find optimal images for to find optimal images for
teachingteaching
•• Share images among lecturersShare images among lecturers
•• Find visually similar images with varying diagnosesFind visually similar images with varying diagnoses
7©2005 Hôpitaux Universitaires de Genève
myPACS (http://www.mypacs.net/)
8©2005 Hôpitaux Universitaires de Genève
MIRC – Medical Image Resource Center
•• http://http://mircmirc..rsnarsna.org/.org/
•• RRadiological adiological SSociety ociety NNorth orth AAmericamerica
•• Ten+ databases are made available for text-Ten+ databases are made available for text-based search in database fields or as free textbased search in database fields or as free text•• Based on Internet standardsBased on Internet standards
•• Software is open sourceSoftware is open source
•• Goal is to create a Goal is to create a worldwide repositoryworldwide repository of cases of casesfor teachingfor teaching
•• Visual retrieval would be a good complement toVisual retrieval would be a good complement tothe textthe text•• Multi-lingual retrieval is currently impossibleMulti-lingual retrieval is currently impossible
9©2005 Hôpitaux Universitaires de Genève
CasImage (http://www.casimage.com/)
10©2005 Hôpitaux Universitaires de Genève
Research
•• Optimize the Optimize the selection of casesselection of cases for research for research
•• Find visually similar casesFind visually similar cases
•• Browse databases through example casesBrowse databases through example cases
•• Find misclassified casesFind misclassified cases
•• Include Include visual features into research studiesvisual features into research studies
•• Find unknown connectionsFind unknown connections
•• Features need to have a rather high levelsFeatures need to have a rather high levels
•• Correspond roughly to diseasesCorrespond roughly to diseases
•• Visual data miningVisual data mining
•• Visual knowledge managementVisual knowledge management
11©2005 Hôpitaux Universitaires de Genève
Diagnostic aid
•• Case-based reasoningCase-based reasoning
•• Evidence-based medicineEvidence-based medicine
•• Supply Supply similar casessimilar cases as a help for practitioners as a help for practitioners•• Has shown to help inexperienced practitionersHas shown to help inexperienced practitioners
•• Aisen Aisen et al., et al., RadiologyRadiology
•• This is possible in fields where visual low-levelThis is possible in fields where visual low-levelsimilarity is importantsimilarity is important•• High resolution lung CTHigh resolution lung CT
•• Dermatology, Pathology, MammographyDermatology, Pathology, Mammography
•• Fractures (treatment planning)Fractures (treatment planning)
•• Problem: Advances in medical imaging equipmentProblem: Advances in medical imaging equipment
12©2005 Hôpitaux Universitaires de Genève
Example: case-based reasoning
?
Emphysema
Micro nodules
Emphysema
Macro nodules
13©2005 Hôpitaux Universitaires de Genève
Assert
•• Diagnostic aid on lung Diagnostic aid on lung CTsCTs
14©2005 Hôpitaux Universitaires de Genève
Dermatology
•• ABCDABCD rule (Asymmetry, Border, Color, rule (Asymmetry, Border, Color,
Differential structures)Differential structures)
•• Hair removal, boundary detection, textureHair removal, boundary detection, texture
analysis, analysis, ……
15©2005 Hôpitaux Universitaires de Genève
UPittsburg, Pathology; IGDS Rutgers
16©2005 Hôpitaux Universitaires de Genève
Mammography
•• Less image retrieval, but rather Less image retrieval, but rather detection ofdetection of
regionsregions with abnormal characteristics with abnormal characteristics
•• micro calcificationsmicro calcifications
•• Local analysis is importantLocal analysis is important
•• Large databases with Large databases with preclassified preclassified image regionsimage regions
existexist
•• England: England: MammogridMammogrid
17©2005 Hôpitaux Universitaires de Genève
Case-based rather than image-based retrieval
•• Currently the input is mostly one imageCurrently the input is mostly one image
•• MD might have MD might have several imagesseveral images (RX, CT, (RX, CT, ……) for a) for a
same patientsame patient
•• Cases stored in the patient record also oftenCases stored in the patient record also often
have more than one imagehave more than one image
•• Plus other data: text, numerical values (lab)Plus other data: text, numerical values (lab)
•• Also, Also, entire seriesentire series (CT, MRI) as an input and not (CT, MRI) as an input and not
selected imagesselected images
•• Slice selection based on what a Slice selection based on what a ““normalnormal”” image would image would
be likebe like
18©2005 Hôpitaux Universitaires de Genève
Other applications
•• Parameter settingsParameter settings for segmentation, etc. for segmentation, etc.
•• Based on a large number of known, well-segmentedBased on a large number of known, well-segmented
casescases
•• Show me if this case needs further attention,Show me if this case needs further attention,
dissimilaritydissimilarity retrieval against healthy cases retrieval against healthy cases
•• Needs a large number of healthy casesNeeds a large number of healthy cases
•• Create a model for a Create a model for a ““healthyhealthy”” image image
•• ……
19©2005 Hôpitaux Universitaires de Genève
Conclusions
•• Image retrieval is at the moment mainly anImage retrieval is at the moment mainly an
academicacademic problem problem
•• Information explosions is happening in the medicalInformation explosions is happening in the medical
domain (multimedia in digital form)domain (multimedia in digital form)
•• We need We need toolstools and we need to imagine how to use and we need to imagine how to use
themthem
•• There are There are many applicationsmany applications for image retrieval for image retrieval
•• We need to start the clinical We need to start the clinical integrationintegration
•• Visual systems will not replace text butVisual systems will not replace text but
complement itcomplement it