advanced interaction techniques for medical intervention and diagnosis
DESCRIPTION
Final thesis presentation at KU Leuven (Master of Science in Electrical Engineering)TRANSCRIPT
![Page 1: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/1.jpg)
Advanced Interaction Techniques for Medical Intervention and Diagnosis
Chris DesomerLars Jacobs
Prof. Dr. Ir. P. Suetens Prof. Dr. Ir. T. Tuytelaars
Dr. Ir. S. De BuckDr. Ir. P. SlagmolenDr. Ir. J. GemmekeDr. Ir. M. Pedersoli
Supervisors:
Advisors:
Assessors:
![Page 2: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/2.jpg)
07/04/2023 Slide 2 of 32
Outline• Context• Autostereoscopic visualisation• Gesture interface• Evaluation• Discussion & conclusion• Demo
![Page 3: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/3.jpg)
Context
![Page 4: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/4.jpg)
07/04/2023 Slide 4 of 32
Introduction (I)
![Page 5: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/5.jpg)
07/04/2023 Slide 5 of 32
Introduction (II)
![Page 6: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/6.jpg)
07/04/2023 Slide 6 of 32
Problem statement
![Page 7: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/7.jpg)
07/04/2023 Slide 7 of 32
Purpose • Implementation of autostereoscopic visualisation
• Implementation of gesture interface
• Added value?
![Page 8: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/8.jpg)
Autostereoscopic visualisation
![Page 9: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/9.jpg)
07/04/2023 Slide 9 of 32
3D perception • Humans sees 3D if each eye gets another image
![Page 10: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/10.jpg)
07/04/2023 Slide 10 of 32
Lenslet
![Page 11: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/11.jpg)
07/04/2023 Slide 11 of 32
Multiview
![Page 12: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/12.jpg)
07/04/2023 Slide 12 of 32
2D+Depth
![Page 13: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/13.jpg)
07/04/2023 Slide 13 of 32
Real time aspects • Network optimisation
• Code optimisationo Block based instead of pixel basedo memcpy(), memset()
• Hardware optimisation
![Page 14: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/14.jpg)
07/04/2023 Slide 14 of 32
4D visualisation • 3D + time
• Demo 4D
![Page 15: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/15.jpg)
Gesture interface
![Page 16: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/16.jpg)
07/04/2023 Slide 16 of 32
Hardware/framework gesture interface• Advanced interaction 3D hand/finger detection required• Kinect
o Far field detectiono Frameworks for 3D
hand/finger detection (e.g. Forth)
• Leap Motiono Near field detectiono Designed for 3D hand/finger
detection
![Page 17: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/17.jpg)
07/04/2023 Slide 17 of 32
Comparison Kinect versus Leap MotionKinect Leap Motion
Price € 250 € 901
Position in the OR Outside sterile zone Inside sterile zone2
Interaction zone 57° horizontal Forth: 2m distance from sensor gives a
range of ≈ 2m
150° vertical 0,25 m distance from sensor gives a range of
≈ 2mResolution Kinect: 30 fps
Kinect + Forth: 20 fps 115 fps
Accuracy in ideal interaction zone 2cm 0,01mm
Conclusion: Leap Motion
1 Excl. price of sterilization 2 Sterilization can be done by a protective cover for ultrasound transducers
![Page 18: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/18.jpg)
07/04/2023 Slide 18 of 32
Requirements Requirement Gesture Translation Closed hand movement
Scaling Pinch to zoom
Rotation Open hand rotation
Reset Swipe
On/off mechanism Virtual push button
Settings Static gesture
![Page 19: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/19.jpg)
07/04/2023 Slide 19 of 32
Leap Motion
Gesture recognition
using heuristics
Action
Flow
Leap Motion
Gesture recognition using
correlation
Action
Data frame
Parameters
![Page 20: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/20.jpg)
07/04/2023 20
Virtual push button
![Page 21: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/21.jpg)
Evaluation
![Page 22: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/22.jpg)
07/04/2023 Slide 22 of 32
Data models• Meshes
• Tracts
• Volume data o Filtering
![Page 23: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/23.jpg)
07/04/2023 Slide 23 of 32
Data models• Meshes
• Tracts
• Volume data o Filtering
![Page 24: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/24.jpg)
07/04/2023 Slide 24 of 32
Data models• Meshes
• Tracts
• Volume data o Filtering
![Page 25: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/25.jpg)
Autostereoscopic visualisation
![Page 26: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/26.jpg)
07/04/2023 Slide 26 of 32
Gesture interfaceGesture Intuitiveness DifficultyTranslation 3.4 3.0
Scaling 4.2 3.0
Rotation 3.5 2.9
Swipe 4.7 4.2
Airbutton 4.3 3.6
Static gestures 3.6 3.5
1: very unintuitive/very difficult5: very intuitive/very easy
![Page 27: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/27.jpg)
07/04/2023 Slide 27 of 32
System Usability Scale (SUS)
50% gives at least a SUS score of 74 (≈70%)25% gives at least a SUS score of 79 (≈88%)
An average system has a SUS score of 68
![Page 28: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/28.jpg)
Discussion & conclusion
![Page 29: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/29.jpg)
07/04/2023 Slide 29 of 32
Discussion (I)• Autostereoscopic visualisation
+ Added value- Low resolution
• Gesture interface+ Dynamic 3D gesture set+ Intuitive- Training needed
![Page 30: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/30.jpg)
07/04/2023 Slide 30 of 32
Discussion (II)• Forensic science
o Virtual crime sceneo No sterility
• Cardiovascular surgery o Combination = added valueo Catheter visualisation
![Page 31: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/31.jpg)
07/04/2023 Slide 31 of 32
Conclusion• First general prototype
o Autostereoscopic visualisationo Gesture interface
• Nexto Application specific prototypeo Optimisations per medical domain
![Page 32: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/32.jpg)
Demo
![Page 33: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/33.jpg)
![Page 34: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/34.jpg)
07/04/2023 Slide 34 of 32
References (I)• Article, R., & Ar, W. I. S. (2013). Augmented Reality in
Surgery, 139.• Dodgson, N. A. (2005). Autostereoscopic 3D displays.
Computer, 38(8), 31–36. doi:10.1109/MC.2005.252• Delude, C., Is Heart Surgery History?, 2007, (‘http://
protomag.com/assets/is-heart-surgery-history?page=4’)• Ector, B. J., Buck, S. De, Sc, M., Ph, D., Heidbuchel, H., &
Ph, D. (2008). Improved Efficiency in the EP Lab with syngo DynaCT Cardiac, (October), 26–31.
![Page 35: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/35.jpg)
07/04/2023 Slide 35 of 32
References (II)• Oikonomidis, I., Kyriazis, N., & Argyros, A. (2011). Efficient
model-based 3D tracking of hand articulations using Kinect. Procedings of the British Machine Vision Conference 2011, 101.1–101.11. doi:10.5244/C.25.101
• Ruijters, D. (2008). Integrating autostereoscopic multi-view lenticular displays in minimally invasive angiography. Workshop proceedings, 87–94. Retrieved from https://lirias.kuleuven.be/handle/123456789/198103
• Sansoni, G., & Docchio, F. (2013). Biomedical 2D and 3D Imaging : State of Art and Future Perspectives, 3–19.
• Solutions, P. (n.d.). 3D Interface Specifications White Paper.
![Page 36: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/36.jpg)
07/04/2023 Slide 36 of 32
2D+Depth
![Page 37: Advanced Interaction Techniques for Medical Intervention and Diagnosis](https://reader033.vdocuments.net/reader033/viewer/2022061104/53f4b0828d7f728e318b48ec/html5/thumbnails/37.jpg)
07/04/2023 Slide 37 of 32
Hardware optimisations Windows 8 laptop
Windows 7 desktop
iMac
CPU Intel Core i7 Q720 @ 1.60GHz
Intel Core i5 2500 @ 3.30GHz
Intel Core i5 4670 @ 3.40GHz
RAM memory 8Gb SO-DIMM DDR3 @ 533MHz
16Gb DDR3 @ 666MHz
32Gb DDR3 @ 1600MHz
GPU AMD Radeon HD6570M 1Gb RAM
AMD Radeon HD6450 1Gb RAM
NVIDIA GeForce GTX 780M 4GB RAM
Refresh rate ± 2.5 fps ± 9 fps ± 20 fps