Virtual and augmented reality

Gabriel Hanssen Kiss Research coordinator FOR, St Olavs hospital

Researcher CIUS, NTNU

Overview

• Introduce VR/AR

• Medical applications for VR

• AR (in the OR)

• HoloLens

• Demo

VR vs. AR

VR vs. AR

• Computer generated environment• Fully immersive experience• Completely artificial world

• Augments the real world• User not cut off from reality

VR - hardware

VR – principle

1. Lenses 2. Display 3. Processing and tracking

Is it the same as the real-world?

• Depends on hardware, but there is a catch!

Vergence accommodation conflict, Hoffman et al.

Surgical training

NSALK - Norwegian National Advisory Unit on Advanced Laparoscopic Surgery

Surgical simulators

Li et al. - Application of virtual reality technologyin clinical medicine

Virtual operating room

• HTC Vive• Student training• Role play• Remote simulation setup

F. Lindseth et al.

Virtual operating room

F. Lindseth et al.

Pain management morphine or VR?

Hoffman, H.G et al.

Phobia

Be Fearless app

Stroke patients – spatial neglect

Fordell H. et al.

Detection

Fordell H. et al.

Rehabilitation

Fordell H. et al.

Experience ageing – macular hole

https://youtu.be/pOW7oG6bIFI

VR in the OR

GYN - FOR, 2015

+ Improved 3D perception- No direct eye contact with the surgical field

Augmented reality

Lifewire

= computer-generated content overlaid on a real world environment

Augmented reality

Augmented reality is it new?

Sutherland et al., 1968

Early example

State et al.

Liver surgery

Fraunhofer, MEVIS

Neurosurgery

D. Low et al.

Vein finder

Vein finder

https://youtu.be/3xeM-jfJsWg

HoloLens – “the leader in mixed reality”

Microsoft

Liver surgery planning

Intervensjonssenteret

AR in remote emergency room

University of Maryland – ultrasound acquisitions

Intraoperative AR

Scopis medical

https://youtu.be/xvbWE4OsKxY

HeartPad

Illustration by Steve Ford

HeartPad

Illustrations by Steve Ford

Model fusion under acquisition

Fusion principle

Real-time fusion in the echo lab

Student test

Student test

HoloLens implementation

HoloLens implementation

HoloLens implementation

Plans

HoloLens alternatives

Google

Apple - rumor

ODG

Meta

Sony, SmartEye glasses

Demo

Demo