![Page 1: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/1.jpg)
Gaze-Dependent Simulation of Light Perception
in Virtual Reality
Research Division of Computer Graphics
Institute of Visual Computing & Human-Centered Technology
TU Wien, Austria
ISMAR 2020
Laura R. Luidolt1
Michael Wimmer1
Katharina Krösl2
1TU Wien, 2VRVis Forschungs-GmbH1 2
![Page 2: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/2.jpg)
Motivation ▪
Overview ▫
Methodology ▫
Evaluation ▫
Conclusion ▫
Introduction
L. R. Luidolt2
<brightness range
tone mapping
![Page 3: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/3.jpg)
Motivation ▪
Overview ▫
Methodology ▫
Evaluation ▫
Conclusion ▫
Introduction
L. R. Luidolt3
![Page 4: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/4.jpg)
Motivation ▪
Overview ▫
Methodology ▫
Evaluation ▫
Conclusion ▫
Introduction
▪ → Perceptual algorithms necessary!
▪ Medically based
▪ Account for viewing direction, pupil size
L. R. Luidolt4
![Page 5: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/5.jpg)
Motivation ▪
Overview ▫
Methodology ▫
Evaluation ▫
Conclusion ▫
Contribution
▪ Post-processing workflow
▪ Accurate simulation of light perception in VR/AR
▪ Medically-based, perceptual effects
▪ In real-time VR/AR
▪ Following optometrist advice
▪ Eye tracking for measuring light incidence
▪ Pilot user study, comparison of
▪ Real-world low-light situation
▪ And VR simulation
L. R. Luidolt5
![Page 6: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/6.jpg)
Motivation ▫
Overview ▪
Methodology ▫
Evaluation ▫
Conclusion ▫
L. R. Luidolt6
Visual adjustment to
bright and dark
Adaptation of rods
and cones over time
Temporal Eye Adaptation
Colorful patterns
when viewing bright
light sources
Scattering of light in
the eye
PerceptualGlare
Blurred details in low
light scenes
Rods not present in
fovea (point of
sharpest vision)
Visual Acuity Reduction
Color shift towards
blue in low light
scenes
Rods more sensitive
to longer wavelength
light than cones
Scotopic Color Vision
Based on Krawczyk et al., 2005and Ritschel et al., 2009
![Page 7: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/7.jpg)
![Page 8: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/8.jpg)
Motivation ▫
Overview ▪
Methodology ▫
Evaluation ▫
Conclusion ▫
L. R. Luidolt0 0 2 1,.4 -0.02 0.15 2.2 2.98
Visual adjustment to
bright and dark
Adaptation of rods
and cones over time
Temporal Eye Adaptation
Colorful patterns
when viewing bright
light sources
Scattering of light in
the eye
PerceptualGlare
Blurred details in low
light scenes
Rods not present in
fovea (point of
sharpest vision)
Visual Acuity Reduction
Color shift towards
blue in low light
scenes
Rods more sensitive
to longer wavelength
light than cones
Scotopic Color Vision
Based on Krawczyk et al., 2005and Ritschel et al., 2009
![Page 9: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/9.jpg)
![Page 10: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/10.jpg)
Motivation ▫
Overview ▪
Methodology ▫
Evaluation ▫
Conclusion ▫
L. R. Luidolt10
Visual adjustment to
bright and dark
Adaptation of rods
and cones over time
Temporal Eye Adaptation
Colorful patterns
when viewing bright
light sources
Scattering of light in
the eye
PerceptualGlare
Blurred details in low
light scenes
Rods not present in
fovea (point of
sharpest vision)
Visual Acuity Reduction
Color shift towards
blue in low light
scenes
Rods more sensitive
to longer wavelength
light than cones
Scotopic Color Vision
Based on Krawczyk et al., 2005and Ritschel et al., 2009
![Page 11: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/11.jpg)
![Page 12: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/12.jpg)
Motivation ▫
Overview ▪
Methodology ▫
Evaluation ▫
Conclusion ▫
L. R. Luidolt12
Visual adjustment to
bright and dark
Adaptation of rods
and cones over time
Temporal Eye Adaptation
Colorful patterns
when viewing bright
light sources
Scattering of light in
the eye
PerceptualGlare
Blurred details in low
light scenes
Rods not present in
fovea (point of
sharpest vision)
Visual Acuity Reduction
Color shift towards
blue in low light
scenes
Rods more sensitive
to longer wavelength
light than cones
Scotopic Color Vision
Based on Krawczyk et al., 2005and Ritschel et al., 2009
![Page 13: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/13.jpg)
![Page 14: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/14.jpg)
Motivation ▫
Overview ▪
Methodology ▫
Evaluation ▫
Conclusion ▫
L. R. Luidolt14
Visual adjustment to
bright and dark
Adaptation of rods
and cones over time
Temporal Eye Adaptation
Colorful patterns
when viewing bright
light sources
Scattering of light in
the eye
PerceptualGlare
Blurred details in low
light scenes
Rods not present in
fovea (point of
sharpest vision)
Visual Acuity Reduction
Color shift towards
blue in low light
scenes
Rods more sensitive
to longer wavelength
light than cones
Scotopic Color Vision
Based on Krawczyk et al., 2005and Ritschel et al., 2009
![Page 15: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/15.jpg)
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▪
Glare ▫
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Temporal Eye Adaptation
L. R. Luidolt15
▪ 𝐿𝑖 = 𝐿𝑖−1 + (𝑌 − 𝐿𝑖−1) ∙
1 − 𝑒− ൗft 𝜏(𝑌)
▪ Target luminance Y
▪ Temporally filtered
luminance Li of frame i
▪ Photoreceptor adaptation
times τ
![Page 16: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/16.jpg)
L. R. Luidolt16Image adapted fromcommons.wikimedia.org/wiki/File:Eyesection.svg
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Perceptual Glare
![Page 17: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/17.jpg)
L. R. Luidolt17Image adapted fromcommons.wikimedia.org/wiki/File:Eyesection.svg
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Perceptual Glare
![Page 18: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/18.jpg)
L. R. Luidolt18Image adapted fromcommons.wikimedia.org/wiki/File:Eyesection.svg
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Perceptual Glare
𝑀 𝑥, 𝑦 =1
𝜆𝑑 2
1
𝑁∙ ℱ 𝑃(𝑥, 𝑦) ∙ 𝑒
i 𝜋𝜆𝑑
𝑥2+𝑦22
After Ritschel et al., 2009
![Page 19: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/19.jpg)
Diffraction on the retina of a
single wavelength light source
Monochromatic PSF
Combination of multiple
wavelengths to simulate spectral
light
Spectral PSF
L. R. Luidolt19
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Perceptual Glare
![Page 20: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/20.jpg)
Diffraction on the retina of a
single wavelength light source
Monochromatic PSF
Combination of multiple
wavelengths to simulate spectral
light
Spectral PSF
L. R. Luidolt20
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
Perceptual Glare
![Page 21: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/21.jpg)
Perceptual Glare
L. R. Luidolt21
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
· (1 - ) + ·
=
![Page 22: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/22.jpg)
Perceptual Glare
L. R. Luidolt22
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▪
VA reduction ▫
Color shift ▫
Evaluation ▫
Conclusion ▫
![Page 23: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/23.jpg)
Visual Acuity Reduction
L. R. Luidolt23
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▫
VA reduction ▪
Color shift ▫
Evaluation ▫
Conclusion ▫
▪ 𝜎 𝐿 = max(1 − 𝐿, 0)
▪ Gaussian variance 𝜎
▪ Pixel’s lightness L
![Page 24: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/24.jpg)
Scotopic Color Vision
L. R. Luidolt24
OnOff
Motivation ▫
Overview ▫
Methodology ▼
Adaptation ▫
Glare ▫
VA reduction ▫
Color shift ▪
Evaluation ▫
Conclusion ▫
![Page 25: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/25.jpg)
Evaluation
Qualitative user study with 5 participants
Motivation ▫
Overview ▫
Methodology ▫
Evaluation ▪
Conclusion ▫
L. R. Luidolt25
![Page 26: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/26.jpg)
Real-time VR/AR post-processing workflow
Using eye tracking
Based on medical research
Pilot user study
ConclusionMotivation ▫
Overview ▫
Methodology ▫
Evaluation ▫
Conclusion ▪
L. R. Luidolt26
▪ temporal eye adaptation▪ perceptual glare▪ visual acuity reduction▪ scotopic color vision
Related article: “CatARact: Simulating Cataracts in Augmented Reality”, Krösl et al., 2020
![Page 27: Gaze-Dependent Simulation of Light Perception in Virtual Reality · 2020. 10. 30. · Gaze-Dependent Simulation of Light Perception in Virtual Reality Research Division of Computer](https://reader035.vdocuments.net/reader035/viewer/2022081403/60a28de730813213f912d493/html5/thumbnails/27.jpg)
Laura R. Luidolt, [email protected]
Michael Wimmer, [email protected]
Katharina Krösl, [email protected]
Gaze-Dependent Simulation of Light
Perception in Virtual Reality
Than
k yo
u f
or
you
r at
ten
tio
n!