![Page 1: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/1.jpg)
FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS
Di Tang1, Zhe Zhou2, Yinqian Zhang3, Kehuan Zhang1
The Chinese University of Hong Kong1
Fudan University2
The Ohio State University3
![Page 2: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/2.jpg)
Face-based Authentication Will Become Popular
2
Online payment
Door entrance
ATM withdraw
Phone unlock
![Page 3: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/3.jpg)
� Easy-obtained faces
3
Face Recognition Is Not Enough
![Page 4: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/4.jpg)
4
![Page 5: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/5.jpg)
� Easy-obtained faces
� High-resolution printers/screens
� Powerful CPUs/GPUs
� Developed technologies
5
Face Recognition Is Not Enough
![Page 6: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/6.jpg)
6
Liveness Detection Is Necessary
Detect whether the subject under authentication is a real human
![Page 7: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/7.jpg)
7
Liveness Detection Is Hard to Be Done Right
Texture extraction methods:
- Local Binary Pattern (LBP) - 2D Fourier Spectra
- …
High-resolution screen will fail it. ---- It can outputs any patterns you want
![Page 8: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/8.jpg)
8
Challenge-response protocols:
- Eye blink
- Expression
- Head movement
- Speaking
Liveness Detection Is Hard to Be Done Right
![Page 9: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/9.jpg)
9
Human Reaction Time
Machines can do109 flops, in 260MS
www.humanbenchmark.com
![Page 10: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/10.jpg)
10
Machines Are Powerful
3D reconstruction
Face morphing
![Page 11: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/11.jpg)
11
Machines Are Powerful
Expression synthesizing
![Page 12: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/12.jpg)
12
Fundamental Problem ?
![Page 13: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/13.jpg)
13
Fundamental Problem ?
No strong security guarantee!
Details
Trembling Ability
Precision
![Page 14: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/14.jpg)
14
Weakness of Human Reactions
Limited speed Uncertainty Smart device + Screen can fail it
2D dynamic attacks (e.g., Media-based Facial Forgery)
![Page 15: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/15.jpg)
15
What We Want to Do?
Solid stone to build a secure protocol
Human reaction
Relieve threats from 2D dynamic attacks
![Page 16: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/16.jpg)
16
Light reflection
Non-digital physical
![Page 17: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/17.jpg)
17
Features of Light Reflection
Fastest in the universe -- No computers can generate fake responses at the same speed, no matter how powerful it will be Without human reaction Can capture rich information -- 3D shape -> eyes, nose -- Texture -> skin vs. non-skin
![Page 18: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/18.jpg)
E: Illumination R: Reflectance S: Sensor response function λ : Wave length x : position of a given point
We will separately consider R,G,B channels. There are no inter-effect among them, if we use the raw data (before AWB).
Reflection Model
18
![Page 19: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/19.jpg)
19
E: Incoming light R: Reflectance
Get reflectance: Get illumination:
The reflections is determined by incoming light Without knowing the incoming light, it is impossible to pre-calculate the reflected light.
Reflection Model
To check face To check time
![Page 20: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/20.jpg)
20
Things to verify: 1. Response time 2. Face information 3. Expressions
Design
![Page 21: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/21.jpg)
21
� Challenging!!
� Reflections happen at speed of light
� But camera is not
� Limited by the refreshing speed
� à around 30 fps
� Does it mean powerful attackers with high speed camera and displaying devices can bypass?
Verifying the Timing is Difficult
![Page 22: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/22.jpg)
22
WORKING DETAILS OF CAMERA
Working Details of Camera
![Page 23: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/23.jpg)
23
WORKING DETAILS OF CAMERA
Working Details of Camera
column column column
column
Col 0
Col 1
Col 2
Col 3
Col N-1
Col N
Anytime, there are always sensors awake
![Page 24: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/24.jpg)
24
Detecting tiny differences in time is possible
![Page 25: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/25.jpg)
25
Both camera and LCD monitor work in a scanning pattern. So what will happen?
Working Details of Screen
Assumption: No modification can be added to the buffer that is being displayed
![Page 26: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/26.jpg)
26
Partially Captured Images
Camera Screen
![Page 27: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/27.jpg)
27
How to verify?
![Page 28: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/28.jpg)
Lighting challenge Background challenge
28
Challenges
![Page 29: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/29.jpg)
29
Response and Challenge
Get challenges:
Lighting area Lighting area
Mirror
![Page 30: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/30.jpg)
30
Calculate the Location
The Challenge image (with lighting area)
Corresponding region
Camera
Forgery -> Delay -> Wrong location
Accumulation:
![Page 31: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/31.jpg)
31
Get reflectance:
Put it into a Neural network for classification
Face Feature Verification
![Page 32: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/32.jpg)
Evaluation
![Page 33: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/33.jpg)
33
SENSITIVITY TO FAKE RESPONSES
Sensitivity to Forged Responses
![Page 34: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/34.jpg)
34
Timing: Camera VS. Mirror
Mirror’s
Laptop’s
![Page 35: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/35.jpg)
35
Face Feature
![Page 36: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/36.jpg)
36
Robustness
![Page 37: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/37.jpg)
Our method will force adversaries to use “3D Dynamic Attack” which is more expensive
Our method could not handle 3D dynamic attack
twins, silicone masks
31
Discussion
![Page 38: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/38.jpg)
Our implementation just used 8 different colors Our implementation needs several seconds to accomplish once authentication Using ‘albedo curve’ may handle 3D dynamic attacks Combine with face recognition algorithm could enhance efficiency and effectiveness
32
Discussion
![Page 39: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/39.jpg)
Face Flashing protocol Effective and efficient method on timing and face verifications
Prototype and empirical evaluations 33
Summary
![Page 40: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/40.jpg)
Q & A
![Page 41: FACE FLASHINGwp.internetsociety.org/ndss/wp-content/uploads/sites/25/...FACE FLASHING: A SECURE LIVENESS DETECTION PROTOCOL BASED ON LIGHT REFLECTIONS Di Tang1, Zhe Zhou2, Yinqian](https://reader031.vdocuments.net/reader031/viewer/2022022014/5b419e697f8b9a472b8b493b/html5/thumbnails/41.jpg)
Thanks