Anil Jain

Michigan State University

Feb 10, 2020

http://biometrics.cse.msu.edu/

Fingerprints: Learning Representation for Encrypted Matching

Outline• Biometrics is mainstream

• Growing concerns about privacy issues

• Points of Attack in a biometric system

• Privacy-preserving matching of fingerprints

• Learn a Fixed-length representation for fingerprints

• Implement matching in fully homomorphic encrypted

domain

• Compare with minutiae-based fingerprint SDKs

Biometrics is Mainstream

1:1 match (authentication); 1: N match (search)

Mobile Authentication Border Crossing Forensics

Applications in Israel

https://www.gov.il/en/service/biometric_smart_id_request

ID card with Face and two index fingerprints Speed Gate: West Bank Check posts

https://www.npr.org/2019/08/22/752765606/face-recognition-lets-palestinians-cross-israeli-checkposts-fast-but-raises-conc

Biometrics Recognition System

• Template: A compressed and salient representation of image• False accept rate (FAR): Proportion of imposters accepted• False reject rate (FRR): Proportion of genuine users rejected

FeatureExtractor

TemplateDatabase

Authentication Enrollment

Similarity

computation(Threshold)

Yes/No

Preprocessor Preprocessor

Points of Attack on Biometric Systems

SensorFeature

Extractor MatcherApplication Device

(e.g.,cash dispenser)

StoredTemplates

1. FakeBiometric

2. ReplayOld Data

3. OverrideFeature Extractor

Yes/No

8. Override Final Decision

5. OverrideMatcher

4. SynthesizedFeature Vector

7. Interceptthe Channel

6. ModifyTemplate

Ratha, Connell, Bolle, “Enhancing security and privacy in biometrics-based authentication systems”, IBM Systems Journal, 2001

Security of Biometric Template

R. Cappelli ; D. Maio ; A. Lumini ; D. Maltoni, “Fingerprint Image Reconstruction from Standard Templates”, IEEE Trans. PAMI, 2007

GOODIX IN-DISPLAY FINGERPRINT SENSORTM

ARM TrustZone®Serial Peripheral Interface (SPI)

Encrypt Raw Data

To secure fingerprint,

processing is done in TEE

(Trusted Execution

Environment):

• Image pre-processing

• Feature extraction

• Alignment and recognition

Match on Device

Template attacks are less of a concern; stored in secure chip & never leaves device

Biometrics: Privacy Concerns

• Data collection and use

• User consent; retention policy; data sharing

• What recourse you have if you are incorrectly recognized?

• Data security

• How will the data (template) be protected?

• Government regulations

• Lack of privacy laws with teeth (exception: Illinois BIPA, GDPR)

Security v. Privacy

Argument used by individual agencies or the government as a whole

Social Good v. Privacy

“Aadhaar gives dignity to the marginalized. Dignity tothe marginalized outweighs privacy” - Justice Sikri,Indian Supreme Court (Sept 2018)

Kenya’s High Court Delays Biometrics ID Program

A Kenyan being photographed for a national ID, NY Times, Jan 29, 2020)

….until the government enacts laws to protect the security of the data and prevent discrimination against minorities

Facebook Looses Privacy Suit

• Facebook to pay $550 million for violation of 2008 Biometric

Information Privacy Act (BIPA); settlement would require

user consent for face tagging in photos

• It was fined $5 billion by FTC for lack of privacy & security

measures (USA Today, July 24, 2019)

Fingerprints

Global Level-1

FeaturesLocal Level-2 Features (Minutiae)

cores

deltas

ridge-flow

Fingerprint Alignment

Align fingerprints prior to Comparison

Enrolled fingerprint

Fingerprint Comparison

Similarity = 0.9Query fingerprint

16

Limitations of Minutiae Representation

• # minutiae in different impressions of the same finger can be different

• Large scale matching is computationally expensive

• Not amenable to matching in encrypted domain without a loss of accuracy

22 minutiae detected 28 minutiae detected

Fixed Length Representation of Fingerprints:Training

Alignment

Network

Texture

Network

Engelsma, Cao, Jain, "Learning a Fixed-Length Fingerprint Representation", IEEE Trans. Pattern Analysis and Machine Intelligence, 2019

Longitudinal Training Data

Stem

NetworkMinutiae

Network

Minutiae Map

128x128x6

Fixed-Length

Representation

(192-dim)

DeepPrint Architecture

Fixed Length Representation of Fingerprints: Testing

Probe Fingerprint Aligned Fingerprint

Alignment

Network

Stem

Network

Texture

Network

Minutiae

Network

Fixed-Length

Representation

(192-dim)

EncryptedMatching

Encrypt

DecryptScores

score = 0.96

Encrypted

Database

Encrypted Templates have been enrolled into the database offline

DeepPrint omparison

Use cosine distance to compute similarities

Genuine Pair: s = 0.78 Imposter Pair: s = 0.55

Fully Homomorphic Encryption (FHE)

• FHE: supports multiplications and additions in

the encrypted domain

• DeepPrint match score is computed with 192

multiplications and 191 additions (cosine

distance between 192-dim vectors)

• Legacy minutiae matching requires sorting and

comparison operations (not supported by FHE)

DeepPrint Authentication PerformanceAlgorithm / Database FVC 2004 DB1A

FAR = 0.1%NIST SD4

FAR = 0.01%NIST SD14

FAR = 0.01%

Verifinger (unencrypted) 96.75% 99.7% 99.89%

DeepPrint (unencrypted) 97.5% 97.9% 98.55%

DeepPrint + (encryption) 97.0% 96.9% 97.3%

1) Reporting True Accept Rate

2) Encryption converts 32-bit feature values to 8-bits (for faster match speed);

results in slight drop in authentication accuracy in the encrypted domain

3) Utilized open-source encryption code from [1]

[1] Vishnu Boddeti, “Secure Face Matching Using Fully Homomorphic Encryption”, BTAS 2018

Code: https://github.com/human-analysis/secure-face-matching

1) Verifinger is a minutiae matcher; matching in the encrypted domain is not possible

2) Experiments done on an Intel Core i9-7900X CPU @ 3.30 GHz with 64 GB of RAM

3) Utilized open-source encryption code from [1]

DeepPrint Matching SpeedMetric DeepPrint Verifinger

Unencrypted(matches / sec)

10,000,000 50

Encrypted (matches / sec)

790 N.A.

Template Size 200 bytes 1.5 – 23 kilobytes

[1] Vishnu Boddeti, “Secure Face Matching Using Fully Homomorphic Encryption”, in BTAS 2018

Code: https://github.com/human-analysis/secure-face-matching

Matching Failure Examples

Minutiae-Matcher False Reject

distorted fingerprint43 minutiae detected

wet fingerprint44 minutiae detected

DeepPrint False Reject

No minutiae required for DeepPrint match

Partial, non-overlapping

fingerprints

Matching Failure Examples

Successful match with unencrypted DeepPrint representation; fails to match with encrypted DeepPrint representation

Score drops from 0.88 to 0.84 due to loss in feature value precision (32 to 8 bits)

Summary

Chongqing: World’s Most Heavily Surveilled; 2.58m cameras; 15.35 million people – 1 camra/6 residentshttps://www.theguardian.com/cities/2019/dec/02/big-brother-is-watching-chinese-city-with-26m-cameras-is-worlds-most-heavily-surveilled

• Biometrics is here to stay; use cases are growing• Stake holders: government, corporations, researchers, citizens• Challenge: Legislation to separate lawful v. unlawful uses• These algorithms are not all-knowing, They’re flawed, they’re

biased, and that kind of deployment in secret, and without protections is troubling. (ACLU Winter 2020)