ongoing face recognition vendor test (frvt) part 1 ... face recognition vendor test (frvt) part ......

NISTIR XXXX Draft

Ongoing Face RecognitionVendor Test (FRVT)Part 1: Verification

Patrick GrotherMei Ngan

Kayee HanaokaInformation Access Division

Information Technology Laboratory

This publication is available free of charge from:https://www.nist.gov/programs-projects/face-recognition-vendor-test-frvt-ongoing

FRVT - FACE RECOGNITION VENDOR TEST - VERIFICATION 1

DISCLAIMER

Specific hardware and software products identified in this report were used in order to perform the evalua-tions described in this document. In no case does identification of any commercial product, trade name, orvendor, imply recommendation or endorsement by the National Institute of Standards and Technology, nordoes it imply that the products and equipment identified are necessarily the best available for the purpose.

ABOUT THIS REPORT

This report is a draft NIST Interagency Report, and is open for comment. It documents the verification-trackof the ongoing Face Recognition Vendor Test. The report will be updated continuously as new algorithms areevaluated, as new datasets are added, and as new analyses are included. Comments and suggestions shouldbe directed to [email protected].

2017/08/25 13:00:41 FNMR(T) False non-match rateFMR(T) False match rate


ContentsDISCLAIMER 1

1 CHANGELOG 6

2 METRICS 112.1 CORE ACCURACY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 DATASETS 123.1 CHILD EXPLOITATION IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.2 VISA IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.3 MUGSHOT IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.4 SELFIE IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.5 WEBCAM IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.6 WILD IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 RESULTS 144.1 TEST GOALS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.2 TEST DESIGN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.3 FAILURE TO ENROL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174.4 RECOGNITION ACCURACY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.5 GENUINE DISTRIBUTION STABILITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.5.1 EFFECT OF BIRTH PLACE ON THE GENUINE DISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.5.2 EFFECT OF AGE ON GENUINE SUBJECTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.6 IMPOSTOR DISTRIBUTION STABILITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.6.1 EFFECT OF BIRTH PLACE ON THE IMPOSTOR DISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.6.2 EFFECT OF AGE ON IMPOSTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

List of Tables1 ALGORITHM SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 FALSE NON-MATCH RATE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 FAILURE TO ENROL RATES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

List of Figures

1 PERFORMANCE SUMMARY: FNMR VS. TEMPLATE SIZE TRADEOFF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 PERFORMANCE SUMMARY: FNMR VS. TEMPLATE TIME TRADEOFF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 EXAMPLE IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

(A) VISA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14(B) MUGSHOT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14(C) WEBCAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14(D) SELFIE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14(E) WILD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 ERROR TRADEOFF CHARACTERISTIC: VISA IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 ERROR TRADEOFF CHARACTERISTIC: VISA IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 ERROR TRADEOFF CHARACTERISTIC: MUGSHOT IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 ERROR TRADEOFF CHARACTERISTIC: SELFIE IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 ERROR TRADEOFF CHARACTERISTIC: SELFIE IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 ERROR TRADEOFF CHARACTERISTIC: WILD IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2410 ERROR TRADEOFF CHARACTERISTICS: CHILD EXPLOITATION IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25



11 ERROR TRADEOFF CHARACTERISTICS: CHILD EXPLOITATION IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2612 SEX AND RACE EFFECTS: MUGSHOT IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2713 SEX EFFECTS: VISA IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2814 ERROR TRADEOFF CHARACTERISTIC: WILD IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2915 FALSE MATCH RATE CALIBRATION: VISA IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3016 FALSE MATCH RATE CONCENTRATION: VISA IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3117 FALSE MATCH RATE CALIBRATION: MUGSHOT IMAGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3218 EFFECT OF COUNTRY OF BIRTH ON FNMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3419 EFFECT OF SUBJECT AGE ON FNMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3620 IMPOSTOR DISTRIBUTION SHIFTS FOR SELECT COUNTRY PAIRS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3921 ALGORITHM 3DIVI-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4022 ALGORITHM AYONIX-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4123 ALGORITHM CYBEREXTRUDER-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4224 ALGORITHM DERMALOG-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4325 ALGORITHM DERMALOG-002 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4426 ALGORITHM DERMALOG-003 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4527 ALGORITHM DIGITALBARRIERS-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4628 ALGORITHM DIGITALBARRIERS-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4729 ALGORITHM ID3-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4830 ALGORITHM ID3-002 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4931 ALGORITHM INNOVATRICS-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5032 ALGORITHM INNOVATRICS-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5133 ALGORITHM ISITYOU-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5234 ALGORITHM ITMO-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5335 ALGORITHM ITMO-002 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5436 ALGORITHM MORPHO-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5537 ALGORITHM NEUROTECHNOLOGY-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5638 ALGORITHM NEUROTECHNOLOGY-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5739 ALGORITHM NTECHLAB-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5840 ALGORITHM NTECHLAB-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5941 ALGORITHM RANKONE-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6042 ALGORITHM RANKONE-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6143 ALGORITHM RANKONE-002 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6244 ALGORITHM SAMTECH-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6345 ALGORITHM TONGYITRANS-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6446 ALGORITHM TONGYITRANS-002 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6547 ALGORITHM TUPEL-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6648 ALGORITHM VCOG-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6749 ALGORITHM VCOG-002 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6850 ALGORITHM VIGILANTSOLUTIONS-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6951 ALGORITHM VIGILANTSOLUTIONS-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7052 ALGORITHM VISIONLABS-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7153 ALGORITHM VOCORD-001 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7254 ALGORITHM VOCORD-002 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7355 ALGORITHM YITU-000 CROSS REGION FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7456 ALGORITHM 3DIVI-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7557 ALGORITHM AYONIX-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7658 ALGORITHM CYBEREXTRUDER-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7759 ALGORITHM DERMALOG-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7860 ALGORITHM DERMALOG-002 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7961 ALGORITHM DERMALOG-003 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8062 ALGORITHM DIGITALBARRIERS-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8163 ALGORITHM DIGITALBARRIERS-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8264 ALGORITHM ID3-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8365 ALGORITHM ID3-002 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8466 ALGORITHM INNOVATRICS-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8567 ALGORITHM INNOVATRICS-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86



68 ALGORITHM ISITYOU-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8769 ALGORITHM ITMO-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8870 ALGORITHM ITMO-002 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8971 ALGORITHM MORPHO-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9072 ALGORITHM NEUROTECHNOLOGY-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9173 ALGORITHM NEUROTECHNOLOGY-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9274 ALGORITHM NTECHLAB-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9375 ALGORITHM NTECHLAB-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9476 ALGORITHM RANKONE-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9577 ALGORITHM RANKONE-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9678 ALGORITHM RANKONE-002 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9779 ALGORITHM SAMTECH-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9880 ALGORITHM TONGYITRANS-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9981 ALGORITHM TONGYITRANS-002 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10082 ALGORITHM TUPEL-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10183 ALGORITHM VCOG-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10284 ALGORITHM VCOG-002 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10385 ALGORITHM VIGILANTSOLUTIONS-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10486 ALGORITHM VIGILANTSOLUTIONS-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10587 ALGORITHM VISIONLABS-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10688 ALGORITHM VOCORD-001 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10789 ALGORITHM VOCORD-002 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10890 ALGORITHM YITU-000 CROSS COUNTRY FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10991 IMPOSTOR COUNTS FOR CROSS COUNTRY FMR CALCULATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11092 ALGORITHM 3DIVI-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11293 ALGORITHM AYONIX-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11394 ALGORITHM CYBEREXTRUDER-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11495 ALGORITHM DERMALOG-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11596 ALGORITHM DERMALOG-002 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11697 ALGORITHM DERMALOG-003 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11798 ALGORITHM DIGITALBARRIERS-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11899 ALGORITHM DIGITALBARRIERS-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119100 ALGORITHM ID3-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120101 ALGORITHM ID3-002 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121102 ALGORITHM INNOVATRICS-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122103 ALGORITHM INNOVATRICS-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123104 ALGORITHM ISITYOU-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124105 ALGORITHM ITMO-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125106 ALGORITHM ITMO-002 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126107 ALGORITHM MORPHO-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127108 ALGORITHM NEUROTECHNOLOGY-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128109 ALGORITHM NEUROTECHNOLOGY-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129110 ALGORITHM NTECHLAB-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130111 ALGORITHM NTECHLAB-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131112 ALGORITHM RANKONE-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132113 ALGORITHM RANKONE-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133114 ALGORITHM RANKONE-002 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134115 ALGORITHM SAMTECH-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135116 ALGORITHM TONGYITRANS-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136117 ALGORITHM TONGYITRANS-002 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137118 ALGORITHM TUPEL-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138119 ALGORITHM VCOG-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139120 ALGORITHM VCOG-002 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140121 ALGORITHM VIGILANTSOLUTIONS-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141122 ALGORITHM VIGILANTSOLUTIONS-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142123 ALGORITHM VISIONLABS-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143124 ALGORITHM VOCORD-001 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144



125 ALGORITHM VOCORD-002 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145126 ALGORITHM YITU-000 CROSS AGE FMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146



1 Changelog

2017-08-22

. Added results for three additional algorithms, rankone-002, neurotechnology-001, and itmo-002.

. The algorithms dermalog-001 and rank-001 have been retired - FRVT lists only two algorithms per organization

. The algorithm tupel-001 has been retired as it is not operable on all datasets

. Clarified the tradeoff Figures 1 and 2 plot only genuine comparison durations.

. Corrected image type label in section 3.5

2017-08-07

. Added results for 5 new algorithms

. Added Figure 3 giving simulated example images.

. Added Figure 1 showing an alternative view of the same tradeoff data in Figure 2

. Added Figure 5 showing accuracy on visa images just for low FMR.

. Added Figure 20 showing impostor distribution shifts from certain country pairs. Section 4.6.1 in this and priorreports documents high false match rates for individuals from certain countries. That effect, however, is often notconfined to anomalously high impostor scores in the tails of the distribution, but arises from systematic shifts ofthe whole distribution. These shifts sometimes reach 2.

2017-07-29

. Added results for 8 new algorithms

. Added results for a child-exploitation dataset

. Added Table 2 a standalone tabulation of false non-match rates

. We have received additional CPU algorithms - Results should appear August 4, 2017

. We have received additional GPU algorithms - Results to appear as computational resources are released from theFace Recognition Prize Challenge

2017-06-19

. Added five new algorithms, three of which remain in-process

. Added results for a wild dataset of images similar to non-cooperative photojournalism images

. Added Table 3 a standalone tabulation of failure to enrol rates

. Added Fig. 2 showing tradeoff between FNMR, template size, template generation time, and match duration.

. Added Fig. 16 showing how FMR is concentrated in certain images.

. Restated cross-region false match rates at nominal FMR = 0.0001 instead of 0.001

. Improved DET legends.


https://www.nist.gov/programs-projects/face-recognition-prize-challenge

FR

VT

-FA

CE

RE

CO

GN

ITIO

NV

EN

DO

RT

EST

-V

ER

IFICA

TIO

N7

Developer Short Seq. Validation Config1 Template GPU Comparison Time (ns)3

Name Name Num. Date Data (KB) Size (B) Time (ms)2 Genuine Impostor

1 3DiVi 3divi 000 2017-03-16 169360 8512 0 13285 52 Yes 1378 20 2375 192 Ayonix ayonix 000 2017-06-22 58505 111036 0 118 2 No 2621 23 3620 263 Cyberextruder cyberex 001 2017-08-02 121211 6256 0 28893 25 No 61083 16 81079 194 Dermalog dermalog 002 2017-02-22 0 131043 0 381 0 No 1822169 114 2122105 1465 Dermalog dermalog 003 2017-07-10 0 121043 0 9121 22 No 1922957 93 2222808 1316 Digital Barriers barriers 000 2017-05-31 157794 172056 0 5104 0 No 1413232 166 1713226 1467 Digital Barriers barriers 001 2017-07-20 236934 182056 0 14294 1 No 1312311 164 1512347 1978 ID3 Technology id3 001 2017-08-04 225574 10520 0 10238 19 No 51058 32 71049 289 ID3 Technology id3 002 2017-08-04 225574 9520 0 18482 34 No 71100 59 61048 3210 Innovatrics innova 000 2017-07-25 0 3146 0 19578 5 No 104964 63 114665 26211 Innovatrics innova 001 2017-07-25 0 7284 0 23645 5 No 115506 131 124975 30812 Is It You isityou 000 2017-06-26 48010 2919200 0 8113 5 No 30237517 1318 30237374 127913 ITMO University itmo 001 2017-06-12 1923215 3137997 0 32959 18 No 2028901 1492 2327517 18614 ITMO University itmo 002 2017-08-07 1923215 254162 0 21611 17 No 127423 96 137451 9415 Morpho morpho 000 2017-07-11 100806 1116 0 7109 1 No 4993 31 51000 3416 Neurotechnology neurotech 000 2017-03-22 62129 287148 0 22611 48 No 2874288 2194 2872879 264017 Neurotechnology neurotech 001 2017-08-07 280771 212718 0 26881 46 No 2769356 684 2769140 57918 N-Tech Lab ntech 000 2017-03-13 191530 222906 1 12278 13 No 2230787 142 2430846 7719 N-Tech Lab ntech 001 2017-05-10 691296 276744 1 20587 11 No 2567692 833 2667486 24420 Rank One Computing rankone 000 2017-03-21 0 2144 0 482 9 No 2439932 468 148722 17121 Rank One Computing rankone 002 2017-08-18 0 5224 0 275 1 No 2669113 3802 1369 2622 Samtech InfoNet Limited samtech 000 2017-05-02 109774 162056 0 11262 2 No 94550 26 104541 2823 TongYi Transportation Technology tongyi 001 2017-04-01 625339 202058 0 15310 20 No 1617769 74 1917750 6324 TongYi Transportation Technology tongyi 002 2017-07-15 625336 192058 0 16356 35 No 2129816 281 2017799 12725 VCognition vcog 001 2017-03-28 86103 234126 0 6108 17 Yes 1516320 197 1816426 42526 VCognition vcog 002 2017-06-12 3229434 3261504 5 17357 25 No 31296154 3077 31296436 418327 Vigilant Solutions vigilant 000 2017-03-30 352218 3031540 0 27884 23 No 1718201 94 1613030 8328 Vigilant Solutions vigilant 001 2017-06-13 344685 151544 0 30921 2 No 3644 13 4649 1629 VisionLabs visionlabs 001 2017-06-12 343661 4204 0 31943 8 No 81395 45 91148 5330 Vocord vocord 001 2017-04-21 616989 266194 0 29908 16 No 321094730 64282 321107193 6652331 Vocord vocord 002 2017-06-07 918292 141330 0 25782 36 Yes 2983063 517 2983072 71432 Shanghai Yitu Technology yitu 000 2017-05-23 2211068 244130 0 24672 2 No 2335352 114 2537848 1773

Notes1 The size of configuration data does not capture static data included in the libraries. We do not include the size of the libraries because

some algorithms include common ancilliary libraries for image processing (e.g. openCV) or numerical computation (e.g. blas).2 The median template creation times are measured on Intel RXeon RCPU E5-2630 v4 @ 2.20GHz processors or, in the case of GPU-enabled

implementations, NVidia Tesla K40.3 The median comparison durations, in nanoseconds, are estimated using std::chrono::high resolution clock which on the machine in (2)

counts clock ticks of duration 1ns. Precision is somewhat worse than that however. The value is the median absolute deviation times1.48 for Normal consistency.

Table 1: Summary of algorithms and properties included in this report. The red superscripts give ranking for the quantity in that column.

2017/08/

2513:00:41

FNM

R(T)

Falsenon-m

atchrate

FMR

(T)False

match

rate


FALSE NON-MATCH RATE (FNMR)Algorithm CONSTRAINED, COOPERATIVE LESS CONSTRAINED, NON-COOPERATIVEName VISA VISA MUGSHOT WEBCAM SELFIE WILD CHILD EXP

FMR 1E-06 0.0001 0.0001 0.0001 0.0001 0.0001 0.01

1 3divi-000 0.133 7 0.029 6 0.037 9 0.002 7 0.055 14 0.547 4 0.553 32 ayonix-000 0.487 27 0.230 28 0.309 28 0.172 27 0.360 28 0.807 20 0.843 163 cyberextruder-001 0.255 18 0.076 20 0.165 22 0.029 19 0.144 24 0.853 23 - 314 dermalog-002 0.315 22 0.122 23 0.241 26 0.109 26 0.179 26 0.709 12 0.985 21

5 dermalog-003 0.280 20 0.112 22 0.202 25 0.041 23 0.115 23 0.693 9 0.845 186 digitalbarriers-000 0.463 26 0.161 25 0.184 24 0.045 24 0.170 25 0.741 17 0.771 147 digitalbarriers-001 0.502 28 0.155 24 0.041 12 0.029 20 0.115 22 0.678 8 - 258 id3-001 0.250 17 0.063 18 0.036 8 0.002 8 0.040 10 0.765 18 - 32

9 id3-002 0.239 16 0.057 17 0.032 7 0.003 10 0.037 8 0.810 21 - 2310 innovatrics-000 0.191 12 0.034 10 0.046 17 0.001 4 0.040 9 0.720 13 - 2411 innovatrics-001 0.183 11 0.034 9 0.043 13 0.001 5 0.043 11 0.643 6 - 2812 isityou-000 0.703 30 0.414 30 0.680 30 0.690 31 - 32 1.000 31 - 22

13 itmo-001 0.441 24 0.174 27 0.256 27 0.069 25 0.199 27 0.932 28 - 2714 itmo-002 0.287 21 0.050 15 0.047 18 0.036 21 0.069 20 0.892 25 - 2615 morpho-000 0.134 8 0.026 4 0.028 4 0.007 13 0.012 2 0.893 26 0.846 1916 neurotechnology-000 0.237 15 0.052 16 0.062 19 0.005 12 0.052 12 0.943 29 0.845 17

17 neurotechnology-001 0.222 14 0.044 13 0.046 16 0.001 6 0.017 7 0.817 22 - 2918 ntechlab-000 0.086 6 0.027 5 0.044 14 0.003 9 0.014 5 0.367 2 0.533 219 ntechlab-001 0.083 5 0.025 3 0.030 5 0.003 11 0.014 4 0.319 1 0.472 120 rankone-000 0.276 19 0.071 19 0.177 23 0.021 17 0.092 21 0.723 14 0.787 15

21 rankone-002 0.217 13 0.049 14 0.032 6 0.000 2 0.052 13 0.705 11 - 3022 samtech-000 0.443 25 0.161 26 0.044 15 0.021 18 0.063 18 0.878 24 0.765 1323 tongyitrans-001 0.072 4 0.038 12 0.041 11 0.009 14 0.063 16 0.704 10 0.743 924 tongyitrans-002 0.066 3 0.030 7 0.039 10 0.010 15 0.063 17 0.725 15 0.746 10

25 vcog-001 0.892 31 0.409 29 - 32 0.302 28 0.427 29 - 32 0.686 626 vcog-002 0.903 32 0.504 32 0.692 31 0.559 30 0.666 31 0.778 19 0.752 1127 vigilantsolutions-000 0.688 29 0.415 31 0.595 29 0.401 29 0.643 30 0.915 27 0.894 2028 vigilantsolutions-001 0.348 23 0.105 21 0.101 21 0.016 16 0.061 15 0.729 16 0.730 8

29 visionlabs-001 0.180 10 0.030 8 0.024 3 0.001 3 0.014 6 0.591 5 0.561 430 vocord-001 0.141 9 0.035 11 0.063 20 0.036 22 0.069 19 0.654 7 0.695 731 vocord-002 0.034 2 0.013 1 0.019 2 - 32 0.012 3 0.948 30 0.762 1232 yitu-000 0.033 1 0.021 2 0.017 1 0.000 1 0.012 1 0.431 3 0.586 5

Table 2: FNMR is the proportion of mated comparisons below a threshold set to achieve the FMR given in the header on the fourthrow. FMR is the proportion of impostor comparisons at or above that threshold. Note that the webcam and selfie values apply toimages collected on the same day, and that will often yield optimisticlly low FNMR values.



3div

i0

ayon

ix0

cybe

rex1

derm

alo2

derm

alo3

digi

tal0

digi

tal1

id31

id32

inno

vat0

inno

vat1

isity

ou0

itmo1

itmo2

mor

pho0

neur

ote0

neur

ote1

ntec

hla0

ntec

hla1

rank

one0

rank

one2

sam

tech

0

tong

yit1

tong

yit2

vigi

lan0

vigi

lan1

visi

onl1

voco

rd1

voco

rd2

yitu

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.10

0.20

0.30

0.40

0.50

0.60

0.70

100

200

300

400

500

600

700

800

1000

2000

3000

4000

5000

7000

9000

3000

050

000

Tem

plat

e si

ze (

byte

s)

FNMR at FMR = 0.0001

3456lo

g10

(mat

ch_n

ano

sec)

log

10(t

emp

late

_mill

isec

)

1.5

2.0

2.5

Figu

re1:

The

poin

tssh

owfa

lse

non-

mat

chra

tes

(FN

MR

)ve

rsus

the

dura

tion

ofth

ete

mpl

ate

gene

rati

onop

erat

ion.

FNM

Ris

the

geom

etri

cm

ean

ofFN

MR

valu

esfo

rvi

saan

dm

ugsh

otim

ages

(fro

mFi

gs.4

and

6)at

afa

lse

mat

chra

te(F

MR

)of0

.000

1.Te

mpl

ate

gene

rati

onti

me

isa

med

ian

esti

mat

edov

er64

0x

480

pixe

lpor

trai

ts.T

hesi

zeof

the

poin

tsen

code

ste

mpl

ate

gene

rati

onti

me

-w

hich

span

son

eor

der

ofm

agni

tude

.Th

eco

lor

ofth

epo

ints

enco

des

one-

to-o

nege

nuin

ete

mpl

ate

com

pari

son

dura

tion

-whi

chsp

anth

ree

orde

rsof

mag

nitu

de.



3div

i0

cybe

rex1

derm

alo2

derm

alo3

digi

tal0

digi

tal1

id31

id32

inno

vat0

inno

vat1

isity

ou0

itmo1

itmo2

mor

pho0

neur

ote0

neur

ote1

ntec

hla0

ntec

hla1

rank

one0

rank

one2

sam

tech

0

tong

yit1

tong

yit2

vcog

2

vigi

lan0

vigi

lan1

visi

onl1

voco

rd1

voco

rd2

yitu

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.10

0.20

0.30

0.40

0.50

0.60

0.70

5060

7080

9010

020

030

040

050

060

070

080

090

010

00Te

mpl

ate

gene

ratio

n du

ratio

n (m

illis

econ

ds)

FNMR at FMR = 0.0001

3456lo

g10

(mat

ch_n

ano

sec)

log

10(t

emp

late

_byt

es)

3 4

Figu

re2:

The

poin

tssh

owfa

lse

non-

mat

chra

tes

(FN

MR

)ve

rsus

the

dura

tion

ofth

ete

mpl

ate

gene

rati

onop

erat

ion.

FNM

Ris

the

geom

etri

cm

ean

ofFN

MR

valu

esfo

rvi

saan

dm

ugsh

otim

ages

(fro

mFi

gs.4

and

6)at

afa

lse

mat

chra

te(F

MR

)of0

.000

1.Te

mpl

ate

gene

rati

onti

me

isa

med

ian

esti

mat

edov

er64

0x

480

pixe

lpor

trai

ts.T

hesi

zeof

the

poin

tsen

code

ste

mpl

ate

size

-w

hich

span

two

orde

rsof

mag

nitu

de.

The

colo

rof

the

poin

tsen

code

son

e-to

-one

genu

ine

tem

plat

eco

mpa

riso

ndu

rati

on-

whi

chsp

anth

ree

orde

rsof

mag

nitu

de.



2 Metrics

2.1 Core accuracy

Given a vector of N genuine scores, u, the false non-match rate (FNMR) is computed as the proportion below somethreshold, T:

FNMR(T ) = 1 1N

Ni

H(ui T ) (1)

where H(x) is the unit step function, and H(0) taken to be 1.

Similarly, given a vector of N impostor scores, v, the false match rate (FMR) is computed as the proportion above T:

FMR(T ) =1

N

Ni

H(vi T ) (2)

The threshold, T, can take on any value. We typically generate a set of thresholds from quantiles of the observed impostorscores, v, as follows. Given some interesting false match rate range, [FMRL,FMRU ], we form a vector of K thresholdscorresponding to FMR measurements evenly spaced on a logarithmic scale

Tk = Qv(1 FMRk) (3)

where Q is the quantile function, and FMRk comes from

log10 FMRk = log10 FMRL +k

K[log10 FMRU log10 FMRL] (4)

Error tradeoff characteristics are plots of FNMR(T) vs. FMR(T). These are plotted with FMRU 1 and FMRL as lowas is sustained by the number of impostor comparisons, N. This is somewhat higher than the rule of three limit 3/Nbecause samples are not independent, due to re-use of images.



3 Datasets

3.1 Child exploitation images

. The number of images is O(104).

. The number of subjects is O(103).

. The number of subjects with two images O(103).

. The images are operational. They are taken from ongoing investigations of child exploitation crimes. The imagesare arbitrarily unconstrained. Pose varies considerably around all three axes, including subject lying down. Res-olution varies very widely. Faces can be occluded by other objects, including hair and hands. Lighting varies,although the images are intended for human viewing. Mis-focus is rare. Images are given to the algorithm withoutany cropping; faces may occupy widely varying areas.

. The images are usually large from contemporary cameras. The mean interocular distance (IOD) is 70 pixels.

. The images are of subjects from several countries, due to the global production of this imagery.

. The images are of children, from infancy to late adolescence.

. All of the images are live capture, none are scanned. Many have been cropped.

. When these images are input to the algorithm, they are labelled as being of type EXPLOITATION - see Table 4 ofthe FRVT API.

3.2 Visa images




. The images have geometry in reasonable conformance with the ISO/IEC 19794-5 Full Frontal image type. Pose isgenerally excellent.

. The images are of size 252x300 pixels. The mean interocular distance (IOD) is 69 pixels.

. The images are of subjects from greater than 100 countries, with significant imbalance due to visa issuance patterns.

. The images are of subjects of all ages, including children, again with imbalance due to visa issuance demand.

. Many of the images are live capture. A substantial number of the images are photographs of paper photographs.

. When these images are input to the algorithm, they are labelled as being of type ISO - see Table 4 of the FRVTAPI.

3.3 Mugshot images






. The images have geometry in reasonable conformance with the ISO/IEC 19794-5 Full Frontal image type.

. The images are of variable sizes. The median IOD is 104 pixels. The mean IOD is 123 pixels.

. The images are of subjects from the United States.

. The images are of adults.

. The images are all live capture.

. When these images are input to the algorithm, they are labelled as being of type mugshot - see Table 4 of theFRVT API.

3.4 Selfie images

. The number of images is below 500.

. The number of subjects is below 500.

. All subjects have a selfie image, and a portrait image.

. The portrait images are in reasonable conformance with the ISO/IEC 19794-5 Full Frontal image type.

. The selfie images vary: taken with camera above and below eye level, with one hand or two hands. Pitch anglesvary more than yaw angles, which are frontal. Some perspective distortion is evident.

. The images have mean IOD of 140 pixels.




. When these images are input to the algorithm, they are labelled as being of type WILD - see Table 4 of the FRVTAPI.

3.5 Webcam images

. The number of images is below 1500.

. The number of subjects is below 1500.

. All subjects have a webcam image, and a portrait image.

. The portrait images are in reasonable conformance with the ISO/IEC 19794-5 Full Frontal image type.

. The webcam images are taken with camera at a typical head height, with mild pitch angles, low yaw angles, butsome variation in range, such that low perspective distortion is sometimes evident.

. The images have mean IOD of 68 pixels (sd=12).




. When these images are input to the algorithm, they are labelled as being of type MUGSHOT - see Table 4 of theFRVT API.



(a) Visa (b) Mugshot (c) Webcam (d) Selfie (e) Wild

Figure 3: The figure gives simulated samples of image types used in this report.

3.6 Wild images




. The images include many photojournalism-style images. Images are given to the algorithm using a variable butgenerally tight crop of the head. Resolution varies very widely. The images are very unconstrained, with wide yawand pitch pose variation. Faces can be occluded, including hair and hands.


. All of the images are live capture, none are scanned.

. When these images are input to the algorithm, they are labelled as being of type WILD - see Table 4 of the FRVTAPI.

4 Results

4.1 Test goals

. To state overall accuracy.

. To compare algorithms.

4.2 Test design

Method: For visa images:

. The comparisons are of visa photos against visa photos.

. The number of genuine comparisons is O(104).

. The number of impostor comparisons is O(1010).



. The comparisons are fully zero-effort, meaning impostors are paired without attention to sex, age or other covari-ates. However, later analysis is conducted on subsets.

. The number of persons is O(105).

. The number of images used to make 1 template is 1.

. The number of templates used to make each comparison score is two corresponding to simple one-to-one verifica-tion.

For mugshot images:

. The comparisons are of mugshot photos against mugshot photos.



. The comparisons are fully zero-effort, meaning impostors are paired without attention to sex, age or other covari-ates.




For selfie images:

. The comparisons are of selfie photos against portrait photos.


. The number of impostor comparisons is O(108) selfies are compared with portraits of O(106) other subjects.





For webcam images:

. The comparisons are of webcam photos against portrait photos.


. The number of impostor comparisons is O(109) webcams are compared with portraits of O(106) other subjects.







For child exploitation images:

. The comparisons are of unconstrained child exploitation photos against others of the same type.







. We produce two performance statements. First, is a DET as used for visa and mugshot images. The second isa cumulative match characteristic (CMC) summarizing a simulated one-to-many search process. This is done asfollows.

We regard M enrollment templates as items in a gallery.

These M templates come from M > N individuals, because multiple images of a subject are present in thegallery under separate identifiers.

We regard the verification templates as search templates.

For each search we compute the rank of the highest scoring mate.

This process should properly be conducted with a 1:N algorithm, such as those tested in NIST IR 8009. Weuse the 1:1 algorithms in a simulated 1:N mode here to a) better reflect what a child exploitation analyst does,and b) to do show algorithm efficacy is better than that revealed in the verification DETs.



4.3 Failure to enrol

Algorithm Failure to Enrol Rate1

Name CHILD-EXPLOIT MUGSHOT SELFIES VISA WEBCAM WILD

3divi-000 0.2019 12 0.0019 18 0.0202 18 0.0008 12 0.0020 15 0.2070 19ayonix-000 0.0000 1 0.0109 31 0.0751 29 0.0137 35 0.0109 21 0.0000 2cyberextruder-001 - 32 0.0036 22 0.0376 20 0.0029 24 0.0205 26 0.5730 32dermalog-002 0.9109 26 0.0045 26 0.0954 32 0.0013 15 0.0471 30 0.3979 26

dermalog-003 0.0434 6 0.0007 7 0.0000 1 0.0025 23 0.0007 6 0.0701 11digitalbarriers-000 0.5469 23 0.0043 24 0.0925 30 0.0019 21 0.0184 25 0.5170 31digitalbarriers-001 - 32 0.0044 25 0.0925 31 0.0018 20 0.0232 27 0.4593 30id3-001 - 32 0.0043 23 0.0260 19 0.0043 32 0.0014 12 0.2090 20

id3-002 - 32 0.0030 21 0.0202 17 0.0032 26 0.0020 14 0.1496 14innovatrics-000 - 32 0.0013 11 0.0087 13 0.0004 8 0.0027 16 0.3337 24innovatrics-001 - 32 0.0013 12 0.0087 15 0.0004 9 0.0027 17 0.3337 25isityou-000 0.4714 21 0.0022 20 0.0665 27 0.0010 13 0.0116 23 0.4586 29

itmo-001 - 32 0.0114 32 0.0694 28 0.0047 33 0.0546 32 0.7410 36itmo-002 - 32 0.0068 29 0.0636 26 0.0029 25 0.0498 31 0.6985 34morpho-000 0.0000 2 0.0000 1 0.0000 4 0.0000 2 0.0000 4 0.0000 5neurotechnology-000 0.0000 4 0.0000 3 0.0000 5 0.0000 3 0.0000 5 0.0163 8

neurotechnology-001 - 32 0.0000 2 0.0000 2 0.0000 1 0.0000 1 0.0163 7ntechlab-000 0.2496 15 0.0015 16 0.0058 11 0.0016 18 0.0007 7 0.1099 13ntechlab-001 0.0926 10 0.0009 8 0.0029 7 0.0005 10 0.0007 8 0.0584 10rankone-000 0.0187 5 0.0005 5 0.0000 6 0.0003 6 0.0007 10 0.2349 21

rankone-002 - 32 0.0001 4 0.0000 3 0.0000 4 0.0000 3 0.0855 12samtech-000 0.5474 24 0.0052 27 0.0491 24 0.0042 31 0.0252 28 0.7023 35tongyitrans-001 0.0000 3 0.0068 28 0.0462 22 0.0040 29 0.0055 19 0.0000 3tongyitrans-002 0.3609 19 0.0078 30 0.0462 23 0.0040 30 0.0055 20 0.0000 4

vcog-001 0.1579 11 - 32 0.0058 10 0.0018 19 0.0000 2 - 32vcog-002 0.2209 13 0.0021 19 0.0087 14 0.0019 22 0.0007 9 0.1672 16vigilantsolutions-000 0.5580 25 0.0018 17 0.0462 21 0.0007 11 0.0109 22 0.5927 33vigilantsolutions-001 0.3585 18 0.0010 10 0.0116 16 0.0004 7 0.0048 18 0.3262 23

visionlabs-001 0.2699 16 0.0014 13 0.0058 12 0.0014 17 0.0020 13 0.1803 17vocord-001 0.4732 22 0.0158 33 0.0520 25 0.0038 28 0.0348 29 0.4494 28vocord-002 0.3782 20 0.0015 14 0.0029 9 0.0037 27 0.0171 24 0.1992 18yitu-000 0.3475 17 0.0015 15 0.0029 8 0.0013 16 0.0014 11 0.1591 15

Table 3: FTE is the proportion of failed template generation attempts. Failures can occur because the software throws an exception,or because the software electively refuses to process the input image. This would typically occur if a face is not detected. FTE ismeasured as the number of function calls that give a non-zero error code, OR that give a small template. This is defined as onewhose size is less than 0.3 times the median template size. This second rule is needed because some algorithms incorrectly fail toreturn a non-zero error code when template generation fails.1The effects of FTE are included in the accuracy results of this report by regarding any template comparison involving a failed templateenrollment to produce a low similarity score. Thus higher FTE results in higher FNMR.



4.4 Recognition accuracy

Core algorithm accuracy is stated via:

. Cooperative subjects

The summary table of Figure 2;

The visa image DETs of Figures 4 and 5;

The mugshot DETs of Figure 6 ;

The selfie-portrait DETs of Figure 7;

The webcam-portrait DETs of Figure 8;

. Cooperative subjects

The photojournalism DET of Figure 9

The child-exploitation DET of Figure 10;

The child-exploitation CMC of Figure 11.

Figure 15 shows dependence of false match rate on algorithm score threshold. This allows a deployer to set a thresholdto target a particular false match rate appropriate to the security objectives of the application.

Figure 17 likewise shows FMR(T) but for mugshots, and specially four subsets of the population.

Note that in both the mugshot and visa sets false match rates vary with the ethnicity, age, and sex, of the enrollee andimpostor - see section 4.6.



isit0

rank

0

rank

2ne

ur0

ntec

0

ntec

1

itmo1

ayon

0

vcog

1

vcog

2

vigi

0

vigi

1di

gi0

itmo2

derm

2

derm

3

tong

1

tong

2

sam

t0

yitu

0

visi

1

voco

1

voco

2

3div

0

mor

p0

inno

0

0.00

3

0.01

0

0.03

0

0.10

0

0.30

0 1e

071e

06

1e

051e

04

1e

031e

02

1e

011e

+00

Fals

e m

atch

rat

e (F

MR

)

False nonmatch rate (FNMR)

Dat

aset

: V

ISA

FN

MR

@ F

MR

=0.0

001

and

Alg

ori

thm

0.50

2 v

cog_

002

0.41

5 v

igila

ntso

lutio

ns_0

00

0.41

3 i

sity

ou_0

00

0.40

7 v

cog_

001

0.22

9 a

yoni

x_00

0

0.17

3 i

tmo_

001

0.16

0 d

igita

lbar

riers

_000

0.16

0 s

amte

ch_0

00

0.15

4 d

igita

lbar

riers

_001

0.12

1 d

erm

alog

_002

0.11

1 d

erm

alog

_003

0.10

5 v

igila

ntso

lutio

ns_0

01

0.07

5 c

yber

extr

uder

_001

0.07

0 r

anko

ne_0

00

0.06

2 i

d3_0

01

0.05

6 i

d3_0

02

0.05

2 n

euro

tech

nolo

gy_0

00

0.05

0 i

tmo_

002

0.04

8 r

anko

ne_0

02

0.04

3 n

euro

tech

nolo

gy_0

01

0.03

8 t

ongy

itran

s_00

1

0.03

5 v

ocor

d_00

1

0.03

4 i

nnov

atric

s_00

0

0.03

3 i

nnov

atric

s_00

1

0.03

0 v

isio

nlab

s_00

1

0.03

0 t

ongy

itran

s_00

2

0.02

9 3

divi

_000

0.02

7 n

tech

lab_

000

0.02

6 m

orph

o_00

0

0.02

5 n

tech

lab_

001

0.02

1 y

itu_0

00

0.01

3 v

ocor

d_00

2

Figu

re4:

For

the

visa

imag

es,d

etec

tion

erro

rtr

adeo

ff(D

ET

)cha

ract

eris

tics

show

ing

fals

eno

n-m

atch

rate

vs.f

alse

mat

chra

tepl

otte

dpa

ram

etri

cally

onth

resh

old,

T.Th

esc

ales

are

loga

rith

mic

inor

der

tosh

owm

any

deca

des

ofF

MR

.


FRVT - FACE RECOGNITION VENDOR TEST - VERIFICATION 20is

it0

rank

0

rank

2

neur

0

ntec

0

ntec

1

ayon

0

vcog

1

vcog

2

vigi

0

vigi

1di

gi0

derm

2

derm

3

tong

1

tong

2

sam

t0

yitu

0

visi

1

voco

1

voco

2

3div

0m

orp0

inno

0

0.01

0.03

0.10

0.30 1

e07

1e

061e

05

1e

04Fa

lse

mat

ch r

ate

(FM

R)


Dat

aset

: V

ISA

FN

MR

@ F

MR

=0.0

001

and

Alg

ori

thm

0.50

2 v

cog_

002

0.41

5 v

igila

ntso

lutio

ns_0

00

0.41

3 i

sity

ou_0

00

0.40

7 v

cog_

001

0.22

9 a

yoni

x_00

0

0.17

3 i

tmo_

001

0.16

0 d

igita

lbar

riers

_000

0.16

0 s

amte

ch_0

00

0.15

4 d

igita

lbar

riers

_001

0.12

1 d

erm

alog

_002

0.11

1 d

erm

alog

_003

0.10

5 v

igila

ntso

lutio

ns_0

01

0.07

5 c

yber

extr

uder

_001

0.07

0 r

anko

ne_0

00

0.06

2 i

d3_0

01

0.05

6 i

d3_0

02

0.05

2 n

euro

tech

nolo

gy_0

00

0.05

0 i

tmo_

002

0.04

8 r

anko

ne_0

02

0.04

3 n

euro

tech

nolo

gy_0

01

0.03

8 t

ongy

itran

s_00

1

0.03

5 v

ocor

d_00

1

0.03

4 i

nnov

atric

s_00

0

0.03

3 i

nnov

atric

s_00

1

0.03

0 v

isio

nlab

s_00

1

0.03

0 t

ongy

itran

s_00

2

0.02

9 3

divi

_000

0.02

7 n

tech

lab_

000

0.02

6 m

orph

o_00

0

0.02

5 n

tech

lab_

001

0.02

1 y

itu_0

00

0.01

3 v

ocor

d_00

2

Figu

re5:

For

the

visa

imag

es,

but

now

just

for

low

FM

R,

dete

ctio

ner

ror

trad

eoff

(DE

T)

char

acte

rist

ics

show

ing

fals

eno

n-m

atch

rate

vs.

fals

em

atch

rate

plot

ted

para

met

rica

llyon

thre

shol

d,T.

The

scal

esar

elo

gari

thm

icin

orde

rto

show

man

yde

cade

sof

FM

R.



3div

0

digi

0

cybe

1

digi

1

rank

0

rank

2

neur

0

neur

1 nte

c0

ntec

1

mor

p0

visi

1

vigi

1

vigi

0

tong

1

tong

2

derm

2

derm

3

ayon

0

voco

1

voco

2

sam

t0

yitu

0

vcog

2

itmo1

itmo2

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.10

0.20

0.30

0.40 1

e05

1e

041e

03

1e

021e

01

1e+

00Fa

lse

mat

ch r

ate

(FM

R)


Dat

aset

: M

UG

SH

OT

FN

MR

@ F

MR

=0.0

001

and

Alg

ori

thm

0.69

1 v

cog_

002

0.67

9 i

sity

ou_0

00

0.59

5 v

igila

ntso

lutio

ns_0

00

0.30

9 a

yoni

x_00

0

0.25

5 i

tmo_

001

0.24

0 d

erm

alog

_002

0.20

1 d

erm

alog

_003

0.18

3 d

igita

lbar

riers

_000

0.17

6 r

anko

ne_0

00

0.16

5 c

yber

extr

uder

_001

0.10

0 v

igila

ntso

lutio

ns_0

01

0.06

3 v

ocor

d_00

1

0.06

1 n

euro

tech

nolo

gy_0

00

0.04

7 i

tmo_

002

0.04

6 i

nnov

atric

s_00

0

0.04

5 n

euro

tech

nolo

gy_0

01

0.04

4 s

amte

ch_0

00

0.04

4 n

tech

lab_

000

0.04

3 i

nnov

atric

s_00

1

0.04

1 d

igita

lbar

riers

_001

0.04

0 t

ongy

itran

s_00

1

0.03

9 t

ongy

itran

s_00

2

0.03

7 3

divi

_000

0.03

6 i

d3_0

01

0.03

2 i

d3_0

02

0.03

1 r

anko

ne_0

02

0.03

0 n

tech

lab_

001

0.02

8 m

orph

o_00

0

0.02

4 v

isio

nlab

s_00

1

0.01

9 v

ocor

d_00

2

0.01

7 y

itu_0

00

Figu

re6:

For

the

mug

shot

imag

es,d

etec

tion

erro

rtr

adeo

ff(D

ET

)cha

ract

eris

tics

show

ing

fals

eno

n-m

atch

rate

vs.f

alse

mat

chra

tepl

otte

dpa

ram

etri

cally

onth

resh

old,

T.Th

esc

ales

are

loga

rith

mic

inor

der

tosh

owde

cade

sof

FM

R.



3div

0

neur

0

rank

0

vcog

1

ayon

0

ntec

0

ntec

1

derm

2

sam

t0

tong

1

tong

2

vigi

0

voco

1

voco

2yi

tu0

mor

p0

inno

0

0.01

0.03

0.10

0.30 1

e05

1e

041e

03

1e

021e

01

Fals

e m

atch

rat

e (F

MR

)


Dat

aset

: S

EL

FIE

SF

NM

R @

FM

R=0

.000

1an

d A

lgo

rith

m

1.00

0 i

sity

ou_0

00

0.66

6 v

cog_

002

0.64

3 v

igila

ntso

lutio

ns_0

00

0.42

7 v

cog_

001

0.36

0 a

yoni

x_00

0

0.19

9 i

tmo_

001

0.17

9 d

erm

alog

_002

0.17

0 d

igita

lbar

riers

_000

0.14

4 c

yber

extr

uder

_001

0.11

5 d

erm

alog

_003

0.11

2 d

igita

lbar

riers

_001

0.09

2 r

anko

ne_0

00

0.06

9 i

tmo_

002

0.06

9 v

ocor

d_00

1

0.06

3 s

amte

ch_0

00

0.06

3 t

ongy

itran

s_00

1

0.06

3 t

ongy

itran

s_00

2

0.05

8 v

igila

ntso

lutio

ns_0

01

0.05

5 3

divi

_000

0.05

2 n

euro

tech

nolo

gy_0

00

0.05

2 r

anko

ne_0

02

0.04

3 i

nnov

atric

s_00

1

0.04

0 i

d3_0

01

0.04

0 i

nnov

atric

s_00

0

0.03

7 i

d3_0

02

0.01

7 n

euro

tech

nolo

gy_0

01

0.01

4 n

tech

lab_

000

0.01

4 n

tech

lab_

001

0.01

4 v

isio

nlab

s_00

1

0.01

2 m

orph

o_00

0

0.01

2 v

ocor

d_00

2

0.01

2 y

itu_0

00

Figu

re7:

For

the

selfi

e-to

-por

trai

tco

mpa

riso

ns,d

etec

tion

erro

rtr

adeo

ff(D

ET

)ch

arac

teri

stic

ssh

owin

gfa

lse

non-

mat

chra

tevs

.fa

lse

mat

chra

tepl

otte

dpa

ram

etri

cally

onth

resh

old,

T.Th

esc

ales

are

loga

rith

mic

inor

der

tosh

owse

vera

ldec

ades

ofF

MR

.Cau

tion

:The

FNM

Rva

lues

here

are

opti

mis

tic

stat

emen

tsof

accu

racy

beca

use

the

imag

epa

irs

wer

eco

llect

edon

the

sam

eda

y.Th

isis

know

nac

ross

biom

etri

csto

give

bett

erac

cura

cy,a

ndis

oper

atio

nally

rele

vant

only

insp

ecia

lcas

es.



vcog

1

vigi

0

3div

0

tong

1

sam

t0

rank

0

neur

0

ntec

0

ntec

1

derm

2

yitu

0

voco

1

mor

p0

digi

0

0.00

1

0.00

3

0.01

0

0.03

0

0.10

0

0.30

0 1e

051e

04

1e

031e

02

1e

01Fa

lse

mat

ch r

ate

(FM

R)


Dat

aset

: W

EB

CA

MF

NM

R @

FM

R=0

.000

1an

d A

lgo

rith

m

0.68

9 i

sity

ou_0

00

0.55

7 v

cog_

002

0.40

0 v

igila

ntso

lutio

ns_0

00

0.30

1 v

cog_

001

0.17

1 a

yoni

x_00

0

0.10

9 d

erm

alog

_002

0.06

9 i

tmo_

001

0.04

5 d

igita

lbar

riers

_000

0.04

1 d

erm

alog

_003

0.03

6 v

ocor

d_00

1

0.03

6 i

tmo_

002

0.02

9 d

igita

lbar

riers

_001

0.02

9 c

yber

extr

uder

_001

0.02

1 s

amte

ch_0

00

0.02

0 r

anko

ne_0

00

0.01

6 v

igila

ntso

lutio

ns_0

01

0.01

0 t

ongy

itran

s_00

2

0.00

9 t

ongy

itran

s_00

1

0.00

7 m

orph

o_00

0

0.00

5 n

euro

tech

nolo

gy_0

00

0.00

3 i

d3_0

02

0.00

3 n

tech

lab_

001

0.00

3 n

tech

lab_

000

0.00

2 3

divi

_000

0.00

2 i

d3_0

01

0.00

1 i

nnov

atric

s_00

0

0.00

1 i

nnov

atric

s_00

1

0.00

1 n

euro

tech

nolo

gy_0

01

0.00

1 v

isio

nlab

s_00

1

0.00

0 r

anko

ne_0

02

0.00

0 y

itu_0

00

Figu

re8:

For

the

web

cam

-to-

port

rait

com

pari

sons

,det

ecti

oner

ror

trad

eoff

(DE

T)c

hara

cter

isti

cssh

owin

gfa

lse

non-

mat

chra

tevs

.fal

sem

atch

rate

plot

ted

para

met

rica

llyon

thre

shol

d,T.

The

scal

esar

elo

gari

thm

icin

orde

rto

show

seve

rald

ecad

esof

FM

R.C

auti

on:T

heFN

MR

valu

eshe

rear

eop

tim

isti

cst

atem

ents

ofac

cura

cybe

caus

eth

eim

age

pair

sw

ere

colle

cted

onth

esa

me

day.

This

iskn

own

acro

ssbi

omet

rics

togi

vebe

tter

accu

racy

,and

isop

erat

iona

llyre

leva

nton

lyin

spec

ialc

ases

.



id3_

1

id3_

2

rank

0

rank

2

sam

t0

neur

0

visi

1yi

tu0

vigi

0

cybe

1

ayon

0di

gi0

digi

1

tong

1to

ng2

derm

2

derm

33d

iv0

ntec

0

ntec

1

voco

1

voco

2

mor

p0

inno

0

inno

1

vcog

2

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9 1e

05

1e

041e

03

1e

021e

01

Fals

e m

atch

rat

e (F

MR

)


Dat

aset

: W

ILD

FN

MR

@ F

MR

=0.0

01an

d A

lgo

rith

m

1.00

is

ityou

_000

0.93

itm

o_00

1

0.89

itm

o_00

2

0.88

vi

gila

ntso

lutio

ns_0

00

0.87

sa

mte

ch_0

00

0.86

vo

cord

_002

0.83

cy

bere

xtru

der_

001

0.80

ne

urot

echn

olog

y_00

0

0.78

m

orph

o_00

0

0.76

ay

onix

_000

0.71

di

gita

lbar

riers

_000

0.70

to

ngyi

tran

s_00

2

0.69

to

ngyi

tran

s_00

1

0.67

ne

urot

echn

olog

y_00

1

0.66

ra

nkon

e_00

0

0.66

di

gita

lbar

riers

_001

0.65

de

rmal

og_0

02

0.62

id

3_00

2

0.62

ra

nkon

e_00

2

0.62

vo

cord

_001

0.61

vi

gila

ntso

lutio

ns_0

01

0.59

de

rmal

og_0

03

0.59

vc

og_0

02

0.58

id

3_00

1

0.57

in

nova

tric

s_00

0

0.56

in

nova

tric

s_00

1

0.47

3d

ivi_

000

0.40

vi

sion

labs

_001

0.36

yi

tu_0

00

0.27

nt

echl

ab_0

00

0.24

nt

echl

ab_0

01

Figu

re9:

For

the

wild

imag

eco

mpa

riso

ns,d

etec

tion

erro

rtr

adeo

ff(D

ET

)ch

arac

teri

stic

ssh

owin

gfa

lse

non-

mat

chra

tevs

.fa

lse

mat

chra

tepl

otte

dpa

ram

etri

cally

onth

resh

old,

T.Th

esc

ales

are

loga

rith

mic

inor

der

tosh

owse

vera

ldec

ades

ofF

MR

.



3div

0

digi

0

rank

0

visi

1

neur

0

ntec

0

ntec

1

tong

1

sam

t0

vcog

1

vigi

0

vigi

1

ayon

0

voco

1

voco

2

yitu

0

mor

p0

0.3

0.4

0.5

0.6

0.7

0.8 0

.001

0.01

00.

100

Fals

e m

atch

rat

e (F

MR

)


Dat

aset

: C

hild

Exp

loit

atio

nF

NM

R @

FM

R=0

.000

1an

d A

lgo

rith

m

1.00

is

ityou

_000

0.98

de

rmal

og_0

02

0.89

vi

gila

ntso

lutio

ns_0

00

0.85

m

orph

o_00

0

0.84

de

rmal

og_0

03

0.84

ne

urot

echn

olog

y_00

0

0.84

ay

onix

_000

0.79

ra

nkon

e_00

0

0.77

di

gita

lbar

riers

_000

0.76

sa

mte

ch_0

00

0.76

vo

cord

_002

0.75

vc

og_0

02

0.75

to

ngyi

tran

s_00

2

0.74

to

ngyi

tran

s_00

1

0.73

vi

gila

ntso

lutio

ns_0

01

0.69

vo

cord

_001

0.69

vc

og_0

01

0.59

yi

tu_0

00

0.56

vi

sion

labs

_001

0.55

3d

ivi_

000

0.53

nt

echl

ab_0

00

0.47

nt

echl

ab_0

01

Figu

re10

:Fo

rch

ildex

ploi

tati

onim

ages

,de

tect

ion

erro

rtr

adeo

ff(D

ET

)ch

arac

teri

stic

ssh

owin

gfa

lse

non-

mat

chra

tevs

.fa

lse

mat

chra

tepl

otte

dpa

ram

etri

cally

onth

resh

old,

T.Th

esc

ales

are

loga

rith

mic

inor

der

tosh

owm

any

deca

des

ofF

MR

.Acc

urac

yis

poor

beca

use

man

yim

ages

have

adve

rse

qual

ity

char

acte

rist

ics,

and

beca

use

dete

ctio

nan

den

rollm

entf

ails

.



rank

0

rank

1

digi

0

neur

0

yitu

0

ntec

0

ntec

1

tong

1

sam

t0

3div

0

vigi

0

vigi

1

vcog

1

vcog

2

derm

1

voco

1

voco

2

mor

p0

isit0

itmo1

ayon

0

0.2

0.3

0.4

0.5

0.6

0.7

0.8

13

1030

100

Ran

k at

whi

ch h

ighe

st s

corin

g m

ate

is fo

und

True positive identification rate (TPIR) aka "hit rate"

Dat

aset

: C

hild

Exp

loit

atio

nC

MC

@ R

ank=

1an

d A

lgo

rith

m

0.59

nt

echl

ab_0

01

0.52

nt

echl

ab_0

00

0.48

3d

ivi_

000

0.47

vi

sion

labs

_001

0.46

ne

urot

echn

olog

y_00

0

0.46

yi

tu_0

00

0.43

vc

og_0

01

0.39

vo

cord

_002

0.38

to

ngyi

tran

s_00

2

0.37

vo

cord

_001

0.36

vi

gila

ntso

lutio

ns_0

01

0.35

m

orph

o_00

0

0.34

ra

nkon

e_00

0

0.34

to

ngyi

tran

s_00

1

0.33

de

rmal

og_0

03

0.32

ra

nkon

e_00

1

0.29

sa

mte

ch_0

00

0.27

itm

o_00

1

0.27

di

gita

lbar

riers

_000

0.24

vc

og_0

02

0.23

ay

onix

_000

0.19

vi

gila

ntso

lutio

ns_0

00

0.18

is

ityou

_000

0.05

de

rmal

og_0

01

0.05

de

rmal

og_0

02

Figu

re11

:For

child

expl

oita

tion

imag

es,c

umul

ativ

em

atch

char

acte

rist

ics

(CM

C)s

how

ing

true

posi

tive

iden

tific

atio

nra

tevs

.ran

k.Th

isis

sim

ulat

ion

ofa

one-

to-m

any

sear

chex

peri

men

t-

see

disc

ussi

onin

sect

ion

4.2.

The

scal

esar

elo

gari

thm

icin

orde

rto

show

the

effe

ctof

long

cand

idat

elis

ts.

Acc

urac

yis

poor

but

muc

him

prov

edre

lati

veto

the

1:1

DET

sof

Fig.

10be

caus

ea

sear

chca

nsu

ccee

dif

any

ofa

subj

ects

seve

rale

nrol

led

imag

esm

atch

esth

ese

arch

imag

ew

ith

ahi

ghsc

ore.



3div

i_00

0ay

onix

_000

cybe

rext

rude

r_00

1de

rmal

og_0

02de

rmal

og_0

03di

gita

lbar

riers

_000

digi

talb

arrie

rs_0

01id

3_00

1id

3_00

2in

nova

tric

s_00

0in

nova

tric

s_00

1is

ityou

_000

itmo_

001

itmo_

002

mor

pho_

000

neur

otec

hnol

ogy_

000

neur

otec

hnol

ogy_

001

ntec

hlab

_000

ntec

hlab

_001

rank

one_

000

rank

one_

002

sam

tech

_000

tong

yitr

ans_

001

tong

yitr

ans_

002

vcog

_002

vigi

lant

solu

tions

_000

vigi

lant

solu

tions

_001

visi

onla

bs_0

01vo

cord

_001

voco

rd_0

02

yitu

_000

0.01

0.03

0.05

0.10

0.20

0.30

0.01

0.03

0.05

0.10

0.20

0.30

0.01

0.03

0.05

0.10

0.20

0.30

0.01

0.03

0.05

0.10

0.20

0.30

0.01

0.03

0.05

0.10

0.20

0.30

0.01

0.03

0.05

0.10

0.20

0.30 1

e05

1e

041e

03

1e

021e

01

Fals

e m

atch

rat

e (F

MR

)


Sex

F M

Rac

e

B W

Figu

re12

:Fo

rth

em

ugsh

otim

ages

,err

ortr

adeo

ffch

arac

teri

stic

sfo

rw

hite

fem

ales

,bla

ckfe

mal

es,b

lack

mal

esan

dw

hite

mal

es.

The

grey

lines

corr

espo

ndto

fixed

thre

shol

ds,s

how

ing

how

both

FNM

Ran

dFM

Rva

ryat

one

oper

atin

gth

resh

old.

Impo

rtan

t:M

any

ofth

epl

ots

will

naiv

ely

bere

adas

sayi

ngw

hite

sgi

ves

low

erer

ror

rate

sth

anbl

acks

beca

use

the

blue

trac

eslie

bene

ath

the

red

ones

.H

owev

er,t

his

ism

isle

adin

gan

din

com

plet

e:Th

egr

eylin

essh

owth

etr

aces

are

gene

rally

shif

ted

hori

zont

ally

.Th

usfo

rth

ede

rmal

og-0

01al

gori

thm

FNM

Rfo

rw

hite

sis

high

erth

anfo

rbl

acks

ata

fixed

thre

shol

dbu

t,at

the

sam

eti

me,

FMR

ishi

gher

for

blac

ks-

see

Figu

re17

.As

acce

ssco

ntro

lsys

tem

sal

mos

talw

ays

oper

ate

ata

fixed

thre

shol

d,th

ena

ive

inte

rpre

tati

onis

inco

rrec

t.



MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

M

M

MM

MM

M

M

MM

MM

MM

MM

MM

MM

MM

MM

M

M

MM

MM

M

M

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

M

M

MM

MM

M

M

MM

MM

MM

MM

MM

MM

MM

MM

M

M

MM

MM

M

M

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

MM

M

M

MM

MM

M

M

MM

MM

MM

MM

MM

MM

MM

MM

MM

3div

i_00

0ay

onix

_000

cybe

rext

rude

r_00

1de

rmal

og_0

02de

rmal

og_0

03di

gita

lbar

riers

_000

digi

talb

arrie

rs_0

01id

3_00

1id

3_00

2in

nova

tric

s_00

0in

nova

tric

s_00

1is

ityou

_000

itmo_

001

itmo_

002

mor

pho_

000

neur

otec

hnol

ogy_

000

neur

otec

hnol

ogy_

001

ntec

hlab

_000

ntec

hlab

_001

rank

one_

000

rank

one_

002

sam

tech

_000

tong

yitr

ans_

001

tong

yitr

ans_

002

vcog

_001

vcog

_002

vigi

lant

solu

tions

_000

vigi

lant

solu

tions

_001

visi

onla

bs_0

01vo

cord

_001

voco

rd_0

02yi

tu_0

00

0.01

0.02

0.04

0.06

0.08

0.10

0.20

0.30

0.40

0.60

0.80

0.01

0.02

0.04

0.06

0.08

0.10

0.20

0.30

0.40

0.60

0.80

0.01

0.02

0.04

0.06

0.08

0.10

0.20

0.30

0.40

0.60

0.80

0.01

0.02

0.04

0.06

0.08

0.10

0.20

0.30

0.40

0.60

0.80

0.01

0.02

0.04

0.06

0.08

0.10

0.20

0.30

0.40

0.60

0.80

0.01

0.02

0.04

0.06

0.08

0.10

0.20

0.30

0.40

0.60

0.80

1e

041e

03

1e

021e

01

1e

041e

03

1e

021e

01

Fals

e m

atch

rat

e (F

MR

)


fmr_

no

min

al

0.00

001

0.00

003

0.00

010

0.00

030

0.00

100

0.00

300

Figu

re13

:Fo

rth

evi

saim

ages

,FN

MR

and

FMR

atsi

xop

erat

ing

poin

tsal

ong

the

DET

char

acte

rist

ic.

At

each

poin

ta

line

isdr

awn

betw

een

(FM

R,F

NM

R) M

AL

Ean

d(F

MR

,FN

MR

) FE

MA

LE

show

ing

how

whi

chse

xha

slo

wer

FMR

and/

orFN

MR

.The

M

labe

lden

otes

mal

e,th

eot

her

end

ofth

elin

eco

rres

pond

sto

fem

ale.

The

six

oper

atin

gth

resh

olds

are

sele

cted

togi

veth

eno

min

alfa

lse

mat

chra

tes

give

nin

the

lege

nd,a

ndar

eco

mpu

ted

over

alli

mpo

stor

pair

sre

gard

less

ofag

e,se

x,an

dpl

ace

ofbi

rth.

The

plot

ted

FMR

valu

esar

ebr

oadl

yan

orde

rof

mag

nitu

dela

rger

than

the

nom

inal

rate

sbe

caus

eFM

Ris

com

pute

dov

erde

mog

raph

ical

ly-m

atch

edim

post

orpa

irs

i.ein

divi

dual

sof

the

sam

ese

x,fr

omth

esa

me

geog

raph

icre

gion

(see

sect

ion

4.6.

1),a

ndth

esa

me

age

grou

p(s

eese

ctio

n4.

6.2)

.



3div

i_00

0ay

onix

_000

cybe

rext

rude

r_00

1de

rmal

og_0

02de

rmal

og_0

03di

gita

lbar

riers

_000

digi

talb

arrie

rs_0

01id

3_00

1id

3_00

2in

nova

tric

s_00

0in

nova

tric

s_00

1is

ityou

_000

itmo_

001

itmo_

002

mor

pho_

000

neur

otec

hnol

ogy_

000

neur

otec

hnol

ogy_

001

ntec

hlab

_000

ntec

hlab

_001

rank

one_

000

rank

one_

002

sam

tech

_000

tong

yitr

ans_

001

tong

yitr

ans_

002

vcog

_002

vigi

lant

solu

tions

_000

vigi

lant

solu

tions

_001

visi

onla

bs_0

01vo

cord

_001

voco

rd_0

02

yitu

_000

0.1

0.3

0.5

0.1

0.3

0.5

0.1

0.3

0.5

0.1

0.3

0.5

0.1

0.3

0.5

0

ongoing face recognition vendor test (frvt) part 1 ... face recognition vendor test (frvt) part ......

Documents