template generator 2g - nist · 2019-02-15 · 2g ongoing minex: template generator report card 5 3...

Ongoing MINEX Report Card

Template Generator 2G

Last Updated: October 9, 2015

2G Ongoing MINEX: Template Generator Report Card 1

Participant DetailsCompany: Suprema Inc.Date Submitted: 2/28/2008Date Validated: 10/21/2008Date Completed: 11/3/2008

Library Size (bytes) MD5 Checksumlibminex.dll 184832 c6c56cf8693fc8a8760d4b85e0377778

NOTE: NIST plans to decertify Windows-based libraries in MINEX III.

Compliance Test ResultsThe following presents PIV compliance results per the criteria detailed in NIST Special Publication 800-76-2:Biometric Specifications for Personal Identity Verification.

PIV: PASS• All certified matchers must be able to match templates from this template generator with an

FNMRFMR(0.01) ≤ 0.01 using two fingers (4.5.2.2-3). 3• Minutia density plots derived from generated templates do not exhibit a periodic, grid-like, or geometric

structure without reasonable justification. 3

Notes• This report will be updated as new matching algorithms and template generators pass the compliance

test. These updates will not change the PASS/FAIL decision above.

• NIST reserves the right to decertify a template generator if it later discovers the template generator violatesPIV specifications in some previously undetected way.

1 Last Updated: October 9, 2015

http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-76-2.pdf




Contents

Participant Details 1

Compliance Test Results 1

Notes 1

1 Introduction 3

2 Methodology 32.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Accuracy Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.3 Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3 Results 53.1 Single Finger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.2 Two Finger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.3 Minutia Counts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.4 Minutia Density Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 Performance Tables 12

5 References 20

List of Figures1 MINEX Interoperability Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 DET (Single Finger) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 DET (Left Index) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 DET (Right Index) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 FNMR @ FMR = 0.01 (Single Finger) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 DET (Two Finger) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 FNMR @ FMR = 0.01 (Two Finger) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Minutia Counts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 2D Minutia density plots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

List of Tables1 Single finger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Right index finger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Left index finger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 Two finger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18



1 IntroductionTesting is performed at a NIST facility. Each participant’s submission is validated by NIST before undergoingfull testing to ensure it operates correctly. If the matcher passes the validation procedure, it is then used to com-pare standard fingerprint templates. Performance is assessed against templates created by a template generatorsubmitted by the participant as well as templates created by other compliant template generators.

2 MethodologyTesting is performed at a NIST facility. Each participant’s submission is validated by NIST before undergo-ing full testing to ensure it operates correctly. If the template generator passes the validation procedure, per-formance is assessed by using MINEX compliant matching algorithms to compare templates created by thetemplate generator. These matchers were submitted to the ongoing MINEX program by various participants.

2.1 DatasetTesting is performed over a single dataset of sequestered fingerprint images. The images were collected by U.S.Visit at ports of entry into the United States. They consist of Live-scan plain impressions of left and right indexfingers. WSQ [1] compression was applied to all images at a ratio of 15:1. The most recent capture of eachsubject was treated as the authentication sample, and the next most recent as the enrolled sample.

The dataset was divided into 123 962 mated and 124 994 non-mated subject pairings. Since both left and rightindex fingerprints are available for each subject, this provides 247 924 mated and 249 988 nonmated single-fingercomparisons (after database consolidation). This also means that when left and right index fingers are fused atthe score level [2, 7], the sets condense to 123 962 mated and 124 994 nonmated comparison scores.

2.2 Accuracy MetricsCore matching accuracy is presented in the form of Detection Error Tradeoff (DET) plots [6], which show thetrade-off between the False Match Rate (FMR) and the False Non-Match Rate (FNMR) as a decision thresholdis adjusted. Formally, let mi (i = 1 . . .M ) be the ith mated comparison score, and nj (j = 1 . . . N ) the jthnon-mated comparison score. Then the statistics are

FMR(τ) =1

N

N∑j=1

1{nj ≥ τ}, (1)

FNMR(τ) =1

M

M∑i=1

1{mi < τ}. (2)

where 1{A} is the indicator [3] of event A. Equations 1 and 2 define the curve parametrically with the decisionthreshold, τ , as the free parameter. In some figures and tables, FNMR is presented as a function of FMR. Thisrelationship is determined by

FNMRFMR(α) = minτ

{ FNMR(τ) | FMR(τ) ≤ α }, (3)

which reads as the smallest FNMR that can be achieved while maintaining an FMR less than or equal to α, thetargeted FMR. This method of relating the two error statistics ensures FNMR is well-defined for all 0 ≤ α ≤ 1.It also imposes a natural penalty on matching algorithms that produce heavily discretized scores.



Figure 1: MINEX Interoperability Test Setup

2.3 InteroperabilityInteroperability is tested in a manner similar to Scenario 1 from the MINEX Evaluation Report [4] (see Figure1). An enrolment template is prepared using submission X. Submission Y is used to prepare the authenticationtemplate and perform the match. The authentication template is always prepared by the same submission usedto compare the templates. However, enrolment templates need not originate from the same submission. Whenthey do, we refer to as ”native” mode.


http://www.nist.gov/customcf/get_pdf.cfm?pub_id=150619


3 ResultsThis section details the performance of template generator 2G. Sections 3.1 and 3.2 present accuracy results forsingle finger and two finger matching respectively. Section 3.3 presents information on the number of minutiathe template generator finds in the samples.

3.1 Single FingerSinge finger comparison results show the combined results for left and right index comparisons. For reference,NIST Special Publication 800-76-2 requires that the template generator achieve an accuracy of FNMRFMR(0.01) ≤0.01 against all compliant matchers.

Last Updated: Sep 10, 2015

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

4S

4S4S

4S4S

4S

4S4S

4S4S

4S

4S4S

4S4S

4H

4H

4H

●

●

●

●

●

●

●

●

●

●

●

●● ● ● ● ● ●

1I

1I1I

1I1R

1I

1R1R

2T2T

2T2T

3J 3J 3J 3J 3J 3J

0.001

0.002

0.005

0.010

0.020

0.050

0.100

0.200

0.0001 0.0002 0.0005 0.0010 0.0020 0.0050 0.0100 0.0200 0.0500 0.1000 0.2000FMR

FN

MR

Single FingerTemplate Generator = 2G, Num Mated = 247924, Num Nonmated = 249988

Figure 2: Single finger DET statistics for template generator 2G. Each box shows the distribution of FNMRs at a fixedFMR across different matchers. The whisker ends show the minimum and maximum FNMRs. The brown curve showsthe DET curve when the matcher and template generators were submitted by the same participant. The orange DET curveshows pooled performance when all matchers compare templates created by 2G.




●

●●

●●

●●

●● ●

●

●●

●

●

●

●

●

4S4S 4S 4S 4S 4S

4S 4S4S 4S 4S

4S4S

4H4H

4H

4H

4H

●

●●

●●

●

●●

●●

●● ● ● ● ● ● ●

1R1I

1R1R 1R 1R

1R1R

1R 1R1R 3J 3J 3J 3J 3J 3J 3J

0.0005

0.0010

0.0020

0.0050

0.0100

0.0200

0.0500

0.1000

0.0001 0.0002 0.0005 0.0010 0.0020 0.0050 0.0100 0.0200 0.0500 0.1000 0.2000FMR

FN

MR

Right Index FingerTemplate Generator = 2G, Num Mated = 123962, Num Nonmated = 124994

Figure 3: Right Index Finger DET statistics for template generator 2G. Each box shows the distribution of FNMRs ata fixed FMR across different matchers. The whisker ends show the minimum and maximum FNMRs. The brown curveshows the DET curve when the matcher and template generators were submitted by the same participant. The orange DETcurve shows pooled performance when all matchers use templates created by 2G.


●●

●●

●●

●●

●●

●

●●

●●

●

●

●

4S4S 4S 4S 4S 4S

4S 4S 4S 4S 4S4S

4S4S

4S4T

4S

4S

●

●●

●●

●

●●

●●

●●

● ● ● ● ● ●

1I1I 1I

1I 1I 1I1I 2T 2T 2T 2T 2T 2T 3J 3J 3J 3J 3J

0.001

0.002

0.005

0.010

0.020

0.050

0.100

0.200

0.0001 0.0002 0.0005 0.0010 0.0020 0.0050 0.0100 0.0200 0.0500 0.1000 0.2000FMR

FN

MR

Left Index FingerTemplate Generator = 2G, Num Mated = 123962, Num Nonmated = 124994

Figure 4: Left Index Finger DET statistics for template generator 2G. Each box shows the distribution of FNMRs at afixed FMR across different template generators. The brown curve shows the DET curve when the matcher and templategenerators were submitted by the same participant. The orange DET curve shows pooled performance when all matchersuse templates created by 2G.




0.0562

0.0819

0.0440

0.0812

0.0578

0.0715

0.0580

0.06280.0628

0.1341

0.0778

0.0408

0.1223

0.0657

0.0778

0.0819

0.1102

0.0429

0.0685

0.0926

0.0719

0.0495

0.0327

0.0642

0.0725

0.0708

0.0523

0.0226

0.0605

0.0447

0.0604

0.0652

0.0510

0.0913

0.0380

0.0811

0.0210

0.0510

0.0249

0.0610

0.0266

0.0682

0.0407

0.0475

0.0643

0.0190

0.0292

0.0406

0.0497

0.0265

1I1R2C2J

3QC

2AF

3W1W1J2S2N1C2T2I

4L1T3N4Q2R1G1F4I

3D3J1EGA

2Y4A3O4X2O4P3G

D2G1L4O4U3T2Q4T4J4Z4H3A3Z4S

0.02 0.04 0.06 0.08 0.10FNMR

Mat

cher

Single FingerTemplate Generator = 2G, Num Mated = 247924, Num Nonmated = 249988

Figure 5: Single Finger FNMRs at FMR=0.0001 when MINEX compliant matchers compare templates created by tem-plate generator 2G. Each box represents uncertainty about the true FNMR. The box edges mark the 50% confidenceintervals while the whiskers mark the 90% confindence intervals. The numbers on the right show the actual computedFNMRs.



3.2 Two FingerThis section presents accuracy when different MINEX compliant matchers compare templates created by tem-plate generator 2G. Two finger fusion is achieved by averaging the scores for left and right index fingers foreach person.


●Conformance Requirement

●●

●●

●●

●

●● ●

●

●

●

●

●

●

●

●

4S 4S4S

4S 4S4S

4S4S

4S 4S 4S

4T

4T

4T

4T

4T

4T

4T

●

●●

●●

●

●

●

●●

●

● ● ● ● ● ● ●

1W1W

1W1W

1W1W

1R2J

2J2J

3J3J 3J 3J 3J 3J 3J 3J

0.0001

0.0002

0.0005

0.0010

0.0020

0.0050

0.0100

0.0200

0.0500

0.1000

0.0001 0.0002 0.0005 0.0010 0.0020 0.0050 0.0100 0.0200 0.0500 0.1000 0.2000FMR

FN

MR

Two FingerTemplate Generator = 2G, Num Mated = 247924, Num Nonmated = 249988

Figure 6: Two Finger DET statistics for template generator 2G. Each box shows the distribution of FNMRs at a fixedFMR across different matchers. The whisker ends show the minimum and maximum FNMRs. The brown curve showsthe DET curve when the matcher and template generators were submitted by the same participant. The orange DET curveshows pooled performance when all matchers use templates created by 2G. Score-level fusion is achieved by averaging thescores for left and right index fingers.




0.0032

0.0044

0.0021

0.0044

0.0025

0.0041

0.0024

0.00280.0028

0.0040

0.0033

0.0020

0.0037

0.00270.0027

0.0044

0.0036

0.0013

0.0013

0.0043

0.0031

0.0014

0.0013

0.0029

0.0033

0.0044

0.0021

0.0011

0.0026

0.0016

0.0046

0.0013

0.0018

0.0033

0.0014

0.0046

0.0007

0.0033

0.0005

0.0015

0.0007

0.0033

0.0013

0.0030

0.0012

0.00050.0005

0.0010

0.0034

0.0010

3J3W

C2A

F2T2J1C1I

1R2C4X4A1J

3Q4L2S

A2N4P2R1F1G1T

1W3DG

1E2YD

1L3O3G4I

2O3T3N2Q2G4O2I

4Q3A4U4Z3Z4J4H4T4S

0.0004 0.0006 0.0008 0.0010 0.0020 0.0040 0.0060FNMR

Mat

cher

Two FingerTemplate Generator = 2G, Num Mated = 247924, Num Nonmated = 249988

Figure 7: Two Finger FNMR at FMR=0.01 when different matchers compare templates created by template generator2G. Each box represents uncertainty about the true FNMR. The box edges mark the 50% confidence intervals while thewhiskers mark the 90% confindence intervals. The numbers on the right show the actual computed FNMRs. Score-levelfusion is achieved by averaging the scores for left and right index fingers.



3.3 Minutia CountsThis section presents information relating to the number of minutia the template generator finds in fingerprintimages. The relative number of minutia found in common fingerprint images has been shown to influencematching outcomes [8, 5].

0.00

0.01

0.02

0.03

0 25 50 75 100 125Minutiae Count

AverageTemplate Generator 2G

Figure 8: Probability distribution of the number of minutia the template generator found in the samples. The averageprobability distribution shows the combined distribution of minutia counts across all compliant template generators.



3.4 Minutia Density PlotsMinutia density plots show where the template generator tends to find minutia in fingerprint images. They are2D histograms where the degree of illumination at an (x, y) coordinate indicates how frequently the softwarelocated a minutiae point at that location. The purpose of showing minutia density plots is to determine whetherthe template generator exhibits regional preference when locating minutia.

Some template generators produce minutia that exhibit a periodic structure, but this template generatordoes not. This is an indication that the template generator is departing from the minutia placement require-ments of INCITS 378, clause 5. The expected pattern is a locally uniform distribution, and the appearance oflocal structure indicates systematic non-conformance with the standard. Given such behavior negatively affectsinteroperability[8], developers are asked to determine the cause of such behavior – for example, as an artifactof a tilebased image processing algorithms applied to the input fingerprint image and to resubmit correctedalgorithms.

(a) Minutia density plot for 251 874 368x368 right indexes. (b) Minutia density plot for 251 874 368x368 left indexes.

(c) Minutia density plot for 123 120 500x500 right indexes. (d) Minutia density plot for 123 120 500x500 left indexes.

Figure 9: 2D Minutia density plots.



4 Performance TablesThe following tables present accuracy numbers, including estimates of uncertainty in the form of 90% confi-dence bounds. These tables are provided because most of the figures in the main body of the report do notpresent actual accuracy numbers.

Table 1: Single finger FNMRs at various FMRs when 2G and MINEX compliant matchers comparetemplates created by template generator 2G.

Matcher FNMR @ FMR=0.01 FNMR @ FMR=0.001 FNMR @ FMR=0.0001A 0.0242± 0.0005 0.0357± 0.0006 0.0562± 0.0008B − − −C 0.0277± 0.0005 0.0506± 0.0007 0.0819± 0.0009D 0.0185± 0.0004 0.0297± 0.0006 0.0440± 0.0007F 0.0278± 0.0005 0.0501± 0.0007 0.0812± 0.0009G 0.0257± 0.0005 0.0334± 0.0006 0.0578± 0.00081C 0.0274± 0.0005 0.0462± 0.0007 0.0715± 0.00091E 0.0230± 0.0005 0.0373± 0.0006 0.0580± 0.00081F 0.0234± 0.0005 0.0407± 0.0007 0.0628± 0.00081G 0.0235± 0.0005 0.0407± 0.0007 0.0628± 0.00081I 0.0314± 0.0006 0.0641± 0.0008 0.134± 0.0011J 0.0238± 0.0005 0.0466± 0.0007 0.0778± 0.00091L 0.0172± 0.0004 0.0279± 0.0005 0.0408± 0.00071R 0.0314± 0.0006 0.0637± 0.0008 0.122± 0.0011T 0.0239± 0.0005 0.0414± 0.0007 0.0657± 0.00081W 0.0231± 0.0005 0.0451± 0.0007 0.0778± 0.00092A 0.0277± 0.0005 0.0506± 0.0007 0.0819± 0.00092C 0.0277± 0.0005 0.0585± 0.0008 0.110± 0.0012G 0.0137± 0.0004 0.0263± 0.0005 0.0429± 0.00072I 0.0158± 0.0004 0.0348± 0.0006 0.0685± 0.00082J 0.0282± 0.0005 0.0544± 0.0007 0.0926± 0.00102N 0.0248± 0.0005 0.0419± 0.0007 0.0719± 0.00092O 0.0119± 0.0004 0.0239± 0.0005 0.0495± 0.00072Q 0.0130± 0.0004 0.0218± 0.0005 0.0327± 0.00062R 0.0231± 0.0005 0.0411± 0.0007 0.0642± 0.00082S 0.0249± 0.0005 0.0432± 0.0007 0.0725± 0.00092T 0.0343± 0.0006 0.0516± 0.0007 0.0708± 0.00082Y 0.0176± 0.0004 0.0304± 0.0006 0.0523± 0.00073A 0.0100± 0.0003 0.0151± 0.0004 0.0226± 0.00053D 0.0239± 0.0005 0.0399± 0.0006 0.0605± 0.00083G 0.0154± 0.0004 0.0285± 0.0005 0.0447± 0.00073J 0.0304± 0.0006 0.0420± 0.0007 0.0604± 0.00083N 0.0158± 0.0004 0.0350± 0.0006 0.0652± 0.00083O 0.0184± 0.0004 0.0325± 0.0006 0.0510± 0.00073Q 0.0299± 0.0006 0.0585± 0.0008 0.0913± 0.00103T 0.0136± 0.0004 0.0239± 0.0005 0.0380± 0.00063W 0.0276± 0.0005 0.0499± 0.0007 0.0811± 0.00093Z 0.0086± 0.0003 0.0138± 0.0004 0.0210± 0.00054A 0.0246± 0.0005 0.0370± 0.0006 0.0510± 0.00074H 0.0077± 0.0003 0.0152± 0.0004 0.0249± 0.0005



Table 1: (continued)

Matcher FNMR @ FMR=0.01 FNMR @ FMR=0.001 FNMR @ FMR=0.00014I 0.0168± 0.0004 0.0339± 0.0006 0.0610± 0.00084J 0.0079± 0.0003 0.0152± 0.0004 0.0266± 0.00054L 0.0244± 0.0005 0.0393± 0.0006 0.0682± 0.00084O 0.0127± 0.0004 0.0253± 0.0005 0.0407± 0.00074P 0.0203± 0.0005 0.0313± 0.0006 0.0475± 0.00074Q 0.0141± 0.0004 0.0279± 0.0005 0.0643± 0.00084S 0.0065± 0.0003 0.0111± 0.0003 0.0190± 0.00054T 0.0092± 0.0003 0.0169± 0.0004 0.0292± 0.00064U 0.0120± 0.0004 0.0223± 0.0005 0.0406± 0.00074X 0.0251± 0.0005 0.0355± 0.0006 0.0497± 0.00074Z 0.0094± 0.0003 0.0165± 0.0004 0.0265± 0.0005



Table 2: Right index finger FNMRs at various FMRs when 2G and MINEX compliant matchers com-pare templates created by template generator 2G.




Table 3: Left index finger FNMRs at various FMRs when 2G and MINEX compliant matchers comparetemplates created by template generator 2G.




Table 4: Two finger FNMRs at various FMRs when 2G and MINEX compliant matchers comparetemplates created by template generator 2G.




5 References[1] Jonathan N. Bradley, Christopher M. Brislawn, and Thomas Hopper. FBI wavelet/scalar quantization stan-

dard for gray-scale fingerprint image compression. In SPIE, Visual Information Processing II, 1961. 3

[2] Patrick Grother Elham Tabassi, George W. Quinn. When to fuse two biometrics. In IEEE Computer Society onComputer Vision and Pattern Recognition, Workshop on Multi-Biometrics, 2006. 3

[3] Robert Fontana, Giovanni Pistone, and Maria Rogantin. Cliassification of two-level factorial fractions. Jour-nal of Statistical Planning and Inference, 87:149–172, 2000. 3

[4] P. Grother, M. McCabe, C. Watson, M. Indovina, W. Salamon, P. Flanagan, E. Tabassi, E. Newton, and C. Wil-son. Performance and Interoperability of the INCITS 378 Fingerprint Template. Technical report, NIST, 2006.4

[5] Olaf Henniger and Dirk Scheuermann. Minutiae template conformance and interoperability issues. InArslan Bromme, Christoph Busch, and Detlef Huhnlein, editors, BIOSIG, volume 108 of LNI, pages 25–32.GI, 2007. 10

[6] A. Martin, G. Doddington, T. Kamm, M. Ordowski, and M. Przybocki. The DET curve in assessment ofdetection task performance. In Proc. Eurospeech, pages 1895–1898, 1997. 3

[7] George W. Quinn. Evaluation of latent fingerprint technologies: Fusion. In NIST Latent Fingerprint TestingWorkshop Recognition, Workshop, 2009. 3

[8] Elham Tabassi, Patrick Grother, Wayne Salamon, and Craig Watson. Minutiae interoperability. In ArslanBromme, Christoph Busch, and Detlef Huhnlein, editors, BIOSIG, volume 155 of LNI, pages 13–30. GI, 2009.10, 11


template generator 2g - nist · 2019-02-15 · 2g ongoing minex: template generator report card 5 3...

Documents