a cross-sensor evaluation of three commercial iris cameras for iris biometrics ryan connaughton and...

A Cross-Sensor Evaluation of Three Commercial Iris Cameras for Iris

Biometrics

Ryan Connaughtonand

Amanda Sgroi

June 20, 2011CVPR Biometrics Workshop

Computer Vision Research LabDepartment of Computer Science & Engineering

University of Notre Dame

Objectives

Compare three commercially available sensors

– Does one sensor consistently out-perform the others?

– What factors impact sensor performance the most?

Observe performance of sensors in a cross-sensor scenario

– What kind of performance can we expect from a cross-sensor system?

– What is the relationship between single-sensor and cross-sensor performance? 2

Overview of Experiment Strategy

Collect images for the same subjects using all 3 sensors under similar conditions

Use 3 different matching algorithms to perform matching experiments

• In a single-sensor context

• In a cross-sensor context

Analyze performance of sensors in each scenario

3

Sensors

4

Sensor

Iris-to-Sensor Distance

Wavelength(s) of NIR Illumination

Type of Illumination (cross or direct)

Acquisition Instructions

S1 8 to 12 inches 820 nm Both (same time) Sensor Prompt

S2 10 to 14 inches

770 and 870 nm Both (different times)

Sensor Prompt

S3 13 inches 870 and 760 nm Cross * Operator

*Speculation

Image Examples

5

S1 S2 S3

Same Subject, Same Session Images

Data Collection Results

23,444 Iris Images acquired, spanning 510 subjects (1,020 unique irises)

6

The Matching Algorithms

A1

- Similarity Score

- Asymmetric Scores

A2

- Distance Score

- Asymmetric Scores

A3

- Distance Score

- Symmetric Scores

7

8

Segmentation Information

9

Segmentation Information

Note: Image information using A1 is not easily accessible and is thus not included.

Match and Non-Match Comparisons

10

The Experiments

S1 vs S1

S2 vs S2

S3 vs S3

S1 vs S2

S1 vs S3

S2 vs S3

11

Single-Sensor &

Cross-Session

Cross-Sensor &

Cross-Session

These experiments were repeated for all 3 matchers

12

ROC Curves Using A1

13

ROC Curves Using A2

14

ROC Curves Using A3

15

TAR's at FAR = 0.01

A1 A2 A3

S1 vs S1 0.9997 (1) 0.9898 (2) 0.9949 (1)

S2 vs S2 0.9993 (3) 0.9937 (1) 0.9857 (4)

S3 vs S3 0.9978 (6) 0.9819 (6) 0.9818 (5)

S1 vs S2 0.9995 (2) 0.9890 (3) 0.9858 (3)

S1 vs S3 0.9986 (4) 0.9848 (5) 0.9870 (2)

S2 vs S3 0.9984 (5) 0.9856 (4) 0.9807 (6)

Numbers in parentheses indicate ranking within the corresponding matching algorithm

16

Sensor Rankings @ FAR = 0.01

A1 A2 A3

1 S1 vs S1 S2 vs S2 S1 vs S1

2 S1 vs S2 S1 vs S1 S1 vs S3

3 S2 vs S2 S1 vs S2 S1 vs S2

4 S1 vs S3 S2 vs S3 S2 vs S2

5 S2 vs S3 S1 vs S3 S3 vs S3

6 S3 vs S3 S3 vs S3 S2 vs S3

Brackets indicate that performance difference at FAR=0.01 may not be statistically significant

17

Single-Sensor Conclusions

S3 consistently performed the worst for all matchers

S1 was best for 2 of 3 matchers

Best overall performance was achieved using S1 sensor with A1 matcher (TAR=0.9997 @ FAR=0.01)

18

Cross-Sensor Conclusions

A1: Cross-sensor performance was between performance of individual sensors

A2: In general, cross-sensor performance was between performance of individual sensors

– S1 vs S2 actually performed slightly worse than either single sensor

A3: Individual sensor performance is not a good predictor of cross-sensor performance

– S1 vs S3 appears to perform better than S1 vs S2

19

General Conclusions

Sensors and matching algorithms should be evaluated in combination, not separately

In some cases, adding a new and “better” sensor for cross-sensor matching will increase performance, but in some cases it will degrade performance

Single-sensor performance is not always a reliable predictor of cross-sensor performance

20

Future Work

Which results are statistically significant?

What factors have the largest effect on performance?

– Pixels on the iris

– Dilation ratio

– Occlusion

– Contact Lenses

– Order of sensors during acquisition

– Focus or other quality metrics

– Illumination

21

Thanks!

Questions?

Acknowledgments: This work is sponsored under IARPA BAA 09-02 through the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-10-2-0067. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of IARPA, the Army Research Laboratory, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.