University of Notre Dame biometrics researchers Kevin Bowyer and Estefan Ortiz have released a paper that points to errors in the National Institute of Standards and Technology (NIST) report on how iris aging affects the accuracy of iris recognition. In addition to describing specific methodological errors in the NIST report, Bowyer and Ortiz presented a list of suggestions to be addressed in a revised version of the NIST report.
In July, NIST released a report titled “IREX VI: Temporal Stability of Iris Recognition Accuracy.” The IREX VI report analyzed an iris recognition data set acquired from a successful, ongoing Canadian border-crossing application. Based on analysis of this data set, the IREX VI report concluded that its “best estimate of iris recognition aging” is so small that there should be no concern about the possibility of iris recognition accuracy degrading over time. This result could be considered surprising, since it appears to contradict results of previous research by the Notre Dame research group, as well as work by researchers at Michigan State University, Clarkson University, West Virginia University, Warsaw University of Technology and elsewhere.
However, Bowyer and Ortiz pointed to various flaws in the IREX VI report that may explain why it reached conclusions that appear to be at odds with previous research. One factor that may explain the different conclusions is simply that the IREX VI report defined “iris aging” in a way that is fundamentally different from the ISO standard definition of “iris template aging” that guided previous researchers. Previous researchers have attempted to measure the change in error rate for iris recognition over time, without excluding any possible causes of the increase in error rate. In contrast, IREX VI focused on only the change in error rate that can be tied to change in the appearance, and specifically excluded any change related to change in pupil dilation. It is possible that NIST researchers could find a small aging effect for the phenomenon that they study, and all for all previous research that studied the more general iris template aging to also be correct.
Bowyer and Ortiz also pointed to methodological errors in the regression analysis used in the IREX VI report. One is that the data set used in the IREX VI report is “truncated,” meaning that all data points with a value above a threshold were deleted. The IREX VI regression analysis did not take account of this, resulting in an estimate for iris aging that is biased to be lower than it should be in reality. A second error is that the IREX VI report used one result of its regression analysis, taken “out of context,” to represent the effect of iris aging, when in fact the effect of aging may be present in several results in the IREX VI regression model. This could also result in an estimate of iris aging that is smaller than it is in reality. A third error is that the data set used in IREX VI is a mixture of data points resulting from first, second and third attempts at iris recognition in the border-crossing application, and this mixture could introduce bias that caused the estimated effect of iris aging to be smaller than it is in reality.
Bowyer and Ortiz presented a list of suggestions for improving a revised IREX VI report. The suggestions include obtaining a new version of the data set used in the analysis, and using regression analysis methods appropriate to the data set.
Notre Dame research has played a role in previous IREX reports issued by the National Institute of Standards. For instance, previous Notre Dame research on the effects of varying pupil dilation on the accuracy of iris recognition is discussed in the IREX I and the IREX III reports.
Bowyer is the Schubmehl-Prein Professor and the chair of the Department of Computer Science and Engineering at the University of Notre Dame, as well as co-editor of the “Handbook of Iris Recognition.” Ortiz is a Ph.D. candidate in Notre Dame’s Department of Computer Science and Engineering.
The Bowyer and Ortiz report:
Kevin W. Bowyer and Estefan Ortiz,
Making Sense of the IREX VI Report,
Computer Vision Research Lab Technical Report, December 23, 2013.
The IREX VI report:
P. Grother, J. R. Matey, E. Tabassi, G. W. Quinn and M. Chumakov,
IREX VI: Temporal Stability of Iris Recognition Accuracy,
NIST Interagency Report 7948, version dated July 24, 2013.
Contact: Kevin W. Bowyer, 574-631-9978, kwb@cse.nd.edu
Originally published by at news.nd.edu on January 13, 2014.