Critical Analysis of Adaptive Biometric Systems

Norman Poha, Ajita Rattani∗ and Fabio Roli

Department of Computing, FEPS, University of Surrey, Guildford, UKa

Department of Electrical and Electronic Engineering, University of Cagliari

Piazza d’Armi, Cagliari, Italy

normanpoh@ieee.org,{ajita.rattani,roli}@diee.unica.it

December 18, 2012

Abstract

Biometric based person recognition poses a challenging problem because of large variability in biometric sample

quality encountered during testing and a restricted number of enrollment samples for training. Solutions in the form

of adaptive biometrics have been introduced to address this issue. These adaptive biometric systems aim to adapt

enrolled templates to variations in samples observed during operations. However, despite numerous advantages, few

commercial vendors have adopted auto-update procedures in their products. This is due in part to the limited under-

standing and limitations associated with existing adaptation schemes. In view that the topic of adaptive biometrics has

not been systematically investigated, this paper works toward filling this gap by surveying the topic from a growing

body of the recent literature and by providing a coherent view (critical analysis) of the limitations of the existing

systems. In addition, we have also identified novel research directions and proposed a novel framework. The overall

aim is to advance the state-of-the-art and improve the quality of discourse in this field.

1 Introduction

While the biometric technology continues to improve, an intrinsic characteristic of this technology is that a sys-

tem’s error rate, e.g., the false accept rate (FAR), false reject rate (FRR) and equal error rate (EER) (the rate at which

FAR is equal to FRR), cannot attain the absolute zero. A major cause of these errors is the compound effect of the

scarcity of training samples during the enrollment phase as well as the presence of substantial sample variations due

to human-sensor interaction and the acquisition environment during operations [1]. Apart from this, being biolog-

ical tissues in nature, biometric traits can be altered either temporarily or permanently, due to ageing [2], diseases

or treatment to diseases. An important consequence of these factors is that a biometric reference 1 (obtained during1A template refers to the biometric sample used for enrolment. The term model refers to a statistical representation derived from one or more

biometric samples. In order for our discussion to cover both types of method, we shall adapt the standard vocabulary that is biometric reference or

1

enrollment) cannot be expected to fully represent a person’s identity.

Solutions in the form of adaptive biometrics have been introduced to address this issue of reference representa-

tiveness [3, 4]. These adaptive biometric systems attempt to update reference galleries by integrating information

captured in input operational samples. The two-fold aim is to continuously adapt the biometric system to the intra-

class variation of the input data as a result of (1) changing acquisition conditions that may have adverse impact on

the system, e.g., pose and illumination changes for face biometrics, and (2) age and life-style related changes that can

cause permanent changes to the biometric trait.

Most of the existing automated adaptive biometric systems have adopted semi-supervised learning [11, 4] for

the purpose of adaptation. Semi-supervised learning is a machine learning scheme based on the joint use of labeled

and unlabeled samples. In other words, input samples are assigned identity labels using enrolled references and the

positively classified samples are used to adapt the references. A commonly adopted adaptation procedure is to augment

the reference set with the newly classified input samples. The efficacy of the system can be gauged by comparing

the obtained performance gain with a traditional biometric system which does not have any adaptation mechanism.

The expected performance gain is dependent on the effective labeling (classification) of the input samples. This is

because misclassification errors will introduce impostor samples into the updated reference set, the result of which

can be counterproductive. An adaptive biometric system may also operate in supervised mode in which biometric

samples are manually labeled [3]. The supervised method represents the best case performance as all the available

positive (genuine) samples are used for adaptation. However, manual intervention may be time consuming and costly.

Therefore, it is generally infeasible to manually update references regularly.

In contrast, an adaptive biometric system has numerous advantages. First, with this system, one no longer needs

to collect a large number of biometric samples during enrollment. Second, it is no longer necessary to re-enrol or

re-train the system (classifier) from scratch in order to cope up with the changing environment [3]. This convenience

can significantly reduce the cost of maintaining a biometric system. Third, the actual observed variations can be in-

corporated into the references. Despite these advantages, to our knowledge, few biometric vendors such as BIOsingle

(fingerprint) and Recogsys (hand geometry) have incorporated automated adaptation mechanism into their technolo-

gies at the time of this writing. This is due in part to the limited understanding and limitations associated with existing

adaptive biometric systems.

The goal of this manuscript is to advance the state-of-the-art in adaptive biometrics by improving the understanding

and drawing on the limitations of the existing adaptive biometric systems. To this aim, critical analysis of the existing

literature is conducted. Based on the findings of the critical analysis, we propose a novel framework that aims to

mitigate some of the limitations and investigate possible future research avenues.

Specific contributions of this manuscript are as follows:

simply reference. A reference is subsequently used for comparing a biometric test/query sample to obtain a similarity score.

2

1. a taxonomy of adaptive biometric systems through a number of key attributes,

2. use of a meta-analysis technique to objectively compare the effectiveness of key attributes across various systems

reported in the literature, and

3. identification of novel research directions based on the findings of the above meta-analysis.

A preliminary version of this manuscript appeared in [3] in the form of critical survey. The current manuscript

substantially differs from [3] in the following ways. First, novel attributes that distinguish an adaptive system from one

another are introduced. Second, meta-analysis is utilized to aid analysis of various state-of-the-art adaptive systems.

Last but not least, a novel framework, as well as research directions, is proposed.

The paper is organized as follows: section 2 formulates the key attributes and conducts the meta-analysis. Section

3 provides the novel framework and research directions. Conclusions are drawn in section 4.

2 Attributes and Critical analysis

2.1 Attributes of the existing adaptive biometric systems

In an attempt to categorize adaptive biometric systems, the most logical way to proceed is to define a number of

key attributes. On surveying the current state-of-the-art, we find that the following attributes are relevant to distinguish

one adaptive biometric system from another:

1. Supervised against Semi-supervised: The foremost attribute in classifying adaptive systems is indisputably on

the basis of whether the data labeling process is supervised [3, 4, 5, 9] or unsupervised [12, 13, 4]. While

in supervised adaptation, samples are manually labeled, in the unsupervised case, they are inferred by the

system. The latter approach is generally referred to as semi-supervised learning because the enrolment biometric

reference (template) is effectively labeled but the potential operational biometric samples that are used for

adaptation are unlabeled. As mentioned before, supervised adaptation represents the best case scenario, i.e.,

resulting in the best possible performance because all available genuine samples are used for the process of

adaptation. Therefore, it is generally useful to report both strategies when comparing different adaptive methods.

2. Self- against Co-train: For an automated adaptive systems based on semi-supervised learning, self- [12, 13, 14]

and co-training [17, 4] are the commonly adopted schemes for adaptation. In self-training, the system updates

itself by adding only highly confidently classified input samples as additional data for training. A sample is

said to be highly confidently classified if its matching score on comparison with the enrolled templates is above

a stringent operating threshold. The reason to adopt highly confidently classified samples for adaptation is to

avoid impostor intrusion into the updated template set.

3

On the other hand, a co-training based scheme utilizes the mutual and complementary help of the two biometrics

to update the references. Intuitively, one system is expected to assign correct labels to biometric query samples

that are difficult for another system. Consider an example of face and fingerprint co-training system. While on its

own, the face sub-system may have difficulty in labeling a query sample in difficult conditions, the fingerprint

sub-system may classify the associated fingerprint sample with very high confidence. In this case, the face

system can benefit from the high confidence of the fingerprint by incorporating the additional face samples for

training. Therefore, two systems operating at high threshold can still help each other to identify difficult samples

exhibiting large intra-class variations.

3. Verification against Identification: These adaptive biometric system can also be differentiated on the basis of the

system’s basic mode of operation i.e., verification (input sample is compared to the references of the claimed

identity) or identification (input samples are matched to the references of all the users in the database and then

the correct identity is determined among the top most retrievals) [1]. Accordingly, the performance gain will be

measured using EER or rank-one performance metrics, respectively.

4. Level of adaptation: In addition to the adaptation at the reference level, the process of adaptation can also take

place at score or decision level where the matching score or decision functions are adapted to the variations of

the input biometric samples. For instance, Reference [5] uses biometric sample quality to adapt the matching

score so as to render the final accept/reject decision independent of the input sample quality.

5. Online against offline: Online adaptive systems [12, 13] adapt themselves as soon as the input data is available

after the recognition process. On the other hand, offline methods [15, 16, 4] adapt themselves after a batch of

input samples have been accumulated over a period of time. Another fine distinction between the two is that

while an online method follows the chronological ordering of the availability of the samples during adaptation,

the offline one may not adhere to such an ordering.

6. Quality against non-quality based: Recent advancement in the biometric community 2 shows that biometric

sample quality has considerable impact on the system performance for various traits like fingerprint, iris, face,

and etc, as well as for fusion [7]. Quality measures quantify the degree of excellence or conformance of bio-

metric samples to some predefined criteria known to influence the system performance.

However, it is only recently that biometric sample quality has been considered for adaptive biometric systems [5,

6]. Quality based adaptation requires maintaining a different set of updated references for each type of condition.

Since a query sample is always acquired under a particular condition, the inference (matching task) requires

identification of its quality condition and matching with the set of references of the same quality type [6]. In

this manuscript, the resultant system is termed condition-adaptive system.

2http://www.itl.nist.gov/iad/894.03/quality/workshop/

4

7. Impostor against Non-impostor attack: Adaptive biometric systems deployed in a real operational environment

are vulnerable to impostor attack where an unauthorized user attempts to gain access to the system. However,

early studies did not assume impostor attack during the system’s operation [9, 12, 14, 15]. This is evident by the

fact that the data used for adaptation (called the adaptation set) contained only genuine samples for adaptation.

Later on, this limitation was identified and impostor samples were introduced in the adaptation set to simulate

update process in a real operational environment [16, 21, 23, 28, 4].

In the next section, we will quantify existing adaptive biometric systems based on the mentioned attributes using

meta-analysis.

2.2 Meta Analysis: a tool for critical analysis

Meta-analysis is a quantitative method for analyzing results from multiple papers on the same subject [8]. It has the

property [8] of synthesizing (summarizing) results from multiple independent experiments in a quantitative manner.

Therefore, we adopted meta-analysis as a tool to perform critical analysis and to compare existing adaptive biometric

systems based on the identified key attributes (in section 2.1).

To this end, we divided the existing adaptive systems based on the key attributes i.e., supervised vs. semi-

supervised, self- vs. co-train, online vs. offline, and quality vs. non-quality based. Furthermore, we distinguished

these systems on the basis of inclusion or exclusion of impostor attacks during the adaptation process i.e., impostor

vs. non-impostor attack. The effect of a system’s different mode of operation (i.e., verification or identification) is

mitigated by standardizing the performance metrics (explained in detail in the following subsection).

Then we use meta-analysis to validate the following hypotheses:

1. Is supervised adaptation better than semi-supervised?

2. Can co-training outperform self-training?

3. Is offline adaptation better than its online counter-part?

4. Can quality-based adaptation outperform its non-quality based counterpart?

5. Is there any performance bias if one excludes non-match (impostor) samples in the adaptation set (i.e., no

impostor attack)?

As it turns out, there are already 23% of existing papers addressing the second hypothesis above. Nevertheless,

it would still be interesting to infer the expected difference in the performance gain of co-training over self-training

based on the existing studies.

However, if we would like to infer whether or not the use of biometric sample quality can outperform a non-quality

based adaptive system (hypothesis 4), none of the selected papers directly tested this hypothesis. Only meta-analysis

5

can offer the possibility of utilizing the diverse experiments performed by independent researchers to test hypothesis

4 without recourse to direct experimentation.

In summary, irrespective of the different protocols and data sets, meta-analysis offers a means to objectively

quantify and compare different adaptive biometrics in our survey, as the first recourse. The resultant set of hypotheses

may then be subject to direct testing, i.e., explicit comparison of two adaptive systems on a common data set, if clear

inferences cannot be drawn.

For the purpose of meta-analysis, we selected a total of 22 research papers based on the criteria that these papers

provided sufficient details regarding the obtained performance and clearly stated different attributes of the experiments

(as listed in section 2.1); they are: [9, 12, 13, 14, 15, 17, 16, 20, 21, 5, 22, 23, 18, 24, 19, 25, 10, 26, 27, 28, 29, 30].

Since each paper reported several experiments (with a median of 4), a total of 103 experiments are available for

meta-analysis.

We characterize and summarize the outcome of these 103 experiments 3, based on pre-identified attributes, using

meta-analysis. A generalized linear model (GLM) with linear output is trained with a data table containing one

experiment per line. This model takes a set of attributes (as binary variables) in order to predict the performance

gain as its output. In the following sections, we first explain how a generalized linear model (GLM) can be used

to characterize the outcome of one of the 103 experiments and how the attributes are encoded. We then present the

experimental protocols. The final subsection presents the findings of the meta-analysis.

2.2.1 Standardizing the Performance Statistics

Proceeding to meta-analysis is not straightforward since the performance quoted by each paper is not consistent.

In particular, there are two types of performance metric that are systematically quoted: Equal Error Rate (EER) and

rank-one recognition performance. EER quantifies the probability of error at an operating threshold where the rate of

false acceptance is equal to that of false rejection. It is often quoted in a biometric verification scenario. The rank-

one recognition performance, on the other hand, is quoted in a biometric identification scenario. It is defined as the

probability of a target user is indeed ranked the top from a gallery of registered users.

In order to handle the different metrics used, we opted to derive a secondary metric called performance gain. It

is defined as the amount of improvement with respect to the baseline system as well as the primary target metric one

would like to achieve.

For EER, it is defined as:

Perf. gain =EERb − EERa

EERb − 0(1)

where EERa is the EER of the adaptive system, EERb is the EER of the baseline system, and the zero value in the

denominator is the target EER value which one would like to achieve. For the rank-one recognition performance, we

3The data used for meta-analysis is available in the following link: https://sites.google.com/site/ajitarattaniitaly/resources

6

used the following performance gain definition, instead:

Perf. gain =Perfa − Perfb1− Perfb

(2)

where Perfa denotes the performance of the adaptive system whereas Perfb is the performance of the baseline system,

and the unit value in the denominator is the target rank-one recognition performance.

Despite the differences in definition of the EER and the rank-one recognition metrics, the performance gain metric

has the following properties in both cases. First, a positive performance gain implies improvement over the baseline

system. Second, the maximum performance gain can be almost equal to one. This can be simply verified by the fact

that Perfa ≤ 1 and EERa >= 0. Therefore, given that an adaptive biometric system is always reported to be better

than its baseline counterpart, the performance gain will be bounded between 0 and 1. If a value of 1 is registered,

then the target metric is achieved (that is zero for EER and one for rank-one recognition performance). Therefore, the

performance gain we introduced is a viable means to handle the differences in the two primary metrics used (due to

the different mode of operation i.e., verification or identification) by the researchers, allowing the performance gain of

different systems to be compared on equal ground.

2.2.2 Encoding the Attributes of an Experimental Outcome

An experiment is assigned a code of 4 bits in order to represent the following binary attributes, namely:

1. Presence of quality (quality): 1 means yes; and 0, otherwise

2. Use of co-training (co-train): 1 means yes; and 0, otherwise, which implies either self-training or supervised

adaptation

3. Use of supervised adaptation (supervised): 1 means yes; and 0, otherwise, which implies semi-supervised

adaptation (co-training or self-training)

4. Presence of non-match samples in the data set reserved for adaptation (impostor attack): 1 means pres-

ence; and 0, otherwise.

For instance, an experiment coded as 1101 implies that the experiment involves an adaptive biometric system that

uses biometric sample quality, relies on co-training hence, cannot be supervised and contains non-match samples

(impostor attacks) in the adaptation data set.

In order to compare different attributes, two types of meta-analysis experiments are performed, namely single-

factor and multi-factor analysis. In the former, only one of the four attributes (as explained earlier) is considered,

whereas in the latter, all four attributes are considered at the same time. In order to carry out the two types of

experiments, we used a generalized linear model with linear output that estimates performance gain using eq.(1) or (2).

7

Apart from these binary attributes, we also collected other contextual variables that may impact on the generalization

performance of our analysis. These variables are the database size, number of samples used for adaptation, modality

involved etc. Collected data for the contextual variables 4 demonstrate that face is the commonly adopted modality

followed by fingerprint in the existing studies. The adopted adaptation procedure is the same irrespective of the

modality involved. Existing adaptive studies have handled short-to-medium term temporal variations without explicitly

considering the ageing effect over long time span. This is evident by the fact that the adopted databases are collected

over 14-15 weeks in most of the studies.

However, since these contextual variables are not used in inference, they are not considered in fitting the general-

ized linear model (GLM). By inference, we mean that the model will be used to predict a novel but valid combination

of attributes not necessarily represented by the data table. Therefore, our primary goal is to study the attributes that

are likely to dictate the performance gain of an adaptive biometric system. The influence of contextual variables are

not of interest here but this can be a subject of future investigation. Next, we explain how the generalized linear model

is trained.

2.2.3 Training with Generalized Linear Model (GLM)

The generalized linear model [31] takes a set of four attributes as independent variables in order to predict the

performance gain as its output. If a ≡ [a1, . . . aN ] is a vector of binary attributes encoded as a binary string, and

w ≡ [w1, . . . wN ] is the weight vector of real numbers whose elements are associated with those in a, then, GLM

produces

y = waT + w0 (3)

as output. The training process involves estimating the vector of coefficients w including a bias term, w0 ∈ R.

After training, the weights {w0, . . . , wN} are obtained. The GLM is inferred by enumerating a subset of valid

attributes {a}. In the single-factor analysis, eq. (3) is then invoked to consider only an attribute (N = 1) which

can take either a 1 or a 0. The performance gain inferred by both cases, along with their respective upper and lower

confidence intervals, are then compared. In the multi-factor analysis scenario, the multi-dimensional attribute a is

enumerated but invalid combinations are excluded. The performance gain of each valid attributes in {a} is then

compared.

Next, we shall report the findings of single-factor meta-analysis followed by that of multi-factor one.

2.2.4 Findings of Single-factor Meta-analysis

The result for single-factor analysis is shown in Figure 1.

4Collected data is available in the tabulated form (excel file) in the following link: https://sites.google.com/site/ajitarattaniitaly/resources

8

• Supervised adaptation is likely to outperform (about 22.2% more performance gain) semi-supervised method to

adaptation such as self-training and co-training. As mentioned before, performance of the supervised adaptation

can be considered as best case as the references are adapted to all the available genuine samples [4]. This is

in contrary to methods based on semi-supervised learning in which only selective (mostly highly confidently

classified) samples are used for adaptation. Our meta-analysis findings show large variance in the performance

gain of the supervised method such that it overlaps significantly with that of semi-supervised one, indicating

also the effectiveness of the latter. However, for the real time deployment of automated methods (based on

semi-supervised learning), their performance should be equal to their supervised (manual) counterparts. Thus,

further indicating the need of effective adaptation schemes for automated systems.

• Co-training is likely to boost the performance gain by about 25.3 % in comparison to its self-training counterpart.

• The use of biometric sample quality appears to be much better than not using this information. According to our

findings, adaptive biometric systems considering quality measurements resulted in about 47% more performance

gain.

• Including impostor samples in the adaptation set can result in lesser performance gain (16% lesser in our ex-

periments) than if the samples were not present. Since an automated adaptive system deployed in operational

environment is vulnerable to impostor attack, it is unrealistic not to include impostor samples in the adapta-

tion set. As a consequence, our exercise here shows that not including impostor samples in adaptation set can

over-estimate the performance gain.

2.2.5 Findings of Joint-factor Meta-analysis

Figure 2 summarizes performance gain for the joint-factor meta-analysis scenario spanned by four binary at-

tributes: quality, co-train, supervised, and impostor attack. For instance, 0001 implies that an

adaptive system that does not use biometric sample quality, that is based on self-training (hence, not supervised),

and the system has been tested with non-match samples (impostor attack) in the adaptation data set. The attribute

impostor attack is always true as this strategy reports a less biased performance gain, as explained before.

The first three attributes are then enumerated, excluding invalid combinations. For instance, it is not possible that

co-training and supervised adaptation to be present at the same time, as co-training is a semi-supervised

learning strategy; hence, cannot be supervised. Note that the adaptive systems considering supervised adaptation and

quality at the same time (quoted as 1011) are managing the updated references on the basis of quality type. The query

biometric samples are matched to the references of the same quality type [5]. Adaptive systems based on co-training

exploit mutual and complementary information of the bi-modal system for template adaptation as well as testing. On

the other hand, existing studies on supervised adaptation have been reported only for single biometric modality. This

explains the superiority of co-training over supervised adaptation.

9

20 25 30 35 40 45 50 55 60 65

quality

co−train

supervised

impostor

Performance gain (%)

Figure 1: The performance gain for a given attribute obtained by the trained generalized linear model. A blue (red)bar denotes a 95% confidence interval around the expected performance gain, denoted as circle (square), when a givenattribute is present (absent).

These findings suggest that there is a natural increase in performance as one exploits co-training, supervised

adaptation and biometric sample quality systematically. To sum-up, our meta-analysis findings (both single and joint

factor analysis), support the conjecture that quality and co-train are important attributes for the design of automated

adaptive biometric systems.

3 Novel Framework and Research Directions

In this section, we propose a novel framework and set some future research directions that are motivated by the

findings of the meta-analysis.

The results of both the single and joint factor analysis indicate that quality and co-training are important ingredients

when designing an adaptive biometric system. Furthermore, when analyzing the contextual variables of the reported

systems, such as the adopted database size and the number of samples, we found that these systems did not consider

the notion of time in order to account for the ageing effects which may induce temporal performance variation over a

long time span. Motivated by this, we shall propose a novel system that can make use of quality and further include

the notion of time in a single framework. This proposed novel framework is termed as condition-and age-adaptive

system.

10

0 10 20 30 40 50 60 70 80

1101

1011

1001

0101

0011

0001

Performance gain (%)

Figure 2: The performance gain along with the confidence intervals of various configurations spanned by four binaryattributes: quality, co-train, supervised, and impostor (see text).

3.1 Framework for condition and age adaptive system

Existing adaptive biometric systems have not considered the ageing effect explicitly. A possible reason for this is

that, the effect of ageing is often considered to be very different from that caused by biometric sample quality. As a

consequence, methods that aim to address ageing often assume that the biometric sample is free from noise, that is,

images are often well aligned and acquired in controlled conditions.

In practice, however, an adaptive biometric system has to deal with both the aspects (i.e., adaptation to ageing

and quality conditions) for the life-long learning and coping under non-stationary conditions caused by changes in

biometric sample quality. Two separate strategies are needed in order to handle variations caused by biometric sample

quality and those caused by ageing because while the former can cause dramatic changes to the captured biometric

features almost instantaneously, age-related changes are, in comparison, a much more slower and irreversible process.

However, beyond a certain limit of time, the variation due to age-related factors will dominate over that due to the

quality-related ones. This is illustrated in Figure 3, where one image is taken under a somewhat controlled condition,

another with a significantly different quality (head pose) but taken at the same time, and then another taken after two

years.

Thus in order to cope with the changes in the quality as well as temporal variations in the input sample, we propose

a possible framework called a condition and age adaptive system.

We have adopted a Bayesian approach for the formulation of the system. This choice is appropriate because

biometric features are generated by a stochastic process. As a result, no two consecutive samples obtained from a

biometric trait are exactly the same. Thus the uncertainty at the feature space can be characterized using a distribution

11

Figure 3: Illustrating an example of face images taken at different quality conditions (left versus middle) and over time(left versus right).

defined over the feature space. Indeed, a number of state-of-the-art face and speaker recognition classifiers are based

on Bayesian formulation, e.g., [32] and [33]. Furthermore, the state-of-the-art online template update method used in

the fingerprint literature [12, 13, 14] can be interpreted using a Bayesian framework [6].

Next, we introduce the bayesian framework and explain the proposed system.

3.1.1 Bayesian Framework and notations

The recursive formulation of Bayesian estimation allows one to update the parameters of an “old” or initial model

with a new ones given only the latest sample. Thus, given a sequence of observations collected over time, (x1, . . . , xT )

or (x1 : xT ), one can estimate a statistical model parameterized by θ, p(x|θ), in the following way (ignoring the

normalizing factor in each step since we are only interested in maximizing the function with respect to θ):

p(θ|x1 : xT ) ∝T∏

i=1

p(xi|θ)p(θ)

∝T∏

i=2

p(xi|θ)p(θ|x1)

∝T∏

i=3

p(xi|θ)p(θ|x1, x2)

∝...

∝ p(xT |θ)p(θ|x1 : xT−1) (4)

This recursive formulation implies that in order to calculate the optimal value of θ given all previously observed

T samples, one only needs to use the parameter calculated up to T − 1 to do so. The above recursive formulation

shows the benefit of learning for density-based classifiers as an example, leading to finding the optimal value of θ.

This recursive formulation is known as true recursive Bayesian learning. The right- hand term, p(θ|x1 : xT ), is a

reproducing density and the term p(θ) is a conjugate prior [34]. Although the above adaptation is well established and

appears to be sound, it does not consider biometric sample quality nor the ageing effect.

A theoretical framework for model (reference) adaptation using biometric sample quality has been proposed in

12

[6] but did not consider the time effect. The model is henceforth referred to a condition adaptive system. In the

subsequent section, we shall propose a theoretical formulation of condition and age adaptive system that considers

both aspects, hence will be capable of life-long learning (age adaptive) and learning under non-stationary environment

causing concept drift (condition adaptive).

Let x be a biometric feature vector; j ∈ N, the user’s identity; Qu ∈ {Q0, Q1, . . . , QQ}. The condition in

which a biometric sample is captured, with Q0 being the enrolment (controlled) condition and Qu|u ̸= 0 being

other uncontrolled conditions. Each condition Qu is due to a number of factors. Let these factors be enumerated by

(f (1), . . . , f (F)). For instance, for the face biometrics, f (1) is lighting; f (2) corresponds to facial expression types;

f (3) indicates the presence of glasses, f (4) estimates the head pose, and etc. Then, each Qu is a compound effect of

these factors, i.e., Qu = (f (1), . . . , f (F)). It is arguable that, in practice, the condition Q is countable but the total

number of conditions, i.e., Q+1 (including the enrolment condition Q0), cannot be determined exactly. This number,

however, is not impossible to estimate. For instance, it can be estimated by clustering quality measures, or by manual

annotation [6].

Let t ∈ N, the “time” at which a sample is captured. This notion of time is discrete; it is loosely defined such that

two samples that are close in time (say a few seconds apart) will have the same t value. The rationale for using this

definition of t is that the appearance of each biometric trait does not change, as a result of ageing, at the same rate.

Using the above notation, the feature distribution of person j can be completely specified by p(x|j,Q, t).

Let x(j,Q, t) be a sample drawn from p(x|j,Q, t). We shall refer to x(j,Q0, t0) as a reference or model where t0 is

the time at which this sample is obtained; and p̂(x|θ(j,Q0, t0)), a model with parameter θ(j,Q0, t0) that approximates

the true density p(x|j,Q0, t0). Q0 implies that the sample is taken under controlled conditions, that is one in which

all the quality-related factors have been carefully controlled, i.e., F (1) = F(1)0 , . . . , F (F) = F

(F)0 . The notation also

allows us to describe non-ideal samples, for instance, non-frontal head poses, presence of glasses, as may be captured

during enrollment i.e, {x(j,Qu, t0)} for u ̸= 0. The distribution defined over these samples is written as p(x|j,Qu, t0)

for each u; and their corresponding approximated model, as p̂(x|θ(j,Qu, t0)).

Let y ∈ R be a matching score. Furthermore, let j∗ be the claimed identity and x ≡ x(j,Q, t∗) be a query sample

taken at time t∗ from an unknown person j under an unknown condition state Q. For simplicity and without loss of

generality, we also write x ≡ x(j,Q, t) but write in full in order to emphasize a particular state, e.g., x(j,Q, t∗) to

emphasize a given time t∗ and x(j,Q∗, t) to emphasize a given state Q∗.

13

We can then define the following modes of operation:

enrol : biometric trait → x(j,Q0, t0)

matchtq : x, x(j∗, Q0, t0) → y

matchbayes : x, p(x|·, Q, t), p(x|j∗, Q, t),→ y

adaptcond : p̂(x|θ(j,Q, t)), x(j,Q∗, t) → p̂(x|θ(j,Q∗, t))

adapttime : p̂(x|θ(j,Q, t)), x(j,Q, t∗) → p̂(x|θ(j,Q, t∗)),

where Q∗ is a short hand for Q = Q∗ to emphasize that Q assumes a particular quality state. The same convention is

adopted for t∗.

The operation “matchtq” produces a similarity or a dissimilarity measure between a reference, denoted as x(j∗, Q0, t0),

and a query sample, x(j,Q, t), with the unknown identity j. On the other hand, the operation “matchbayes” typically

takes a query sample and a pair of densities (one representing the universal background or “world” model and another

representing the client-specific model) as input and produces a likelihood ratio or a posterior probability as output. For

instance, the state-of-the-art speaker verification system computes:

matchbayes(x) = log(p̂(x|θ(j∗, Q, t))

p̂(x|θ(·, Q, t))) (5)

as output, where p̂(x|θ(·, Q, t)) is the density of the general population, also known as the universal background or

“world” model:

p̂(x|θ(·, Q, t) =∑j ̸=j∗

p̂(x|θ(j,Q, t)P (j)

where P (j) weighs the contribution of client-specific density p̂(x|θ(j,Q, t) to the final general-population density.This

approach can be traced back to Neyman-Pearson theorem. An equivalent, alternative formulation is to invoke the

Bayes rule, which computes matchbayes(x) = P (j|θ′(j∗, Q, t)) as output, noting that the parameter θ′(j∗, Q, t) is not

the same as the θ’s used before if the posterior probability is approximated using a discriminative classifier such as

a multi-layer perceptrons. Accordingly, the condition adaptation (adaptcond) operation adapts an existing model to

a new quality condition, Q∗, represented by a given input sample taken in a different condition. On the other hand,

temporal adaptation (adapttime) updates an existing model to a new one given the most current sample taken at time

t∗.

A non-adaptive biometric system is neither adaptive to the age-related factors nor the quality related ones. These

systems can be defined as

match(x) = logp(x|θ(j∗, Q0, t0))

p̂(x|θ(·, Q0, t0))

14

Based on the mentioned notation, next we will describe the proposed method.

3.1.2 Towards a Condition and Age Adaptive System

In order to describe the proposed condition and age-adaptive system, first, we will present a condition-adaptive

system and an age-adaptive one separately. The condition adaptive system has been proposed in [6]. However,

different from [6] which considers only condition-adaptive systems, we further introduce the age-adaptive system and

the proposed mixture of both condition- and age-adaptive systems here.

Accordingly, a condition-adaptive system operates in two modes: adaptation mode and matching (comparison)

mode. In the adaptation mode, the system adapts its model using one or more samples taken from an unseen condition

Q.

Adaptation:

p̂(x|j∗, Q, ·) = adaptcond(p̂(x|j∗, Q0, t0), x) for all Q

p̂(x|·, Q, ·) = adaptcond(p̂(x|·, Q0, t0), x) for all Q

The result is a new model that will operate optimally on the novel condition. We assume for now that the query sample,

x, has already been identified to belong to the claimed identity j∗ for now.

The matching mode of a condition-adaptive system will be slightly more complicated, since there are several

models each of which can only operate optimally under a given condition Q. Intuitively, the model with the same

condition as the query sample will be chosen or weighed more heavily than the rest. Formally, this weight is called

the posterior probability of a condition Q given the observation – a set of quality measures, q-assess(x). The posterior

probability is written as P (Q|q-assess(x)). The inference using the log-likelihood ratio is computed as follows:

Matching:

match(x) = log

∑Q P (Q|q-assess(x))p̂(x|j∗, Q, t0)∑Q P (Q|q-assess(x))p̂(x|·, Q, t0)

(6)

= logp̂(x|j∗, t0, q-assess)p̂(x|·, t0, q-assess)

. (7)

The sum over Q implies that this variable is marginalized because it is not observable. This summation in eq. (6)

shows that if one were to estimate the density p̂(x|j∗, t0, q-assess) correctly, one will need sufficient number of quality

states Q+ 1. It is recommended that Q be determined by clustering the quality measures, q-assess [6].

The above formulation of condition-adaptive systems can be found in [6]. An experiment is realized in [5] using

samples that are manually labeled, providing the most favorable scenario for adaptation. Nevertheless, the performance

15

gain of 30% is significant.

An age-adaptive system does not have any mechanism to handle variation in biometric sample quality. Almost all

reported literature assumes that a query sample is always captured in the controlled condition, Q0. The goal of an age-

adaptive system is to retain a number of time-dependent models, i.e., p̂(x,Q0, tu) at various point in time {tu|u ∈ U}.

The set U reflects the time window in such a way that models that are “too old” will be eventually abandoned[12, 14].

During inference, a more recent model is given more important consideration than the rest of the models. The two

modes of operation are computed as follow:

Adaptation:

p̂(x|j,Q0, tu) = adapttime(p̂(x|j,Q0, t0), x(j,Q, tu)

for u ∈ U

Matching:

match(x) = log

∑u∈U f(t∗ − tu)p̂(x|j,Q0, t∗)∑u∈U f(t∗ − tu)p̂(x|·, Q0, t∗)

where the function f(·) is a decreasing non-negative function. The argument for function f cannot be negative since

t∗ > tu.

Although the age-adaptive model can be easily replaced with a synthesized model using image-based regression

the synthesized model does not take into account of the person-specific variation. These include facial surgery, the use

of make-up products such as Botulinum Toxin Type A injections that can make a person look a lot younger than they

actual are, and other life-style related changes such as diet regime and weight-lifting exercise. In contrast, by adapting

the model to the actually observed samples, an age-adaptive system could possibly be a better solution.

A major weakness of the age-adaptive system is that it does not consider variation in biometric sample quality due

to changing acquisition conditions. This can be remedied by either restoring a query sample to the same enrollment

condition or by allowing the system to adapt to different quality as well as age-related conditions. These systems will

be described next.

A condition- and age-adaptive system contains a set of references that vary in time as well as in conditions. During

inference, the more recently adapted models (references) with the matching conditions are given higher weights than

the rest of the models during inference. The system operates in two modes, as follow:

16

Adaptation:

p̂(x|j,Q, tu) = adapttime(p̂(x|j,Q, t), x(j,Q, tu)

for all Q and u ∈ U

p̂(x|·, Q, tu) = adapttime(p̂(x|·, Q, t), x(j,Q, tu)

for all Q and u ∈ U

Matching:

match(x) = log

∑u

∑Q f(t∗ − tu)P (Q|q-assess(x))p̂(x|j,Q, t∗)∑

u

∑Q f(t∗ − tu)P (Q|q-assess(x))p̂(x|·, Q, t∗)

where the function f is a non-negative decreasing function so that more important weights are given to more recent

samples.

In the proposed condition and age adaptive system, the input samples may be labeled and added to the reference

set using either co-training or self-training. However, the updated reference set will be managed and inferences will

be drawn (for input samples) considering the quality and the notion of time using the proposed framework.

However, one of the challenges at the moment is the lack of large-scale, longitudinal and multi-modal biometric

database, capturing quality as well as long term temporal variations. The available databases capturing long term

ageing effect (for instance MORPH face database 5) do not provide adequate number of samples with changes in

quality conditions (like pose or illumination changes for face) and vice versa. Thus the evaluation of the proposed

framework remains a part of future work. Nevertheless, the proposed framework is a step ahead to the field of adaptive

biometric systems and will provide important incentives, ideas and future directions to the research community.

3.2 New Research Directions

Our findings related to impostor against non-impostor attack suggests that classification errors in the labeling

process can result in sub-optimal performance of an adaptive biometric system. This is because classification error

(false acceptance) cause adapting user references with the impostor samples. Thus increasing the vulnerability to

template security and undermining the integrity of adaptive biometric systems. To this front, modeling and early

stoppage of impostor attack is an important research direction to be pursued. Developed solutions will allow vendors

to adopt auto-update procedures in their commercial biometric products.

Furthermore, there is a need for incorporating robust labeling scheme in the adaptive biometric systems. This

is supported by our findings related to supervised against semi-supervised methods to adaptation where supervised

scheme generalizes better than semi-supervised one. These results indicate that the use of confidently classified (la-

5http://www.faceaginggroup.com/projects-morph.html

17

beled) input samples (as used by most of the existing automated systems) may not be an efficient strategy for adap-

tation. Thus emphasizing the need for more robust labeling schemes that are capable of correctly classifying genuine

(with substantial variations) as well as impostor samples.

Furthermore, our findings related to supervised against semi-supervised and online against offline mode of adap-

tation need direct testing via single experimental framework. This is because of large variance and overlap in the

obtained performance gain on comparing adaptive systems based on these mentioned attributes. As a consequence,

our results do not allow us to state the conjecture firmly.

4 Conclusion

This manuscript has worked towards advancing the state-of-the-art related to adaptive biometric systems. This

has been achieved by identifying key attributes related to adaptive biometric systems followed by the comparison and

critical analysis. Meta-analysis has been used as a tool to perform comparison, critical analysis and to draw on the

limitations of the existing systems. Specifically, meta-analysis has been used to gauge the relative impact of single and

joint attributes on the generalization behavior (performance gain) of adaptive biometric systems. Our meta-analysis

findings has generally supported our conjecture that biometric sample quality and co-training are important ingredients

for an adaptive biometric system. Furthermore, a novel framework termed condition-and-age adaptive system has been

proposed and future avenues have been set. Collection of a large scale, longitudinal multibiometric database capturing

both quality and age related variations will be useful for future work in this field.

5 Acknowledgement

This work has been partially supported by Regione Autonoma della Sardegna ref. no. CRP2-442 through the

Regional Law n.7 for Fundamental and Applied Research, in the context of the funded project “Adaptive biometric

systems: models, methods and algorithms”. Rattani was partly supported by a grant awarded to Regione Autonoma

della Sardegna, PO Sardegna FSE 2007-2013, L.R. 7/2007 “Promotion of the scientific research and technological

innovation in Sardinia”. Poh was partially supported by Biometrics Evaluation and Testing (BEAT), an EU FP7

project with grant no. 284989.

References

[1] Ross, A., Nandakumar, K., Jain, A. K.: ‘Handbook of Multibiometrics’ (Springer Verlag, 2006)

[2] Lantinis, A.: ‘A Survey of the effects of aging on biometric identity verification’, Int. Journal of Biometrics,

2010, 2, (1) , pp. 34-52

18

[3] Rattani, A., Freni, B., Marcialis, G.L., Roli, F.: ‘Template Update Methods in Adaptive Biometric Systems: A

Critical Review’. Proc. 3rd Int. Conf. on Biometrics, Sardinia, Alghero, 2009, pp. 847-856

[4] Rattani, A.: ‘Adaptive Biometric System based on Template Update Procedures’. PhD thesis, University of

Cagliari, Italy, 2010

[5] Poh, N., Kittler, J., Marcel, S., Matrouf, D., Bonastre, J-F.: ‘Model and Score Adaptation for Biometric Sys-

tems: Coping With Device Interoperability and Changing Acquisition Conditions’. Proc. Int. Conf. on Pattern

Recognition, Istambul,Turkey, 2010, pp. 1229-1232

[6] Poh, N., Wong, R., Kittler, J., Roli, F.: ‘Challenges and Research Directions for Adaptive Biometric Recognition

Systems’. Proc. 3rd Int. Conf. on Biometrics, Sardinia, Alghero, 2009, pp. 753-764

[7] Poh, N., Bourlai, T., Kittler, J.: ‘Quality-based Score Normalisation with Device Qualitative Information for

Multimodal Biometric Fusion’, IEEE Trans. on Systems, Man, Cybernatics Part B : Systems and Humans, 2010,

40, pp. 539-554

[8] Hedges, L.V., Olkin, I.: ‘Statistical Methods for Meta-Analysis’ (Academic, New York, 1985)

[9] Uludag, U., Ross, A., Jain, A.: ‘Biometric template selection and update: a case study in fingerprints’, Pattern

Recognition, 2004, 37, (7), pp. 1533-1542

[10] Ozawa, S., Toh, S. L., Abe, S., Shaoning, P., Kasabov, N.: ‘Incremental learning for online face recognition’.

Proc. 5th IEEE Int. Joint Conf. on Neural Networks, Montreal, Canada, 2005, pp. 3174-3179

[11] Zhu, X.: ‘Semi-supervised learning literature survey’, Computer Science TR 1530,2008

[12] Jiang, X., Ser, W.: ‘Online Fingerprint Template Improvement’, IEEE Trans. on Patter Analysis and Machine

Intelligence, 2002, 24, (8), pp. 1121-1126

[13] Ryu, C., Hakil, K., Jain, A. K.: ‘Template adaptation based fingerprint verification’. Proc. 18th Int. Conf. on

Pattern Recognition, HongKong, 2006, pp. 582-585

[14] Liu, X., Chen, T., Thornton, S. M.: ‘Eigenspace updating for non-stationary process and its application to face

recognition’, Pattern Recognition, 2003, 36, (9), pp. 1945-1959

[15] Roli, F., Marcialis, G. L. : ‘Semi-supervised PCA-based face recognition using self training’. Proc. joint IAPR

Int workshop on S+SSPR06, HongKong, China, 2006, pp. 560-568

[16] Rattani, A., Marcialis, G.L., Roli, F. : ‘Biometric template update using the graph mincut: a case study in face

verification’. Proc. 6th IEEE Biometric Symposium, Tampa, USA, 2008, pp. 23-28

19

[17] Roli, F., Didaci, L., Marcialis, G.L. : ‘Template co-update in multimodal biometric systems’. Proc. IEEE/IAPR

Int. Conf. on Biometrics, Seoul, Korea, 2007, pp. 1194-1202

[18] Gayar, N. E., Shaban, S. A., Hamdy, S. : ‘Face recognition with semi-supervised learning and multiple clas-

sifiers’. Proc. 5th WSEAS Int. Conf. on Computational Intelligence, Man-Machine Systems and Cybernetics,

Venice, Italy, 2006, pp. 296-301

[19] Rattani, A., Marcialis, G.L., Roli, F.: ‘Capturing large intra-class variations of biometric data by template coup-

date’. Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops, Anchorage,

Alaska, USA, 2008, pp. 1-6

[20] Amayeh, G., Bebis, G., Nicolescu, M.: ‘Improving Hand-Based Verification Through Online Finger Template

Update Based on Fused Confidences’. Proc. 3rd IEEE Int. Conf. on Biometrics: Theory, applications and sys-

tems, Washington, DC, USA, 2009, pp. 352-357

[21] Pavani, S.K., Sukno, F.M., Butakoff, C., Planes, X., Frangi, A.F.: ‘A Confidence Based Update Rule for Self-

updating Human Face Recognition Systems’. Proc. Int. Conf. on Biometrics, Alghero, Italy, 2009, pp. 151-160

[22] Garcia, R., Perales, J.F.: ‘Adaptive Templates in Biometrical Authentication’. Proc. 16th Int. Conf. in Central

Europe on Computer Graphics, Visualization and Computer Vision, Czech Republic, 2008

[23] Rattani, A., Marcialis, G. L., Roli, F.: ‘Self Adaptive Systems: An Experimental Analysis of the Performance

Over Time’. Proc. IEEE Workshop on Computational Intelligence in Biometrics and Identity Management

CIBIM, Paris, France, 2011, pp. 36-43

[24] Martinez, C., Fuentes, O.:‘Face recognition using unlabeled data’, Computacin y Sistemas, Iberoamerican Jour-

nal of Computer Science Research, 2003, 7, (2), pp. 123-129

[25] Rattani, A., Marcialis, G.L., Roli, F.: ‘Temporal analysis of biometric template update procedures in uncontrolled

environment’. Proc. 16th Int. Conf. on Image analysis and processing, Ravenna, Italy, 2011, pp. 595-604

[26] Freni, B., Marcialis, G.L., Roli, F.:‘Replacement algorithms for fingerprint template update’. Proc. 8th Int. Conf.

on Image Analysis and Recognition, Portugal, 2008, pp. 884-893

[27] Giot, R., Dorizzi, B., Rosenberger, C.:‘Analysis of Template Update Strategies for Keystroke Dynamics’. Proc.

IEEE Workshop on Computational Intelligence in Biometrics and Identity Management, Paris,France, 2011, pp.

21-28

[28] Marcialis, G. L., Rattani, A., Roli, F.:‘Biometric template update: an experimental investigation on the relation-

ship between update errors and performance degradation in face verification’. Proc. in Joint IAPR Int’l Workshop

on SSPR+SPR08, Orlando,Florida, USA, 2008, pp. 684-693

20

[29] Zuo, J., Nicolo, F., Schmid, N.:‘Adaptive Biometric Authentication Using Non-linear Mappings on Quality

Measures and Verification Scores’. Proc. in IEEE Int. Conf. on Image Processing, Hong Kong, 2010, pp. 4077-

4080

[30] Guerra-Casanova, J., Sanchez-Avila, C., de Santos Sierra, A., del Pozo, G. B.: ‘Score optimization and template

updating in a biometric technique for authentication in mobiles based on gestures’, Journal of Systems and

Software, 2011, 84, (11), pp. 2013-2021

[31] Collett, D.:‘Modelling Binary Data’ (Chapman and Hall /CRC Press, 2nd edition, 2002)

[32] Cardinaux, F., Sanderson, C., Bengio, S.:‘User Authentication via Adapted Statistical Models of Face Images’,

IEEE Trans. on Signal Processing, 2006, 54, (1), pp. 361-373

[33] Reynolds, D. A., Quatieri, T., Dunn, R.:‘Speaker Verification Using Adapted Gaussian Mixture Models’, Digital

Signal Processing, 2000, 10, pp. 19-41

[34] Duda, R. O., Hart, P. E., Stork, D. G.:‘Pattern Classification and Scene Analysis’ (Digital Signal John Wiley and

Sons, New York, 2001)

21