off-line identification with handwritten signature images ... · matic handwritten signature...

Off-line Identification With Handwritten Signature Images: Survey and Perspectives

Robert Sabourin 1, Rejean Plamondon2 , and Guy Lorette3

1 Laboratoire Scribens, Ecole de Technologie Superieure, Departement de Genie de la Production Automatisee, 4750 Henri-Julien, Montreal QC, Canada H2T 2C8

2 Laboratoire Scribens, Ecole Poly technique de Montreal, Departement de Genie Electrique, C.P. 6079, Succ. "A," Montreal QC, Canada H3C 3A7

3 IRISA, Universite de Rennes 1, Campus de Beaulieu, 35042 Rennes Cedex, France

The first part of this paper presents a survey of the literature on automatic handwritten signature verification systems using binary or graylevel images, and focuses primarily on preprocessing techniques, feature or primitive extraction methods, comparison processes, and performance evaluation. With these previous studies in mind, we propose, in the second part of this paper, an image-understanding system based on the extraction of a novel representation of handwritten signature images. This approach is text insensitive. A structural match between a reference primitive set Pr and a test primitive set Pt takes into account the geometric shape and spatial relations between primitives. Finally, the local comparison of gray levels between pairs of primitives next to each node of the static solution path N results in a pseudo dynamic similarity measure iJd(Pr, Pt). This scheme allows the elimination, with a certain degree of success, of skilled forgeries such as tracings and photocopies, showing marked gray-level dissimilarity along the signature line.

1 Introduction

Off-line signature verification problems are generally considered to be more difficult than on-line ones [7]. A signature image can easily be copied optically or mechanically. Moreover, timing and dynamic information are highly degraded in a static specimen. Although part of this information can be recovered by expert document analysts using specific skills and techniques [23, 24, 25], most of their methods cannot be implemented easily in a computer environment. A few papers proposing ways of dealing with this problem have been reported in the literature. Sakai et al. have proposed an interactive hybrid system to collect and display the static and dynamic features of a test and a reference signature on a CRT screen, using a color code [3]. This system was used to help a clerk examine specific features of a signature, and the final decision about the specimen's authenticity was left to him. Ammar et al. have tried recently to extract indirectly and use pressure information from a signature image [1, 2]. Apart from this recent paper, most of the work done in this area thus far has been concerned with static feature analysis only.

H. S. Baird et al. (eds.), Structured Document Image Analysis© Springer-Verlag Berlin Heidelberg 1992

220 R. Sabourin et al.

The design of off-line signature verification systems requires the solution of five types of problems: data acquisition, preprocessing, feature or primitive extraction, the comparison process, and performance evaluation.

2 Data Acquisition

Different types of acquisition devices are used for off-line signature verification systems, including TV vidicon cameras, CCD matrix cameras, and electrooptical scanners. The size of the image may be up to 512 x 512 pixels in area and with up to 256 gray levels of quantization. Off-line data entry systems may also be simulated by ordering (X,Y) coordinate pairs issued from a digitizing tablet as with a raster scanning.

3 Preprocessing

This step deals with the preparation of the relevant information: nonuniformity correction for sensor elements, localization of the signature in the picture, extraction of the signature from the background, slicing, solving thresholding and filtering problems, segmentation, and data reduction.

Since most of the applications covered by these studies are related to the automatic validation and processing of specific paper forms (such as checks), the finding of a signature in this context is not a difficult problem and may be solved with the help of a window operator [4]. However, as suggested by Nagel [4, 5, 6], this problem might become more serious if the form standard permits a large number of variations. If no artifice is used (e.g. signing with special ink), the problem of extracting a signature image from its background is not a trivial one. Thresholding and slicing techniques have been proposed [1, 6, 8] but are generally insufficient in practice in a noisy background for which restoration techniques are also necessary [1].

Indeed, the efficiency of slicing and thresholding techniques is directly related to the selection of the cut point from a gray-level histogram of the picture. The problem is then reduced to finding a point on the frequency distribution that separates the peaks of a bimodal curve, where the peak representing the signature is almost lost in noise. The use of the Laplacian or the Sobel operator has been proposed as a border or stroke detector. In the first case, the threshold is fixed to select pixels above the 85% valve in the cumulative Laplacian distribution [5, 6]. In the second, the modulus gradient vector of the gray-level signature is used, with the threshold fixed at the point where the modulus variations in the pixel distribution function become less than 1% [8]. This latter approach offers the advantage of being independent of the thickness of the strokes and of the overall length of the signature.

However, for practical cases where specimens are noisy, these techniques have to be improved. A four-step preprocessing operation has recently been proposed by Ammar et al. [1] (background equalization and reduction, noise reduction by averaging, automatic thresholding and image extraction). This approach has

Off-line Identification of Handwritten Signatures 221

been found to be successful in removing the overlapping in a signature as well as in the signing line, at the cost, however, of eliminating some parts of the signature in about 2% of samples.

Depending on the features needed at the next stage, the preprocessing algorithms can perform more specific operations on the extracted image. Prescreening operators are used to eliminate points that are judged to contain little information on writer identity [14, 15, 16]. Window operators have been used by Brocklehurst to improve the image [17]. A tracking algorithm has been applied by Nagel and Rosenfeld to identify and label all connected components in the picture [6]. Based on the hypothesis that high-pressure regions are indicated by higher gray levels in an image, a high pressure threshold selection algorithm has been proposed by Ammar et al. [1] to extract the high-pressure regions of a signature. This mechanism is based on the evaluation of a pressure factor, i.e. the ratio of the high-pressure area to the binary signature area. Recently, Sabourin and Plamondon [10, 11] have proposed a new version of a centroidal linkage region-growing-with-merging algorithm to perform signature segmentation. Using the statistics of directional data, atomic regions characterized by local uniformity in the orientation of the gradient are extracted. This scheme allows the extraction of signature areas characterized by gray levels with a very low signal/noise ratio.

4 Feature or Primitive Extraction

The division of feature or primitive extraction methods into those involving textsensitive and those involving text-insensitive features, as has been proposed for document expertise [18], cannot be strictly applied to signatures since the number of characters or symbols in a specimen is limited and often irrelevant (in an alphabetical context). Similarly, since the time information is not available from the image data, the function approach to feature or primitive extraction is not a really an appropriate distinction for static techniques [7]. Attempts at feature or primitive selection are better categorized by their global or local approaches to the problem, although an efficient system will probably incorporate both global and local feature or primitive extraction [13]. Moreover, none of the feature or primitive sets selected thus far preserves enough information to allow a reconstruction of the handwritten specimens.

In the global approach, the signature image is taken and processed as a whole; features or primitives are deduced from the global aspect of a signature. In other cases, features are evaluated from local parameter values which are averaged over the signature. In both cases, structural aspects encountered in signatures are taken into account in the local and global descriptions. Nemcek and Lin [16] have used the Hadamard transform on binary images as a means for data reduction and feature selection, the resulting spectrum constituting the feature set. A Sobel operator has been applied by Sabourin and Plamondon [8] to graylevel handwritten signature images, and the angular behavior of the resulting intensity gradient (assumed to reflect the stroke orientation) has been used as


a feature set. A close-fitting polygon has also been computed to evaluate the overall shape of a signature and to serve, once normalized, as basic features [15]. Sabourin and Plamondon have proposed a feasibility study on the use of some graphometric techniques, and, in particular, the rhythmic line of the signature has been used [9].

Chuang [21] has divided the signature into upper, middle, and lower zones, initial and end strokes (a process familiar to graphoanalysts), and performed a sequential analysis of each sub-image to extract features related to each resulting zone and its proportionalities. A single study has been reported to date describing the extraction of some global dynamic features from a static image. Indeed, Ammar et at. [1, 2] have worked with a set of features mainly related to pressure measurements, as extracted from gray-level images: the vertical position of the baseline of the high-pressure and of the binary image, a pressure factor and threshold, the highest gray level in the extracted signature image, the dynamic range on the gray scale for a signature, and its area measured as the number of pixels.

In the second approach, a set of local attributes is computed. Many of these features are known to be stable, according to document examiner expertise [23, 24, 25]. However, most of the studies dealing with local features also incorporate some global ones. For example, Brocklehurst [17] has proposed the use of the overall length of the signature (global) with a set of local features likes slope measurements, the distance from the left-hand end of the designated space provided for a signature to the point at which the signature begins, concavity measures, etc. Nagel and Rosenfeld [4, 5,6] have also worked with characteristics extracted from signatures segmented into vertical and horizontal zones. Assuming that a signature spelling was known, these authors detected tall letters in a specimen. Two global features (the ratio of signature width to short-stroke height and to long-stroke height) and two types of local features (the ratio of the height of a long stroke to the height of the short strokes immediately preceding it and the slope features of appropriate long letters) were retained for their studies, the number of these local features being dependent on the number of tall letters in a signature.

Most of the work done so far in the area of off-line signature verification has been concerned mainly with the statistical approach. Few studies deal with structural approaches using primitives. In his work, Requier [30] has used the linear strokes of signatures as primitives in a simulated off-line verification system. Nouboud et at. [19, 20] used as primitives the segments of the polygonal envelope of a signature. In a recent paper, Ammar et at. [22] described signature features and relations among them using a character string (the Global Description of a Signature (GDS)) and a hierarchical tree structure as a local description.

5 Comparison Processes and Performance Evaluation

As for the on-line system [7], the comparison process is based on the assumption that feature sets or structural descriptions extracted from genuine signatures


are more stable than those of forgeries: that is, that intrapersonal variations are smaller than interpersonal variations. So, an unknown signature may be recognized or rejected according to its similarity to the reference signature representation first specified by an indirect identifier.

To evaluate this similarity, three approaches have been tried, two involving statistical or data analysis and one involving structural analysis. Considering signature verification as a non-standard two-class problem (since the class of forgeries cannot be specified), a first group of researchers [1, 6, 21] has used weighted distances. These methods compute a distance measured between the reference set and the test set. The decision is taken according to the value of this distance criterion with respect to a decision threshold, this threshold being pre-established with the help of a training set of genuine signatures. Assuming some a priori probability [16], or by limiting their studies to random forgeries [8], some authors have used maximum likelihood classifiers and linear discriminant functions. This second approach also assumes statistical interdependence of gaussian distributed features. The third approach is typically a structural one. Requier [30] has proposed a method using graph and subgraph isomorphism for signature verification. Nouboud et al. [19, 20] have used dynamic programming to compare segments related to the polygonal envelope of a signature.

Table 1 summarizes the results obtained thus far by the research groups involved in off-line signature verification. Since the performance evaluation method differs greatly from one approach to another, these experimental results are again difficult to compare. The table shows the type of input data used in each author's study, a description of the data base used for the training or test experiments or both, the number and type of features with the comparison method, the error rates as reported according to type I (cd and type II (c2) definitions. The last column reports a few comments that might be helpful in analyzing the results.

Generally speaking, it has been observed that type I and type II errors of the order of a few percent can be obtained at the present time for systems working with simple and random forgeries. Only one system has been tested with skilled forgeries, and these specimens were not reliably detected [17]. According to expert document analysts, skilled forgery detection will not be efficient with these systems as long as dynamic and static features are not exhaustively extracted from a written specimen [23, 24, 25]. This is corroborated by observations from human perception experiments that knowledge of the drawing methods can greatly influence the recognizability of distorted characters [26].

Some authors have commented on their system failures [16, 17], two major causes of which are reported: first, scanning problems or signal quantization errors, resulting in incomplete specimen acquisition, due to the thinness of the trace or the color of the ink used [16]; second, excessive natural variations in the signatures of a few users [17].

6 ANew Approach

Incorporating some knowledge from expert document analysts [23, 24, 25] seems to us the best way of obtaining a significant improvement in the performance of


Table 1. Performance of static signature verification systems.

AUTHORS INPUT IMAGE TRAINING AND/OR TEST COMPARISON ERROR COMMENTS RESOLUTION DATA BASE METHOD RATES

FEATURE (Specimens x Writers)

DESCRIPTION

=6A, 256 x 1024 grid m ~e~uIBee) ~!¥~~~~ f1 • 6" ~~:e~~~~e!~~¥e~;~d 256 gra~ levels f2 • 4"

FUKUMURA 7 g loba pressure 200 forgeries with thresho ld for (1986) [I] features (20 S x 10 imitators) maximizing performances

BROCKLE- 60 pixe ls/cm 2820 genu i nes f1 • 5" 6 S/W as reference HURST binary !60 S x 47 W) f2 • 5" acceptation training (1985~ 7 local equa 1 number of thresho ld at 95" true

17] cMracterist ics unseen forgeries) acceptat i on

CHUANG 100 x 300 grid 2400 genuines weighted f1 • 20X 3 S/W as refarence (1977) binary (6 S x 400 W) distance f2 • 20X freehand forgeries

1600 forgeries t~~:~~Uo~i~~~i~~n25" g loba 1 features (4 S/user) [21] genuine acceptat ion

NAGEL, 500 pixels/inch n ~e~uln~s + 5S x lW) ~!¥~~~ f1 • 8" freehand for~eries ROSENFELD 64 gray leve Is f2 • OX average resu t of single (1977) 2 t~pes of loca 1 14 forgeries f1 - 12" and double jacknife

[6] g 10 a 1 features (9 5 x I W + 55 x lW) f2 - OX method

NEMCECK, 128 x 256 grid 600 ~enu i nes maximum f 1 - 11" 30 S/W training LIN binary (40 x 15 W) likelihood f2 - 41" 10 S/W test (1974) 14 features from 120 forgeries classifier freehand forgeries

[16] Hademar spectra 110 users imitated training threshold at

Sluser, 4 imitators 96" genu ine acceptat ion

NOUBOUD 384 x 510 1000 genuines dynamic f1 • 2" with persona 1 threshold (1989~ 64 gray levels ~10 S x 100 W) ~'i~gram- f2 • 8"

20] sig. envelope S/W for reference

PHELPS reso lut ion not 3 genuines similarity incomp lete resu Its (1982) sracified ~3 S x 1 W) of over-

[15] c ose-fitting forgeries lapping polygon (3 S x 2 W) area

SABOURIN 128 x 512 grid 760 ~enu ines maximum f1 • 1.5" random forgeries on ly PLAMONDON 256 ~ray leve Is (40 x 19 W) likelihood f2 - 7" 15 S/W for tra ining (1986) angu ar histogram and linear 25 S/W for test

[8] image gradient discrim.

SABOURIN 800 ~enu i nes nearest f1 -0.57" 5 ref ./signer PLAMONDON (40 x 20 W) neighbour f2 -0.03" (1987) claSSifier

[9] 63 genuines from 1 W nearest f1 • 3.5" features ext racted 117 simulated neighbour f2 • 0" interactively at a

rhythmic 1 ine forgeries from 6 classifier graphic terminal amateur forgers

6 ref./signer

17 sk i lled for~eries f1 - 1.8" 6 ref ./signer from 1 professIonal f2 -47.1" forger

SABOURIN 128 x 512 grid 224 ~enuines, pen II structura 1 f1 -0.0" 3 S/W as reference PLAMONDON 256 gray-levels FB x B W) matching f2 =1.34" random forgeries (1990~ 60 tracing forgeries B - .0

14] set of primitives 110 Sluser, pen II! (static) f2 - 100X comparison threshold 10 Sluser, pen 12 f2 ·98.9" se lected from the random 24 ~hotocoPies forgeries experiment,

(28 luser) f2 - 96" tracing forgeries from 1 writer

224 ~enuines, pen II structura 1 f1 '8.93" 3 S/W as reference (28 x 8 W) matching f2 -I. 79% randomBf~r~~Oies 1.60 traCing forgeries ! 10 S/user. pen II! (stat ic + f2 -6.25" compar i son thresho ld 10 51 user , pen '2 pseudo- f2 -78.8" se lected from the random 24 ~hotocoPies dynamic) forgeries experiment,

(28 luser) f2 =61.2" tracing forgeries from 1 writer


Automatic Handwritten Signature Verification Systems (AHSVS) [13]. As stated in [12, 13], Locard [25] has proposed many characteristics belonging to genuine handwritten signatures which are imperceptible to the average forger and very difficult to imitate. These characteristics are related to local variation in aspect ratio, orientation, relative position, shading, etc., between pairs of characters. This scheme seems to be more powerful than the individual letter's shape or general design which is very easily perceived by the average forger and therefore easier to imitate [23, 24, 25].

In the light of these previous studies [1,6,8,9,15,16,17,20,21]' we propose an image-understanding system [10, 11, 12, 13, 14, 32] based on the extraction of a novel signature representation from gray-level images [10, 11] for automatic signature verification. By an image-understanding system we mean the separation of the local and global evaluation functions [28] for the analysis of the handwritten signature and the detection of skilled forgeries. The approach considered in the present work [10, 11, 12, 13, 14] is text insensitive because no attempt is made to segment specific letters from the semantic part of the signature. Two classes of features, static and pseudo-dynamic, are taken into account for the representation of the scene. The former includes the geometric shape and spatial relations (N,S,E,O,N-E,N_O,S-E,S_O, and ADJ_TO) between some primitives extracted from the signature line [12]. The latter is associated with the gray-level variation inside the primitive [14]. By considering simultaneously these two classes of features, and the separation from the local interpretation of the primitives followed by the global interpretation of the scene, we have designed a general-purpose image-understanding system for the interpretation of handwritten signature images. This system is view-point dependent [27] and tailor-made for this purpose.

The spatial sampling of handwritten signatures produces an image format of 128 x 512 pixels. The preprocessing stage of the AHSVS (see Figure 1) is responsible for producing the signature image where each pixel from the graylevel image has a label Ls(m, n) signifying that the pixel at location (m, n) is either a background or a signal picture element. The preprocessing phase is subdivided into two stages: the gradient computation and the background elimination processes [10, 11]. The gradient at location (m, n) is evaluated over the entire gray-level image with the Sobel operator. The resulting filtered image is then analyzed by the background elimination process. Because the handwritten signature is characterized by a high gradient activity, the density function F(IV'I) is computed and a threshold T is automatically settled. Finally, those pixels at location (m, n) where the gradient activity is high, that is to say 1V'I(m, n) 2: T, are associated with a label Ls(m, n) = 1, and Ls(m, n) = 0 otherwise.

The signature image is thereafter analyzed by the primitive extraction process that is responsible for the production of the primitive sets necessary for the structural analysis of the handwritten signature by the comparison process. The strategy adopted here takes into account the collinearity of neighboring signalpixels (labeled as Ls(m, n) = 1) in the directional plane 8(m, n) of the gradient space. This task is performed in two steps [10].

First, the signal-pixels are merged into atomic regions characterized by the

226

.... 10 NUMBER ........ / ........ --/ /. .... /' ",I

10 NUMBER ~\ ENROLMEI'IT \ PROCESS /

'- /' .,. J REFERENCE

SET

R. Sabourin et al.

Fig. 1. The data-flow diagram of the proposed AHSVS using gray-level images.

uniformity of a local property, i.e. the orientation 8( m, n) of their gradient vectors. The resulting atomic regions belong to the signature line or represent spurious noise elsewhere in the scene. The initials, "ar", depicted in Figures 2(a) and 2(b) (reference and test images), will serve as a case study to illustrate the various stages of the proposed AHSVS. The enlargement of the letter "r" (Figure 2( a)) depicted in Figure 2( c) shows the atomic region partitions in the gradient space.

The second step of the primitive extraction process is responsible for generating the primitive set [11]. The High-Level-Merging process (HLM) starts with the elimination of sparse atomic regions. The HLM process continues with a hierarchical merging strategy in growing collinear atomic regions. The growth is therefore limited by the resulting circular variance R. The latter sub-process is used repeatedly in varying a growing constraint Rm , considering a fixed directional constraint em related to the local organization in the directional data located at the common border of neighboring atomic regions. This scheme acts as a zoom procedure focusing the attention of the HLM process on the biggest homogeneous regions from the atomic region set. The resulting primitive set is therefore characterized by a collection of arbitrarily shaped primitives. The degree of merging in the HLM process depends on the enrollment flag status (Figure 1). When this status has a TRUE value, the resulting set is a reference primitive set Pr where the growth is limited only by the presence of directional discontinuities along the handwritten signature line. In the case of a test primitive set Pt,


a) b)

QJ\.

c) d)

e) IIIrIIIIICI PBIIIl'l'IVI SEl' Pr

Fig. 2. Reference (a) and test (b) images. The atomic region set is shown in the gradient space (C) with the enlargement of the letter "r" from (a), followed by the corresponding reference primitives (d). Finally, the representation of the reference (e) and the test (f) primitive sets Pr and Pt.

the HLM process limits the growth depending on a circular constraint Rm (for example, the reference primitives Pri E Pr depicted in Figure 2(d), resulting from the application of the HLM process to the atomic regions of Figure 2(c)). The final results obtained from the extraction of the reference primitive set Pr from the reference image (Figure 2(a)) and the test primitive set Pt from the test image (Figure 2(b)), are shown in Figures 2(e) and 2(f) with cardinalities of N r = 6 and Nt = 11 primitives. Examples of reference primitive sets Pr taken from handwritten signature images are also depicted in Figure 3.

The comparison process is therefore responsible for the structural match between a reference primitive set Pr and a test primitive set Pt, resulting in a global similarity measure {}(Pr, Pt).

The first stage of the comparison process, the Local Interpretation of Primitives (LIP), is related to the final merging of test primitives Ptv E Pt. The LIP process is responsible for the labeling of all test primitives Ptv E Pt, given a reference primitive set Pr. The A* algorithm previously proposed [29] is replaced by a partially informed best-first BF* strategy governed by a new evaluation function f(n) = f(g(n), h(n)), where heuristics are embedded in f(n) [12]. The


\ a)

b)

Fig. 3. Examples of reference primitive sets Pr for two writers enrolled in the proposed AHSVS data base [13].

evaluation function f(n) is now defined by the relation

f(n) = [WI * (1 - SHAPE(Ptv:n , Pri))] + [W2 * AREA(Ptv:n , Pri)]

+[W3 * DIST(Ptv:n , Pri)] + [W4 * A_R(Ptv:n , Pri)],

where Lk=I Wk = 1, and 0 ~ f(n) ~ l.

(1)

Each term used in the definition of the evaluation function f(n) is related in some way to graphometric features used by the expert document analysts [13]. The first term SHAPE(Ptv:n,Pri) gives the similarity of shape between the collection of test primitives Ptv:n E Pt and the reference primitive Pri E Pro The shape factor is related to the measure of planar shapes using binary shape matrices obtained from the polar sampling of the silhouette of primitives Ptv:n and Pri. The next term AREA(Ptv:n , Pr;) is related to the difference in area between the test primitive subset Ptv:n centered around the test primitive Ptv and tentatively interpreted as reference primitive Pri' The relative difference in area is justified by the fact that all test and reference images for each writer are taken under similar experimental conditions. The AREA(Ptv:n, Pri) feature should permit the rapid elimination of primitives with the same shape but with


great dissimilarity in proportion. In the same way, DIST(Ptv :n , Pri) enables discrimination between two primitives on the basis of their local position. Two primitives similar in shape and equally prominent must be discriminated finally by their local positions. For example, if two lines with the same orientation have to be matched, DIST(Ptv :n , Pri) acts as a local constraint in the intelligent local template matching, limiting the merge of test primitives on the basis of their respective X positions from the start of the handwritten signature. This is also carried out by the term A_R(Ptv :n , Pri) related to the relative differencein-aspect ratio between Ptv :n and Pri.

The local confidence rating between a test primitive subset Ptv :'! E Pt, tentatively labeled as primitive Pri E Pr, is given by

(2)

where 0 :::; Fi(V) :::; l. The local confidence rating Fi(V) takes a value near unity for a perfect match

between the subset of test primitives Ptv :'! E Pt in the neighborhood of test primitive Ptv , and the reference primitive Pri E Pr. The local interpretation of Ptv as Pri terminates with the set of primitives

Ptv :'! = Ptv , ... , Pt'! E Pt (3)

The second stage of the comparison process, the Global Interpretation of the Scene (GIS), completes the analysis with the evaluation of a solution path N, produced by the structural matching of the primitive sets Pr and Pt, and followed by the validation of the resulting solution with the comparison of corresponding pseudo-dynamic features taken along the solution path N.

The first stage of the GIS process involves the evaluation of a static similarity fJs(Pr, Pt) between the test primitive set Pt and a reference primitive set Pr. The state space graph G' = (V', E') is defined as an oriented graph where nodes (states) are grouped in k ~ 2 disjoint subsets Vi,l :::; i :::; k (phases). So, the set of nodes V' may be defined as the union of all subsets, as V' = VI U V2 U ... U Vi u ... Vk. As previously stated [12, 14], a greedy algorithm seems a sufficient search mechanism for the global interpretation of handwritten signature images. This scheme is computationally less costly than the dynamic programming used in [29]. The static cost (similarity) at node v E Vi may be defined as

(4)

The factor Ri-I,i (u, v) represents the similarity in spatial relations between pairs of reference primitives Pri-I and Pri taken from the reference handwritten signature image represented by the primitive set Pr, and the spatial relations between pairs of corresponding test primitives Ptu and Ptv from the test primitive set Pt. Finally, node v E Vi, where the accumulated cost COSTi(V) is maximum, is pushed on list Ni = V. Assuming the use of a greedy algorithm at the GIS stage, the cost COSTi(V) is therefore defined by the expression [12, 14]:

(5)

230

where u = N i - 1 , V E Vi, < u,v >E EI, 3::; i::; k -1, and

Ni = argmax COSTi(V) vEV,

Note also that

0::; Ri - 1,i(U, v) ::; 1

0::; }Ci(V) ::; 1

0::;

(the similarity in spatial relations)

(the local confidence rating)

(the normalization factor)

Moreover, the initial conditions are as follows:

COST1(1) = 0

N1 = S

COST2(V) = st2 * }C2(V), 'Iv E V2 N2 = argmax COST2(v)

VE V2

The final step is therefore:

Nk = t

COSTk(t) = COSTk-1(Nk-d

R. Sabourin et al.

(6)

(7)

(8)

(9)

(10)

(11) (12)

(13)

(14)

(15)

The normalization factor defined by sti = AREA(Pr;)/AREA(Pr) takes into account the ratio, in area, of the reference primitives related to each phase (Pri == Vi) E VI to the entire area of the reference primitive set Pro This permits an additional penalty for primitives not yet included in the definition of VI. Finally, sti emphasizes prominent reference primitives in set VI in the evaluation of a global static similarity measure iJs(Pr, Pt), defined as follows:

(16)

where of course 0 ::; iJs(Pr, Pt) ::; 1. A pseudo-dynamic similarity measure iJd(Pr, Pt) is therefore computed at

the end of the GIS process with the use of the resulting solution path N = S, Vb V2,"', Vk-1, t. A pseudo-dynamic feature is evaluated from the local analysis of the gray levels inside pairs of primitives Vi = Pri, Ptv :"{ located on the solution path N (Figure 4). The pseudo-dynamic feature adopted here is the proportion of "black-labeled pixels" obtained from a discriminant analysis of gray-level histograms computed for each of the primitives Pri E Pr and Ptv :"{ E Pt, where Pri, Ptv :"{ E N [14]. This feature is called pseudo dynamic because it reflects the effect of the writing process dynamics, which produces gray-level variations along the signature line. The pseudo-dynamic similarity measure iJd(Pr, Pt) is defined as:

k-1 iJd(Pr, Pt) = (st2 * }C2( v) * Dg (v)) + Z)sti * Si-1,i( u, v) * D? (v)) (17)

i=3


RIlIRINCE TlST

~~ ~~

~~ ~£

~~ ~A Q~ &£ ~~ @1~

~~ ~~ Fig. 4. Primitives pairs located on static solution path N and considered in the evaluation of the pseudo-dynamic similarity measure iJd(Pr, Pt).

where u = Ni - l E Vi-I. V = Ni E Vi, INI = k, Nl = S, Nk = t, and

o~ Fi(V) ~1 (the local confidence rating) (18) 0::; Si-l,i(U,V) ::; 1 (the static similarity measure) (19) 0::; Di(V) ~1 (the pseudo - dynamic similarity measure) (20)

O~ (J ~5 (the scaling factor) (21)

O~ ili ::;1 (the normalization factor) (22)

The pseudo-dynamic factor D i ( v) acts as an additional penalty when the cost of the static solution path N is revalued. Define the pseudo-dynamic features

DYN(i) = wi(ki) DYN(v) = wv(k;)

(23) (24)

where wi(ki) (wv(k;)) represents the proportion of black pixels defined at graylevel threshold k; (k;), which is evaluated automatically with a discriminant


analysis of the gray level density functions proposed by Otsu in [31], and computed from pixels belonging to the reference primitive Pri (Ptv:'Y)'

Di (v) is defined as the relative similarity in the proportion of black pixels between pseudo-dynamic features DYN(i) and DYN(v) belonging to the primitive pair Pri, Ptv:'Y = Vi E N, i.e.

D _ _ IDYN(i) - DYN(v)1 ,(v) -1 MAX(DYN(i),DYN(v)) (25)

with of course 0 :S Di (v) :S 1. The resulting pseudo-dynamic cost Di (v) is used in equation (17) for the eval

uation of the final similarity measure 'l3d (Pr, Pt) between the test and reference primitive sets Pt and Pr.

An experiment was conducted [14] with 224 genuine specimens from eight writers, 224 random forgeries, 160 tracing forgeries from one forger using two kind of tracing pens and 224 photocopies of the genuine specimens. Let WI be the class of genuine handwritten signatures and W2 be defined as the class of forgeries. A test image from class WI or W2 is compared to three reference signature images for a specific writer enrolled in the AHSVS, and identified by a specific ID number (Figure 1). The best similarity 'l3 d (Pr, Pt) obtained from the comparison process is therefore dispatched to the decision process. The probability functions of resulting maximum similarity measures 'l3 d (Pr, Pt) are therefore computed considering an equal a priori probability for each class WI and W2, say P[WI] = P[W2] = 0.5. A decision threshold Tra is settled with the help of random forgeries at the value corresponding to the total minimum error rate Etmin [14].

In the static case ({3 = 0 in equation (17)), experimental results with random forgeries show a type I error rate of 101 = 0.0%, a type II error rate of 102 = 1.34%, resulting in a total minimum error rate of Etmin = 0.67% for the proposed AHSVS. Considering the same decision threshold Tra , tracing forgeries and photocopies were not eliminated (see Table 1). Using the proposed pseudo dynamic scheme with {3 = 5 in equation (17), a new decision threshold Tra is computed with the random forgeries. Using this threshold (Table 1), tracing forgeries (pen number 1) showing a thicker signature line than the genuines can be eliminated with a type II error rate of 102 = 6.25%, at the expense of a higher type I error rate of 101 = 8.93%. The corresponding type II error rate for the random forgeries experiment is now equal to 102 = 1.79%.

7 Conclusion

A new scheme is advocated for the design and analysis of automatic handwritten signature verification systems using gray-level images [32]. The use of a novel handwritten signature representation allows local analysis of gray levels along the signature line, enabling the elimination of skilled forgeries such as tracings and the photocopies showing great gray level dissimilarity along the signature line. The concept of combining the structural matching of gray-level images with the separately considered local interpretation of primitives and the global


interpretation of the scene is not new in the field of computer vision [27, 29], but major improvements to the method have been presented [10, 11, 12, 13, 14, 32].

Acknowledgements

This work was supported by grant A0915 to Rejean Plamondon from NSERC CANADA, and by FIR grant to Robert Sabourin from Ecole de Technologie Superieure. The preparation of this paper was completed when Rejean Plamondon was a Fellow of the Netherlands Institute for Advanced Studies.

References

[1] M. Ammar, Y. Yoshida, and T. Fukumura, "A New Effective Approach For OffLine Verification of Signatures by Using Pressure Features," Proc. 8th ICPR, Paris, France, pp. 566-569, October 1986.

[2] M. Ammar, Y. Yoshida, and I. Fukumura, "Feature Extraction and Selection for Simulated Signature Verification," Proc. 3rd Int'l Symp. on Handwriting and Computer Applications, Montreal, pp. 167-169, 1987.

[3] T. Sakai, T. Kanade, and Y. Ariki, "Multi-Feature Display of On-line Signatures by Color TV Display," Proc. 2nd Int'l Joint ConI on Pattern Recognition, pp. 303-304, Copenhagen, 1974.

[4] R. N. Nagel and A. Rosenfeld, "Steps Toward Handwritten Signature Verification," Proc. 1st Int'l Joint ConI Pattern Recognition, pp. 59-66, 1973.

[5] R. N. Nagel, "Computer Detection of Freehand Forgeries," Ph.D. thesis, Univ. Maryland, 1976.

[6] R. N. Nagel and A. Rosenfeld, "Computer Detection of Freehand Forgeries," IEEE Trans. on Computers, vol. C-26, no. 9, pp. 895-905, September 1977.

[7] R. Plamondon and G. Lorette, "Automatic Signature Verification and Writer Identification, The State of The Art," Pattern Recognition, Vol. 22, No.2, pp. 107-131, 1989.

[8] R. Sabourin and R. Plamondon, "Preprocessing of Handwritten Signatures From Image Gradient Analysis," Proc. 8th ICPR, Paris, France, pp. 576-579, October 1986.

[9] R. Sabourin and R. Plamondon, "On the Implementation of Some Graphometric Techniques for Interactive Signature Verification: A Feasibility Study," Proc. The Third Int'l Symp. on Handwriting and Computer Applications, Montreal, Canada, pp. 160-162, 1987.

[10] R. Sabourin and R. Plamondon, "Segmentation of Handwritten Signature Images Using The Statistics of Directional Data," Proc. 9th ICPR, Rome, Italy, pp. 282-285, November 1988.

[11] R. Sabourin and R. Plamondon, "Segmentation of Handwritten Signature Images: A Structural Approach," Tech. Report TR89a, Ecole de Technologie Superieure, Montreal QC, Canada, 1989.

[12] R. Sabourin and R. Plamondon, "Structural Interpretation of Handwritten Signature Images," Tech. Report TR90a, Ecole de Technologie Superieure, Montreal QC, Canada, 1990.


[13] R. Sabourin and R. Plamondon, "Progress in the Field of Automatic Handwritten Signature Verification Systems Using Gray-level Images," Proc. International Workshop on Frontiers in Handwriting Recognition, Montreal, April 1990.

[14] R. Sabourin and R. Plamondon, "Steps Toward Efficient Processing of Handwritten Signature Images," Vision Interface 90, Halifax, Nova Scotia, May 1990.

[15] R. I. Phelps, "A Holistic Approach to Signature Verification," Proc. ICPR, vol. 2, p. 1187, 1982.

[16] W. F. Nemcek and W. C. Lin, "Experimental Investigation of Automatic Signature Verification," IEEE Trans. on SMC, pp. 121~126, January 1974.

[17] E. R. Brocklehurst, "Computer Methods of Signature Verification," NPL Report DITC 41/84, pp. 1~12, May 1984. Also in: J. of the Forensic Science Society, Vol. 25, pp. 445~457, 1985.

[18] V. Klement, R. D. Naske, and K. Steinke, "The Application of Image Processing and Pattern Recognition Techniques to the Forensic Analysis of Handwriting," Int'l Con/. on Security Through Science and Engineering, Berlin, pp. 5~11, September 1980.

[19] F. Nouboud, "Contribution it l'etude et it la Mise Au Point d'un Systeme d'Authentification de signatures Manuscrites," These de Doctorat, Universite de Caen, France, 1988.

[20] F. Nouboud, F. Cuozzo, R. Collot, and M. Achemlal, "Authentification de Signatures Manuscrites par Programation Dynamique," Proc. Congres PIX 1M, Paris, pp. 345~960, 1988.

[21] P. C. Chuang, "Machine Verification of Handwritten Signature Images," Proc. Int'l Con/. on Crime Countermeasures, Sciences and Eng., pp. 105~1O9, 1977.

[22] M. Ammar, Y. Yoshida, and I. Fukumura, "Description of Signature Images and Its Application to their Classification," Proc. 9th ICPR, Rome, Italy, pp. 23~26, 1988.

[23] W. R. Harrison, Suspect Documents, Their Scientific Examination, Chicago, Nelson-Hall, 1981.

[24] O. Hilton, Scientific Examination of Questioned Documents, Elsevier, NY, revised edition, 424 Pages, 1982.

[25] E. Locard, Les Faux en Ecriture et leur Expertise, Payot, Paris, 1959. [26] J. J. Freyd, "Representing The Dynamics of a Static Form," Memory Cognition

11, pp342~346, 1983. [27] T. O. Binford, "Survey of Model-Based Image Analysis Systems," Int'l J. of

Robotics Research, Vol. 1, No.1, Spring 1982. [28] M. A. Fischler and R. A. Elschlasser, "The Representation and Matching of Pic

torial Structures," IEEE Trans on Computers, C-22, No.1., pp. 67~92, 1973. [29] M. D. Levine, "A Knowledge-Based Computer Vision System," In E. M. Riseman

and A. R. Hanson (eds.), Computer Vision Systems, Academic Press, New York, 1978.

[30] J. P. Requier and G. Lorette, "Reconnaissance de Signatures Manuscrites," Actes du 2e Congres A FCET-IRIA , Reconnaissance des Formes et Intelligence Artificielle, Tome III, pp. 298~307, 1979.

[31] N. Otsu, "A Threshold Selection Method from Gray-Level Histograms," IEEE Trans. on SMC, Vol. SMC-9, No.1, pp. 62~66, 1979.

[32] R. Sabourin, "Une Approche de Type Comprehension de Scene Appliquee au Probleme de la Verification Automatique de I'Identite par l'Image de la Signature Manuscrite," Ph.D. Thesis, Ecole Poly technique de Montreal, 1990.