The adaptative local Hausdorff-distance map as a new dissimilarity measure ´ Etienne Baudrier ∗ , Gilles Millon, Fr´ed´eric Nicolier, Su Ruan Centre de Recherche en STIC (CReSTIC) IUT de Troyes, 9, rue de Qu´ebec, 10026 TROYES CEDEX, FRANCE {e.baudrier, g.millon, f.nicolier, s.ruan}@iut-troyes.univ-reims.fr

Abstract The similarity measure process is generally composed of two steps: an image signature extraction and a signature comparison to take a decision about the image similarity. In the scope of binary images, we propose to replace the information mining by a new straight image comparison which does not require a priori knowledge. With this end, a local Hausdorff distance is proposed to measure locally the dissimilarities in an adaptative way. A local-distance map which contains the dissimilarity spatial layout is then built. As an application, the algorithm has been successfully tested and compared with other methods on an ancient illustration database. Key words: binary images, Hausdorff distance, similarity measures, dissimilarity layout

Introduction Image retrieval is an active domain (Smeulders et al., 2000). Retrieving images by their content, as opposed to meta-data, has become an important activity. It is classically composed of two stages: firstly, an information mining, which results in an image signature and secondly, a signature distance measure used to decide on image similarity. In this process, conspicuous features must be captured by the signature of each image in order to be as discriminating as possible in some user-defined sense (Lecce and Gerriero, 1999). Generally, the signature contains color, shape or texture information of one image. But the choice of the signature attributes is not easy and depends on the processed images (Antani et al., 2002; Smeulders et al., 2000). In the scope of binary ∗ Corresponding author, tel: 0(33)3.25.41.71.11, fax: 0(33)3.25.42.71.14

Preprint submitted to Elsevier Science

16 December 2004

images, we propose to replace this awkward information mining by a straight image comparison based on a modified Hausdorff distance (HD) producing a local-distance map (Huttenlocher and Rucklidge, 1993; Veltkamp and Hagedoorn, 2000). The second stage can be then replaced by a decision process based on the distance map. While a priori knowledge on discriminating features is required by information mining before comparing images. Our process first expresses dissimilarities straightly from the image comparison before making a decision. This process developed for binary images can be adaptable to pattern recognition.

State of the art Content based image retrieval (CBIR) has become an active research field since the early 1980s (Chan and Fu, 1980). The research has developed various methods to retrieve visual information by its content based on a subsystem which extracts features or makes pattern recognition. Features, that are generally color, shape and texture, are used to select good matches in response to the user’s query (Smeulders et al., 2000). Once extracted, the features are compared with a similarity measure. Similarity measures are designed according to the features which are treated separately or aggregated in a vector. They are commonly based on the histogram comparison (Brunelli and Mich, 2001) and when features are in a vector multidimensional space, a distance measure is used to estimate their similarity. The best known are the Euclidean distance, the Chamfer distance, the Hausdorff distance and the Mahalanobis distance (Veltkamp and Hagedoorn, 1999). In this process, the choice of the features is an awkward operation which depends on the application and on the user’s query (Antani et al., 2002), and moreover it results in a loss of spatial information coming from the feature extraction. In the case of binary images which can be real images or the result of an edge extraction, the situation is easier. Indeed, the content is easily extracted, considering the black pixels to be the content and the white ones to be the background. In Klette and Zamperoni (1987), several measures of correspondence between binary images are described and compared. This study shows that distancebased measures perform better comparisons of binary images than measures based on set memberships. The most common distance-based measures are the Euclidean distance, the distance measures that are based on a 1 - 1 correspondence and the measures based on the Hausdorff distance. The measures based on a 1 - 1 correspondence comprise the bottleneck distance, minimum weight matching, uniform matching and minimum deviation matching (Veltkamp and Hagedoorn, 1999). They are used in graph theory and imply to find a correspondence between the points of the two images. So it is necessary for images to have the same number of points or to determine a number k of points to match, which is delicate. The Hausdorff distance on which our measure is based, is a max-min distance and it doesn’t have this inconvenience. It is widely used, from face recognition (Zhao et al., 2004) to match binary patterns in images (Huttenlocher et al., 1993). 2

Nevertheless, for binary images, the shape extraction is deemed difficult and comes down to extract connected pixels. So local information is difficult to access to. A good way to access to local representation is to measure a local distance. Recently Wang et al. (2004) have exploited the know characteristics of the Human Visual System (HVS) to produce a Structural Similarity (SSIM) Index. This index of local measures could be seen as a measure of local dissimilarity but it is done through a 8 × 8 window independently of the content. In the case of binary images, this study will show the possibility of giving an adaptative measure of the local distance. We present an overview of the Hausdorff distances (HD) in section 1. Secondly the notion of local HD is introduced and its properties are exposed in sections 2,3. It enables to determine automatically the size of the local HD’s window in function of the dissimilarity (section 4). Thank to this criterion, the map of local distances is defined using a parameter-free algorithm in sections 4,5. Finally, qualitative results and an application in content based image retrieval (CBIR) are presented in section 6 before concluding.

1

Overview of the Hausdorff distance

1.1 Dissimilarity measure over binary images: the choice of the Hausdorff distance Among dissimilarity measures over binary images, the Hausdorff distance (HD) has often been used in the content-based retrieval domain and is known to have successful applications in object matching (Huttenlocher and Rucklidge, 1993; Kwon et al., 2001) or in face recognition (Tak`acs, 1998; Jesorsky et al., 2001). It can be computed quickly using Voronoi diagrams (Borgefors, 1986). Let’s have a brief review of the definition and of some properties related to the HD. Originally meant as a measure between two point collections A and B in a metric space E (whose underlying distance is d), it can be viewed as a dissimilarity measure between two binary images A and B, considering A and B respectively the black pixels of A and B. For finite sets of points, the HD can be defined as (Huttenlocher et al., 1993): Definition 1 (Hausdorff distance) Given two non-empty finite sets of points A = (a1, . . . , an) and B = (b1, . . . , bm) of R2 , and an underlying distance d, the HD is given by DH (A, B) = max (h(A, B), h(B, A)) µ

¶

where h(A, B) = max min d(a, b) , a∈A

b∈B

3

(1) (2)

h(A, B) is the so-called directed Hausdorff distance. For images, we use the same notation: DH (A, B) = DH (A, B). The interest of this measure comes firstly from its metric properties: nonnegativity, identity, symmetry and triangle inequality. These properties correspond generally to our intuition for shape resemblance. Indeed, a pattern is identical to itself. And the order of comparison does not generally matter; for the case where the order of comparison is important, the property of the HD according to which the directed distance is not symmetrical can be exploited. Finally, the triangular inequality prevents some unknown patterns to be similar to two dissimilar patterns at the same time. Moreover, the HD is a match methodology without point-to-point correspondence, so it is robust to local non-rigid distortions. Another source of interest is the following property: Proposition 2 (Translation) Let v be a vector of R2 , Tv translation of vector v and A a non-empty finite set of points, then DH (A, Tv A) = kvk.

(3)

PROOF. As A = T−v (Tv A), it is equivalent to show DH (A, Tv A) = kvk and h(A, Tv A) = kvk. Let’s prove this last point. • Let’s show that h(A, Tv A) 6 kvk. ∀a ∈ A, minb∈B d(a, b) 6 d(a, Tv a) 6 kvk So maxa∈A minb∈B d(a, b) 6 kvk. • Now, let show that h(A, Tv A) > kvk. Suppose that h(A, Tv A) = r < kvk S then Tv A ⊂ a∈A B(a, r). Absurd! 2

It implies that for a small translation, the Hausdorff distance is small, which matches our expectation for a dissimilarity measure.

1.2 Some modified versions of the Hausdorff distance The classical HD has good properties but it measures the most mismatched points between A and B, and as a consequence it is sensitive to noise (Paumard, 1997). Indeed considering two images containing the same pattern and one point added to the first image, far from the pattern, then the HD will measure the distance between the pattern and the point. Several modifications of the HD have been proposed to improve it such as the 4

partial HD (Huttenlocher et al., 1993), the modified HD (MHD) (Dubuisson and Jain, 1994), the censored HD (CHD) (Paumard, 1997), the ”doubly” modified Hausdorff distance (M2HD) (Tak`acs, 1998), the least trimmed squared HD (LTS-HD) (Sim et al., 1999) and the weighted Hausdorff distance (WHD) (Lu et al., 2001). The next definitions have mainly been presented in Zhao et al. (2004). The directed distance of the partial Hausdorff distance is defined by Huttenlocher et al. (1993): th hK (A, B) = Ka∈A d(a, B) (4) th where Ka∈A denotes the Kth ranked value of d(a, B). The partial Hausdorff distance method yields good results for an impulse noise case. The directed distance of the MHD is defined in (Dubuisson and Jain, 1994):

hM HD (A, b) =

1 X d(a, B) Na a∈A

(5)

where Na = card(A). The MHD measure does not require any parameter as the partial Hausdorff distance, and the method adapts only to a Gaussian noise case. The directed distance of the WHD is defined in (Zhao et al., 2004): hW HD (A, B) =

1 X w(a) · d(a, B) Na a∈A

(6)

P

where a∈A w(a) = Na . An image can be divided into different parts, and the contribution of the different parts to the image matching is different, therefore the Hausdorff distance should be different. The WHD has been used in the Chinese character image matching (Lu et al., 2001; Lu and Tan, 2002) and in face recognition Guo et al. (2001). The directed distance of the CHD is defined by Paumard (1997): th hk,l (A, B) = Pa∈A Qth b∈B d(a, b)

(7)

th where P th denotes the P th ranked value of Qth b∈B d(a, b), with Qb∈B representing the Qth ranked value of the underlying distance set. Since the CHD ranks the underlying distance, the effect of the impulse noise to the image is reduced.

The directed distance of the M2HD is defined by Tak`acs (1998): hM =

1 X d(a, B) Na a∈A ´

³

(8)

with d(a, B) = max I minb∈NBa d(a, b), (1 − I)P where NBa is a neighborhood of the point a in the set B and I indicates if there exists a point b ∈ NBa . 5

The directed distance of the LTS-HD is defined in (Sim et al., 1999): hLT S (A, B) =

H 1 X d(a, B)(i) H 1

(9)

where H denote h × NA , as in the partial HD case is, and d(a, B)(i) represents the ith distance value in the sorted sequence d(a, B)(1) 6 d(a, B)(2) 6 · · · 6 d(a, B)(Na ) . The measure hLT S (A, B) is minimized by remaining distance values, after large distance values are eliminated. Even if the object is occluded or degraded by noise, this matching scheme yields good results.

1.3 Discussion

It is noticeable that except for the MHD, at least one arbitrary parameter has to be determined: k for the partial HD, the size of the neighborhood NBa for the M2HD, the weighted function w for the WHD or the rank (i) in the LTS-HD. The parameter must be chosen to make the measure as discriminating as possible and it depends obviously on the kind of images, and even sometimes on the compared images in the same application (e.g. more or less dark or noisy images). The MHD measure does not require any parameter. However its matching performance is not as good as the partial HD and the CHD, due to the summation operator over all distances, some of which might be computed from outliers. Moreover, these measures are global and cannot account for local dissimilarities. Indeed, the principle of HD is to be a ”max min” distance and it means that the value of the HD between two images is reached for at least one couple of points. But it doesn’t say if the value is reached in several parts or only for one pair, which corresponds to different degrees of dissimilarity. These remarks motivate us to design a local and parameter-free HD in the next section.

2

Local HD measure

In this section, the notion of local dissimilarity is first discussed, then a naive definition of a HD measure in a local window is presented. The use of the HD in a local window implies some modifications to make it consistent when window size varies. A necessary modification is brought. It finally leads to a fully consistent definition. In all this section, A and B design two non-empty finite sets of points of R2 , and W a convex closed subset of R2 . 6

2.1 What is a local dissimilarity? Producing locally a dissimilarity measure implies to compare the two images locally. It can be done thanks to a sliding window. The parts of both images viewed through this window are compared with a dissimilarity measure. The sliding-window size plays an important rˆole: it should fit the local dissimilarity so that the distance can give a local measure. As the dissimilarity size is a high-level notion related to semantics, it is not desirable to give a precise definition of it in this low-level study. Nevertheless, here is a rough idea of it: if the pixels located in the sliding-window belong to coarse features, the window should be big enough to grasp feature’s dissimilarities. Similarly, for fine features, a window ”bigger” than the features will include unwanted information on dissimilarities. Therefore, it is necessary to adapt the size of the window to obtain precise measures The meaning of ”local” in ”local dissimilarity” has also to be clarified. We make the assumption that a local dissimilarity must concern information involving the central pixels in the window, i.e. pixels whose distance from the central pixel is less than sliding step p (in our case, as p = 1, there is only the central pixel of the window). 2.2 Naive definition It consists in modifying the definition of the global measure (def. 1) by introducing a subset standing for the window: Definition 3 (HD in a window (naive)) HDW (A, B) = max (hW (A, B), hW (B, A)) µ

where hW (A, B) = max

a∈A∩W

¶

min d(a, b) .

b∈B∩W

(10) (11)

But this definition is not available for the possible case where one set (for example B) has no point in W and the other one has. So as to make the value consistent when W grows (see fig. 1 (a)), the distance to the frontier of W must be taken into account to improve the definition. 2.3 Modification of the naive definition Indeed, if there are points of A outside W and close to its frontier (fig. 1 (a)), then if W grows and includes them, the new HDW will be equal to the distance to these points. The most restrictive case will be when there are points of A 7

Discrete case Points of A

Points of B

2n+1

2n+1

2n+3

2n+1

W

W

W

W Naive HD non defined

Naive HD = n+1

Modified local HD = 2n Modified local HD = n

Distance to Fr(W) = n

Distance to Fr(W) = n+1

Final local HD = n−1

(a)

Final local HD = n

(b)

Fig. 1. Examples of critical cases for the local HD. (a) If B ∩ W = φ, the naive measure is not defined, to define it and to make it consistent where W increases, the distance to the frontier is proposed. (b) The definition has then to be modified to be consistent when the W slides in R2

all over the frontier of W . Then to make it consistent with a bigger W , the computation of HDW must include the distance of the points of the two sets A and B to the edges of W . Thus, if B has no point in W , the directed distances have the following definition: Definition 4 (First modification) There are two cases: • If A and B are both non-empty HDW (A, B) = max (hW (A, B), hW (B, A)) µ

where hW (A, B) = max

a∈A∩W

(12)

¶

min d(a, b) .

(13)

b∈B∩W

• If exactly one of the two subsets A and B is empty (for example B), Ã

hW (A, B) = max

a∈A∩W

!

µ

min d(a, w) , hW (B, A) = max w∈W

w∈F r(W )

¶

min d(a, w) .

a∈A∩W

Remark 5 In the case where there is no point of A nor of B in W, as the two subsets are equal, the measure is prolonged by 0. The frontier comes from the topology defined by the metric d. In the application (section 6), the distance is the one associated to the norm L∞ . 2.4 Local Hausdorff distance Once this case is clarified, the definition in the global case has to be modified to bring a consistency when the window slides over the images. Indeed taking 8

two cases in the definition will bring sharp variations of the distance value when the window slides and would not reflect the intuition (see fig 1(b)). Then the distance to the frontier of W has to be taken into account in the definition. Definition 6 (Final version: the Hausdorff Distance in a window) Let A, B be two bounded sets of points of R2 , and W a convex closed subset of R2 . HDW (A, B) = max (hW (A, B), hW (B, A)) "

where hW (A, B) = max

a∈A∩W

3

Ã

min

!#

min d(a, b), min d(a, w)

b∈B∩W

w∈F r(W )

.

(14) (15)

HDW properties

In this section, useful properties are demonstrated. The criterion proposed in sec. 4 is based on these properties: • the HDW is between 0 and HD(A, B), • it is non-decreasing when embedded windows W are considered. 3.1 General properties HDW is non-negative and symmetric by definition. Proposition 7 (Identity) Let A, B be two bounded sets of points of R2 , and W a convex closed subset of R2 . HDW (A, B) = 0 ⇐⇒ A ∩ W = B ∩ W PROOF. (⇐) If then

A∩W =B∩W

minb∈B∩W (d(a, b)) = 0 ³

´

and so min minb∈B∩W d(a, b), minw∈F r(W ) d(a, w) = 0 and then by (15)

hW (A, B) = 0,

in the same way,

dW (B, A) = 0

and so by (14)

HDW (A, B) = 0.

(⇒) If HDW (A, B) = 0 9

(16)

then both directed distances are equal to zero, and as the distance to the points of F r(W ) is never equal to zero, it remains to the property of the classical Hausdorff distance which is a metric. 2

Proposition 8 (Boundary) Let x ∈ R2 and r > 0, and let define W = B(x, r) then HDW (A, B) 6 HD(A, B).

PROOF. Suppose HDW (A, B) > HD(A, B). Without loss of generality, suppose

HDW (A, B) = hW (A, B)

i.e.

³

³

´´

= maxa∈A∩W min minb∈B∩W d(a, b), minw∈F r(W ) d(a, w)

Then

∃ a0 ∈ A ∩ W reaching the maximum: ³

´

hW (A, B) = min minb∈B∩W d(a0 , b), minw∈F r(W ) )d(a0 , w) . There are two cases: ³

(1) If

´

min minb∈B∩W d(a0 , b), minw∈F r(W ) d(a0 , w) = minb∈B∩W d(a0 , b),

then ∃ b0 ∈ B ∩ W/ minb∈B∩W d(a0 , b) = d(a0 , b0 ). For the point a0 , there are two pieces of information: • this point is more distant from the edges of W than d(a0 , b0 ). Indeed, if it is not the case, we have Ã

min

!

min d(a0 , b), min d(a0 , w) =

b∈B∩W

w∈F r(W )

min d(a0 , w),

w∈F r(W )

which is not the hypothesis. • W ⊃ B(a0 , d(a0 , b0 )) (because W is also a ball) and by definition of b0 , there is no other point of B in B(a0 , d(a0 , b0 )) than b0 . With this last fact, it is easy to compute that for a0 , minb∈B d(a0 , b) = d(a0 , b0 ) i.e. so now in this case (1),

d(a0 , b0 ) is one of the minimum, d(a0 , b0 ) 6 maxa∈A (minb∈B d(a, b)) . HDW (A, B) = d(a0 , b0 ),

so HDW (A, B) 6 HD(A, B) This is contradictory to the hypothesis.

10

.

³

´

(2) If min minb∈B∩W d(a0 , b), minw∈F r(W ) d(a0 , w) we note

= minw∈F r(W ) d(a0 , w),

minw∈F r(W ) d(a0 , w) = r0 ,

so

B(a0 , r0 ) ⊂ W

and

B ∩ B(a0 , r0 ) = φ.

So

minb∈B (a0 , b) > r0

and then

maxa∈A minb∈B d(a, b) > r0 ,

now in this case (2),

r0 = HDW (A, B),

So HD(A, B) > HDW (A, B) which is contradictory to the hypothesis. 2

So when the window W slides all over the two images, the values in the produced dissimilarity map will remain between 0 and HD(A, B). But what happens when the size of the window W is increasing? It seems intuitive that the value is increasing too. This is the object of the following proposition.

3.2 Property depending on the window-size Proposition 9 (growth) Let V = B(xv , rv ) and W = B(xw , rw ) be two close discs such as V ⊂ W then HDV (A, B) 6 HDW (A, B).

PROOF. The difference with the previous property is the distance to the edges in HDW . First point: V ⊂ W implies that for all point v ∈ V , d(v, F r(V )) 6 d(v, F r(W )).

³

´

³

´

So hV (A, B) = maxa∈A∩V min minb∈B∩V (d(a, b), minv∈F r(V) d(a, v)

6 maxa∈A∩V min minb∈B∩V d(a, b), minw∈F r(W) d(a, v) . The demonstration follows the same way as the previous one: suppose HDV (A, B) > HDW (A, B) then without loss of generality, we can suppose HDV (A, B) = hV (A, B). Then ∃ a0 ∈ A ∩ V reaching the maximum: Ã

hV (A, B) = min

!

min d(a0 , b), min d(a0 , v) .

b∈B∩V

There are two cases: 11

v∈F r(V )

³

(1) Let hV (A, B) = min minb∈B∩V d(a0 , b), minv∈F r(V ) d(a0 , v)

´

= minb∈B∩V (d(a0 , b). Then ∃ b0 ∈ B ∩ V / minb∈B∩V (d(a0 , b)) = d(a0 , b0 ). For the point a0 , there are two pieces of information: • minv∈F r(V ) d(a0 , v) > d(a0 , b0 ), • V ⊃ B(a0 , d(a0 , b0 )) and B(a0 , d(a0 , b0 )) ∩ B = b0 . With this last fact, it is easy to compute that d(a0 , b0 ) is one of the minima for b ∈ B ∩ W , i.e. minb∈B∩W d(a0 , b) = d(a0 , b0 ). Let’s prove now that d(a0 , b0 ) 6 minv∈F r(V ) d(a0 , v). By hypothesis, we have d(a0 , b0 ) 6 minv∈F r(V ) d(a0 , v) and as a0 ∈ V and V ⊂ W , we have minw∈F r(W ) d(a0 , w) > minv∈F r(V ) d(a0 , v) i.e. minw∈F r(W ) d(a0 , w) > d(a0 , b0 ) ³

´

and then min minb∈B∩W d(a0 , b), minw∈F r(W ) d(a0 , w) > d(a0 , b0 ) ³

´

now, hW (A, B) = maxa∈A∩W min(minb∈B∩W d(a0 , b), minw∈F r(W ) d(a0 , w) ), so hW (A, B) > d(a0 , b0 ). This is contradictory to the hypothesis. ³

(2) Let hV (A, B) = min minb∈B∩V d(a0 , b), minv∈F r(V ) d(a0 , v)

´

= minv∈F r(V ) d(a0 , v). We note minv∈F r(V ) d(a0 , v) = r0 , it means that the disc B(a0 , r0 ) ⊂ V and B(a0 , r0 ) ∩ B = φ. So, on the one hand

minb∈B∩W d(a0 , b) > r0

and on the other hand r0 = minv∈F r(V ) d(a0 , v) 6 minw∈F r(W ) d(a0 , w) then min(minb∈B∩W d(a0 , b), minw∈F r(W ) d(a0 , w)) > r0 so now so

hW (A, B) > r0 r0 = hV (A, B) hW (A, B) > hV (A, B),

which is in contradiction with the hypothesis. 2

4

A parameter-free adaptative local HD

According to subsection 2.1, we need to precise the notion of local dissimilarity mathematically to define the criterion for the optimal window size. This is the 12

object of the next subsection.

4.1 Characterization of the measure of local dissimilarity In all the paragraph, A and B design two non-empty finite sets of points of R2 . The next property specifies the conditions on A and B where HDW (A, B) is maximum. Then, the notion of local dissimilarity in a HD way is defined. This enables finally to define an optimal size for a window B(x, r) in function of A and B. Lemma 10 (maximum value) Let x ∈ R2 and r > 0, and let a window B(x, r) then supA,B HDB(x,r) (A, B) is reached only when there is exactly one point of A (resp. B) at the center of W and no point of B (resp. A) except maybe on F r(W ). Then HDB(x,r) (A, B) = r.

PROOF. Without loss of generality, we can focus on hW (A, B). For points of A³ in W that are not at the center of ´W , minwinF r(W ) d(a, w) < r then min minb∈B∩W (d(a, b), minwinF r(W ) d(a, w) < r. For a0 at the center of W , ³

´

min minb∈B∩W (d(a0 , b), minwinF r(W ) d(a0 , w) = r and it is reached for the points on³ F r(W ). For the other points of A in W ´ if there are any of them, this min minb∈B∩W (d(a0 , b), minwinF r(W ) d(a0 , w) is below r, and so the max over the points of A is r. 2

The aim here is to give a criterion for the window size. As it is explained at the beginning of the first paragraph, we assume a local dissimilarity to be related to features including the central pixel in W . We also want the window size to be with no other dissimilarity (i.e. related to others features) in it. Thus the measure must concern • a central point: if it is not involved, the window can be moved to put one of the involved point at its center, • and an edge points: if none of the edge points is involved, the window can be shrunk. Lemma 10 implies that the measure in the window reaches its maximum value. From which the definition: Definition 11 (local measure) A window W = B(x, r) is said to give a local measure when the measure of the HD in the window B(x, r) is maximum: HDB(x,r) (A, B) = r. 13

We want to know if there is a maximum local measure. So, let x ∈ R2 and r > 0, let’s define: Definition 12 (local-measure set) The local-measure set R is given by R = {r > 0/HDB(x,r) (A, B) = r}.

(17)

When R is non-empty, it is bounded by HD(A, B) (prop 9), so it has a upper bound rmax . From which the definition: Definition 13 (Maximum local measure) For x ∈ R2 fixed, if R is non empty, rmax = sup(R) is named the optimal ray and for this ray, HDB(x,rmax ) is said to give the maximum local measure rmax .

4.2 Criterion for the size of the window B(x, r) The aim here is firstly to provide a criterion to compute rmax and secondly to design an algorithm so as to compute numerically HDB(x,rmax ) (A, B). Proposition 14 (nature of the subset R) Let x ∈ R2 , there are two possibilities: • R=φ • R = [0, rmax ].

PROOF. This is a consequence of Lemma 10 • If x belongs neither to A nor to B or if x belongs to A and B, then by lemma 10, HDW will never reach its maximum value. • if x belongs only to one of the two subsets A and B, for example A. Then, as B is finite, ∃ bmin ∈ B ∩ W , the closest point to x (for the distance d). By lemma 10, we know that for 0 6 r 6 d(a0 , bmin) = rm , W will give a local measure and for r > rm , as minb∈B∩W (a0 , b) = rm < r, HDW cannot have its maximum value. So R is the interval [0, rm ] and rm = rmax . 2 Criterion 15 Let A and B be two non-empty finite sets of points of R2 and x ∈ R2 , as regards the local HD measure in x, the optimal radius r for the window W = B(x, r) is rmax = max({r/HDB(x,r) (A, B) = r}). r>0

14

(18)

We have now a criterion to find the maximal size of W = B(x, r) in order to measure the local dissimilarity. Thus, the study has been presented for a fixed point of R2 , we will now consider the whole plan.

5

Local distance map

5.1 Definition Definition 16 (Local Distance map (LDMap)) Let A and B design two non-empty finite sets of points of R2 , the local distance map LDM ap is defined by  2

∀x ∈ R , LDM ap(x) =

  HDB(x,r max ) (A, B) if R 6= φ   0

if R = φ.

(19)

Corollary 17 (Maximum value in the LDMap) The value v = HD(A, B) is reached at least once in the LDMap: max(LDM ap(A, B)) = HD(A, B).

(20)

PROOF. As A and B are finite, definition 1 implies that HD(A, B) is reached for two points, for example a0 and b0 : h(A, B) = d(a0 , b0 ). Lemma 10 implies that for a0 , rmax = d(a0 , b0 ).

5.2 The discrete case Now we will use the LDMap in the practical case of the comparison of two digital images in order to obtain a dissimilarity map. To enable the computation near the borders of the images, the images are extended by white pixels (color of the background). The discrete case brings practical issues: the topological objects have to be made clear. • B(x, r): for a pixel x = [i, j] and r > 0, B(x, r) = {y/d(x, y) < r}. • F r(B(x, r)): we consider that the frontier is between the pixels. For example, the frontier of B(x, n) is the line between the sets {d(x, y) = n − 1} and {d(x, y) = n}. Let’s define C(x, n) = {y/d(x, y) ∈]n, n + 1]}. The distance of a point z ∈ B(x, n) to the frontier is equal to the distance to the pixels just after the frontier. Thus, for a point z in B(x, n) and on the side of B(x, n), its distance to the frontier is equal to 1

15

5.3 Algorithm The algorithm for calculating the local HD map id proposed below. Algorithm 1 (Computation of LDMap) Compute DH (A, B) For each pixel x do n := 3 While HDB(x,n) (A, B) = n and n 6 HD(A, B) n := n + 1 EndWhile Return LDM ap(x) = HDB(x,n−1) (A, B) = n − 1 EndFor

5.4 Computational complexity Let’s consider two m × m images. The computation of the LDMap begins with the computation of the global HD, and then a recursive loop to compute the maximal local HD (alg. 1): • For each pixel, the size of the window is increased step by step from 3 2 to rmax 6 HD(A, B). It remains to compute rmax comparisons, which is 2 majorized by HD(A, B) . • The computation is done for all the m × m pixels. Thus the complexity for the LDMap is O(m2 HD(A, B)2 ). Remark 18 rmax is a variable that is majorized here by HD(A, B), but most of the time, this bound is not reached. Moreover, this bound cannot be reached at the same time for every pixels.

5.5 Qualitative results Some images and their LDMaps are presented here. Figure 2 contains simple patterns (a vertical line, a horizontal one and a square) and their corresponding LDMaps. The darker the pixel in the LDMap, the higher the local distance. Let’s give some comments about the comparison of the vertical and the horizontal lines: for the pixel where they cross, the value is equal to zero and for the other black pixels, the more distant from a line, the bigger the distance in the LDMap. The value of the global HD is 11. The comparison between the vertical line and the square results in the same global HD (= 11) but the 16

10

10

20

20

30

30

40

40

50

50

60

60 20

40

60

20

40

60

(C)

(B)

10

10

10

10

20

20

20

8

30

30

30

6

40

40

40

4

50

50

50

60

60

60

20

40

60

20

40

60

20

LDMap(A,B)

(A)

2

40

60

0

LDMap(A,C)

Fig. 2. Three simple patterns and their LDMaps. A vertical line, a horizontal one, a square and their LDMaps. The darker the pixel, the higher the distance measure . 10

10

10

20

20

20

30

30

30

40

40

40

50

50

50

60

60

60

70

70

70

80

80

80

90

90

90

16

14

12

10

8

6

20

40

60

80

20

40

60

80

20

40

60

80

4

Fig. 3. Letters CO and ET and their LDMap illustrating their local dissimilarities.

spatial layout shows that this value is reached in numerous pixels (belonging to the square’s vertical sides) while it is reached only four times in the comparison of the two lines (at their ends). Figure 3 illustrates the notion of local distance. Each compared image contains two letters. The first two letters (”c” and ”e”) are similar and the high distances (in black) are located ”on” the place of the line of the ”e” that is not shared with the ”c”. The comparison of the ”o” and the ”t” results in the same behavior. Figure 4 offers an elaborate way to save time during the holidays. Indeed the images come from a ten-error game: the second image is a copy of the first one, but with ten differences (the ten errors). They have been photographed from slightly different points of view and straight comparison C = |B − A| doesn’t allow to find the ten errors. The LDMap (image D) highlights most of the errors (we have circled the 10 errors in black to highlight them), except the black stripe on the tracksuit (that has become two black stripes). The reason is the anisotropy of the ”error”: it is long and not wide. The window stops its growth as soon as it meets points of the stripes and so it is blind to 17

Fig. 4. The ten-error game. The two images to compare (A and B), the absolute difference C = |B − A| and their LDMap (D) where we have circled the errors in black.

the length of the ”error”. 5.6 Limitation Figure 4 shows with the tracksuit’s stripe(s),if the features have a small dissimilarity in one direction but a large one in another one, the large dissimilarity can not be captured. Indeed, in this case, the local HD isotropy makes it stop as soon as the small dissimilarity is measured. As a consequence, if the images comprise numerous close lines, the local HD measures will be low, as if they were similar even if the lines don’t have the same direction in the two images. This limitation can be prevented by pre-processing the image with oriented filters in order to measure dissimilarities in some selected directions. 5.7 Comparison We present here some results regarding the advantages of the LDMap with respect to the modified HD existing in the literature. Figure 5 presents two 18

10

10

20

20

30

30

40

40

50

50

60

60 20

40

60

20

(D)

40

60

(E) 10

10

10

10

20

20

20

8

30

30

30

6

40

40

40

4

50

50

50

60

60

60

20

40

60

20

(A)

40

2

60

20

40

60

0

LDMap(A,E)

LDMap(A,D)

Fig. 5. LDMap from the comparison of the image A (vertical line) with the image D (shifted image A) and the image E (dotted shifted image A).

method comparison

HD

partial HD

MHD

WHD (threshold)

AD

11

11

11

11

AE

11

11

11

11

Table 1 Table of results for several HD measures of the pair A, D and A, E. The results remain equal although the line in E has been dotted.

simple object’s comparisons that results in the same HD value 11. As shown in the table 1, the comparisons of the image A with the images C and D give the same value for the partial HD, the MHD, the WHD, although in the first case, two plain lines are compared and in the second one, a plain line is compared with a dashed one. The LDMap accounts clearly for this distinction and enables a potential decision to distinguish these two cases.

6

Application: Content Based Image Retrieval

In order show the interest of the proposed distance map with quantitative results, it has to be included in a global classification process. The aim of this section is only to compare the dissimilarity map to other methods. The test database is composed of digitalized ancient illustrations provided by Troyes’ library in the framework of the project ANITA (Seulin et al., 2003). 19

6.1 The global classification process It is composed of two stages: firstly the construction of a LDMap between two images, secondly a decision using the Support Vector Machines (SVM) based on the obtained distance map. During the acquisition, the ancient impressions have been rescaled and binarized. The chosen size is 64×64 pixels to save computation time. Instead of a comparison of two image signatures, the distance map allows a direct dissimilarity comparison including spatial information on the dissimilarities. This piece of information is exploited in the decision step.

Decision - Local-Distance-Map classification The comparison process results in a local-distance map. It can be classified in two classes: Csim class for similar images and Cdissim class for dissimilar images. A SVM method is used to classify the LDMaps into these two classes. A first study on the different SVM methods has been carried out and shows that the most efficient choice to deal with our data was the classical SVM (C-SVM) with a linear kernel (A¨ıt Aou¨ıt, 2004).

6.2 Experiments The proposed method is tested on the database of digitalized ancient illustrations. These images, originally printed in books, have strong contrast which allows to binarize them with almost no loss. This database is composed of 68 images, some of them illustrating the same scene. the objective is to retrieve illustrations of the same scene. First 2380 LDMaps were built, 103 of which are classified in Csim thanks to a manual comparison of the impressions. Examples of ancient impressions and their LDMaps are given in fig. 6. Impression 1 and 2 come from very similar wooden stamps and impression 3 illustrate the same scene with differences in the way of illustrating the grass and the helmets. Even if the global HD doesn’t reflect the similarity degree (HD(imp1, imp2) = 25, HD(imp1, imp3) = 15), DLM ap(imp1, imp3) is locally darker (so with higher values) than DLM ap(imp1, imp2) where the illustration differs. The impression 4 illustrates a distinct scene and high values (comparing to those of the other LDMaps) can be found all over DLM ap(imp1, imp2). Our objective is to test the method’s efficiency in assessing local dissimilarities. The experiments was carried out by the following way: first a supervised machine learning is made on a set of 30 LDMaps in Csim and 60 in Cdissim . Then, the test is done on a distinct set of 45 LDMaps of Csim and 100 of Cdissim . The choice of the sets in each class is randomized. Secondly, we apply 20

Imp 1 30 20 10

Imp 2

LDMap(Imp1,Imp2) 30 20 10

Imp 3

LDMap(Imp1,Imp3) 30 20 10

Imp 4

LDMap(Imp1,Imp4)

Fig. 6. Ancient impressions and their LDMaps. Here are 4 ancient impressions. Imp. 1, Imp. 2 and Imp. 3 illustrate the same scene with a different kind of grass and helmets in Imp. 3. Imp. 4 illustrates a distinct scene.

21

Successful retrieval

HDW

HD

HSD

found in Csim

95%

46%

66%

found in Cdissim 84% 70% 78% Table 2 Results for DH,W , the global HD and for the LDMap used with the absolute difference.

three classification methods. Finally, we compare the results obtained manually and automatically. The three classifications are the following: our method based on the LDMaps; another one based on the global HD instead of the LDMap and the last one - referred as HSD - using the distance map, but with the simple difference locally (with a fixed window size) instead of the HD: HSDW (A, B) = |A ∩ W − B ∩ W |. The results are summarized in table 2. They show the efficiency of the local distance map both concerning spatial information (comparison with the global HD) and concerning the ability of the local HD to catch the local dissimilarities (comparison with the HSD).

7

conclusion

This paper proposes a method to make an adaptative measure of the local dissimilarities between two images. With this end, a local Hausdorff distance is defined and its properties of boundary and growth are proved. This enables to provide a method that makes the HD measure fit the image local dissimilarity automatically. The result is a map containing the local distances and their spatial layout. These pieces of information can be exploited to distinguish images that are not discriminated by global similarity measures but have different contents. It can also be efficiently used as input for a supervised decision, as the application to the ancient wooden stamps base has shown it. Further works will explore the gray-level case and study the use of the LDMap for assessment of non-linear filters properties.

References A¨ıt Aou¨ıt, D., september 2004. Classification d’images par la m´ethode des support vector machines : ´etude et applications. Master’s thesis, University of Reims-Champagne-Ardenne, Troyes. Antani, S., Kasturi, R., Jain, R., April 2002. A survey on the use of pattern recognition methods for abstraction, indexing and retrieval of images and video. Pattern Recognition 35 (4), 945–965. Borgefors, G., 1986. Distance transformations in digital images. Comput. Vision Graph. Image Process. 34 (3), 344–371. 22

Brunelli, R., Mich, O., 2001. Histograms analysis for image retrieval. Pattern Recognition 34 (8), 1625–1637. Chan, N.-S., Fu, K.-S., 1980. Query-by-pictorial-example. IEEE Transactions on Software Engineering 6 (6), 519–524. Dubuisson, M.-P., Jain, A. K., October 1994. A modified Hausdorff distance for object matching. In: Proc. 12th IAPR International Conference on Pattern Recognition. pp. 566–568. Guo, B., Lam, K., Siu, W., Yang, S., 2001. Human face recognition using a spatially weighted Hausdorff distance. In: The 2001 IEEE International Symposium on Circuits and Systems. pp. 145–148. Huttenlocher, D. P., Klanderman, D., Rucklidge, W. J., September 1993. Comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence 15 (9), 850–863. URL citeseer.nj.nec.com/huttenlocher93comparing.html Huttenlocher, D. P., Rucklidge, W. J., 1993. A multi-resolution technique for comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 705–706. Jesorsky, O., Kirchberg, K. J., Frischholz, R. W., 2001. Robust Face Detection Using the Hausdorff Distance. In: Bigun, J., Smeraldi, F. (Eds.), Audio- and Video-Based Person Authentication - AVBPA 2001. Vol. 2091 of Lecture Notes in Computer Science. Springer, Halmstad, Sweden, pp. 90–95. URL citeseer.nj.nec.com/jesorsky01robust.html Klette, R., Zamperoni, P., November 1987. Measures of correspondence between binary patterns. Image and Vision Computing 5 (4), 287–295. Kwon, O.-K., Sim, D.-G., Park, R.-H., 2001. Robust hausdorff distance matching algorithms using pyramidal structures. Pattern Recognition 34 (10), 2005–2013. Lecce, A. D., Gerriero, V., 1999. An evaluation of the effectiveness of image features for image retrieval. Journal of Visual Communication and Image Representation 10 (4), 351–362. Lu, Y., Tan, C., 1115 August 2002. Word spotting Chinese document images without layout analysis. In: Proceeding of 16th International Conference on Pattern Recognition. Vol. 3. pp. 57–60. Lu, Y., Tan, C., Huang, W., Fan, L., 1013 September 2001. An approach to word image matching based on weighted Hausdorff distance. In: Proc. 6th Internat. Conf. on Document Anal. Recogn. pp. 921–925. Paumard, J., October 1997. Robust comparison of binary images. Pattern Recognition Letters 18 (10), 1057–1063. Seulin, R., Morel, O., Millon, G., Nicolier, F., may 2003. Range image binarization: application to wooden stamps analysis. In: International Conference on Quality Control by Artificial Vision (QCAV) - IEEE. Vol. 5132. SPIE, Gatlinburg, Tenessee, USA, pp. 252–258. Sim, D.-G., Kwon, O.-K., Park, R.-H., 1999. Object matching algorithms using robust Hausdorff distance measures. IEEE Transaction on Image Processing 8 (3), 425–429. 23

Smeulders, A. W. M., Worring, M., Santini, M., Gupta, S., Jain, R., 2000. Content based image retrieval at the end of the early years. IEEE Transcrition on Pattern Analysis and Machine Intelligence 22, 1349–1380. Tak`acs, B., december 1998. Comparing faces using the modified Hausdorff distance. Pattern Recognition 31 (12), 1873–1881. Veltkamp, R., Hagedoorn, M., 1999. State-of-the-art in shape matching. Tech. Rep. UU-CS-1999-27, Utrecht University, the Netherlands. URL citeseer.nj.nec.com/veltkamp99stateart.html Veltkamp, R. C., Hagedoorn, M., 2000. Shape similarity measures, properties, and constructions. Wang, Z., Bovik, A. C., Sheikh, H. R., Simoncelli, E. P., 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13 (4). Zhao, C., Shi, W., Deng, Y., 2004. A new Hausdorff distance for image matching. Pattern Recognition Letters.

24