Stripe: Image Feature Based on a New Grid Method and Its

users' demand in real time, especially for online users. So we developed a new image feature based on a new region division method of images, and named it.
221KB taille 2 téléchargements 151 vues
Stripe: Image Feature Based on a New Grid Method and Its Application in ImageCLEF Bo Qiu, Daniel Racoceanu, Chang Sheng Xu, and Qi Tian Institute for Infocomm Research, A-star, 21, Heng Mui Keng Terrace, 119613, Singapore {qiubo, visdaniel, xucs, tian}@i2r.a-star.edu.sg

Abstract. There have been many features developed for images, like Blob, image patches, Gabor filters, etc. But generally the calculation cost is too high. When facing a large image database, their responding speed can hardly satisfy users’ demand in real time, especially for online users. So we developed a new image feature based on a new region division method of images, and named it as ‘stripe’. As proved by the applications in ImageCLEF’s medical subtasks, stripe is much faster at the calculation speed compared with other features. And its influence to the system performance is also interesting: a little higher than the best result in ImageCLEF 2004 medical retrieval task (Mean Average Precision — MAP: 44.95% vs. 44.69%), which uses Gabor filters; and much better than Blob and low-resolution map in ImageCLEF 2006 medical annotation task (classification correctness rate: 75.5% vs. 58.5% & 75.1%). Keywords: Stripe, image feature, image retrieval, image annotation.

1 Introduction In the medical field, with the increasingly important needs from clinicians, researchers, and patients, interactive or automatic image retrieval and annotation based on content are being paid more and more attention. An open medical image collection has been published by ImageCLEF since 2003 and the realistic scenarios are used to test performance of different retrieval and annotation systems [1]. ImageCLEF1 is a part of Cross language Evaluation Forum (CLEF) and includes two parts: Adhoc (for historic photographs) and ImageCLEFmed (for medical image retrieval and annotation). In 2005, 13 groups from the world joined its medical image retrieval task, and 12 groups joined the annotation task. Whatever the methods they used, the common basic problem is how to choose and calculate image features. Some features (like Blob, image patches, etc.) are powerful but the computational cost is too high, so as to be difficult to be used for online purpose when facing an open dataset. Some others are a little weak for all kinds of medical images. To explore more efficient features will be a permanent objective for the researchers in this field. 1

http://ir.shef.ac.uk/imageclef/

H.T. Ng et al. (Eds.): AIRS 2006, LNCS 4182, pp. 489 – 496, 2006. © Springer-Verlag Berlin Heidelberg 2006

490

B. Qiu et al.

In this section firstly we will review the features reported in the past literature in ImageCLEF. Then we will give our solution for a fast and efficient image feature. 1.1 Image Features Used in ImageCLEF In [2], based on some small regions/blocks like 5x5 or 3x3, HSV histogram (16x4x4), Canny edges (8 directions) and texture (first 5 DCT coefficients, each quantized into 3 values) are used. Then the three components are connected to a feature vector. In [3], all the images are resized into 256x256 at first. Then they are divided into smaller blocks with the size 8x8. Average gray values of the blocks are taken to form the image feature vector. In [4] and [5], RGB color quantization is processed and the image pixels are classified as border or interior according to 4-neighbor connection in the 4x4x4=64 color space. Then color histogram is computed based on border pixels or interior pixels and used as the feature vector. Besides of this, global texture histogram is computed from local edge pattern descriptors, using a Sobel filtering, and edge map of a whole image is divided into 4 equal squares and projected to vertical and horizontal axes to form the feature vector. In [6], 32x32 down-scaled low resolution map (DLSM) of each image is used as layout feature. Further more, color histogram, global texture descriptors, and invariant feature histogram are also used, especially for Tamura features, which include coarseness, contrast, directionality, line-likeness, regularity, and roughness. In [7], Gabor features and Tamura features are used. In [8], there are following features: global texture features based on co-occurrence matrix, histogram of edge direction based on Canny edge detection, global shape features based on 7 invariant moments. In [9], Tamura features, in which only 3 features are chosen: coarseness, contrast, directionality, are quantized into 6x8x8=384 bins based on the normalized image 256x256, ignoring the aspect ratio. In [10], 4 features are used: 8x8 DLSM, global gray histogram by dividing the image into 9 sections, coherent moment based on pixel clustering and 8-neighbor connection, color histogram. In [11], after partitioning the images successively into four equally sized regions (4 times), color histogram and Gabor filters are used in the different scales, and global and local features are both considered. In [12], 16x16 DLSM is used, with regional feature Blob, and 3 texture features: contrast, anisotropy, and polarity. Image patches [13] and random sub-windows [14] are also used in ImageCLEF and they had very good results in the annotation subtask. Both of them directly use pixel gray values of the small blocks. For image patches, salient points are extracted at first in the work of [13]. According to the last report of ImageCLEF’s committee, in visual-only medical image retrieval task, Blob features reached the highest position in 2005; in annotation task, image patches got the No.1 position in 2005. But whether Blob or image patches, their computation cost is very high. This forces researchers to keep on searching more efficient features. We put forward the ‘stripe’.

Stripe: Image Feature Based on a New Grid Method and Its Application

491

1.2 Stripe: An Efficient Feature Based on a New Grid Method of Images By dividing an image into different kinds of grids, we can extract features based on the cells of grids. To make it easily handled, histogram of each cell is calculated and we call the cells ‘stripes’, as shown in Fig.1. For one image, all its stripes’ histogram queues in order and forms a feature vector. R1 R2

R1 R2

Rn Rn

(a)

(b)

(d)

(e)

(c)

Fig. 1. Stripes’ kinds: (a) horizontal stripes (R1, R2, … , Rn); (b) vertical stripes; (c) squarecirque stripes (R1, R2, … , Rn); (d) left-tilt stripes; (e) right-tilt stripes

Compared with other image features, stripes have following characteristics: ¾ ¾ ¾ ¾

Faster at the calculation speed; More adaptive to different sizes of images without resizing them; More suitable for ‘vague’ matching where for medical images, they are so different even in a same class, and precise shape matching is impossible but statistical features can match well Without losing the information of spatial positions

2 Stripe Generation Stripe is a kind of local feature of images. To generate it, firstly we divide an image into regions; secondly we choose suitable features to represent the regions. Considering the division of an image as shown in Fig.1, we can define different ‘stripes’: Horizontal, vertical, square-cirque, left-tilt, right-tilt.

492

B. Qiu et al.

2.1 Horizontal and Vertical Stripes In this case, we use straight lines equally to divide an image into some horizontal and vertical regions. For each division, the widths of all the regions are probably equal while sometimes except the last region, because of different image sizes in an image database. But for all the regions, the bins of histogram are the same, starting from 2 at least. The number of bins will be decided by experiments and training dataset. It should be bigger if the image gray values vary a lot in the regions. For example, for Fig.1 (a), there are n stripes: R1~Rn. For each stripe the histogram has m bins. When using percentage denotation (P1, P2, … , Pm), only the first (m-1) bins are needed because the total sum should be 1. Then the feature vector will be (P1,1…Pn,m-1) which is derived from the following queue: (R1 P1,1 P1,2 … P1,m-1

R2 P2,1 P2,2 … P2,m-1

… …

Rn) Pn,1 Pn,2 … Pn,m-1

2.2 Square-Cirque Stripes As shown in Fig.1 (c), the stripes start from the center/core of the image. When users give the number of stripes in an image, the widths of the stripes are correspondingly defined. The size of the core stripe (R1) will be adjusted to satisfy the rule that all the other stripes have the same width in the horizontal and vertical directions separately. This kind of stripe is designed to simulate some disc-like medical images like skull, heart, etc. It is useful to distinguish skull from chest x-ray.

(a)

(b)

Fig. 2. Medical images: (a) skull; (b) chest

On this sense, square-cirque stripes can provide some shape information. How to generate the square-cirque feature vector is similar to section 2.1. 2.3 Left- and Right-Tilt Stripes Tilt stripes are designed to track the rotation of an image. Fig.3 gives two rotation examples. Statistics following the direction of tilt angles will make the feature vector more meaningful. And it can decrease the influence of the white border regions. Though there are different rotation angles for different images, only ±45° lines are chosen to form the tilt stripes in our program. Surely for more precisely calculation, more angles should be considered. But this will increase the computation cost. Why choosing ±45° lines? Besides the advantage of geometrical calculation without the need to judge by pixels, another one is that if rotate the image in a 90° angle, the lefttilt program can be reused to calculate right-tilt stripes.

Stripe: Image Feature Based on a New Grid Method and Its Application

(a)

493

(b)

Fig. 3. Rotation vs. normal: (a) hand; (b) finger

How to generate the tilt feature vector is similar to section 2.1 when the stripes queue is up-to-bottom. 2.4 Stripe Feature Vector for an Image For an image, its feature vector is formed when connected all the stripe vectors together. For example, given an image w x h (w: width, h: height), in our program users just need to define two parameters: Ns—the number of stripes for each kind, and Nb—the number of histogram bins. Then the length of the last feature vector of the image will be Ns x (Nb-1) x 4.5. Ns should be even because the square-cirque stripes are symmetric to axes. Then it is clear that the image sizes won’t influence the feature vector’s length, so that it is no need to resize all the images to a fixed width and height.

3 Application of Stripes in ImageCLEF Tasks We apply the stripe feature method into some tasks of ImageCLEF. The first experiment is based on the medical image retrieval task in 2004; the second experiment is based on the medical annotation task in 2006. The effect of stripes is compared with other features and the system performance is evaluated correspondingly. 3.1 ImageCLEF 2004: Medical Image Retrieval Task According to [15], near to 9000 images are available in this task, and 26 topics are included. In [16], it says in all published results based on the same frame, the best result is from GIFT, with its MAP 44.69%. By Blob features and PCA as described in [17], we reached a higher MAP 45.35%. • Precision Using the same dataset and PCA method in [17], when STRIPE features take place of Blob features, the best MAP is 44.95%, which is 0.58% higher than GIFT, but 0.88% less than Blob. • Time Though Blob’s result is slightly better, its computation cost is very high. In our case to process 8717 images the total time is longer than one week, with 13 computers working simultaneously. The worst PC has a 1.1G CPU and 128M RAM. However in the case of STRIPES, when using the ‘worst’ PC (1.1G CPU + 128M RAM), the longest process costs 35 minutes with the parameter Ns=8, Nb=30.

494

B. Qiu et al.

1 0.9

0.9 0.8

0.7

0.7

0.6

0.6 MAP

Average Precision

0.8

1 5 10 20 100

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

Blob

GIFT

GIFT Blob stripe

0 0

stripe

5

10

methods

(a)

15 Topic

20

25

30

(b) 1 GIFT Blob stripe

0.9 0.8

Precision

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

0.2

0.4

0.6

0.8

1

Recall

(c) Fig. 4. Comparison: (a) MAPs at recall points (5, 10, 20, 100) for 3 methods; (b) MAPs of 26 topics; (c) PR-curve

Obviously for future online application, STRIPE has great advantage at the speed compared with Blob. In Fig.4 the comparison in MAP and PR-curve is shown. As we can see, in Fig.4(a), at the first 5 and 10 recall points, MAP of STRIPE is higher than the others’; in Fig.4(b) and (c), STRIPE performs better than GIFT and close to Blob. DLSM 16x16

75.1%

1

0.5

0

0

20

40

60 Blob

80

60 STRIPE

80

60

80

100

120

100

120

100

120

58.5%

1

0.5

0

0

20

40

75.5%

1

0.5

0

0

20

40

Fig.5. Correctness rates of 3 methods: DLSM, Blob, STRIPE, for all 116 classes

Stripe: Image Feature Based on a New Grid Method and Its Application

495

3.2 ImageCLEF 2006: Medical Image Annotation Task2 Slightly different from the task in 2005, which is described in [12], in this year the annotation task includes 116 classes, and 1000 images are provided as development set, whose ground truth is known. The training set’s volume is the same as before, including 9000 images, as well as for testing set, including 1000 images. Our experiment is based on the training set and development set. The STRIPE features, Blob features, and DLSM features (16x16) are used separately. And we use SVM as the classifier. At last the correctness rates are 75.5%, 58.5%, and 75.1%. In Fig.5 the correctness for each class is shown.

4 Conclusions and Future Work Stripe features have great advantage at computation speed, which is very interesting in online applications. With the comparison in two tasks from ImageCLEF, the reliability of the stripe features is also proved. It can enhance the system’s performance to a higher degree. For future work, more kinds of stripes should be mined, like more rotation angles for tilt stripes. And more features can be considered besides histogram.

References 1. P. Clough, H. Müller, T. Deselaers , M. Grubinger, T. Lehmann, J. Jensen, W. Hersh, The CLEF 2005 Cross-Language Image Retrieval Track, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 2. Gareth J. F. Jones, K. McDonald, Dublin City University at CLEF 2005: Experiments with the ImageCLEF St Andrew's Collection, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 3. Yih-Chen Chang, Wen-Cheng Lin, Hsin-Hsi Chen, Combining Text and Image Queries at ImageCLEF2005, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 4. R. Besançon, C. Millet, Merging Results from Different Media: Lic2m Experiments at ImageCLEF 2005, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 5. R. O. Stehling, M. A. Nascimento, and A. X. Falcao, A compact and efficient image retrieval approach based on border/interior pixel classification, CIKM '02: Proceedings of the eleventh international conference on information and knowledge management, McLean, Virginia, USA, 2002 6. T. Deselaers, T. Weyand, D. Keysers, W. Macherey, H. Ney, FIRE in ImageCLEF 2005: Combining Content-based Image Retrieval with Textual Information Retrieval, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 7. D. Petkova, L. Ballesteros, Categorizing and Annotating Medical Images by Retrieving Terms Relevant to Visual Features, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 8. Md. M. Rahman, B. C. Desai, P. Bhattacharya, Supervised Machine Learning based Medical Image Annotation and Retrieval, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 2

http://www-i6.informatik.rwth-aachen.de/~deselaers/imageclef06/medicalaat.html

496

B. Qiu et al.

9. M. O Güld, C. Thies, B. Fischer, T. M. Lehmann, Combining Global features for Contentbased Retrieval of Medical Images, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 10. Pei-Cheng Cheng, Been-Chian Chien, Hao-Ren Ke, Wei-Pang Yang, [email protected] ImageCLEFmed 2005: Medical Image Retrieval Task, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 11. H. Müller, A. Geissbühler, J. Marty, C. Lovis, P. Ruch, Using medGIFT and easyIR for the ImageCLEF 2005 Evaluation Tasks, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 12. Bo Qiu, Qi Tian, Chang Sheng Xu, Report on the Annotation Task in ImageCLEFmed 2005, Cross Language Evaluation Forum 2005 Workshop, Vienna, Austria, Sep. 2005 13. T. Deselaers, D. Keysers, H. Ney, Improving a Discriminative Approach to Object Recognition using Image Patches, Proc. DAGM 2005, LNCS 3663, pp. 326-333, Vienna, Austria, Springer 14. R. Maree, P. Geurts, J. Piater, L. Wehenkel, Biomedical Image Classification with Random Subwindows and Decision Trees, ICCV2005 workshop-CVBIA 15. W. X., Bo Qiu, Qi Tian, Chang Sheng Xu, et al, Content-based Medical Retrieval Using Dynamically Optimized Regional Features, ICIP2005, Genova, Italy, Sep.11-14, 2005 16. H. Müller, A. Geissbühler and P. Ruch, “Report on the CLEF Experiment: Combining Image and Multilingual Search for Medical Image Retrieval”, 2005 imageCLEF proceedings, Springer Lecture Notes, to appear. 17. P. Clough, M. Sanderson and H. Müller, “The CLEF Cross Language Image Retrieval Track (ImageCLEF) 2004”, the 8th in the series of European Digital Library Conferences, ECDL 2004, September, Bath, UK, 2004