WAM 12.3 INTERPOLATION USING NEURAL NETWORK FOR DIGITAL STILL CAMERAS Jinwook Go and Chulhee Lee Dept. Electrical and Computer Engineering, Yonsei University, KOREA Email: chulhee@ yonsei .ac .kr

ABSTRACT In this paper; we propose a color interpolation method for single-chip digital cameras using artijicial neural networks. Single-chip digital cameras use a color filter array and an interpolation method in order to produce high quality color images from sparsely sampled images. Experiments show that the proposed interpolation algorithm based on neural networks provides a better performance than the conventional interpolation algorithms.

Fig. 1. Bayer CFA pattem. We used the extended delta-bar-delta algorithm to train the neural network for a fast convergence. In order to obtain training and test data, we selected 37 true three-color images. And we used pixels adjacent to each missing pixel in the color filter array as an input vector for the neural network and the corresponding true value as a target value. Fig 2 shows how the pixels for training vectors and target values were chosen. For the red and blue signals, the value of a missing pixel (marked by “r”) is first interpolated from 16 neighbor pixels (Fig. 2a). After this interpolation, the grid pattem of the red and blue signals will be identical with that of the green color. Then, the value of a missing pixel in Fig. 2b is interpolated from 16 neighbor pixels (Fig. 2b). At the boundaries of images, pixels are mirrored to generate an input vector. We used different neural networks for each color, resulting in five neural networks: two networks for the red and blue signals in the grid pattem of Fig. 2a and three networks for the red, green, and blue signals in the grid pattern of Fig. 2b.

INTRODUCTION Digital still cameras have been widely used as input device for multimedia systems and are expected to replace traditional cameras that use films in many areas. For a practical purpose, however, most digital cameras use a single-chip CCD (chargecoupled device) to reduce cost and size. In the single-chip CCD camera, color images are encoded by color filter array (CFA) pattern and a subsequent interpolation process produces fullcolor images. Many researchers have proposed a variety of interpolation methods to increase the resolution of color signals [l-21. And existing interpolation methods are usually derived from general numerical models. In this paper, we present a color interpolation method based on neural networks for a single-chip CCD. Neural networks have been successfully applied in the area of signal and image processing. In particular, due to the characteristics that neural networks can describe systems without using explicit physical and mathematical modeling, they have been used for the calibration of CCD cameras [3], recognition of moving objects in image captured by the CCD camera [4], and event identification in CCD-based detectors. Image interpolation using neural networks has some advantages that the conventional interpolation algorithms lack. For instance, by training selectively for severely degraded parts such as edges, we can expect that the interpolation algorithm based on neural networks provide a better image quality.

Fig. 2. Pixels chosen for training vectors and target values. (a) grid pattern for red and blue signals. (b) grid pattem for green, red, and blue signals.

EXPERIMENTS AND RESULTS G

R

G

R

The training sets were extracted from 20 color images and the resulting neural networks were tested on 17 color images that

G

176 0-7803-6301-9/00$10.000 2000 E E E

were not used for training. Tables I and I1 show the results of the bilinear interpolation, the cubic interpolation, and the proposed interpolation based on neural networks. As can be seen, the proposed interpolation method outperforms the conventional interpolation methods. Although the performance of the proposed algorithm is slightly worse than that of the cubic spline interpolation in the red channel in the test images, the overall performance of the proposed method is better than those of the conventional methods. Fig. 3 shows interpolation results for 256x256 House color image in the green channel. As can be seen, the proposed algorithm provides a sharper image, particularly around edges. Table I. Performance comparison (20 training images) (PSNR in dB). Bilinear Cubic spline Neural network

Red 30.986 31.628 31.734

Green 34.034 35.001 36.652

Blue 29.45 29.849 30.052

Table 11. Performance comparison (17 test images) in dB). Bilinear Cubic spline Neural network

Red 31.706 31.998 31.962

Green 33.931 34.463 34.895

Blue 32.18 32.326 32.451

Average 31.49 32.16 32.813 (PSNR

Fig. 3. Performance comparison (greed channel). (a) interpolated image (bilinear), (b) interpolated image (neural networks), (c) difference image (bilinear), (d) difference image (neural networks).

Average 32.605 32.929 33.103

CONCLUSION In t h s paper, we proposed an interpolation method for singlechip CCD with CFA using artificial neural networks. Neural networks are trained to interpolate missing pixels from surrounding pixels. The attractive ability of neural networks to learn and generalize may provide a better image quality for edges and comers. Although further experiments are needed, the interpolation algorithm based on neural networks shows a great potential that it may outperform the conventional interpolation algorithms or supplement them in a significant way, providing high quality color images from the images obtained using a single-chip CCD with CFA.

REFERENCE T. Kuno, H. Sugiura and M. Matoba, “New Interpolation Method Using Discriminated Color Correlation for Digital Still Cameras,” IEEE Trans. Consumer Electronics, pp. 259-267, 1999 J. E. Adams Jr., “Design of Practical Color Filter Array Interpolation Algorithms for Digital Cameras, Part 2,” 1998 International Conference on Image Processing, pp. 488492, 1998 J. Wen and G. Schweitzer, “Hybrid Calibration of CCD Cameras Using Artificial Neural Nets,” 1991 IJCNN, pp. 337-342, 1991 S. Sugiyama, “Recognition of movement in neural networks,” 1994 IEEE International Conference on Systems, Man, and Cybemetics, pp. 2373-2377, 1994

177