ADAPTIVE IMAGE PROCESSING A Computational Intelligence Perspective

©2002 CRC Press LLC

IMAGE PROCESSING SERIES Series Editor: Phillip A. Laplante, Pennsylvania Institute of Technology

Published Titles Image and Video Compression for Multimedia Engineering Yun Q. Shi and Huiyang Sun Multimedia Image and Video Processing Ling Guan, S.Y. Kung, and Jan Larsen Shape Analysis and Classification: Theory and Practice Luciano da Fontoura Costa and Roberto Marcondes Cesar Jr. Adaptive Image Processing: A Computational Intelligence Perspective Stuart William Perry, Hau-San Wong, and Ling Guan

Forthcoming Titles Software Engineering for Image Processing Systems Phillip A. Laplante Digital Data-Hiding and Watermarking with Applications Rajarathnam Chandramouli

©2002 CRC Press LLC

ADAPTIVE IMAGE PROCESSING A Computational Intelligence Perspective Stuart William Perry Hau-San Wong Ling Guan

CRC PR E S S Boca Raton London New York Washington, D.C.

©2002 CRC Press LLC

Library of Congress Cataloging-in-Publication Data Perry, Stuart William. Adaptive image processing : a computational intelligence perspective / Stuart William Perry, Hau-San Wong, Ling Guan. p. cm. -- (Image processing series) Includes bibliographical references and index. ISBN 0-8493-0283-8 (alk. paper) 1. Image processing. 2. Computational intelligence. I. Wong, Hau-San. II. Guan, Ling. III. Title. IV. Series. TA1637 .P46 2001 006.6--dc21

2001052637

This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage or retrieval system, without prior permission in writing from the publisher. The consent of CRC Press LLC does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific permission must be obtained in writing from CRC Press LLC for such copying. Direct all inquiries to CRC Press LLC, 2000 N.W. Corporate Blvd., Boca Raton, Florida 33431. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation, without intent to infringe.

Visit the CRC Press Web site at www.crcpress.com © 2002 by CRC Press LLC No claim to original U.S. Government works International Standard Book Number 0-8493-0283-8 Library of Congress Card Number 2001052637 Printed in the United States of America 1 2 3 4 5 6 7 8 9 0 Printed on acid-free paper

Preface In this book, we consider the application of computational intelligence techniques to the problem of adaptive image processing. In adaptive image processing, it is usually required to identify each image pixel with a particular feature type (eg., smooth regions, edges, textures, etc.) for separate processing, which constitutes a segmentation problem. We will then establish image models to describe the desired appearance of the respective feature types or, in other words, to characterize each feature type. Finally, we modify the pixel values in such a way that the appearance of the processed features conforms more closely with that specified by the feature models, where the degree of discrepancy is usually measured in terms of a cost function. In other words, we are searching for a set of parameters which minimize this function, i.e., an optimization problem. To satisfy the above requirements, we consider the application of computational intelligence (CI) techniques to this class of problems. Here we will adopt a specific definition of CI, which includes neural network techniques (NN), fuzzy set theory (FS) and evolutionary computation (EC). A distinguishing characteristic of these algorithms is that they are either biologically inspired, as in the cases of NN and EC, or are attempts to mimic how human beings perceive everyday concepts, as in FS. The choice of these algorithms is due to the direct correspondence between some of the above requirements with the particular capabilities of specific CI approaches. For example, segmentation can be performed by using NN. In addition, for the purpose of optimization, we can embed the image model parameters as adjustable network weights to be optimized through the network’s dynamic action. In contrast, the main role of fuzzy set theory is to address the requirement of characterization, i.e., the specification of human visual preferences, which are usually expressed in fuzzy languages, in the form of multiple fuzzy sets over the domain of pixel value configurations, and the role of EC is mainly in addressing difficult optimization problems. In this book, we have illustrated the essential aspects of the adaptive image processing problem in terms of two applications: adaptive image restoration and the adaptive characterization of edges in feature detection applications. These two problems are representative of the general adaptive image processing paradigm in that the three requirements of segmentation, characterization and optimization are present. This book consists of eight chapters. The first chapter provides material of an ©2002 CRC Press LLC

introductory nature to describe the basic concepts and current state of the art in the field of computational intelligence for image restoration and edge detection. Chapter 2 gives a mathematical description of the restoration problem from the neural network perspective, and describes current algorithms based on this method. Chapter 3 extends the algorithm presented in Chapter 2 to implement adaptive constraint restoration methods for both spatially invariant and spatially variant degradations. Chapter 4 utilizes a perceptually motivated image error measure to introduce novel restoration algorithms. Chapter 5 examines how model-based neural networks can be used to solve image restoration problems. Chapter 6 examines image restoration algorithms making use of the principles of evolutionary computation. Chapter 7 examines the difficult concept of image restoration when insufficient knowledge of the degrading function is available. Finally, Chapter 8 examines the subject of edge detection and characterization using model-based neural networks.

Acknowledgments We are grateful to our colleagues in the Signal and Multimedia Processing Lab in the University of Sydney, especially Matthew Kyan and Kim Hui Yap, for their contributions and helpful comments during the preparation of this book. Our special thanks to Prof. Terry Caelli for the many stimulating exchanges which eventually led to the work in Chapter 8. We would also like to thank Nora Konopka of CRC Press for her advice and assistance. Finally, we are grateful to our families for their patience and support while we worked on the book.

©2002 CRC Press LLC

Contents 1 Introduction 1.1 The Importance of Vision 1.2 Adaptive Image Processing 1.3 The Three Main Image Feature Classes 1.4 Difficulties in Adaptive Image Processing System Design 1.5 Computational Intelligence Techniques 1.6 Scope of the Book 1.6.1 Image Restoration 1.6.2 Edge Characterization and Detection 1.7 Contributions of the Current Work 1.7.1 Application of Neural Networks for Image Restoration 1.7.2 Application of Neural Networks to Edge Characterization 1.7.3 Application of Fuzzy Set Theory to Adaptive Regularization 1.7.4 Application of Evolutionary Programming to Adaptive Regularization and Blind Deconvolution 1.8 Overview of This Book 2 Fundamentals of Neural Network Image Restoration 2.1 Image Distortions 2.2 Image Restoration 2.2.1 Degradation Measure 2.2.2 Neural Network Restoration 2.3 Neural Network Restoration Algorithms in the Literature 2.4 An Improved Algorith 2.5 Analysis 2.6 Implementation Considerations 2.7 A Numerical Study of the Algorithms 2.7.1 Setup 2.7.2 Efficiency 2.7.3 An Application Example 2.8 Summary ©2002 CRC Press LLC

3 Spatially Adaptive Image Restoration 3.1 Introduction 3.2 Dealing with Spatially Variant Distortion 3.3 Adaptive Constraint Extension of the Penalty Function Model 3.3.1 Motivation 3.3.2 The Gradient-Based Method 3.3.3 Local Statistics Analysis 3.4 Correcting Spatially Variant Distortion Using Adaptive Constraints 3.5 Semi-Blind Restoration Using Adaptive Constraints 3.6 Implementation Considerations 3.7 More Numerical Examples 3.7.1 Efficiency 3.7.2 An Application Example 3.8 Adaptive Constraint Extension of the Lagrange Model 3.8.1 Problem Formulation 3.8.2 Problem Solution 3.8.3 Conditions for KKT Theory to Hold 3.8.4 Discussion 3.9 Summary 4 Perceptually Motivated Image Restoration 4.1 Introduction 4.2 Motivation 4.3 A LVMSE-Based Cost Function 4.3.1 The Extended Algorithm for the LVMSE-Modified Cost Function 4.3.2 Analysis 4.4 A Log LVMSE-Based Cost Function 4.4.1 The Extended Algorithm for the Log LVR-Modified Cost Function 4.4.2 Analysis 4.5 Implementation Considerations 4.6 Numerical Examples 4.6.1 Color Image Restoration 4.6.2 Grayscale Image Restoration 4.6.3 LSMSE of Different Algorithms 4.6.4 Robustness Evaluation 4.7 Summary 5 Model-Based Adaptive Image Restoration 5.1 Model-Based Neural Network 5.1.1 Weight-Parameterized Model-Based Neuron 5.2 Hierarchical Neural Network Architecture 5.3 Model-Based Neural Network with Hierarchical Architecture 5.4 HMBNN for Adaptive Image Processing 5.5 The Hopfield Neural Network Model for Image Restoration ©2002 CRC Press LLC

5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15

5.16 5.17

Adaptive Regularization: An Alternative Formulation 5.6.1 Correspondence with the General HMBNN Architecture Regional Training Set Definition Determination of the Image Partition The Edge-Texture Characterization Measure The ETC Fuzzy HMBNN for Adaptive Regularization Theory of Fuzzy Sets Edge-Texture Fuzzy Model Based on ETC Measure Architecture of the Fuzzy HMBNN 5.13.1 Correspondence with the General HMBNN Architecture Estimation of the Desired Network Output Fuzzy Prediction of Desired Gray Level Value 5.15.1 Definition of the Fuzzy Estimator Membership Function 5.15.2 Fuzzy Inference Procedure for Predicted Gray Level Value 5.15.3 Defuzzification of the Fuzzy Set G 5.15.4 Regularization Parameter Update 5.15.5 Update of the Estimator Fuzzy Set Width Parameters Experimental Results Summary

6 Adaptive Regularization Using Evolutionary Computation 6.1 Introduction 6.2 Introduction to Evolutionary Computation 6.2.1 Genetic Algorithm 6.2.2 Evolutionary Strategy 6.2.3 Evolutionary Programming 6.3 The ETC-pdf Image Model 6.4 Adaptive Regularization Using Evolutionary Programming 6.4.1 Competition under Approximate Fitness Criterion 6.4.2 Choice of Optimal Regularization Strategy 6.5 Experimental Results 6.6 Other Evolutionary Approaches for Image Restoration 6.6.1 Hierarchical Cluster Model 6.6.2 Image Segmentation and Cluster Formation 6.6.3 Evolutionary Strategy Optimization 6.7 Summary 7 Blind Image Deconvolution 7.1 Introduction 7.1.1 Computational Reinforced Learning 7.1.2 Blur Identification by Recursive Soft Decision 7.2 Computational Reinforced Learning 7.2.1 Formulation of Blind Image Deconvolution as an Evolutionary Strategy 7.2.2 Knowledge-Based Reinforced Mutation 7.2.3 Perception-Based Image Restoration ©2002 CRC Press LLC

7.3

7.4

7.5

7.2.4 Recombination Based on Niche-Space Residency 7.2.5 Performance Evaluation and Selection Soft-Decision Method 7.3.1 Recursive Subspace Optimization 7.3.2 Hierarchical Neural Network for Image Restoration 7.3.3 Soft Parametric Blur Estimator 7.3.4 Blur Identification by Conjugate Gradient Optimization 7.3.5 Blur Compensation Simulation Examples 7.4.1 Identification of 2D Gaussian Blur 7.4.2 Identification of 2D Gaussian Blur from Degraded Image with Additive Noise 7.4.3 Identification of 2D Uniform Blur by CRL 7.4.4 Identification of Non-standard Blur by RSD Conclusions

8 Edge Detection Using Model-Based Neural Networks 8.1 Introduction 8.2 MBNN Model for Edge Characterization 8.2.1 Input-Parameterized Model-Based Neuron 8.2.2 Determination of Sub-Network Output 8.2.3 Edge Characterization and Detection 8.3 Network Architecture 8.3.1 Characterization of Edge Information 8.3.2 Sub-Network Ur 8.3.3 Neuron Vrs in Sub-Network Ur 8.3.4 Dynamic Tracking Neuron Vd 8.3.5 Binary Edge Configuration 8.3.6 Correspondence with the General HMBNN Architecture 8.4 Training Stage 8.4.1 Determination of pr∗ for Sub-Network Ur∗ 8.4.2 Determination of wr∗ s∗ for Neuron Vr∗ s∗ 8.4.3 Acquisition of Valid Edge Configurations 8.5 Recognition Stage 8.5.1 Identification of Primary Edge Points 8.5.2 Identification of Secondary Edge Points 8.6 Experimental Results 8.7 Summary References

©2002 CRC Press LLC