Nonlinear Dynamic Filtering for Image Compression S. Tramini(1), M. Antonini(1), M. Barlaud(1), G. Aubert(2) (1)
I3S laboratory, CNRS UPRES-A 6070 University of Nice-Sophia Antipolis, Bât.4 SPI Sophia Antipolis, 250 av. A. Einstein F-06560 Valbonne - France
Abstract Despite advances in the domain of source coding, little recent work has been devoted to the problem of joint coding and decoding. In particular, the design of decoders has been, to our knowledge, little investigated. In this paper, we focus on image decoding and propose sufficient conditions on the design of the decoder, involving a priori assumptions on the solution and knowledge of the coder (transformation and quantization). In our approach, we propose an optimization of the reconstruction filters at the decoder to account for effects due to quantization noise. The solution can be viewed as an inverse problem with optimization of the Transform/Quantization/Decoding structure formulated using a variational approach. This leads to nonlinear filtering. Experiments using this nonlinear inverse dynamic filtering demonstrate PSNR gains over standard linear inverse filtering as well as appreciable visual improvements.
1. Introduction Usually, still-image coding is designed as illustrated on the left hand side of figure 1 (referenced as the CODER). First, the data are transformed using a given operator (or projector) R; second the transformed data are quantized using a scalar or vector quantizer; and, finally, the indices representing the best quantization symbols are transmitted via a channel. The decoder generally applies the inverse operations for decoding the quantized image. However, the quantization process entail a loss of information between the quantized data and the input signal. Thus, simply using the dual transformation R * in the decoder as the synthesis operator, does not permit reconstruction of the original input signal because of the introduction of quantization noise. A lot of works have been done on the design of denoising post processing algorithms, which perform after decoding, on decoded images. However, to our knowledge,
(2)
J.A. Dieudonné laboratory, UMR 6621, University of Nice-Sophia Antipolis, Parc Valrose, 06108 Nice Cedex 2- France
these approaches do not take into account the quantization process and accurate noise modeling. In this paper, we break with this usual approach and introduce a new decoding method which modelizes the coder operations: the goal is to remove the artifacts introduced by quantization (such as ringing artifacts). The method performs during decoding and takes into account the nonlinearity of the quantization process. The solution of this problem can be viewed as an inverse problem using a variational approach. In other words, for a given analysis operator (filter) R and quantizer Q, we optimize the distortion at a given rate by computing a nonlinear reconstruction operator (filter) which minimizes a criterion. This criterion involves a model of the quantization operation subject to constraints. The constraints are imposed using a priori assumptions on the solution, i.e., a priori knowledge about the image to be decoded. Section 2 presents an efficient approach for quantization modeling. The criterion we propose to minimize is then developed in Section 3. It contains terms taking into account the observed data and the quantizer model in the transform domain, and terms containing a priori assumptions on the solution. Finally section 4 gives some experimental results.
2. Reconstruction problem at decoding 2.1 Statement of the problem Let us define Ω as the image domain. A gray scale level image is represented (in continuous variables) by the following function:
f : Ω ⊂ ℜ² ℜ ( x, y ) f ( x , y ).
( Ω is an open bounded set in ℜ² )
which at pixel of coordinates (x,y) gives the intensity f(x,y).
CODER f
p
R
Using (1) and (3) it is possible to show that:
DECODER p*
QQ
codeword index
Non linear filter
QG (Rf ) = (G * t )(Rf ) + (G * ε )(Rf ) = α G Rf + rG .
f*
Equation (4) involves that the quantization model can be written as a gain/additive model defined in [6]. Let us write p * = αRf + r (see fig. 2-model (4)), [6] shows that, for a proper choice of the parameter α (gain factor), it is possible to have a noise r uncorrelated to signal Rf .
bo ok cod e
ε fig.1: coding/decoding scheme.
The intensity observation function p * (x, y ) gives the pixel value of a quantized image for (x, y ) ∈ Ω (see figure 1). The output of a quantizer Q can be defined as an additive noise model (see fig. 2-model (1)) and thus, p * = Q (Rf ) = Rf + ε (Rf ) ,
(1)
where the linear operator R is given and the energy of the quantization noise ε is known. This additive noise model is always valid in the sense that (1) always holds [3]. In the rest of the paper, we assume that the operator R corresponds to a transformation with good decorrelation properties [1] and p * results from an optimal quantization of the transformed data [4]. Then, the main problem at the decoder is to reconstruct f using the knowledge of p * and R. Here, we propose to solve problem (1) as an inverse problem by using nonstationary back-projector. The general idea is to find f which minimizes the following criterion, with R, Q and p * fixed:
J 1 ( f ) = Q (Rf ) − p *
(4)
2
(2)
Ω
2.2 Quantizer model
r
p*
Rf
α Γγ
Rf
Model (1)
p* Model (4)
fig.2: quantization noise models.
Let us calculate α . Considering models (1) and (4), we have: E [Rfε ] = E [(p * − ε )ε ] = E [p *ε ]− E [ε 2 ] = −σ ε2 , and next E [Rfε ] = E [Rf (p * − Rf )] = E [Rf (αRf + r − Rf )] 2 + E [Rf r ] . = (α − 1)σ Rf
Then, the value
2 α = 1 − σ ε2 σ Rf
(5)
satisfies both previous equations and implies that E [Rf r ] = 0 . In order to identify the model αRf + r with (4), and assuming that α G must be independent from Rƒ, we 2 and σ 2 constant in Rƒ. Then, the basic idea is impose σ Rf ε to find a regularizing function G which involves the existence of an additive uncorrelated noise rG . 2 Let us define G (x ) = ae − x b , then from (4) it results that
α G = a bπ and rG = (G * ε )(Rf ) .
The obvious drawback of criterion (2) is that the quantization operator is nonlinear and non differentiable. In order to circumvent this problem, we have proposed in a previous work [5] an approach assuming that ε and f are uncorrelated. But, this is a non realistic approach especially at low bit rate coding. Since the drawbacks of the data term is nonlinearity and non differentiability, an other solution consists to regularize it. Theorem 1 (see Annex A) implies that it is possible to regularize Q using a function G ∈ C ck (ℜ) , such that (G ∗ Q )′ = G ′ ∗ Q . Then, we can express Q (Rf ) as
By identification with (5), we can find easily the constants a and b of the regularizing function G:
QG (Rf ) = (G ∗ Q )(Rf )
f * = arg min (J ( f ) )
G
(3)
(
2 −σ 2 a = σ Rf ε
) (σ
Rf
)
2 . π and b = 1 σ Rf
3. Nonlinear dynamic filtering 3.1 Criterion The main idea is to find f which minimizes a criterion J, with R and p * fixed. The estimated image is given by: f ∈L2 (ℜ )
(6)
where J is the sum of a term measuring the faithfulness of the estimate to the data and a regularization term. Here, we propose to minimize the following criterion:
J ( f ) = J 1 ( f ) + C1 ( f ) + C 2 ( f ) + C 3 ( f ).
(7)
Q ′ (Rf )QG (Rf ) R * G + µ 2 2κ ( f ) Rf Rf
ϕ ′( ∇f ) − λ2 div ∇f + η 2ξ ( f ) f = R * (QG′ (Rf ) + µ 2 2κ ( f ))p * 2 ∇f
Equation (2) and (3) imply that the data driven term is: J1 ( f ) =
∫ (QG (Rf ) −
)
2 p * dΩ
Ω
(10) (8)
,
with Neumann boundary condition
which can also be written using model (4) as:
J1( f ) =
2 ∫ (α G Rf + rG − p * ) dΩ .
(9)
Ω
This first term takes into account the observed data and the quantizer model in the transform domain. Furthermore, the spatio-frequency constraints Ci ( f ) contain a p riori assumptions on the solution. We chose the following ones: C1 ( f ) = λ2 ∫ ϕ ( ∇f ) dΩ C2 ( f ) = η 2 ∫ ( f − f ) Ω
i
4 dΩ
(
C 3 ( f ) = µ 2 ∫ Rf − p * − Ω
∫ ( Rf − p
q 2
+ Rf − p * −
)
q 2 2
4 dΩ
)
4 dΩ. Ω In order to avoid smoothing, we introduce a convex potential ϕ-function in the regularization terms so that edges are preserved [2], [5]. The constraints introduced in the criterion are such that: +
*
+
q 2
− Rf + p * −
q 2 2
• the first constraint C1 ( f ) allows noise removal while preserving edges, • the second constraint C 2 ( f ) ensures positivity of the solution, • the last constraint C 3 ( f ) allows to find the solution f such that Rƒ belongs to the "good" quantization interval [p * − q 2 , p * + q 2 [ for Q (Rf ) = p * . It is important to note that, criterion (8) is in general nonconvex. However, assuming that α G and rG are constant in Rƒ, equation (9) can be simplified to
J1( f ) =
q 1 if ( f < 0 ) 1 if Rf − p * > and κ ( f ) = ξ(f )= 2 î 0 elsewhere î0 elsewhere R* stands for the adjoint integral operator, n represents a vector normal to the boundary ∂Ω of Ω, and div stands for the divergence operator defined by: div( f ) = ∑ ∂f ∂xi . 2 σ Rf
Ω
2
∂f = 0 , where: ∂n ∂Ω
2 ∫ (α G Rf − p * ) dΩ ,
Ω
and J(ƒ) becomes convex. Furthermore, the strict convexity of the data driven term depends on the value of α G which must be different from zero (see Annex B).
3.2 Dynamic Filtering If there exists a solution ƒ to this minimization problem, it verifies J ′( f ) = 0 , that is, the following Euler-Lagrange equations:
Assuming that signal and noise energies, and σ ε2 , are constant in Rƒ and known at decoder (they result from the coder or can be estimated at decoder), it results that α G is constant in Rƒ and known. It is then possible to introduce a simplified version of (10) given by the following equations: R * ((α G2 + µ 2 2κ ( f ))Rf )
ϕ ′(∇f ) − λ2 div ∇f + η 2ξ ( f ) f = R * (α G + µ 2 2κ ( f ))p * 2 ∇f (11) In order to solve these nonlinear equations we can use a deterministic minimization algorithm like ARTUR [2].
4. Experimental Results The experimental results are performed on the image Lena of size 512x512 pixels from RPI site. We use as potential ϕ-function the absolute entropy given in [7] and the equations we solve are given by (11). We assume that image and quantization noise energies are known across the subbands at decoder and constant in Rƒ. The quantization noise energy depends on the quantization steps. Results are given figure 3. On figure 3(a) is shown a coded/decoded image using the method proposed in [4] with the biorthogonal filters 9-7. The compression ratio is 80:1. Decoding is ensured using the linear inverse wavelet transform. Figure 3(b) shows result when the proposed nonlinear decoding method is used instead of the linear one. Our new decoding algorithm yields improvements in visual quality as well as a gain of about 1dB in Peak SNR over the linear method. The result is presented with empirically estimated parameters µ and η, and adaptive λ.
5. Conclusion This paper presents a new decoding method which questions the philosophy of perfect-reconstruction filters when quantization noise is introduced. This method takes into account a quantizer model in the transform domain as well as assumptions on the reconstructed image in a spatiofrequency sense. It permits Peak SNR gain over the "classical" decoding methods and visual improvements. This new algorithm can also works for other coders.
Acknowledgements
[3] A. Gersho, R. Gray, "Vector Quantization and Signal Compression", Boston: Kluwer Academic Publishers, 1990. [4] P. Raffy, M. Antonini, M. Barlaud, "Zerotree Edge Adaptive Coder for Low bit Rate Image Transmission", SPIE, Visual Communication and Image Processing, San Jose, USA, 1997. [5] S. Tramini, M. Antonini, M. Barlaud, "Intraframe Image Decoding based on a Nonlinear Variational Approach", submitted to the International Journal of Imaging Systems and Technology, May 1997. [6] P.H. Westerink, "Subband Coding of Images", phD thesis, Delft University of Technology, 1989. [7] M.E. Zervakis, A.K. Katsaggelos, and T.M. Kwon, "A Class of Robust Entropic Functionals for Image Restoration", IEEE Transactions on Image Processing, Vol.4, No.6, 1995.
The authors thank P. Raffy, a phD student from I3S laboratory, for his fruitful collaboration in coding images.
Annex A The differentiability of a convolution product is given by the following theorem. Theorem 1: Let ϖ ∈ L p (ℜ) and ϑ ∈ C ck (ℜ) the set of functions k-continuously differentiable, with compact support. Then,
ϑ ∗ϖ ∈ C k and (ϑ ∗ϖ )′ = ϑ ′ ∗ϖ .
Annex B It is known that the strict convexity of the data driven term implies the uniqueness of the solution. Convexity is ensured if: ∀f , g ∈ L2 (ℜ ), ∀θ ∈ [0,1] , J ( f + θ (g − f )) ≤ J ( f ) + θ J (g ) − J ( f ) . 1 1 1 1
(
Let us define J 1 ( f ) =
∫ (α G Rf −
Ω
J 1 ( f + θ (g − f )) =
)
2 p * dΩ
)
(a) linear inverse filtering - PSNR=30.18 dB
. Then,
2 ∫ (α G Rf + θα G (Rg − Rf ) − p * ) dΩ
Ω
Assuming that α G is constant, it follows: J 1 ( f + θ (g − f )) = J 1 ( f ) + θ (J 1 (g ) − J 1 ( f )) + θ (θ − 1)α G2 ∫ (Rf − Rg )2 dΩ ≤ J 1 ( f ) + θ (J 1 (g ) − J 1 ( f ))
θ ∈ [0,1]
Ω
which implies that the data driven term is convex. However, the strict convexity is ensured if α G ≠ 0 . This is not always verified in our case especially when quantization noise is high (strong quantization step), implying the non-uniqueness of the solution (if it exists).
References [1] M. Antonini, M. Barlaud, P. Mathieu, and I. Daubechies "Image Coding Using Wavelet Transform", IEEE Transaction on Image Processing, Vol.1, No.2, 1992, pp. 205-220. [2] P. Charbonnier, L. Blanc-Féraud, G. Aubert, M. Barlaud, "Deterministic Edge-Preserving Regularization in Computed Imaging", IEEE Transaction on Image Processing, Vol.5, No.12, 1996.
(b) nonlinear inverse filtering - PSNR=31.21 dB fig.3: coded/decoded images with compression ratio 80:1 (0.1 bpp with zero order entropy estimation). The images are sharpened for a better visualisation of artifacts.
