Image restoration: edge preserving - Jean-Francois Giovannelli

Hidden / latent parameters, segmentation, detection,. . . 2 / 32 .... Variable Delta. Potentiel .... Set of auxiliary variables bpq so that: ϕ(δpq) ←→ δ2 pq. ϕ(δ) = inf b.
530KB taille 2 téléchargements 253 vues
Image restoration: edge preserving — Convex penalties —

Jean-Fran¸cois Giovannelli [email protected] Groupe Signal – Image Laboratoire de l’Int´egration du Mat´eriau au Syst`eme Univ. Bordeaux – CNRS – BINP

1 / 32

Topics Image restoration, deconvolution Motivating examples: medical, astrophysical, industrial,. . . Various problems: Fourier synthesis, deconvolution,. . . Missing information: ill-posed character and regularisation

Three types of regularised inversion 1

Quadratic penalties and linear solutions Closed-form expression Computation through FFT Numerical optimisation, gradient algorithm

2

Non-quadratic penalties and edge preservation Half-quadratic approaches, including computation through FFT Numerical optimisation, gradient algorithm

3

Constraints: positivity and support Augmented Lagrangian and ADMM

Bayesian strategy: a few incursions Tuning hyperparameters, instrument parameters,. . . Hidden / latent parameters, segmentation, detection,. . . 2 / 32

Convolution / Deconvolution y = Hx + ε = h ? x + ε ε x

H

+

y

b = Xb(y) x Restoration, deconvolution-denoising General problem: ill-posed inverse problems, i.e., lack of information Methodology: regularisation, i.e., information compensation Specificity of the inversion / reconstruction / restoration methods Trade off and tuning parameters

Limited quality results 3 / 32

Competition: Adequation to data Compare observations y and model output Hx Unknown: x Known: H and y

Comparison experiment – model x

H

Hx y

Quadratic criterion: distance observation – model output JLS (x) = ky − Hxk

2

4 / 32

Competition: Smoothness prior

Data insufficiently informative Account for prior information 1 2

Question of compromise, of competition Specificity of methods

Here: smoothness of images Quadratic penalty of the gray level “gradient” (or other linear combinations) X P(x) = (xp − xq )2 p∼q 2

= kDxk

5 / 32

Quadratic penalty: criterion and solution

Criterion including quadratic penalty 2

2

ky − Hxk + µ kDxk X 2 = ky − Hxk + µ (xp − xq )2

JPLS (x)

=

p∼q

Restored image bPLS x t

t

=

arg min JPLS (x) x t

bPLS (H H + µD D) x

=

H y

bPLS x

=

(H t H + µD t D)−1 H t y

6 / 32

Computational aspects and implementation Various options and many relationships. . . Direct calculus, compact (closed) form, matrix inversion Algorithms for linear system solution Gauss, Gauss-Jordan Substitution Triangularisation,. . .

Special algorithms, especially for 1D case Recursive least squares Kalman smoother or filter (and fast versions,. . . )

Numerical optimisation gradient descent: descent in the direction opposite to the gradient. . . . . . and various modifications

Diagonalization Circulant approximation and diagonalization by FFT 7 / 32

Solution from quadratic penalty

Input

Data

Quadratic penalty

Quadratic penalty

Circulant approx

Non-circulant

8 / 32

Edge preserving approaches

Limited resolution of linear solutions Desirable: less “smoothing” around “discontinuities” Ambivalence: Smoothing (homogeneous regions) Heightening, enhancement, sharpening (discontinuities, edges)

. . . and new compromise, trade off, conciliation

Algorithmic structure Based on the linear solution (based on quadratic criterion) FFT (Wiener-Hunt) Kalman smoother or filter (and its fast versions, factorised,. . . ) Numerical optimisation (gradient,. . . )

9 / 32

Edge preservation and non-quadratic penalties Restored image still defined as the minimiser. . . b = arg min J(x) x x

. . . of a penalised criterion . . . 2

J(x) = ky − Hxk + µ P(x) . . . once again penalising “small variations” but less penalising for “discontinuities” X ϕ(xp − xq ) P(x) = p∼q

ϕ(δ) = δ 2

ϕ(δ) = . . . ?

Ambivalence: new compromise, trade off, conciliation Smoothing (homogeneous regions) Heightening, enhancement, sharpening (discontinuities, edges) 10 / 32

Typical potentials Again ϕ(δ) ∼ δ 2 for small δ Behaviour for large δ 1

Horizontal asymptote

2

Horizontal parabolic behaviour

3

Oblique (slant) asymptote

4

Vertical parabolic behaviour Alpha = 0.50 et Seuil = 1.00

Q

2.5

Huber H and L G and McC

2

B and Z

Potentiel

1.5

1

0.5

0 −5

−4

−3

−2

−1 0 1 Variable Delta

2

3

4

5

11 / 32

Four major types of potentials 1

Horizontal asymptote ϕ(δ) ∼ 1 [Blake and Zisserman (87), Geman and McClure (87)] ϕ(δ) = αs 2 min 1, (δ/s)2



;

ϕ(δ) = αs 2

(δ/s)2 1 + (δ/s)2

2

Horizontal parabolic behaviour ϕ(δ) ∼ log |δ| [Hebert and Leahy (89)]   ϕ(δ) = αs 2 log 1 + (δ/s)2

3

Oblique (slant) asymptote ϕ(δ) ∼ |δ| Huber, hyperbolic, Log hyperbolic-cosine , fair function ( q  δ2 if | δ | 6 s 2 2 ϕ(δ) = α ; ϕ(δ) = 2αs 1 + [δ/s] − 1 2s | δ | − s 2 if | δ | > s

4

Vertical parabolic behaviour ϕ(δ) ∼ δ 2 Wiener-Hunt solution ϕ(δ) = αδ 2 12 / 32

Potentials with oblique asymptote (L2 / L1 ): details Alpha = 0.50 et Seuil = 1.00 2.5

Potentiel

2 1.5 1 0.5 0 −4

−3

−2

−1

0 1 Variable Delta

(

2

3

2

[δ/s] 2 | δ |/s − 1

4

if | δ | 6 s if | δ | > s

Huber :

ϕ(δ)

=

αs 2

Hyperbolic :

ϕ(δ)

=

2αs 2

LogCosh :

ϕ(δ)

=

2αs 2 log cosh (| δ |/s)

FairFunction :

ϕ(δ)

=

2αs 2 [| δ |/s − log (1 + | δ |/s)]

q

 2 1 + [δ/s] − 1

13 / 32

More general non-quadratic penalties (1) Differences, higher order derivatives, generalizations,. . . X P(x) = ϕ(xn+1 − xn ) n

P(x)

=

X

ϕ(xn+1 − 2xn + xn−1 )

n

P(x)

=

X

=

X

ϕ(α xn+1 − xn + α0 xn−1 )

n

P(x)

ϕ(αtn x)

n

Linear combinations (wavelet, other-stuff-in-et,. . . dictionaries,. . . ) ! X X X t P(x) = ϕ(wn x) = ϕ wnm xm n

n

m

Redundant or not Link with Haar wavelet and other 14 / 32

More general non-quadratic penalties (2) 2D aspects: derivative, finite differences, gradient approximations X P(x) = ϕ(xp − xq ) p∼q

=

X

ϕ(xn+1,m − xn,m ) +

n,m

X

ϕ(xn,m+1 − xn,m )

n,m

Notion of neighborhood and Markov field Any highpass filter, contour detector (Prewitt, Sobel,. . . ) Linear combinations: wavelet, contourlet and other-stuff-in-et,. . .

Other possibilities (slightly different) Enforcement towards a known shape X P(x) = ϕ(xp − x¯p ) p

Separable penalty P(x) =

X

ϕ(xp )

p 15 / 32

Penalised least squares solution A reminder of the criterion and the restored image X 2 J(x) = ky − Hxk + µ ϕs (xp − xq ) p∼q

b = arg min J(x) x x

with ϕs one of the mentioned (non-quadratic) potentials and two hyperparameters: µ and s

Non-quadratic criterion Non-linear gradient No closed-form expression

Two questions Practical computation: numerical optimisation algorithm,. . . Minimiser: existence, uniqueness,. . . continuity 16 / 32

Convexity and existence-uniquiness Convex set RN , RN+ , intervals of RN ,. . . Properties: intersection, convex envelope, projection,. . .

Strictly convex criterion, convex criterion, Θ(u) = u 2 , Θ(u) = kuk2 , Θ(u) = | u |, Huber,. . . Properties: sum of convex function, level sets,. . .

Key result Set of minimisers of convex criterion on a convex set is a convex set Strict convexity unique minimiser

Application ϕ convex =⇒ J convex

In the following developments, potential ϕs Huber or hyperbolic: convex (strict) guarantees In addition: non-convex no guarantee (although, sometimes. . . ) 17 / 32

Half-quadratic enchantement (start) Reminder of the criterion 2

J(x) = ky − Hxk + µ

X

ϕ(xp − xq )

p∼q

Minimisation based on quadratic Original idea of [Geman + Yang, 95] Set of auxiliary variables bpq so that: ϕ(δpq ) ←→ δ 2pq

 ϕ(δ) = inf b

 1 (δ − b)2 + ζ(b) 2

With appropriate ζ(b):

Extended criterion 2 ˜ J(x, b) = ky − Hxk + µ

X1 p∼q

2

2

[(xp − xq ) − bpq ] + ζ(bpq )

und nat¨ urlich: ˜ J(x) = inf J(x, b) b

18 / 32

Legendre-Fenchel Transform (LFT) or Convex Conjugate (CC) Definition of TLF or CC Consider f : R −→ R strictly convex once (or twice) differentiable The TLF or CC is the function f ? : R −→ R defined by: f ? (t) =

sup [ xt − f (x) ] x∈R

Remark f ? (0) = sup [ −f (x) ] = − inf [ f (x) ] x∈R

x∈R

∀t, x ∈ R,

xt − f (x) 6 f ? (t)

∀t, x ∈ R,

f ? (t) + f (x) > xt 19 / 32

LFT: some shift and dilatation / contraction properties f ? (t) =

sup [ xt − f (x) ] x∈R

Vertical: shift-dilatation (α ∈ R and β ∈ R?+ ) ( g (x) = α + β f (x) g ? (t) = β f ? (t/β) − α Horizontal: shift (x0 ∈ R) and dilatation (γ ∈ R?+ ) ( ( g (x) = f (x − x0 ) g (x) = f (γx) g ? (t) = f ? (t) − tx0 g ? (t) = f ? (t/γ) Specific case α = 0, β = 1 /

x0 = 0 /

γ=1 20 / 32

LFT: a first example Quadratic case (α ∈ R and β ∈ R?+ ) Let us consider the function 1 f (x) = α + β(x − x0 )2 2 We look for the CC f ? (t) = sup [ xt − f (x) ] x∈R

 Let us denote gt (x) = xt − f (x) = xt − α + β(x − x0 )2 / 2 The derivative reads: gt0 (x) = t − β(x − x0 ) And the second derivative is: gt (x)00 = −β By nullification of gt0 (x): x¯ = x0 + t/β Then by substitution: f ? (t) = gt (¯ x)

  1 1 2 2 f (t) = (t + βx0 ) − α + βx0 2β 2 ?

21 / 32

LFT: a generic result for explicitation (a) A swiss army formula: Legendre formula f ? (t) = sup [ xt − f (x) ] x∈R

Let us denote gt (x) = xt − f (x) The derivative reads: gt0 (x) = t − f 0 (x) And the second derivative is: gt (x)00 = −f 00 (x) By nullification of gt0 (x): t − f 0 (¯ x)

=

0



=

f

0−1

(t) = χ(t)

Then by substitution: f ? (t) = gt (¯ x ) = t¯ x − f (¯ x ) = tχ(t) − f [χ(t)]

22 / 32

LFT: a generic result for explicitation (b) Derivatives Convex conjugate made explicit f ? (t) = tχ(t) − f [χ(t)] with χ = f

0−1

The derivative reads: 0

f ? (t)

=

χ(t) + tχ0 (t) − χ0 (t) f 0 [χ(t)]

=

χ(t)

=

f

0−1

(t)

And the second derivative is: 00

f ? (t) = χ(t)0 =

1 f [χ(t)] 00

>0

hence f ? is convex

23 / 32

LFT: a key result Double conjugate Transformed twice gives back the original f ?? (x) = f (x) f ?? (t) = sup [ xt − f ? (x) ] Let us note ht (x) = xt − f ? (x) and nullify the derivative: 0 x¯

0

= ht0 (¯ x ) = t − f ? (¯ x) = t − f =

−1

χ

0−1

(¯ x ) = t − χ(¯ x)

0

(t) = f (t)

By substitution f ?? (t) = ht (¯ x)

= x¯t − f ? (¯ x) = x¯t − [¯ x χ(¯ x ) − f (χ(¯ x ))] = x¯t − x¯t + f (t) = f (t) 24 / 32

Outcome for LFT: “un th´eor`eme vivant” Definition Let us consider f : R −→ R strictly convex once (or twice) differentiable f ? (t) =

sup [ xt − f (x) ] x∈R

Properties f ? (t) = tχ(t) − f [χ(t)] with χ = f 0

f? =f f

?00

0−1

0−1

=χ 00

(t) = 1 / f [χ(t)] f ? is convex f ?? (x) = f (x)

25 / 32

A theorem in action: half-quadratic (beginning) Problem statement Consider a potential ϕ, convex or not, and look for ζ such that   ϕ(δ) = inf (δ − b)2 /2 + ζ(b) b∈R

Let us define g such that it is strictly convex: g (δ) = δ 2 /2 − ϕ(δ) Consider its CC: g ? (b)

=

sup [ bδ − g (δ) ]   sup ϕ(δ) − (δ − b)2 /2 + b 2 /2

δ∈R

=

δ∈R

Let us set (reason explained on the next slide):   ζ(b) = g ? (b) − b 2 /2 = sup ϕ(δ) − (δ − b)2 /2 δ∈R 26 / 32

A theorem in action: half-quadratic (middle) Take advantage of g = g ?? g (δ)

=

δ 2 /2 − ϕ(δ)

=

ϕ(δ)

= =

g ?? (δ) sup [ bδ − g ? (δ) ] b∈R 2

δ /2 − sup [bδ − g ? (δ)] b∈R   inf (δ − b)2 /2 + ζ(b)

b∈R

The icing on the cake, we have the minimiser:   ¯ 2 /2 + ζ(b) 0 = (b¯ − δ) + ζ 0 (b) ¯ = g ?0 (b) ¯ −δ 0 = (δ − b) then:

0−1 b¯ = g ? (δ) = g 0 (δ) = δ − ϕ0 (δ)

27 / 32

A theorem in action: half-quadratic (ending)

Reminder: original criterion and extended criterion X 2 J(x) = ky − Hxk + µ ϕ(xp − xq ) p∼q 2 ˜ J(x, b) = ky − Hxk + µ

X1 p∼q

2

2

[(xp − xq ) − bpq ] + ζ(bpq )

Algorithmic strategy: alternating minimisation 1

2

˜ e(b) = arg minx J(x, Minimisation w.r.t. x for fixed b: x b) Quadratic problem ˜ Minimisation w.r.t. b for fixed x: e b(x) = arg minb J(x, b) Separated and explicit update

28 / 32

Image update Non-separable but quadratic ˜ J(x)

2

= ky − Hxk + µ

X1 p∼q

2

2

2

[(xp − xq ) − bpq ] 2

= ky − Hxk + µ ¯ kDx − bk Multivariate system of equations e (H t H + µ ¯D t D) x

= H ty + µ ¯ Dt b

Minimiser e x

=

(H t H + µ ¯D t D)−1 (H t y + µ ¯ D t b)

Numerical aspects Descent algorithm (including inner-iterations) Circulant case and FFT (without inner-iterations) 29 / 32

Auxiliary variables update Separable (parallel computation), explicit (no inner-iterations), easy Each b = bpq

↔ δ = δpq = xp − xq

Update: e b = δ − ϕ0 (δ) Huber: e b = δ [1 − 2α min (1; s/δ)]   2α Geman & McClure: e b =δ 1− (s/δ)2 ... Alpha = 0.40 et Seuil = 2.00

Facteur de contraction

1 0.8 0.6 Huber H and L G and McC B and Z

0.4 0.2 0 −10

−5

0 Variable Delta

5

10

30 / 32

Result L2 / L1 (Huber)

Input

Observations

Quadratic penalty

L2-L1 (Huber) penalty

31 / 32

Synthesis and extensions Synthesis Image deconvolution Also available for non-invariant linear direct model And also available for the case of signal. . .

Edge preserving and non-quadratic penalties Gradient of gray levels (and other transform) Convex (and differentiable) case and also some non-convex cases

Numerical computations Half-quadratic approach: separable and FFT Numerical optimisation (gradient,. . . )

Extensions Including constraints

better image resolution (next lecture)

Hyperparameters estimation, instrument parameter estimation,. . . 32 / 32