## Probabilistic Methods for Inverse Problems in Computer Vision

Z. Chama : Image recovery from the Fourier phase (Fourier Synthesis). â¢ F. Humblot : Super-resolution from a set of lower resolution images. â¢ N. Bali : Source ...

MVIP05

Teheran University, 23-24 Feb. 2005

Probabilistic Methods for Inverse Problems in Computer Vision

\$

Ali MOHAMMAD-DJAFARI Laboratoire des signaux et syst`emes (UMR 08506 CNRS-Suplec-UPS) Sup´elec, Plateau de Moulon 91192 Gif-sur-Yvette Cedex, FRANCE.

&

[email protected] http://djafari.free.fr http://www.lss.supelec.fr 1

%

MVIP05

Teheran University, 23-24 Feb. 2005

Contents

\$

• Inverses problems in computer vision • Summary of different statistical methods • Basics of Bayesian approach • HMM modeling of images • Examples of applications – Single channel image restoration – Fourier synthesis in optical imaging – Multi channel data fusion and joint segmentation – Video movie segmentation with motion estimation – Blind source (image) separation (BSS) – Hyperspectral image segmentation • Bayesian image processing in wavelet domain • Conclusions

&

2

%

MVIP05

Teheran University, 23-24 Feb. 2005

Inverses problems

\$

• General non linear inverse problem: g(s) = [Hf (r)](s) + ²(s), • Linear model: g(s) =

Z

r ∈ R,

s∈S

f (r)h(r, s) dr + ²(s) R

• Discretized version g = h(f ) + ²

or

g = Hf + ²

where g = {g(s), s ∈ S}, ² = {²(s), s ∈ S}

and

f = {f (r), r ∈ R}

• Multi sensor imaging gi =

N X

Aij Hj fj + ²i ,

i = 1, · · · , M

j=1

where A = {Aij , i = 1, · · · , M, j = 1, · · · , N } is an unknown mixing matrix. & 3

%

MVIP05

Teheran University, 23-24 Feb. 2005

Fourier synthesis in optical imaging Z £ ¤ t g(ω) = f (r) exp −jω r dr + ²(ω)

• Non coherent imaging:

G(g) = |g|

−→

g = h(f ) + ²

• Coherent imaging:

G(g) = g

−→

g = Hf + ²

g = {g(ω), ω ∈ Ω},

² = {²(ω), ω ∈ Ω}

?

20

40

f = {f (r), r ∈ R}

20

40

⇐=

60

60

80

80

100

100

120

120 20

&

and

\$

40

60

80

100

120

20

4

40

60

80

100

120

%

MVIP05

Teheran University, 23-24 Feb. 2005

Single channel image restoration

\$

²(x, y)

? f (x, y) -

h(x, y)

- +

Observation model :

- g(x, y) = h(x, y) ∗ f (x, y) + ²(x, y)

g = Hf + ²

? ⇐=

&

5

%

MVIP05

Teheran University, 23-24 Feb. 2005

Color (Multi-spectral) image deconvolution

\$

²i (x, y) fi (x, y) -

h(x, y)

Observation model :

? - + - gi (x, y) = h(x, y) ∗ fi (x, y) + ²i (x, y)

g i = Hfi + ²i ,

i = 1, 2, 3

? ⇐= &

6

%

MVIP05

Teheran University, 23-24 Feb. 2005

Image fusion and joint segmentation

g1 (r)

\$

Fusion ? =⇒

z

g2 (r) gi (r) = fi (r) + ²i (r), g(r) = {gi (r), i = 1, M }, &

i = 1, · · · , M

g i = {gi (r), r ∈ R},

g(r) = f (r) + ²(r), 7

g = {g i (r), i = 1, M }

g =f +²

%

MVIP05

Teheran University, 23-24 Feb. 2005

\$

Blind image separation and joint segmentation

gi (r) = f1 (r)

j=1

Aij fj (r) + ²i (r)

g(r) = {gi (r), i = 1, M }

? f2 (r)

PN

g1 (r)

g(r) = Af (r) + ²(r), g = {g i (r), i = 1, M }

Separation

g i = {gi (r), r ∈ R},

⇐= g2 (r)

g = Af + ²

f3 (r) &

8

%

MVIP05

Teheran University, 23-24 Feb. 2005

X ray Tomography 3D

\$

2D Projections

80

60 f(x,y)

y 40

20

0 x −20

−40

−60

−80 −80

gφ (r1 , r2 ) =

Z

f (x, y, z) dl Lr1 ,r2 ,φ

−60

−40

gφ (r) =

−20

Z

0

20

40

60

80

f (x, y) dl Lr,φ

Forward problem: f (x, y) or f (x, y, z) −→ gφ (r) or gθ,φ (r1 , r2 ) &

Inverse problem: gφ (r) or gφ,φ (r1 , r2 ) −→ f (x, y) or f (x, y, z) 9

%

MVIP05

Teheran University, 23-24 Feb. 2005

X ray Tomography and Radon Transform

\$

150

100

y

f(x,y)

f (x, y) -

50

0

- g(r, φ)

TR

x

−50

g(r, φ) =

−100

−150

−150

−100

phi

−50

0

50

100

150

g(r, φ) =

ZZ

Z

f (x, y) dl

Lr,φ

f (x, y) δ(r − x cos φ − y sin φ) dx dy D 60

p(r,phi)

40 315

20 270

?

225

0

180

=⇒

135 90

−20

−40

45

−60

0

−60

r

&

10

−40

−20

0

20

40

60

%

MVIP05

Teheran University, 23-24 Feb. 2005

3D Computed Tomography / 3D Shape from shadows

3D Computed Tomography

&

\$

11

%

MVIP05

Teheran University, 23-24 Feb. 2005

3D Computed Tomography / 3D Shape from shadows

3D Computed Tomography

z

z

y

y

x

&

\$

x

12

%

MVIP05

Teheran University, 23-24 Feb. 2005

Deterministic methods

\$

Data matching • Observation model

i = 1, . . . , M −→ g = H(f )²

gi = hi (f ) + ²i ,

• Misatch between data and output of the model ∆(g, H(f )) • Examples: – LS

b = arg min {∆(g, H(f ))} f f

2

∆(g, H(f )) = kg − H(f )k =

X

|gi − hi (f )|

2

i

– Lp – KL

p

∆(g, H(f )) = kg − H(f )k = ∆(g, H(f )) =

X i

X

p

|gi − hi (f )| ,

1