.
Inverse problems, Deconvolution and Parametric Estimation Ali Mohammad-Djafari Laboratoire des Signaux et Syst`emes, UMR8506 CNRS-SUPELEC-UNIV PARIS SUD 11 SUPELEC, 91192 Gif-sur-Yvette, France http://lss.supelec.free.fr Email:
[email protected] http://djafari.free.fr A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
1/102
Contents ◮ Invese problems examples: ◮
◮ ◮ ◮ ◮
◮
Deconvolution, Image restoration, Image reconstruction, Fourier synthesis, ... Classification of Invesion methods: Analytical, Parametric and Non Parametric algebraic methods Regularization theory Bayesian inference for invese problems Full Bayesian with hyperparameter estimation Two main steps in Bayesian approach: Prior modeling and Bayesian computation Priors which enforce sparsity ◮ ◮ ◮
◮
◮
Heavy tailed: Double Exponential, Generalized Gaussian, ... Mixture models: Mixture of Gaussians, Student-t, ... Gauss-Markov-Potts
Computational tools: MCMC and Variational Bayesian Approximation Some results and applications ◮
X ray Computed Tomography, Microwave and Ultrasound imaging, Sattelite Image separation, Hyperspectral image processing, Spectrometry, CMB, ...
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
2/102
Contents ◮ ◮
◮ ◮ ◮ ◮
◮
Invese problems examples Classification of Invesion methods: Analytical, Parametric and Non Parametric algebraic methods Regularization theory Bayesian inference for invese problems Full Bayesian with hyperparameter estimation Two main steps in Bayesian approach: Prior modeling and Bayesian computation Priors which enforce sparsity ◮ ◮ ◮
◮
◮
Heavy tailed: Double Exponential, Generalized Gaussian, ... Mixture models: Mixture of Gaussians, Student-t, ... Gauss-Markov-Potts
Computational tools: MCMC and Variational Bayesian Approximation Applications: X ray Computed Tomography, Microwave and Ultrasound imaging, ...
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
3/102
Direct and indirect observation ◮
Direct observation of a few quantities are possible: length, time, electrical charge, number of particles
◮
For many others, we only can measure them by transforming them. Example: Thermometer transforms variation of temeprature f to variation of length g.
◮
Relating measurable quantity g to the desired quantity f is called Forward modeling: g = H(f ).
◮
Predicting the measurements g if we knew the desired quantity f and the measurement system is called Forward problem.
◮
Infering on the desired quantity f from the measurement g is called Inverse problem.
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
4/102
Inverse problems : 3 main examples ◮
Example 1: Measuring variation of temperature with a therometer ◮ ◮
◮
Example 2: Seeing outside of a body: Making an image using a camera, a microscope or a telescope ◮ ◮
◮
f (t) variation of temperature over time g(t) variation of length of the liquid in thermometer
f (x, y) real scene g(x, y) observed image
Example 3: Seeing inside of a body: Computed Tomography usng X rays, US, Microwave, etc. ◮ ◮
f (x, y) a section of a real 3D body f (x, y, z) gφ (r) a line of observed radiographe gφ (r, z)
◮
Example 1: Deconvolution
◮
Example 2: Image restoration
◮
Example 3: Image reconstruction
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
5/102
Measuring variation of temperature with a therometer ◮
f (t) variation of temperature over time
◮
g(t) variation of length of the liquid in thermometer
◮
Forward model: Convolution Z g(t) = f (t′ ) h(t − t′ ) dt′ + ǫ(t) h(t): impulse response of the measurement system
◮
Inverse problem: Deconvolution Given the forward model H (impulse response h(t))) and a set of data g(ti ), i = 1, · · · , M find f (t)
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
6/102
Measuring variation of temperature with a therometer Forward model: Convolution Z g(t) = f (t′ ) h(t − t′ ) dt′ + ǫ(t) 0.8
0.8
Thermometer f (t)−→ h(t) −→
0.6
0.4
0.2
0
−0.2
0.6
g(t)
0.4
0.2
0
0
10
20
30
40
50
−0.2
60
0
10
20
t
30
40
50
60
t
Inversion: Deconvolution 0.8
f (t)
g(t)
0.6
0.4
0.2
0
−0.2
0
10
20
30
40
50
60
t
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
7/102
Instrumentation Input f (t)
✲
Impluse response h(t)
Output g(t)
◮
Ideal Instrument
◮
A linear and time invariant instrument is characterized by its impulse response h(t).
◮
Ideal Instrument
◮
Forward problem: f (t), h(t) −→ g(t) = h(t) ∗ f (t) Two linked problems in instrumentation:
◮
◮ ◮
Inversion: Identification:
A. Mohammad-Djafari,
g(t) = f (t)
✲
h(t) = δ(t)
does not exist.
does not exist.
g(t), h(t) −→ f (t) g(t), f (t) −→ h(t)
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
8/102
Ex1: Isolators resistivity against lightning strike An instrument giving the possibility to apply very high voltage to simulate lightning strike 1.2 Signal réel
Tension (MV)
1 0.8 Signal restauré
0.6
Signal issu du diviseur THT
0.4 0.2 0 −0.2 0
0.5
1
1.5
2
Temps (ms)
edf– Les Renardi`eres
A. Mohammad-Djafari,
Real and Estimated
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
9/102
Ex2: Radio-astronomy yb(t)
x(t)
0.9
0.9
0.8
0.8
? =⇒
0.7 0.6 0.5 0.4 0.3 0.2
0.6 0.5 0.4 0.3 0.2
0.1
0.1
0 −0.1 0
0.7
0
100
200
300
400
500
600
700
800
900
1000
−0.1 0
100
200
300
400
500
600
700
800
900
1000
Forward model: ǫ(t)
f (t)
A. Mohammad-Djafari,
✲
h(t)
❄ ✲ +
Inverse problems, Deconvolution and Parametric Estimation,
✲ g(t) = h(t) ∗ f (t) +
MATIS SUPELEC,
10/102
Telecommunication: transmission channel compensation ◮
Data transmission System Mo
Flot d’entre
Codeur Filtre
Dem
Modulateur
Ligne
Dmodulateur
Filtre ´ Egaliseur
Flot Dcision de sortie Dcodage
Canal
◮
Channel Model: convolution + noise ǫ(t)
T Canal h(t)
g(t)
Squence transmise
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
Squence reue
MATIS SUPELEC,
11/102
Seeing outside of a body: Making an image with a camera, a microscope or a telescope ◮
f (x, y) real scene
◮
g(x, y) observed image
◮
Forward model: Convolution ZZ g(x, y) = f (x′ , y ′ ) h(x − x′ , y − y ′ ) dx′ dy ′ + ǫ(x, y) h(x, y): Point Spread Function (PSF) of the imaging system
◮
Inverse problem: Image restoration Given the forward model H (PSF h(x, y))) and a set of data g(xi , yi ), i = 1, · · · , M find f (x, y)
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
12/102
Making an image with an unfocused camera Forward model: 2D Convolution ZZ g(x, y) = f (x′ , y ′ ) h(x − x′ , y − y ′ ) dx′ dy ′ + ǫ(x, y) ǫ(x, y)
f (x, y) ✲ h(x, y)
❄ ✲ + ✲
g(x, y)
Inversion: Image Deconvolution or Restoration ? ⇐=
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
13/102
? =⇒
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
14/102
Seeing inside of a body: Computed Tomography ◮
f (x, y) a section of a real 3D body f (x, y, z)
◮
gφ (r) a line of observed radiographe gφ (r, z)
◮
Forward model: Line integrals or Radon Transform Z gφ (r) = f (x, y) dl + ǫφ (r) L
ZZ r,φ f (x, y) δ(r − x cos φ − y sin φ) dx dy + ǫφ (r) =
◮
Inverse problem: Image reconstruction Given the forward model H (Radon Transform) and a set of data gφi (r), i = 1, · · · , M find f (x, y)
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
15/102
Making an image of the interior of a body ◮
f (x, y) a section of a real 3D body f (x, y, z)
◮
gφ (r) a line of observed radiographe gφ (r, z)
◮
Forward model: Line integrals or Radon Transform Z gφ (r) = f (x, y) dl + ǫφ (r) L
ZZ r,φ f (x, y) δ(r − x cos φ − y sin φ) dx dy + ǫφ (r) =
◮
Inverse problem: Image reconstruction Given the forward model H (Radon Transform) and a set of data gφi (r), i = 1, · · · , M find f (x, y)
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
16/102
2D and 3D Computed Tomography 3D
2D Projections
80
60 f(x,y)
y 40
20
0 x −20
−40
−60
−80 −80
gφ (r1 , r2 ) =
Z
f (x, y, z) dl Lr1 ,r2 ,φ
−60
gφ (r) =
−40
Z
−20
0
20
40
60
80
f (x, y) dl
Lr,φ
Forward probelm: f (x, y) or f (x, y, z) −→ gφ (r) or gφ (r1 , r2 ) Inverse problem: gφ (r) or gφ (r1 , r2 ) −→ f (x, y) or f (x, y, z) A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
17/102
Computed Tomography: Radon Transform
Forward: Inverse:
A. Mohammad-Djafari,
f (x, y) f (x, y)
−→ ←−
g(r, φ) g(r, φ)
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
18/102
Microwave or ultrasound imaging Measurs: diffracted wave by the object g(r i ) Unknown quantity: f (r) = k02 (n2 (r) − 1) Intermediate quantity : φ(r)
y
Object
ZZ
r'
Gm (r i , r ′ )φ(r ′ ) f (r′ ) dr ′ , r i ∈ S D ZZ Go (r, r ′ )φ(r ′ ) f (r ′ ) dr ′ , r ∈ D φ(r) = φ0 (r) + g(r i ) =
Measurement
plane
Incident
plane Wave
x
D
Born approximation (φ(r ′ ) ≃ φ0 (r ′ )) ): ZZ Gm (r i , r ′ )φ0 (r ′ ) f (r ′ ) dr ′ , r i ∈ S g(r i ) = D
z
✲
φ0 Discretization : g = H(f ) g = Gm F φ −→ with F = diag(f ) φ= φ0 + Go F φ H(f ) = Gm F (I − Go F )−1 φ0 A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
r
(φ, f )
MATIS SUPELEC,
g
19/102
Fourier Synthesis in X rayZZ Tomography
f (x, y) δ(r − x cos φ − y sin φ) dx dy
g(r, φ) =
G(Ω, φ) = F (ωx , ωy ) = F (ωx , ωy ) = G(Ω, φ) y ✻ s ■
Z
g(r, φ) exp {−jΩr} dr
ZZ
f (x, y) exp {−jωx x, ωy y} dx dy
for r
■
✲
ωy = Ω sin φ ωy ✻
α
✒
f (x, y) φ
ωx = Ω cos φ and
F (ωx , ωy )
x
φ
Ω
✒
✲
ωx
g(r, φ)–FT–G(Ω, φ)
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
20/102
Fourier Synthesis in X ray tomography G(ωx , ωy ) =
ZZ
f (x, y) exp {−j (ωx x + ωy y)} dx dy
v 50 100
u
? =⇒
150 200 250 300 350 400 450 50
100
150
200
250
300
Forward problem: Given f (x, y) compute G(ωx , ωy ) Inverse problem: Given G(ωx , ωy ) on those lines estimate f (x, y) A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
21/102
Fourier Synthesis in Diffraction tomography ωy
y ψ(r, φ)
^ f (ωx , ω y )
FT 1
2 2 1
f (x, y)
x
-k 0
k0
Incident plane wave Diffracted wave
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
22/102
ωx
Fourier Synthesis in Diffraction tomography G(ωx , ωy ) =
ZZ
f (x, y) exp {−j (ωx x + ωy y)} dx dy
v 50
100
150
u
? =⇒
200
250
300 50
100
150
200
250
300
350
Forward problem: Given f (x, y) compute G(ωx , ωy ) Inverse problem : Given G(ωx , ωy ) on those semi cercles estimate f (x, y) A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
23/102
400
Fourier Synthesis in different imaging systems G(ωx , ωy ) = v
ZZ
f (x, y) exp {−j (ωx x + ωy y)} dx dy v
u
v
u
X ray Tomography
Diffraction
v
u
Eddy current
u
SAR & Radar
Forward problem: Given f (x, y) compute G(ωx , ωy ) Inverse problem : Given G(ωx , ωy ) on those algebraic lines, cercles or curves, estimate f (x, y) A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
24/102
Invers Problems: other examples and applications ◮
X ray, Gamma ray Computed Tomography (CT)
◮
Microwave and ultrasound tomography
◮
Positron emission tomography (PET)
◮
Magnetic resonance imaging (MRI)
◮
Photoacoustic imaging
◮
Radio astronomy
◮
Geophysical imaging
◮
Non Destructive Evaluation (NDE) and Testing (NDT) techniques in industry
◮
Hyperspectral imaging
◮
Earth observation methods (Radar, SAR, IR, ...)
◮
Survey and tracking in security systems
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
25/102
Computed tomography (CT) A Multislice CT Scanner Fan beam X−ray Tomography −1
−0.5
0
0.5
g(si ) = 1
Source positions
−1
−0.5
A. Mohammad-Djafari,
0.5
f (r) dli + ǫ(si )
Li
Detector positions
0
Z
1
Discretization g = Hf + ǫ
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
26/102
Positron emission tomography (PET)
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
27/102
Magnetic resonance imaging (MRI) Nuclear magnetic resonance imaging (NMRI), Para-sagittal MRI of the head
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
28/102
Radio astronomy (interferometry imaging systems) The Very Large Array in New Mexico, an example of a radio telescope.
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
29/102
General inverse problems
H
(
model
g
,
measured data
f
,
unknown quantity
z intermediate quantity
,
ǫ )
=
errors and noise
Particular cases: • Implicite model linking f and z :
g = H1 (f , z) + ǫ H2 (f , z) = 0
• Simple non linear model:
g = H(f ) + ǫ
• Linear model with additive noise:
g = Hf + ǫ
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
30/102
0
Time evolution of liquid-solid fusion interface
Solid :
∂Ts ∂t ∂Tl ∂t
2 2 = αs ∂∂xT2s + ∂∂xT2s 2 2 = αl ∂∂xT2l + ∂∂xT2l
Liquid : Energy ∂Tl s conservation ks ∂T v .~n ∂n − kl ∂n = ρLf ~ ~v : speed of solid-liquid interface ~n : normal vector on the interface Observed quantity : Unknown quantity : Intermediate uknown quantity:
A. Mohammad-Djafari,
T1
L
Ts (x, y, t)
solid phase
solid-liquid interface
......
S(x, y, t)
Tl (x, y, t)
y liquid phase
T0
0 x
∂Tl (x,0,t) ∂t
heat flux on the heating surface ∂Ts (x,0,t) ∂t solid-liquid surface evolution S(x, y, t) temperature field Ts (x, y, t) et Tl (x, y, t)
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
31/102
General formulation of inverse problems ◮
General non linear inverse problems: g(s) = [Hf (r)](s) + ǫ(s),
◮
Linear models: g(s) =
Z
r ∈ R,
s∈S
f (r) h(r, s) dr + ǫ(s)
If h(r, s) = h(r − s) −→ Convolution. ◮
Discrete data:Z g(si ) = h(si , r) f (r) dr + ǫ(si ),
i = 1, · · · , m
◮
Inversion: Given the forward model H and the data g = {g(si ), i = 1, · · · , m)} estimate f (r)
◮
Well-posed and Ill-posed problems (Hadamard): existance, uniqueness and stability
◮
Need for prior information
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
32/102
General formulation of inverse problems F
G
H : F 7→ G
Im(
f g 0
Ker(H) f1
g1
f2
g2
H ∗ : G 7→ F < H ∗ g, f >=< g, Hf > ∀f ∈ F, ∀g ∈ G
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
33/102
Inverse problems scientific communities Two communities working on Inverse problems: ◮
Mathematical departments: Analytical methods: Existance and Uniqueness Differential equations, PDE
◮
Engineering and Computer sciences: Algebraic methods: Discretization, Uniqueness and Stability Integral equations, Discretization using Moments method, Galerkin, ...
Two examples: ◮
Deconvolution: Inverse filtering and Wiener filtering
◮
X ray Computed Tomography: Radon transform: Direct Inversion or Filtered Backprojection methods
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
34/102
Differential Equation, State Space and Input-Output A simple electric system
— R ———–— | f (t) ↑ x(t) ↑ C | ————–—–—
↑ g(t)
∂x(t) + x(t), RC = 1 ∂t Differential Equation Modelling ∂x(t) + x(t) = f (t), x(t) = g(t) ∂t State Space Modelling ∂x(t) = −x(t) + f (t) ∂t g(t) = x(t) f (t) = R i(t) + vc (t) = RC
◮
◮
Input-Output Modelling 1 pX(p) = −X(p) + F (p) → X(p) = p+1 F (p) ∂t = −x(t) + f (t) → g(t) = x(t) = h(t) ∗ f (t), h(t) = exp {−t} g(t) = x(t) ◮
∂x(t)
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
35/102
A more complex electric system example — R1 ————— — R2 ——————— | | f (t) ↑ x2 (t) ↑ C1 x1 (t) ↑ C2 | | —————————————————— f (t) =
∂x2 (t) + x2 (t), ∂t
x2 (t) =
↑ g(t)
∂x1 (t) + x1 (t) ∂t
2
◮ ◮
◮
x1 (t) 1 (t) + 2 ∂x∂t Differential Equation model: ∂ ∂t + x1 (t) = f (t) 2 State space model " # ∂x1 (t) −1 1 x1 (t) 0 ∂t = f (t) + ∂x2 (t) 0 −1 x2 (t) 1 ∂t 1 = x1 (t) g(t) 0
Input-Output Model: g(t) = h(t) ∗ f (t)
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
36/102
Design/Control Inverse problems examples Simple Electrical system: a
◮
x(0) = x0 ,
g(t) = x(t)
Design: θ = a = RC ◮ ◮
◮
∂x(t) + x(t) = f (t), ∂t
Forward: Given θ = a and f (t), t > 0, find x(t), t > 0 Inverse: Given x(t) and f (t) find θ = a
Control: f (t) ◮ ◮
Forward: Given θ = a and f (t), t > 0, find x(t), t > 0 Inverse: Given θ = a and x(t), t > 0, find f (t)
More complex Electrical system: f (t) = b
∂x2 (t) + x2 (t), ∂t
x2 (t) = a
∂x1 (t) + x1 (t), ∂t
g(t) = x1 (t)
θ = (a = R1 C1 , b = R2 C2 ) A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
37/102
Design/Control Inverse problems examples Mass-spring-dashpot system m
◮
∂x(t) ∂ 2 x(t) + k = F (t), +c 2 ∂t ∂t
∂x (0) = v0 ∂t
Design: θ = (m, c, k) ◮
◮
◮
x(0) = x0 ,
Forward: Given θ = (m, c, k), x0 , v0 and F (t), t > 0, find x(t), t > 0 Inverse: Given x(t) for t > 0, v0 , F (t) find θ = (m, c, k)
Control: F (t) ◮
◮
Forward: Given θ = (m, c, k), x0 , v0 and F (t), t > 0, find x(t), t > 0 Inverse: Given θ = (m, c, k), v0 and x(t), t > 0, find F (t)
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
38/102
Input-Output model ◮
Linear Systems ◮
◮
◮
Single Input Single Output (SISO) systems Z y(t) = h(t, τ ) u(τ ) dτ
Multi Input Multi Output (MIMO) systems Z y(t) = H(t, τ ) u(τ ) dτ
Linear Time Invariant System ◮
SISO Convolution y(t) = h(t) ∗ u(t) =
◮
MIMO Convolution y(t) =
◮
Z
Z
h(t − τ ) u(τ ) dτ
H(t − τ ) u(τ ) dτ
. . Impulse response h(t) or H(t) = . hij (t) . .
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
. . .
MATIS SUPELEC,
39/102
State space model: Continuous case Dynamic systems: ◮
Single Input Single Output (SISO) system: x(t) ˙ = A x(t) + B u(t) State equation y(t) = C x(t) + D v(t) Observation equation
◮
Multiple Input Multiple Output (MIMO) system: ˙ x(t) = H x(t) + B u(t) State equation y(t) = C x(t) + D v(t) Observation equation H, B, C and D are the matrices of the system.
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
40/102
Analytical methods (mathematical physics) g(si ) =
Z
h(si , r) f (r) dr + ǫ(si ), i = 1, · · · , m Z g(s) = h(s, r) f (r) dr Z w(s, r) g(s) ds fb(r) =
w(s, r) minimizing a criterion: 2
2 Z
Q(w(s, r)) = g(s) − [H fb(r)](s) = g(s) − [H fb(r)](s) ds 2 2 Z Z b = g(s) − h(s, r) f (r) dr ds 2 Z Z Z h(s, r)w(s, r) g(s) ds dr ds = g(s) −
Trivial solution: A. Mohammad-Djafari,
h(s, r)w(s, r) = δ(r)δ(s)
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
41/102
Analytical methods ◮
Trivial solution: w(s, r) = h−1 (s, r) Example: Fourier Transform: Z g(s) = f (r) exp {−js.r} dr h(s, r) = exp {−js.r} −→ w(s, r) = exp {+js.r} Z ˆ g(s) exp {+js.r} ds f (r) =
◮
Known classical solutions for specific expressions of h(s, r): ◮ ◮
1D cases: 1D Fourier, Hilbert, Weil, Melin, ... 2D cases: 2D Fourier, Radon, ...
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
42/102
Deconvolution: Analytical methods Time domain Forward model: g(t) = h(t) ∗ f (t) + ǫ(t) ǫ(t) ❄
f (t) ✲ h(t) ✲ + ✲
g(t)
Deconvolution: g(t) → w(t) =
1 } IF T { H(ω)
→ fb(t)
Deconvolution: g(t) → W (ω) → fb(t) A. Mohammad-Djafari,
Fourier domain G(ω) = H(ω) F (ω) + E(ω) E(ω) ❄
F (ω)✲ H(ω) ✲ + ✲
G(ω)
Inverse filtering b (ω) G(ω) → 1 → F H(ω)
Wiener filtering
G(ω) →
H ∗ (ω) S (ω) |H(ω)|2 + S ǫ (ω)
Inverse problems, Deconvolution and Parametric Estimation,
f
b (ω) →F
MATIS SUPELEC,
43/102
Deconvolution example 0.6
1.8 1.6
0.5 1.4 0.4
1.2 1
0.3
0.8 0.2 0.6 0.4
0.1
0.2 0 0 −0.2
50
100
150
200
250
300
−0.1
50
100
f (t) 1.8
1.8
1.6
1.6
1.4
1.4
1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0 −0.2
150
200
250
300
200
250
300
g(t)
0
50
100
150
200
Inverse Filtering A. Mohammad-Djafari,
250
300
−0.2
50
100
150
Wiener Filtering
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
44/102
Analytical Inversion methods S•
y ✻
r
✒
f (x, y) φ
✲
x
•D g(r, φ) Radon Transform: ZZ g(r, φ) = f (x, y) δ(r − x cos φ − y sin φ) dx dy D Z π Z +∞ ∂ 1 ∂r g(r, φ) f (x, y) = − 2 dr dφ 2π 0 −∞ (r − x cos φ − y sin φ) A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
45/102
X ray Tomography
y
Z
= g(r, φ) = − ln f (x, y) dl Lr,φ ZZ g(r, φ) = f (x, y) δ(r − x cos φ − y sin φ) dx dy
150
100
I I0
f(x,y)
50
D
0
x
−50
−100
f (x, y)✲
−150
−150
phi
−100
−50
0
50
100
✲g(r, φ)
RT
150
60
p(r,phi)
40 315
IRT ?
270 225
20
0
180
−20
135
=⇒
90 45
−60
0 r
A. Mohammad-Djafari,
−40
−60
−40
−20
Inverse problems, Deconvolution and Parametric Estimation,
0
20
40
60
MATIS SUPELEC,
46/102
Analytical Inversion methods S•
y ✻
r
✒
f (x, y) φ
✲
x
Radon:
ZZ
•D Z g(r, φ) = f (x, y) dl L
f (x, y) δ(r − x cos φ − y sin φ) dx dy Z π Z +∞ ∂ 1 ∂r g(r, φ) f (x, y) = − 2 dr dφ 2π 0 −∞ (r − x cos φ − y sin φ) g(r, φ) =
D
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
47/102
Filtered Backprojection method Z π Z +∞ ∂ 1 ∂r g(r, φ) f (x, y) = − 2 dr dφ 2π 0 −∞ (r − x cos φ − y sin φ) ∂g(r, φ) ∂r Z ∞ 1 g(r, φ) ′ Hilbert TransformH : g1 (r , φ) = dr π 0 (r − r ′ ) Z π 1 g1 (r ′ = x cos φ + y sin φ, φ) dφ Backprojection B : f (x, y) = 2π 0 Derivation D :
g(r, φ) =
f (x, y) = B H D g(r, φ) = B F1−1 |Ω| F1 g(r, φ) • Backprojection of filtered projections: g(r,φ)
−→
FT
F1
A. Mohammad-Djafari,
−→
Filter
|Ω|
−→
IFT
F1−1
g1 (r,φ)
−→
Inverse problems, Deconvolution and Parametric Estimation,
Backprojection B MATIS SUPELEC,
f (x,y)
−→
48/102
Limitations : Limited angle or noisy data
60
60
60
60
40
40
40
40
20
20
20
20
0
0
0
0
−20
−20
−20
−20
−40
−40
−40
−40
−60
−60
−60
−60
−40
−20
0
20
Original
40
60
−60
−40
−20
0
20
40
64 proj.
60
−60
−60 −40
−20
0
20
40
16 proj.
◮
Limited angle or noisy data
◮
Accounting for detector size
◮
Other measurement geometries: fan beam, ...
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
60
−60
−40
−20
0
20
40
8 proj. [0, π/2]
MATIS SUPELEC,
49/102
60
Limitations : Limited angle or noisy data −60
−60
−60
−40
−40
−20
−20
−150
−40 −100
f(x,y)
y
−20 −50
0
x
0
50
20
0
0
20
20
40
40
100
40 150
60
60 −60
−40
−20
0
20
40
60
−150
−100
−50
0
50
100
60 −60
150
−40
−20
0
20
40
60
−60
−60
−40
−40
−20
−20
−60
−40
−20
0
20
40
60
−60
−40
−20
0
20
40
60
−150
−100
f(x,y)
y
−50
x
0
50
0
0
20
20
40
40
100
150
60 −150
Original
A. Mohammad-Djafari,
−100
−50
0
50
Data
100
150
60 −60
−40
−20
0
20
40
60
Backprojection Filtered Backprojection
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
50/102
Parametric methods ◮
◮ ◮
f (r) is described in a parametric form with a very few number b which minimizes a of parameters θ and one searches θ criterion such as: P Least Squares (LS): Q(θ) = i |gi − [H f (θ)]i |2 P Robust criteria : Q(θ) = i φ (|gi − [H f (θ)]i |) with different functions φ (L1 , Hubert, ...).
◮
Likelihood :
L(θ) = − ln p(g|θ)
◮
Penalized likelihood :
L(θ) = − ln p(g|θ) + λΩ(θ)
Examples: ◮
◮
Spectrometry: f (t) modelled as a sum og gaussians P f (t) = K a N (t|µk , vk ) θ = {ak , µk , vk } k k=1
Tomography in CND: f (x, y) is modelled as a superposition of circular or elleiptical discs θ = {ak , µk , rk }
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
51/102
Non parametric Zmethods g(si ) =
◮
h(si , r) f (r) dr + ǫ(si ),
i = 1, · · · , M
f (r) is assumed to be well approximated by N X f (r) ≃ fj bj (r) j=1
with {bj (r)} a basis or any other set of known functions Z N X g(si ) = gi ≃ fj h(si , r) bj (r) dr, i = 1, · · · , M j=1
g = Hf + ǫ with Hij = ◮ ◮
Z
h(si , r) bj (r) dr
H is huge dimensional b = arg min {Q(f )} with LS solution : f f P 2 Q(f ) = i |gi − [Hf ]i | = kg − Hf k2 does not give satisfactory result.
A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
52/102
Algebraic methods: Discretization Hij
y ✻
S•
r
✒
f1 fj
f (x, y)
gi
φ
✲
•D g(r, φ) g(r, φ) =
Z
fN
x
P f b (x, y) j j j 1 if (x, y) ∈ pixel j bj (x, y) = 0 else f (x, y) =
f (x, y) dl
gi =
L
N X
Hij fj + ǫi
j=1
g = Hf + ǫ A. Mohammad-Djafari,
Inverse problems, Deconvolution and Parametric Estimation,
MATIS SUPELEC,
53/102
Inversion: Deterministic methods Data matching ◮
◮
◮
Observation model gi = hi (f ) + ǫi , i = 1, . . . , M −→ g = H(f ) + ǫ Misatch between data and output of the model ∆(g, H(f ))
Examples:
– LS
b = arg min {∆(g, H(f ))} f f
∆(g, H(f )) = kg − H(f )k2 =
X
|gi − hi (f )|2
i
– Lp – KL
p
∆(g, H(f )) = kg − H(f )k = ∆(g, H(f )) =
X i
◮
gi ln
gi hi (f )
X
|gi − hi (f )|p ,
1
