'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
BAYESIAN SEGMENTATION OF HYPERSPECTRAL IMAGES
$
Adel Mohammadpour, Nadia Bali and Ali Mohammad-Djafari
&
Laboratoire des Signaux et Syst`emes CNRS-ESE-UPS Sup´elec, Plateau de Moulon 91192 Gif-sur-Yvette, FRANCE.
[email protected] [email protected] [email protected] 1
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
Contents
$
• Introduction to hyperspectral images • Spectral classification methods (Spatial distribution of spectra is neglected) • Spatial classification methods (Spectral shapes of the voxels are neglected) • Modeling for segmentation methods which account for both spatial and spectral structures of the data • Bayesian approach and general MCMC Gibbs sampling • Comparison of the proposed approach with classical methods • Proposed algorithm • Simulation results
• Conclusions &
2
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
Introduction to hyperspectral images
&
3
21-23 Novembre, 2006
$
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
Classification, segmentation and data reduction
$
• Hyperspectral data : g(ω, r) – A set of spectra : gr (ω) – A set of images : gω (r) • Redundancy due to spectral and spatial structure • Main objectif 1 : Find the type of materials in a given position (labeling) – Classification – Segmentation • Main objectif 2 : Data reduction and compression – ACP, ACI and using classification and segmentation for better compression.
– Proposing a method which does data reduction and segmentation at the same time. & 4
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
Generating simulated data
21-23 Novembre, 2006
$
Synthetic data 1:
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
5
&
10
15
20
25
30
35
4 classes, 32 images (64x64) 5
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
Generating simulated data
21-23 Novembre, 2006
$
Synthetic data 2:
7000
6000
5000
4000
3000
2000
1000
0
0
20
&
40
60
80
100
120
8 classes, 112 images (128x128) 6
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
Spectral classification methods
$
Classification
• Each spectral line is considered as a point in a vectorial space • Different classification methods are used: K-means, mixture of gaussians, ...
• Spatial distribution of the spectra is neglected & 7
%
'
A. Mohammad-Djafari & al.
&
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
1
$
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
32 images
0
5
Original
10
15
20
25
30
35
30
35
4 classes 1.4
1.2
1
0.8
0.6
0.4
0.2
0
32 images
Estimated 8
0
5
10
15
20
25
4 classes
%
'
A. Mohammad-Djafari & al.
&
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
7000
$
6000
5000
4000
3000
2000
1000
0
112 images
0
20
Original
40
60
80
100
120
100
120
8 classes 6000
5000
4000
3000
2000
1000
0
112 images
Estimated 9
0
20
40
60
80
8 classes
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
Spatial classification methods
$
• Each image is considered as a point in a vectorial space • Different classification methods are used: K-means, mixture of gaussians, ...
• Spectral structures of the image pixels are neglected & 10
%
'
A. Mohammad-Djafari & al.
&
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
1
$
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
32 images
0
5
Original
10
15
20
25
30
35
30
35
4 classes 1.4
1.2
1
0.8
0.6
0.4
0.2
0
8 images
Estimated 11
0
5
10
15
20
25
4 classes
%
'
A. Mohammad-Djafari & al.
&
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
7000
$
6000
5000
4000
3000
2000
1000
0
112 images
0
20
Original
40
60
80
100
120
100
120
8 classes 7000
6000
5000
4000
3000
2000
1000
0
8 images
Estimated 12
0
20
40
60
80
8 classes
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
Modelling for accounting for both spatial and spectral structures 50
50
50
50
50
50
100
100
100
100
100
100
150
150
150
150
150
150
200
200
200
200
200
200
250
250
250
250
250
250
300
300
300
300
300
300
350
350
350
350
350
350
400
400
400
400
400
400
450
450
450
450
450
450
500
500
500
500
500
100
200
300
400
500
600
100
200
300
400
500
gi (r) = fi (r) + ǫi (r),
600
100
200
300
400
500
600
100
i = 1, · · · , M
200
300
400
500
600
500 100
200
300
400
500
600
100
200
300
400
500
600
Segmentation:
g(r) = {gi (r), i = 1, M }
– Hidden variables rep. regions
g(r) = f (r) + ǫ(r)
z(r) = k, k = 1, · · · , K
gi = {gi (r), r ∈ R}
Rk = {r : z(r) = k}, R = ∪k Rk
g = {gi (r), i = 1, M }
– Homogeneity in regions:
g =f +ǫ &
$
p(fi (r)|z(r) = k) = N (mi k , σi2 k ) 13
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
Modelling for accounting for both spatial and spectral structures g (r) = f (r) + ǫ (r), i = 1, · · · , M i i i p(fi (r)|z(r) = k) = N (mi k , σ 2 ) ik
$
k=4
k=3
k=2
k=1
Prior hypothesis about fi (r) :
k=4 k=3
• Pixels values of fi (r) in different regions of an image are independent. They may share however the same parameters θik = (mi k , σi2 k )
k=2
k=1
k=4 k=2
k=3
k=1
• For pixels values in a given region of an image, two possibilities: – i.i.d.:
p(fj (r)|zj (r) = k) = N (mj k , σj2 k ) p(fj (r), r ∈ Rj k ) = N (mj k 1, σj2 k I)
– Markovien:
&
p(fj (r), r ∈ Rj k ) = N (mj k 1, Σj k ) 14
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
Modelling for accounting for both spatial and spectral structures g (r) = f (r) + ǫ (r), i = 1, · · · , M i i i p(fi (r)|z(r) = k) = N (mi k , σ 2 )
$
ik
k=4
Pixels along the channels represent spectra • A Markovien model for fi (r): p(fi (r)|z(r) = k, fi−1 (r)) =
k=3 k=2 k=1
k=4
N (ψik fi−1 (r), σi2 k )
k=3 k=2
k=1
fik (r) = ψik fi−1,k (r) + ηik
∼ AR(1) k=4
• A Markovien model for the means:
k=2
k=3
k=1
p(fi (r)|zi (r) = k) = N (mi k , σi2 k ) mi k = φk mi−1,k + ηk
&
∼ AR(1) 15
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
Modeling the labels p(fj (r)|zj (r) = k) =
N (mj k , σj2 k )
−→ p(fj (r)) =
X
$
P (zj (r) = k) N (mj k , σj2 k )
k
• Independent Gaussian Mixture model (IGM), where zj = {zj (r), r ∈ R} are assumed to be independent and X Y P (zj (r) = k) = pk , with pk = 1 and p(zj ) = pk k
k
• Contextual Gaussian Mixture model (CGM): zj Markovien X X p(zj ) ∝ exp α δ(zj (r) − zj (s)) r∈R s∈V(r)
which is the Potts Markov random feild (PMRF). The parameter α controls the mean value of the regions’ sizes. & 16
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
Expressions of likelihood, prior and posterior laws
$
gi (r) = fi (r) + ǫi (r) i = 1, · · · , M
−→
g =f +ǫ
θ 1 = {σǫ 2i , i = 1, · · · , M },
−→
Σǫ i = σǫ 2i I
gi = fi + ǫ i , • Likelihood:
p(g|f , θ 1 ) =
M Y
p(g|f , Σǫ i ) =
i=1
• HMM for the images:
M Y
N (f , Σǫ i )
i=1
θ 2 = {(mi k , σi2 k ), j = 1, · · · , M }
– Markovian model for fi |z: p(f |z, θ2 ) = p(f1 |z, mi k , σi2 k )
M Y
p(fi |fi−1 , z, mi k , σi2 k )
i=2
– Markovian model for mi k :
&
p(f |z, θ 2 ) =
M Y
p(fi |z, mi k , σi2 k )
i=1
but
mi k = φk mi−1,k + ηik ∼ AR(1) 17
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
• PMRF for the labels:
p(z) ∝ exp α
X X r∈R s∈V(r)
$
δ(z(r) − z(s))
• Conjugate priors for the hyperparameters θ = (θ 1 , θ 2 ): θ = {{σǫ 2i , i = 1, · · · , M }, {(mi k , σi2 k ), i = 1, · · · , M, k = 1, · · · , K}} p(σǫ i )
= IG(αi0 , βi0 )
p(mi k ) = N (φk mi−1 k , σi2 k 0 ) p(σi2 k )
= IG(αi0 , βi0 )
p(Σi k )
= IW(αi0 , Λi0 )
• Joint posterior law of f , z and θ
&
p(f , z, θ|g) ∝ p(g|f , θ 1 ) p(f |z, θ 2 ) p(z|α) p(θ) 18
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
$
General MCMC sampling scheme p(f , z, θ|g)
∝ p(g|f , θ 1 ) p(f |z, θ2 ) p(z|θ2 ) Q ∝ ( i p(gi |fi , θ1 )) p(f |z, θ2 ) p(z|α) p(θ)
θ = {σ 2 , i = 1, · · · , M } 1 ǫi θ = (θ 1 , θ 2 ) θ 2 = {(mi k , σ 2 ), i = 1, · · · , M, k = 1, · · · , K}} ik
Gibbs sampling:
• Generate samples (f , z, θ)(1) , · · · , (f , z, θ)(N ) using ∼ p(f |g, z, θ) ∝ p(g|f , z, θ) p(f |z)
–
f
–
z ∼ p(z|g, f , θ) ∝ p(g|f , z, θ) p(f |z) p(z|α)
–
θ ∼ p(θ|g, f , z)
• Compute any statistics such as mean, median, variance, ...
&
19
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
Comparison with classical methods
Proposed Observation model: accounts for noise Observation model: no noise gi (r) = fi (r) + ǫi (r), r ∈ R, gi (r) = fi (r) Hidden Markov Model: Ind. Gaussian Mixture model: 2 p(fi (r)|z(r) = k) = N (mi k , σi k ) p(fi (r)|z(r) = k) = N (mi k , σi2 k ) i h P P p(z) ∝ exp α r∈R s∈V(r) δ(z(r) − z(s)) z(r)⊥z(s), s 6= r, Q Q p(f |z) = i p(fi |z) p(f |z) = i p(fi |z) No correlation in diff. channels: Accounts for correlation in diff. channels: mi k ⊥mj k , i 6= j mi k = φk mi−1 k + ηik
&
$
Classical
20
%
'
A. Mohammad-Djafari & al.
Data
Journ´ ees ACI2M, Bordeaux
Original
1
Kmeans1
21-23 Novembre, 2006
Kmeans2
$
BFJsegment
1.4
1.4
1.4
1.2
1.2
1.2
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
5
10
&
15
20
25
30
35
0
0
5
10
15
21
20
25
30
35
0
0
5
10
15
20
25
30
35
0
0
5
10
15
20
25
30
35
%
'
A. Mohammad-Djafari & al.
Data
Journ´ ees ACI2M, Bordeaux
Original
1
21-23 Novembre, 2006
Kmeans1
Kmeans2
1.2
BFJsegment
0.5
1
0.9
0.9 1
0.4
0.8
0.3
0.6
0.2
0.4
0.1
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.2
0
0
−0.1
−0.2
−0.2
0.3
0.2
0.2
0.1
0
$
0.1
0
10
20
&
30
40
50
60
0
10
20
30
22
40
50
60
0
10
20
30
40
50
60
0
0
10
20
30
40
50
60
%
'
A. Mohammad-Djafari & al.
Data
Journ´ ees ACI2M, Bordeaux
Original
6000
Kmeans1
Kmeans2
7000
7000
6000
5000
5000
5000
4000
4000 4000
4000
3000
3000 3000
3000
2000
2000 2000
1000
0
$
BFJsegment
6000
6000
5000
21-23 Novembre, 2006
2000
1000
1000
0
10
20
&
30
40
50
60
0
0
10
20
30
23
40
50
60
0
1000
0
10
20
30
40
50
60
0
0
10
20
30
40
50
60
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
$
DataOriginalKmeans1Kmeans2BFJsegment
&
24
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
$
DataOriginalKmeans1Kmeans2BFJsegment
&
25
%
'
A. Mohammad-Djafari & al.
Journ´ ees ACI2M, Bordeaux
21-23 Novembre, 2006
$
DataOriginalKmeans1Kmeans2BFJsegment
&
26
%
