Structured Models for Image Segmentation Aurelien Lucchi Wednesday February 13th, 2013 Joint work with Yunpeng Li, Kevin Smith, Raphael Sznitman, Radhakrishna Achanta, Bohumil Maco, Graham Knott, Pascal Fua. 1
Image Segmentation ●
Goal: partition an image into meaningful regions with respect to a particular application.
2
Image Segmentation ●
Goal: partition an image into meaningful regions with respect to a particular application.
3
Understanding the Brain
4
Electron Microscopy Data Human brain contains ~100 billion (1011) neurons and 100 trillion (1014) synapses. ●
5 × 5 × 5 μm section taken from the CA1 hippocampus, corresponding to a 1024 × 1024 × 1000 volume (N ≈ 109 total voxels)
Outline 1. CRF for Image segmentation 2. Maximum Margin Training for CRFs Cutting Plane (Structured SVM) 3. Maximum Margin Training of CRFs - Online Subgradient Descent (SGD) 4. SLIC superpixels/supervoxels
9
1. CRF for Image Segmentation
10
Structured Prediction ●
Non structured output ● ●
●
inputs X can be any kind of objects output y is a real number
Prediction of complex outputs
● ●
Structured output y is complex (images, text, audio...) Ad hoc definition of structured data: data that consists of several parts, and not only the parts themselves contain information, but also the way in which the parts belong together 11
Slide courtesy: Christoph Lampert
Structured Prediction for Images
Histograms, Filter responses, ... 12
CRF for Image Segmentation
Maximum-a-posteriori (MAP) solution :
Data (D)
Unary likelihood
Pair-wise Terms
MAP Solution
13 Boykov and Jolly [ICCV 2001], Blake et al. [ECCV 2004] Slide courtesy : Pushmeet Kohli
CRF for Image Segmentation
Maximum-a-posteriori (MAP) solution :
Data (D)
Unary likelihood
Pair-wise Terms
MAP Solution
14 Boykov and Jolly [ICCV 2001], Blake et al. [ECCV 2004] Slide courtesy : Pushmeet Kohli
CRF for Image Segmentation
Pair-wise Terms Favors the same label for neighboring nodes.
15
CRF for Image Segmentation
Maximum-a-posteriori (MAP) solution :
Data (D)
Unary likelihood
Pair-wise Terms
MAP Solution
16 Boykov and Jolly [ICCV 2001], Blake et al. [ECCV 2004] Slide courtesy : Pushmeet Kohli
Energy Minimization ●
MAP inference for discrete graphical models:
●
Dynamic programming –
●
Graph-cuts (Boykov, 2001) –
●
Exact on non loopy graphs Optimal solution if energy function is submodular
Belief propagation (Pearl, 1982) –
No theoretical guarantees on loopy graphs but seems to work well in practice.
●
Mean field (root in statistical physics)
●
... 17
Training a Structured Model ●
First rewrite the energy function as:
Log-linear model
where w is a vector of parameters to be learned from training data and is a joint feature map to map the input-output pair into a linear feature space. 18
Training a Structured Model ●
Energy function is parametrized by vector w
+
-1
1
-1
?
?
1
?
?
19
Training a Structured Model ●
Energy function is parametrized by vector w
+
-1
1
-1
0
1
1
1
0
Low energy
High energy 20
Training a Structured Model ●
Maximum likelihood
●
Pseudo-likelihood
●
Variational approximation
●
Contrastive divergence
●
Maximum-margin framework
21
2. Maximum Margin Training for CRFs Cutting Plane (Structured SVM)
22
Structured SVM
●
Given a set of N training examples with ground truth labels , we can write ≡ Energy for the correct labeling at least as low as energy of any incorrect 23 labeling.
Nov 19, 2012 - inputs X can be any kind of objects. â output y is a real number. â Prediction of complex outputs. â. Structured output y is complex (images, ...
Using the prior data model (5), the prior source model (6) and the prior .... (MFA) then becomes a natural tool for obtaining approximate solutions with lower ...
most independent and piecewise homogeneous. To insure these .... In this algorithm, â¼ represents either argmax or generate sample using or still compute the ...
This paper describes a clustering algorithm for segmenting the color images of natural scenes. The proposed method operates in the 1976 CIE CL*, a*, ...
In this algorithm, we have made a sequence of the priors, posteriors and they then ... For a probability model determination, we can suppose to have mixture of ...
capabilities of this technique are shown in. Section 3. 2. Segmentation based on ... gradient. The following figure (figure 2) illustrates our algorithm: Figure 2.
therefore uses a hybrid co-operation approach and is almost automatic and unsupervised. The performance of ... the application of different algorithms to the same image ..... reduced the research space of the closest point from p in M. (M ¼ C ...
and can be solved using pixel-wise classification and specific classifiers. .... L151 is true if A is greater than 151, and false otherwise. More generally, we can ...
ulations. J. Math. Biol. 43 (2001), 545â560. [4] R. Levins, Some demographic and genetic consequences of environ- mental heterogeneity for biological control.
processing operations such as image registration and archiving. We will use the ... The model-based clustering tree algorithm operates re- cursively on the .... Applications ..... Answers via model-based cluster analysis" , The Computer J our- ... [2
metrical. The 128 Ã 128 observation images of Fig. 2-b have been decomposed up to the 3rd wavelet scale. In Fig. 3, the evolution of a normalized L1 norm er-.
Abstract: Significant progress in control design has been achieved by the ... feature is the result of the idea of the diligent use of nonsmooth criteria of the form.
the vector of exogenous inputs or a test signal, y â Rp2 the vector of measurements and z â Rp1 the controlled or .... the definition fâ(κ) := sup Ïâ[0,â] ..... all t ⥠0, is taken over f1(κ, t) = z(κ, t) â zmax(t), f2(κ, t) = zmin
Feb 12, 2010 - which can also be obtained as the solution that minimizes: J1(f) = g â Hf2 + ...... Cybernetics, Part B, 36(4):849â862, 2006. [44] K. Friston, J.
Broadcasting and television are now entering the era of High Definition (HD) â a transition as ... Several European countries have also used the introduction.
live wire, live lane, and a three-dimensional (3-D) extension of the live-wire method. In this paper, we introduce an ultra-fast live-wire method, referred to as live ...
Kluwer Academic Publ., Santa Fe, NM, K. Hanson edition, 1995. [2] T. Bass, âIntrusion detection systems and multisensor data fusion,â in Comm. of the. ACM, vol.
cantilevers are indeed affected by ferroelectric nonlinearities such as hysteresis ... when the memory and computational power resources are limited, such in ...