Reconstruction of Consistent 3D CAD Models from Point Cloud Data Using a priori CAD Models
Aurélien BEY
(1,2) ,
Raphaëlle CHAINE (2) , Raphaël MARC THIBAULT (1,3) , Samir AKKOUCHE (2) (1) EDF R&D (2) LIRIS - Université Lyon 1 (3) LPPA - Collège De France
August 31th , 2011
(1) ,
Guillaume
Overview
1
Introduction Problem statement
2
Bayesian formulation A probabilistic approach Data fitting term Consistency term Term of compliance with the prior data
3
Solution computation Greedy minimization Solutions space sampler
4
Conclusion Results Prospects
Overview
1
Introduction Problem statement
2
Bayesian formulation A probabilistic approach Data fitting term Consistency term Term of compliance with the prior data
3
Solution computation Greedy minimization Solutions space sampler
4
Conclusion Results Prospects
Introduction
Bayesian formulation
Solution computation
Conclusion
Problem statement : 3D model reconstruction 3D Point Cloud P (LIDAR data)
1 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Problem statement : 3D model reconstruction 3D Point Cloud P (LIDAR data) RANSAC ? Hough ?
Reconstruction
3D CAD model X : assembling of primitive shapes. Cylinders Planes Tori Cones Ellipsoids
1 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Problem statement : 3D model reconstruction 3D Point Cloud P (LIDAR data) 3D CAD model X : assembling of primitive shapes. Cylinders Reconstruction
Planes Tori Cones Ellipsoids
A priori knowledge : expert knowledge (shapes sizes, connectivity, . . . ) existing 3D model M0 (as built 3D model of a twin power plant, 3D model from technical drawings, ...)
1 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Problem statement : 3D model reconstruction 3D Point Cloud P (LIDAR data) 3D CAD model X : assembling of primitive shapes. Cylinders Reconstruction
Planes Tori Cones Ellipsoids
A priori knowledge : expert knowledge (shapes sizes, connectivity, . . . ) existing 3D model M0 (as built 3D model of a twin power plant, 3D model from technical drawings, ...)
Work focused on cylinders. Each cylinder is defined by : its radius [R] its position [p] its axis direction [a] (its length [L]) a
R
p L
1 / 10
Overview
1
Introduction Problem statement
2
Bayesian formulation A probabilistic approach Data fitting term Consistency term Term of compliance with the prior data
3
Solution computation Greedy minimization Solutions space sampler
4
Conclusion Results Prospects
Introduction
Bayesian formulation
Solution computation
Conclusion
Bayesian formulation Maximum a posteriori Search for the most probable configuration X with respect to some a posteriori probability : a posteriori probability
observed data likelihood
z }| { π(X |P, M0 )
z }| { PD (P|X )
∝
a priori probability
z }| { × PP (X |M0 )
The a priori term embeds two independent kinds of knowledge : PP (X |M0 ) = PT (X ) × PG (X |M0 )
Configuration X Consistency expectations Is it relevant ?
Point cloud P Does P fit to X ?
?
Evaluation
?
Shapes well connected ? Prior knowledge
Is X compliant with the a priori expectations ?
X matches M0 ?
A priori 3D model M0
2 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Bayesian formulation Maximum a posteriori Search for the most probable configuration X with respect to some a posteriori probability : a posteriori probability
observed data likelihood
z }| { π(X |P, M0 )
z }| { PD (P|X )
∝
a priori probability
z }| { × PP (X |M0 )
The a priori term embeds two independent kinds of knowledge : PP (X |M0 ) = PT (X ) × PG (X |M0 )
Configuration X Consistency expectations π PT Point cloud P Does P fit to X ?
PD
Evaluation
PP
Prior knowledge
Is X compliant with the a priori expectations ?
PG
A priori 3D model M0
2 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Overall energy
Energy minimization By using the probabilities P(X , P, M0 ) = e −E (X ,P,M0 ) the probability maximization problem is turned into an energy minimization problem : argmax π(X |P, M0 )
=
argmin − log (π(X |P, M0 ))
=
argmin − log (PD (P|X ) × PT (X ) × PG (X |M0 ))
=
argmin ED (P, X ) + ET (X ) + EG (X , M0 ) | {z } X
X
X X
=E (X ,P,M0 )
Global energy Sum of the energies brought by each point in P and each cylinder in X
3 / 10
Overview
1
Introduction Problem statement
2
Bayesian formulation A probabilistic approach Data fitting term Consistency term Term of compliance with the prior data
3
Solution computation Greedy minimization Solutions space sampler
4
Conclusion Results Prospects
Introduction
Bayesian formulation
Solution computation
Conclusion
Data fitting term Data fitting energy Each point in P brings an energy depending on its distance to the shapes in X . 3 types of points, based on a noise estimate : Type Distance to the closest shape Contribution to ED ≤ > and ≤ 3 > 3
Inliers Medium range neighbors Outliers
< 0 (decrease ED ) ω > 0 (increase ED ) 0 (neutral)
∆ED
ω
3
0
3 distance
-1
4 / 10
Overview
1
Introduction Problem statement
2
Bayesian formulation A probabilistic approach Data fitting term Consistency term Term of compliance with the prior data
3
Solution computation Greedy minimization Solutions space sampler
4
Conclusion Results Prospects
Introduction
Bayesian formulation
Solution computation
Conclusion
Consistency term Topological consistency energy connected cylinders axes should intersect angles between connected cylinders should be either flat or right connected cylinders with the same radius are relevant Promote cylinders that fullfill these conditions, penalize those which do not. ET δ α
0 ET 0
R2
δ
R1
ET 0
π 2
α
|R1 − R2 |
5 / 10
Overview
1
Introduction Problem statement
2
Bayesian formulation A probabilistic approach Data fitting term Consistency term Term of compliance with the prior data
3
Solution computation Greedy minimization Solutions space sampler
4
Conclusion Results Prospects
Introduction
Bayesian formulation
Solution computation
Conclusion
Term of compliance with the prior data Energy of similarity with the a priori 3D Model Each cylinder Ci in X is associated with its most similar cylinder C k in M0 . Similarity measurement based on : the axes alignment (axes should be aligned) the cylinders displacement (the cylinders centers should be as close as possible) the cylinders radii (cylinders should have the same radius) Similar cylinders have a small penalty.
R2
R1
EG
d Ck
Ci
σD : Displacement tolerance σR : Geometry change tolerance σA : Rotation tolerance
a 0
a σA
|R1 −R2 | σR
d σD 6 / 10
Overview
1
Introduction Problem statement
2
Bayesian formulation A probabilistic approach Data fitting term Consistency term Term of compliance with the prior data
3
Solution computation Greedy minimization Solutions space sampler
4
Conclusion Results Prospects
Introduction
Bayesian formulation
Solution computation
Conclusion
Optimization algorithm Optimal solution computation Based on an iterative greedy approach. We try to find the optimal choice at each step, by inserting a new elements and removing those that are involved in conflicts. X ←∅
7 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Optimization algorithm Optimal solution computation Based on an iterative greedy approach. We try to find the optimal choice at each step, by inserting a new elements and removing those that are involved in conflicts. X ←∅
Generate a "random" cylinder Ci
7 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Optimization algorithm Optimal solution computation Based on an iterative greedy approach. We try to find the optimal choice at each step, by inserting a new elements and removing those that are involved in conflicts. X ←∅
Generate a "random" cylinder Ci
K ← { cylinders in X that collide with Ci }
7 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Optimization algorithm Optimal solution computation Based on an iterative greedy approach. We try to find the optimal choice at each step, by inserting a new elements and removing those that are involved in conflicts. X ←∅
Generate a "random" cylinder Ci
K ← { cylinders in X that collide with Ci }
Find K ∗ ⊂ K which minimizes E (X \ K ∗ ∪ {Ci }). Let X ∗ ← X \ K ∗ ∪ {Ci }
7 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Optimization algorithm Optimal solution computation Based on an iterative greedy approach. We try to find the optimal choice at each step, by inserting a new elements and removing those that are involved in conflicts. X ←∅
Generate a "random" cylinder Ci
K ← { cylinders in X that collide with Ci }
Find K ∗ ⊂ K which minimizes E (X \ K ∗ ∪ {Ci }). Let X ∗ ← X \ K ∗ ∪ {Ci }
E (X ∗ ) < E (X ) ?
7 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Optimization algorithm Optimal solution computation Based on an iterative greedy approach. We try to find the optimal choice at each step, by inserting a new elements and removing those that are involved in conflicts. X ←∅
Generate a "random" cylinder Ci
K ← { cylinders in X that collide with Ci }
Find K ∗ ⊂ K which minimizes E (X \ K ∗ ∪ {Ci }). Let X ∗ ← X \ K ∗ ∪ {Ci }
E (X ∗ ) < E (X ) ?
yes Accept Ci and remove conflicts : X ← X ∗ 7 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Optimization algorithm Optimal solution computation Based on an iterative greedy approach. We try to find the optimal choice at each step, by inserting a new elements and removing those that are involved in conflicts. X ←∅
Generate a "random" cylinder Ci
K ← { cylinders in X that collide with Ci } no Find K ∗ ⊂ K which minimizes E (X \ K ∗ ∪ {Ci }). Let X ∗ ← X \ K ∗ ∪ {Ci }
E (X ∗ ) < E (X ) ?
no
Stability reached ?
yes End !
yes Accept Ci and remove conflicts : X ← X ∗ 7 / 10
Overview
1
Introduction Problem statement
2
Bayesian formulation A probabilistic approach Data fitting term Consistency term Term of compliance with the prior data
3
Solution computation Greedy minimization Solutions space sampler
4
Conclusion Results Prospects
Introduction
Bayesian formulation
Solution computation
Conclusion
Solutions space sampler Relevant cylinders sampler Stochastic exploration of the solution space. Non uniform sampler favoring cylinders that are likely to decrease E, by using both P and M0 . Speeds up the convergence rate.
Let C i be a cylinder in M0 . Search for a shape in P that is likely to match C i .
8 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Solutions space sampler Relevant cylinders sampler Stochastic exploration of the solution space. Non uniform sampler favoring cylinders that are likely to decrease E, by using both P and M0 . Speeds up the convergence rate.
Let C i be a cylinder in M0 .
P(δ)
Search for a shape in P that is likely to match C i . n
α δ p
0 Ci
Probability P (p, n) ∈ P|C i
Pick a first point p0
Simulate P
δ P(α)
0
π 2
α
8 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Solutions space sampler Relevant cylinders sampler Stochastic exploration of the solution space. Non uniform sampler favoring cylinders that are likely to decrease E, by using both P and M0 . Speeds up the convergence rate.
Let C i be a cylinder in M0 .
P(δ)
Search for a shape in P that is likely to match C i . n
α δ p
0 Ci
Probability P (p, n) ∈ P|C i
Pick a first point p0
Simulate P
δ P(α)
0
π 2
α
N ← a neighborhood of p0 depending on the radius of C i
8 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Solutions space sampler Relevant cylinders sampler Stochastic exploration of the solution space. Non uniform sampler favoring cylinders that are likely to decrease E, by using both P and M0 . Speeds up the convergence rate.
Let C i be a cylinder in M0 .
P(δ)
Search for a shape in P that is likely to match C i . n
α δ p
0 Ci
Probability P (p, n) ∈ P|C i
Pick a first point p0
Simulate P
δ P(α)
0
π 2
α
N ← a neighborhood of p0 depending on the radius of C i
Pick (p1 , . . . , pk−1 ) ⊂ N likely to lie on the same surface as p0 .
Build a cylinder passing through (p0 , . . . , pk−1 ) 8 / 10
Overview
1
Introduction Problem statement
2
Bayesian formulation A probabilistic approach Data fitting term Consistency term Term of compliance with the prior data
3
Solution computation Greedy minimization Solutions space sampler
4
Conclusion Results Prospects
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
0
Iteration
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
Iteration
0
281/76198
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
Iteration
0
331/76198
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
Iteration
0
609/76198
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
Iteration
0
1058/76198
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
Iteration
0
1558/76198
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
Iteration
0
4617/76198
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
Iteration
0
10933/76198
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
Iteration
0
51906/76198
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
Iteration
0
76198/76198
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Introduction
Bayesian formulation
Solution computation
Conclusion
Energies
0
Iteration
Overall energy Data fitting energy Consistency energy Prior data similarity energy
9 / 10
Overview
1
Introduction Problem statement
2
Bayesian formulation A probabilistic approach Data fitting term Consistency term Term of compliance with the prior data
3
Solution computation Greedy minimization Solutions space sampler
4
Conclusion Results Prospects
Introduction
Bayesian formulation
Solution computation
Conclusion
Prospects
Further work handle other shapes (especially planes) use the positions and orientations of the LIDAR sensors to improve the results use the structure in the scene : process groups of shapes instead of shapes alone take the point cloud density changes into account improve the detection of the smallest elements
10 / 10
Thank you for your attention !
