Geometry for Augmented Reality
Content
u
u0
v
(axe optique)
v0
f
1.Introduction 2.Camera models 3.Planar geometry 4.Introduction to 3D geometry
Rc x y
1
2
T. Chateau
T. Chateau
Homogeneous Coordinates
Homogeneous Coordinates
Homogeneous Coordinates
Solu2on: Homogeneous Coordinates
Homogeneous coordinates are a way of representing N-dimensional coordinates with N+1 numbers Therefore, a point in Cartesian coordinates, (X, Y) becomes (x, y, w) in Homogeneous coordinates
⇣Cartesian x y⌘ (x, y, w) $ , w w
Institut Pascal
Homogeneous
(1, 2, 3) = (2, 4, 6) = (1a, 2a, 3a) !
Cartesian
✓
1 2 , 3 3
◆
Homogeneous
T. Chateau
3 Institut Pascal
T. Chateau
4 Institut Pascal
Homogeneous Transformations
Homogeneous Transformations
Homogeneous Transforma2ons R1
T R2
R2
+
P
R1
R1
/
T R2 · P
Rigid Transforma2ons 0
1 0 X R1 sx B Y R1 C Bsy B R C/B @ Z 1 A @sz 0 1
R2
Rotation Translation
R1
0
1 0 X R1 sx B Y R1 C Bsy B R C/B @ Z 1 A @sz 0 1
nx ny nz 0
ax ay az 0
T. Chateau
1 0 R 1 X 2 Px C B Py C B Y R2 C C · Pz A @ Z R2 A 1 1 5 Institut Pascal
Content
PR1 =
✓
R1
RR2 0
R1
nx ny nz 0
TR2 1
◆
ax ay az 0
1 0 R 1 X 2 Px B R2 C Py C C · BY R C Pz A @ Z 2 A 1 1
0
1 0 R X R2 X 1 B Y R2 C B Y R1 B R C=B R @ Z 2 A @ Z 1 1 1 6
T. Chateau
Institut Pascal
Camera Models
Pinhole model 1.Introduction 2.Camera models 3.Planar geometry 4.Introduction to 3D geometry
Image plane Pinhole
T. Chateau
7 Institut Pascal
T. Chateau
Virtual image 8 Institut Pascal
1 C C A
Camera Models
Camera Models
Pinhole model: 3 reference frames
Pinhole model: extrinsic parameter matrix
Y
Y
P
Rc
P
TRw
RW
RW X
u
u0
X
u
Z
v z
v0
Z
z
v0
f
RC
u0
PRC =RC TRW PRW
v
f
RC
x
P
x
y
RC
=
✓
Rc
RRw 0
Rc
TRw 1
◆
PRW
y 9
T. Chateau
10
T. Chateau
Institut Pascal
Camera Models
Institut Pascal
Camera Models
Pinhole model: image plane transforma2on
Pinhole model: perspec2ve projec2on Y
RW
u
Ri
0 1 X P @Y A Z
u0
v X
u
Z
v v0 f
RC
x
0 1 x @y A z
optical axis (axe optique)
v0
u0
z
0
1
0
sxRc f @sy Rc A / @ 0 0 s
0 f 0
0 1 1 X Rc 0 0 B Rc C Y C 0 0A B @ Z Rc A 1 0 1
f
Rc x
0
1 0 su 1/dx 0 @ sv A = @ 0 1/dy s 0 0
10 1 u0 sx v0 A @ sy A 1 s
y
y T. Chateau
11 Institut Pascal
T. Chateau
12 Institut Pascal
Camera Models
Camera Models
Pinhole model: image plane transforma2on ⇢
0
u = x/dx + u0 v = y/dy + v0
1 0 su 1/dx 0 @ sv A = @ 0 1/dy s 0 0
10 1 u0 sx v0 A @ sy A 1 s
•(u0,v0) are pixel based coordinates into the image of
the intersection between the optical axis and the image plane •(dx,dy) are the x and y dimension of one pixel of the physical sensor. 13
T. Chateau
Institut Pascal
Camera Models
Pinhole model: intrinsic parameter matrix 0
1 0 su 1/dx @ sv A = @ 0 s 0
0
0
T. Chateau
1
0
su fx @ sv A = @ 0 s 0
1 u0 v0 A 1
0 f /dy 0 0 fy 0
f @ 0 0
u0 v0 1 u0 v0 1
T. Chateau
0
0 f 0
0 0 1
1 X Rc 0 B Y Rc C C 0 AB @ Z Rc A 0 1 1
0
0 R 1 X c 0 B Y Rc 0 AB @ Z Rc 0 1
1 X Rc 0 B Y Rc C C 0 AB @ Z Rc A 0 1 1
0
14
Institut Pascal
Camera Models
0
fx 0 = @ 0 fy 0 0
fx = f /dx fy = f /dy pixels
0
su f /dx @ sv A = @ 0 s 0
Pinhole model: intrinsic parameter matrix
Mint
1
0 1/dy 0
Pinhole model: global projec2on matrix 0 R 1 0 1 X W su B RW C @sv A = Mint RC TRW B Y R C @Z W A s 1
1
u0 v0 A 1
0 1 0 su m11 @sv A = @m21 s m31
! (fx /fy = dy/dx)
mm 15 Institut Pascal
T. Chateau
m12 m22 m32
P
m13 m23 m33
0 R 1 X W m14 B RW C Y C m24 A B @ Z RW A m34 1 1
16 Institut Pascal
1 C C A
Camera Models
Content
Pinhole model: global projec2on m0atrix 1 RW
0 1 0 su m11 @sv A = @m21 s m31 8 m11 X Rw > > u = < m31 X Rw Rw > > : v = m21 X m31 X Rw
m12 m22 m32
m13 m23 m33
+ m12 Y Rw + m32 Y Rw + m22 Y Rw + m32 Y Rw
1 X m14 B RW C Y C m24 A B @ Z RW A m34 1
+ m13 Z Rw + m33 Z Rw + m23 Z Rw + m33 Z Rw
+ m14 + m34 + m24 + m34
17
T. Chateau
Institut Pascal
Planar transformations
1.Introduction 2.Camera models 3.Planar geometry 4.Introduction to 3D geometry
18
T. Chateau
Institut Pascal
Planar transformations
Homography
Homography
x0 = Rx + T
˜ = (xt , 1)t x
˜ 0 = H˜ x x 1 H = R + TNT d
H: Homography (3x3 matrix) T. Chateau
19 Institut Pascal
T. Chateau
20 Institut Pascal
Planar transformations
Planar transformations
Homography
Homography Estimation
0 h11 0 ˜ / @h21 x h31
˜ 0 = H˜ x x
0 h11 0 ˜ / @h21 x h31
h12 h22 h32
1
h13 ˜ h23 A x h33
1 h13 ˜ h23 A x h33
Build an estimator
ˆ = (h11 , h12 , h13 , h21 , h22 , h23 , h31 , h32 , h33 )t ✓ 21
T. Chateau
h12 h22 h32
Institut Pascal
Planar transformations
T. Chateau
22 Institut Pascal
Planar transformations
Homography Estimation Homography estimation 0 h11 0 ˜ / @h21 x h31
h12 h22 h32
1 h13 ˜ h23 A x h33
We use a set of matches: {xn ; x0n }n=1,..,N
Build an estimator
ˆ = (h11 , h12 , h13 , h21 , h22 , h23 , h31 , h32 , h33 )t ✓ T. Chateau
23 Institut Pascal
T. Chateau
24 Institut Pascal
Planar transformations
Planar transformations
Homography estimation: [Hartley]
Exercice: Homography estimation Given a set of matches: {xn ; x0n }n=1,..,N
ˆ = (h11 , h12 , h13 , h21 , h22 , h23 , h31 , h32 , h33 )t ✓ Write the homography estimation problem as a linear system:
A.✓ˆ = B by setting h33 = 1 25
T. Chateau
Institut Pascal
26
T. Chateau
Institut Pascal
Planar transformations
Planar transformations
Robust homography estimation:
Robust homography estimation: Least square approximation
OUTLIERS
T. Chateau
27 Institut Pascal
T. Chateau
28 Institut Pascal
Planar transformations
Planar transformations
Robust homography estimation:
Robust homography estimation:
Least square approximation
RANSAC RANdom SAmple Consensus - Approach: we want to avoid the impact of outliers, so let’s look for inliers only - Intuition: if an outlier is chosen to compute the current fit, then the resulting line won’t have much support from the rest of the points
29
T. Chateau
Institut Pascal
30
T. Chateau
Institut Pascal
Planar transformations
Planar transformations
Robust homography estimation:
Robust homography estimation:
RANSAC
RANSAC
RANdom SAmple Consensus - Approach: we want to avoid the impact of outliers, so let’s look for inliers only - Intuition: if an outlier is chosen to compute the current fit, then the resulting line won’t have much support from the rest of the points
T. Chateau
31 Institut Pascal
T. Chateau
32 Institut Pascal
Planar transformations
Planar transformations
Robust homography estimation:
Robust homography estimation:
RANSAC
RANSAC
33
T. Chateau
Institut Pascal
34
T. Chateau
Institut Pascal
Planar transformations
Planar transformations
Robust homography estimation:
Robust homography estimation:
RANSAC
T. Chateau
RANSAC
35 Institut Pascal
T. Chateau
36 Institut Pascal
Planar transformations
Planar transformations
Robust homography estimation:
Robust homography estimation: RANSAC
T. Chateau
37
38
T. Chateau
Institut Pascal
Planar transformations
Institut Pascal
3D Vision
Robust homography estimation: exercice
Multi-view 3D reconstruction (intro)
Write a RANSAC Algorithm to estimate an homography. We assume that the outlier rate is up to 50%.
Cam1
Cam2
Epipolar plane T. Chateau
39 Institut Pascal
T. Chateau
40 Institut Pascal
3D Vision
3D Vision
Multi-view 3D reconstruction (intro)
Cam1 Cam1
Cam2
Cam2
Epipolar line 41
T. Chateau
Institut Pascal
3D Vision
42
T. Chateau
Institut Pascal
3D Vision
Essential matrix
Essential matrix
Estimation of essential matrix can be done using: - 5 matches: 5-points algorithm - 8 matches: 8-points algorithm See Nister’s work for further information
T. Chateau
43 Institut Pascal
T. Chateau
44 Institut Pascal
3D Vision
Exercices
Fondamental (Essential) Matrix The fondamental matrix is the key relation in structure from motion algorithms. Further reading in: @Book{Hartley2000, author = "Hartley, R.~I. and Zisserman, A.", title = "Multiple View Geometry in Computer Vision", year = "2000", publisher = "Cambridge University Press, ISBN: 0521623049" }
T. Chateau
45 Institut Pascal
Stereo Two calibrated cameras see the same 3D rigid scene. 1) Write the linear system which gives the estimation of a 3D point from the projection of the point in the two calibrated cameras (in a least square way).
T. Chateau
46 Institut Pascal
