Content
Homogeneous Coordinates
Introduc)on:+ Two+parallel+lines+can+intercept? 1 Before beginning 2 Camera models 3 3D vision
In Euclidean space (geometry), two parallel lines on the same plane cannot intercept In projective space, the train railroad on the side picture becomes narrower while it moves far away from eyes. Finally, the two parallel rails meet at the horizon, which is a point at infinity
2
T. Chateau
T. Chateau
Institut Pascal
4 Institut Pascal
2
4
Geometry for Vision
Content
2014 u
u0
v
(axe optique)
v0
f
Rc
1 Before beginning 1 Homogeneous Coordinates 2 Homogeneous Transformations 2 Camera models 3 3D vision
x y
T. Chateau
1
T. Chateau 1
3 Institut Pascal
3
Homogeneous Coordinates
Homogeneous Coordinates
Introduc)on:+
Solu)on:+Homogeneous+ Coordinates Homogeneous
(1, 2, 3) = (2, 4, 6) = (1a, 2a, 3a) !
Coding+rota)on+and+transla)on+in+the+same+ matrix 0 R 1 0 R1 X f X i @ Y Rf A = @ Y Ri A Z Rf Z Ri
Cartesian
✓
1 2 , 3 3
◆
0 1 a @bA c
Translation 6
T. Chateau
8
T. Chateau
Institut Pascal
Institut Pascal
6
Homogeneous Coordinates
Homogeneous Coordinates
Solu)on:+
Solu)on:+Homogeneous+ Coordinates
Two+parallel+lines+can+intercept?
x y A +B +C =0 w w x y A +B +D =0 w w
Homogeneous coordinates are a way of representing N-dimensional coordinates with N+1 numbers Therefore, a point in Cartesian coordinates, (X, Y) becomes (x, y, w) in Homogeneous coordinates
⇣Cartesian x y⌘ (x, y, w) $ , w w
Ax + By + Cw = 0 Ax + By + Dw = 0
Homogeneous
T. Chateau
8
Solution: w = 0 (x, y, 0) point at infinity 5
T. Chateau
Institut Pascal
5
7 Institut Pascal
7
Homogeneous Coordinates
Homogeneous Transformations
Introduc)on:+
Homogeneous+Transforma)ons
Coding+rota)on+and+transla)on+in+the+same+ matrix
R1
0 R1 X Rf X i @ Y Rf A = R @ Y Ri A Z Rf Z Ri 0
1
0 1 a @bA c
R2
+
PR1 /R1 TR2 · PR2 Rotation Translation
R1
Combining rotation and translation 10
T. Chateau
T R2
1 0 X R1 sx B Y R1 C Bsy B R C/B @ Z 1 A @sz 0 1 0
nx ny nz 0
ax ay az 0
12
T. Chateau
Institut Pascal
1 0 R 1 X 2 Px B R2 C Py C C · BY R C Pz A @ Z 2 A 1 1 Institut Pascal
10
Homogeneous Transformations
12
Homogeneous Coordinates
Introduc)on:+
Into+projec)ve+space
Coding+rota)on+and+transla)on+in+the+same+ matrix 0
1 0 R = @ 0 cos ✓x 0 sin ✓x
1 0 sin ✓x A cos ✓x
0 R 1 0 R1 X f X i @ Y Rf A = R @ Y Ri A Z Rf Z Ri
Rota)on:+ex.+along+x>axis T. Chateau
9
0 R 1 0 X f 1 0 B Y Rf C B0 cos ✓x B R C=B @ Z f A @0 sin ✓x 0 0 1 T. Chateau
Institut Pascal
9
Homogeneous 0 1 Cartesian 0 1 ax x Bay C C P = @y A ! B @ az A z a
0 sin ✓x cos ✓x 0
1 0 R1 X i a B C b C B Y Ri C C · c A @ Z Ri A 1 1 11 Institut Pascal
11
Homogeneous Transformations
Homogeneous Transformations
Transla)on
Any rotation can be written with three simple rotations:
0
B M =B @
T. Chateau
a b C C c A 1
I3 0
- Euler Angles, - Quaternion, ...
1
0
0
14
16
T. Chateau
Institut Pascal
Institut Pascal
14
Homogeneous Transformations
16
Homogeneous Transformations
Rigid+Transforma)ons 1 0 X R1 sx B Y R1 C Bsy B R C/B @ Z 1 A @sz 0 1 0
PR1 =
T. Chateau
✓
R1
RR2 0
R1
nx ny nz 0
TR2 1
◆
ax ay az 0
Exercice:+y>axis+rota)on
1 0 R 1 X 2 Px B R2 C Py C C · BY R C Pz A @ Z 2 A 1 1
1 0 R X 1 X R2 B Y R2 C B Y R1 B R C=B R @ Z 2 A @ Z 1 1 1 0
13
Compute+the+homogeneous+ transforma)on+matrix
1 C C A T. Chateau
Institut Pascal
13
15 Institut Pascal
15
Homogeneous Transformations
Homogeneous Transformations
Combining+transforma)ons 1 Before beginning 2 Camera models 1 pinhole model 2 other models 3 3D vision 0
Tk =0 T1 .1 T2 .2 T3 .3 T4 ...k
1
Tk 18
T. Chateau
20
T. Chateau
Institut Pascal
Institut Pascal
18
Homogeneous Transformations
20
Homogeneous Transformations
Invert+transforma)ons
Exercice zp z1
xp x1
za
ya
T
1
0
T
B AT =B @ 0
T. Chateau
1
T =
s .P ✓ T A nT .P C C= 0 aT .P A 1
A P 0 1
AT .P 1
◆
◆
17
O1Ra = (10, 3, 0)T
T. Chateau 17
x2
Oa
z2 O2
y2
O2Ra = (20, 5, 0)T
Compute :
Institut Pascal
Op
O1 xa
✓
yp
y1
Ra
OpR1 = (5, 4, 2)T
TR1 and
Ra
TR2 19
Institut Pascal
19
Camera Models
Camera Models
Pinhole+model:+extrinsic+parameter+matrix
Pinhole+model
Y
Rc
P
TRw RW
X
u
u0
Z
v z
v0 f
RC
x y 22
T. Chateau
24
T. Chateau
Institut Pascal
Institut Pascal
22
Camera Models
24
Camera Models
Pinhole+model:+3+reference+frames Y
Pinhole+model
P
RW X
u
u0
Z
v
Image plane
z
v0
Pinhole
f
RC
Virtual image
x y
T. Chateau
21
T. Chateau
Institut Pascal
21
23 Institut Pascal
23
Camera Models
Camera Models
Pinhole+model:+perspec)ve+projec)on
Pinhole+model:+image+plane+transforma)on
Y
RW
u
Ri
0 1 X P @Y A Z
u0
v
Rc
X
u
Z
v
0 1 x @y A z
v0 f
RC
optical axis (axe optique)
v0
u0
xRc = X RC f /Z Rc y Rc = Y RC f /Z Rc
z
z
x
Rc x
=f
Rc
⇢
f
y
Ri
y 26
T. Chateau
u = x/dx + u0 v = y/dy + v0 28
T. Chateau
Institut Pascal
Institut Pascal
26
Camera Models
28
Camera Models
Pinhole+model:+extrinsic+parameter+matrix
Pinhole+model:+perspec)ve+projec)on
Y
Rc
Y
0 1 X P @Y A Z
P
TRw RW
RW X
u
u0
u
Z
P
v z
v0 f
RC
x
PRC =
X
✓
Rc
RC
RRw 0
=
RC
Rc
TRW P
TRw 1
RW
◆
v0
PRW
f
RC
x
y T. Chateau
u0
Z
v
0 1 x @y A z
1 0 Rc f z sx @sy Rc A / @ 0 0 s 0
0 f 0
1 0 1 X Rc 0 0 B Rc C Y C 0 0A B @ Z Rc A 1 0 1
y 25
T. Chateau
Institut Pascal
25
27 Institut Pascal
27
Camera Models
Camera Models
Pinhole+model:+image+plane+transforma)on ⇢
0
u = x/dx + u0 v = y/dy + v0
1 0 su 1/dx 0 @ sv A = @ 0 1/dy s 0 0
Pinhole+model:+intrinsic+parameter+matrix+
10 1 u0 sx v0 A @ sy A 1 s
Mint
the intersection between the optical axis and the image plane •(dx,dy) are the x and y dimension of one pixel of the physical sensor.
pixels
30
1 u0 v0 A 1
fx 0 = @ 0 fy 0 0
fx = f /dx fy = f /dy
•(u0,v0) are pixel based coordinates into the image of
T. Chateau
0
! (fx /fy = dy/dx)
mm 32
T. Chateau
Institut Pascal
Institut Pascal
30
Camera Models
32
Camera Models
Pinhole+model:+image+plane+transforma)on u
Ri
Pinhole+model:+intrinsic+parameter+matrix+ 1 0 1/dx su @ sv A = @ 0 0 s 0
u0
v
optical axis (axe optique)
v0
f
Rc x
0
1 0 su 1/dx 0 @ sv A = @ 0 1/dy s 0 0
10 1 u0 sx v0 A @ sy A 1 s
0
29
1 0 fx su @ sv A = @ 0 0 s 0
T. Chateau
Institut Pascal
29
0
f /dx su @ sv A = @ 0 0 s
y T. Chateau
1
0 1/dy 0
1 u0 v0 A 1
0 f /dy 0 0 fy 0
0
f @ 0 0
u0 v0 1 u0 v0 1
0 f 0
0 0 1
0 R 1 X c 0 B Y Rc 0 AB @ Z Rc 0 1
1 X Rc 0 B Y Rc C C 0 AB @ Z Rc A 0 1 1
0
0 R 1 X c 0 B Y Rc 0 AB @ Z Rc 0 1
1 C C A
1 C C A
31
Institut Pascal
31
Camera Models
Camera Models
Pinhole+model:+global+projec)on+matrix 0 R 1 W
0 1 0 m11 su @sv A = @m21 m31 s 8 m11 X Rw > > > : v = m21 X m31 X Rw
m12 m22 m32
m13 m23 m33
+ m12 Y Rw + m32 Y Rw + m22 Y Rw + m32 Y Rw
Pinhole+model:+camera+calibra)on
1
X m14 B RW C Y C m24 A B @ Z RW A m34 1
+ m13 Z Rw + m33 Z Rw + m23 Z Rw + m33 Z Rw
1) built the linear system: 0
B B B B i B Xw B B 0 B B B @
+ m14 + m34 + m24 + m34
i Zw 0
1 0
0 Xwi
0 Ywi
0 i Zw
1
ui Xwi v i Xwi
ui Ywi v i Ywi
i ui Zw i v i Zw
0 m11 C C B m12 CB CB . B ui C CB . B vi C CB . CB C @ m32 A m34
Institut Pascal
34
Camera Models
36
Camera Models
Pinhole+model:+global+projec)on+matrix 0 R 1 0 1 X W su B RW C @sv A = Mint RC TRW B Y R C @Z W A s 1
T. Chateau
C C C C C=0 C C C A
36
T. Chateau
Institut Pascal
0 1 0 m11 su @sv A = @m21 m31 s
1
2) Solve it with a least square linear optimization
34
T. Chateau
Ywi 0
. . . 0 1 . . .
m12 m22 m32
P
m13 m23 m33
Pinhole+model:+camera+calibra)on Given+a+set+of+2D>3P+ matching+points,+ compute: 0 m11 P = @m21 m31
1 0 1 X RW m14 B RW C Y C m24 A B @ Z RW A m34 1 33
T. Chateau
Institut Pascal
33
m12 m22 m32
m13 m23 m33
1 m14 m24 A m34
35 Institut Pascal
35
3D Vision
3D Vision
Planar transformations
Planar transformations
Rc1
Rc2 pc1 : point p in frame 1 pc2 : point p in frame 2 p : point in plane P
H1
Rw
x = (u, v, 1)T
X = (U, V, 1)T
T. Chateau
P2 = RP1 + T
1 H = R + TNT d
X / H1 x 38
40
T. Chateau
Institut Pascal
Institut Pascal
38
3D Vision
40
3D Vision
Planar transformations
X / H1 x
1 Before beginning 2 Camera models 3 3D vision 1 Planar transformations 2 Multi-view 3D reconstruction
T. Chateau
0
1 h11 h12 h13 X / @h21 h22 h23A x h31 h32 h33 37
T. Chateau
Institut Pascal
37
39 Institut Pascal
39
3D Vision
3D Vision
Multi-view 3D reconstruction (intro)
Fondamental Matrix The fondamental matrix is the key relation in structure from motion algorithms. Further reading in:
Cam1
@Book{Hartley2000, author = "Hartley, R.~I. and Zisserman, A.", title = "Multiple View Geometry in Computer Vision", year = "2000", publisher = "Cambridge University Press, ISBN: 0521623049" }
Cam2
Epipolar line 42
T. Chateau
44
T. Chateau
Institut Pascal
Institut Pascal
42
3D Vision
44
3D Vision
Multi-view 3D reconstruction (intro)
Cam1
Fondamental Matrix
Cam2
Cam1
Fondamental Matrix : Epipolar plane T. Chateau
Cam2
xT Fx0 = 0
With F a 3x3 matrix of rank 2.
41
T. Chateau
Institut Pascal
41
43 Institut Pascal
43
Exercices
Stereo Two calibrated cameras see the same 3D rigid scene. 1) Write the linear system which gives the estimation of a 3D point from the projection of the point in the two calibrated cameras (in a least square way). T. Chateau
46 Institut Pascal
46
Exercices
Homography Two cameras see the same plane. 1) How many matches of image points are necessary to compute the Homography between the two images ? 2) Write the linear system which gives the estimation of the homography between the two cameras from matches image points (in a least square way). T. Chateau
45 Institut Pascal
45
