双目立体匹配中的极线约束（Epipolar Constraint），本质矩阵（Essential Matrix），对极几何（2D-2D）

考虑一个SLAM中一个常见的问题：如果两个相机在不同位置拍摄同一个物体，或者一个运动的相机在不同时刻拍摄同一物体，我们有理由相信两张图片中各点存在着某种几何关系，这种关系可以用对极几何来描述。对极几何描述了两帧图像中各像素的射影关系（或者说是各匹配点的几何关系），其与外部的场景本身无关，只与相机的内参以及两图像的拍摄位置有关。

1. The Epipolar constraint

We know that x 1 x_1 x1​ (in homogeneous coordinates) is the projection of 3D coordinate $X$ on the image plane.
λ 1 x 1 = X , λ 2 x 2 = R X + T λ 2 x 2 = R ( λ 1 x 1 ) + T \\lambda_1x_1=X, \\quad \\lambda_2x_2=RX+T \\\\ \\lambda_2x_2=R(\\lambda_1x_1) + T λ1​x1​=X,λ2​x2​=RX+Tλ2​x2​=R(λ1​x1​)+T
Then, we remove the translation by multiplying with T ∧ T^{\\wedge} T∧, the skew matrix.
λ 2 T ∧ x 2 = λ 1 T ∧ R x 1 + 0 \\lambda_2T^{\\wedge}x_2=\\lambda_1T^{\\wedge}Rx_1+0 \\\\ λ2​T∧x2​=λ1​T∧Rx1​+0
Then, we projection onto x 2 x_2 x2​ gives the epipolar constraint:
λ 2 x 2 T T ∧ x 2 = λ 1 x 2 T T ∧ R x 1 \\lambda_2x_2^TT^{\\wedge}x_2=\\lambda_1x_2^TT^{\\wedge}Rx_1 λ2​x2T​T∧x2​=λ1​x2T​T∧Rx1​
As we know, T × x 2 = T ∧ x 2 T\\times x_2=T^{\\wedge}x_2 T×x2​=T∧x2​, is the cross product, which generates a third vector that perpendicular to the plane which is spanned by $T$ and x 2 x_2 x2​.

So, λ 2 x 2 T T ∧ x 2 = 0 \\lambda_2x_2^TT^{\\wedge}x_2=0 λ2​x2T​T∧x2​=0. The dot product of x 2 x_2 x2​ and vector T ∧ x 2 T^{\\wedge}x_2 T∧x2​ is zero. Then, we have:
x 2 T T ∧ R x 1 = 0 x_2^TT^{\\wedge}Rx_1=0 x2T​T∧Rx1​=0
This is called epipolar constraint.

2. Essential matrix $E$

The matrix E = T ∧ R ∈ R 3 × 3 E=T^{\\wedge}R \\in R^{3\\times3} E=T∧R∈R3×3 is essential matrix. Provide the relation between the 2D point coordinates of 3D point in each of the two images and the camera transformation parameters.

This constraint states that the three vector o 1 X ⃗ \\vec{o_1X} o1​X ​, o 2 X ⃗ \\vec{o_2X} o2​X ​ and o 2 o 1 ⃗ \\vec{o_2o_1} o2​o1​ ​ form a plane, and the volume of the plane is 0. This is called the volume spanned by x 2 x_2 x2​, x 1 x_1 x1​, and $R,T$, such that:
v o l u m e = x 2 T ( T × R ) x 1 = 0 x 2 T E x 1 = 0 volume = x_2^T(T\\times R)x_1=0 \\\\ x_2^TEx_1=0 volume=x2T​(T×R)x1​=0x2T​Ex1​=0

The unique solution, need eight point pairs to solve.
x 2 T E x 1 = 0 = a T E s = 0. x_2^TEx_1=0=a^TE^s=0. x2T​Ex1​=0=aTEs=0.
For $n$ point pairs, we can combine this into the linear system.
χ E s = 0 , w i t h χ = ( a 1 , a 2 , . . . , a n ) T . \\chi E^s=0, \\quad with \\ \\ \\chi =(a^1,a^2, ...,a^n)^T. χEs=0,with  χ=(a1,a2,...,an)T.