eric-yyjau / deepf_nocorrs

# Deep Fundamental Matrix Estimation without Correspondences

DeepFMatrix

Try to use Deep Learning to obtain fundamental matrices.

References:

Stereo Image Estimation

DeMoN(CVPR 2017)

H-Matrix Estimation

Deep Image Homography Estimation

Cascaded Lucas-Kanade Networks(CVPR2017)

Direct Regression of H-Matrix

Camera Pose Estimation

VINet

Relative Camera Pose Estimation

PosNet

Geometrics Matching

CNN for Geo-Matching

for training the model:

run the file single_fnet.py
the arguments to be specified are mentioned in the same file.

deepf_nocorrs's Issues

assert np.linalg.matrix_rank(F) == 2

(py36-tf-cuda10) ➜  deepF_noCorrs git:(test) ✗ python data_util.py norm
norm:  norm                                                                               
path_dir: ['calib_velo_to_cam.txt', 'calib_imu_to_velo.txt', '2011_09_26_drive_0001_sync', 'calib_cam_to_cam.txt']
N:  2000                                                                 
path_left: ./data/kitti/2011_09_26/calib_velo_to_cam.txt/image_00/data/
path_right: ./data/kitti/2011_09_26/calib_velo_to_cam.txt/image_01/data/
path_left: ./data/kitti/2011_09_26/calib_imu_to_velo.txt/image_00/data/                                                        path_right: ./data/kitti/2011_09_26/calib_imu_to_velo.txt/image_01/data/
path_left: ./data/kitti/2011_09_26/2011_09_26_drive_0001_sync/image_00/data/
path_right: ./data/kitti/2011_09_26/2011_09_26_drive_0001_sync/image_01/data/
pre-processing image:  ./data/kitti/2011_09_26/2011_09_26_drive_0001_sync/image_00/data/0000000017.png i:  0            
K1: [[984.2439   0.     690.    ]                    
 [  0.     980.8141 233.1966]                                                            
 [  0.       0.       1.    ]], K2: [[989.5267   0.     702.    ]
 [  0.     987.8386 245.559 ]
 [  0.       0.       1.    ]], R1: [[1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]], R2: [[ 0.9993513   0.01860866 -0.03083487]
 [-0.01887662  0.9997863  -0.00842187]
 [ 0.03067156  0.00899847  0.999489  ]], t1: [[ 2.573699e-16]
 [-1.059758e-16]
 [ 1.614870e-16]], t2: [[-0.537     ]
 [ 0.00482206]
 [-0.01252488]]
Traceback (most recent call last):
  File "data_util.py", line 519, in <module>
    data_loader = make_kitti_data_loader(norm=norm)
  File "data_util.py", line 328, in make_kitti_data_loader
    F, pts1, pts2 = get_FMat(left_path, right_path)
  File "/home/yyjau/Documents/deepF_noCorrs/kittiFMat/kitti_fmat.py", line 62, in get_FMat
    F = p.get_F(cam1, cam2)
  File "/home/yyjau/Documents/deepF_noCorrs/parser.py", line 56, in get_F
    assert np.linalg.matrix_rank(F) == 2
  File "<__array_function__ internals>", line 6, in matrix_rank
  File "/home/yyjau/anaconda3/envs/py36-tf-cuda10/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 1876, in matrix_ran
k
    S = svd(M, compute_uv=False, hermitian=hermitian)
  File "<__array_function__ internals>", line 6, in svd
  File "/home/yyjau/anaconda3/envs/py36-tf-cuda10/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 1648, in svd
    s = gufunc(a, signature=signature, extobj=extobj)
TypeError: No loop matching the specified signature and casting was found for ufunc svd_n
(py36-tf-cuda10) ➜  deepF_noCorrs git:(test) ✗

data_util.py norm methods

    # Normalize F-matrices
    if norm == "abs":
        print("[data loader] Use max abs value to normalize the F-matrix")
        Y = Y / (np.abs(Y).max(axis=1)[:,np.newaxis] + 1e-8)
    elif norm == "norm":
        print("[data loader] Use L2 norm to normalize the F-matrix")
        Y = Y / (np.linalg.norm(Y, axis=1)[:,np.newaxis] + 1e-8)
    elif norm == "last":
        print("[data loader] Use last index to normalize the F-matrix")
        Y = Y / (Y[:,-1].reshape(-1)[np.newaxis,1] + 1e-8)
    else:
        raise Exception("Unrecognized normalization methods:%s"%norm)

Which normalization is used by default in the paper?

installation

• packages
  n-CLAIR/nsml-local
  https://github.com/n-CLAIR/nsml-local