opencv 立体校正后的图像未行对齐

mrwjdhj3  于 2022-11-15  发布在  其他
关注(0)|答案(1)|浏览(226)

**问题是什么:**我一直在遵循这个教程(Link)以及openCV文档,但我似乎不能得到我的立体声校正的权利。
**我的问题有何不同:**我的设置是机械修复的,这是this帖子中的问题。
**我所做的:**在MATLAB的Camera Calibrator中计算相机的内在特性和立体参数。然后按照上述教程对两个立体图像进行去扭曲和校正。MATLAB的Stereo Calibrator生成this rectified image view(看起来完全行对齐),我想在python中为相同的图像重现this rectified image view

我在左框中使用了棋盘的六个角,并用一条线将它们连接到右框中的相应角。我的想法是,在一幅未失真且经过校正的图像中,极线必须是水平的,因此这些连接线必须是水平的(平均斜率非常接近于0)。然而它们不是!

如何到达:

left imageright image打印机

import cv2 as cv
    import numpy as np

    # Intrinsics from MATLAB
    distortionCoefficientsL = np.array([0.1112, -0.2270, 0.0014, 7.5801e-04, 0.0835])
    cameraMatrixL = np.array([[1384.3, 0, 933.5327], [0, 1383.2, 532.1460], [0, 0, 1]])
    newCameraMatrixL = cameraMatrixL

    distortionCoefficientsR = np.array([0.0362, -0.1640, -2.2236e-04, 3.4982e-04, 0.1148])
    cameraMatrixR = np.array([[1417.1, 0, 972.7481], [0, 1418.0, 542.9659], [0, 0, 1]])
    newCameraMatrixR = cameraMatrixR

    # Stereo params from MATLAB
    Rot = np.array([[0.9999, 0.0109, 0.0068],[-0.0111, 0.9998, 0.0178],[-0.0066, -0.0179, 0.9998]])
    Trns = np.array([[-96.5080], [-1.0640], [-0.8036]])
    Emat = np.array([[0.0015, 0.7844, -1.0782],[-0.1459, 1.7298, 96.4957],[0.0084, -96.4985, 1.7210]])
    Fmat = np.array([[7.8440e-10, 4.0019e-07, -9.7456e-04],[-7.4317e-08, 8.8188e-07, 0.0677],[4.5630e-05, -0.0706, 3.0555]])

    # Rectification and undistortion
    imgL = cv.imread(‘path to left image’)
    imgR = cv.imread(‘path to right image’)
    grayL = cv.cvtColor(imgL,cv.COLOR_BGR2GRAY)
    grayR = cv.cvtColor(imgR,cv.COLOR_BGR2GRAY)
    imgSize = grayL.shape[::-1]

    R_L, R_R, proj_mat_l, proj_mat_r, Q, roiL, roiR= cv.stereoRectify(newCameraMatrixL, distortionCoefficientsL, newCameraMatrixR, distortionCoefficientsR, imgSize, Rot, Trns, flags=cv.CALIB_ZERO_DISPARITY, alpha=1)

    leftMapX, leftMapY = cv.initUndistortRectifyMap(newCameraMatrixL, distortionCoefficientsL, R_L, proj_mat_l, imgSize, cv.CV_32FC1)
    rightMapX, rightMapY = cv.initUndistortRectifyMap(newCameraMatrixR, distortionCoefficientsR, R_R, proj_mat_r, imgSize, cv.CV_32FC1)
    Left_rectified = cv.remap(imgL,leftMapX,leftMapY, cv.INTER_LINEAR, cv.BORDER_CONSTANT)
    Right_rectified = cv.remap(imgR,rightMapX,rightMapY, cv.INTER_LINEAR, cv.BORDER_CONSTANT)

    grayL = cv.cvtColor(Left_rectified,cv.COLOR_BGR2GRAY)
    grayR = cv.cvtColor(Right_rectified,cv.COLOR_BGR2GRAY)

    font = cv.FONT_HERSHEY_PLAIN
    fontScale = 4

    # Find all chessboard corners at subpixel accuracy
    boardSize = (6,9)
    subpix_criteria = (cv.TERM_CRITERIA_EPS+cv.TERM_CRITERIA_MAX_ITER, 100, 10e-06)
    winSize = (11,11)

    retL, cornersL = cv.findChessboardCorners(grayL, boardSize, cv.CALIB_CB_ADAPTIVE_THRESH+cv.CALIB_CB_FAST_CHECK+cv.CALIB_CB_NORMALIZE_IMAGE)
    retR, cornersR = cv.findChessboardCorners(grayR, boardSize, cv.CALIB_CB_ADAPTIVE_THRESH+cv.CALIB_CB_FAST_CHECK+cv.CALIB_CB_NORMALIZE_IMAGE)

    objp = np.zeros((1, boardSize[0]*boardSize[1], 3), np.float32)
    objp[0,:,:2] = np.mgrid[0:boardSize[0], 0:boardSize[1]].T.reshape(-1, 2)
    objectPoints = []
    imagePointsL = [] 
    imagePointsR = [] 

    slopes = []
    if retR is True and retL is True:
     objectPoints.append(objp)
     cv.cornerSubPix(grayR, cornersR,(3,3),(-1,-1),subpix_criteria)
     cv.cornerSubPix(grayL, cornersL,(3,3),(-1,-1),subpix_criteria)
     imagePointsR.append(cornersR)
     imagePointsL.append(cornersL)
    
     # Get points in 4th row (vertical centre) and display them
     vis = np.concatenate((Left_rectified, Right_rectified), axis=1)
     for i in range(24,30):
         x_l = int(round(imagePointsL[0][i][0][0]))
         y_l = int(round(imagePointsL[0][i][0][1]))
         cv.circle(vis, (x_l, y_l), 7, (0,255,255), -1)
         x_r = int(round(imagePointsR[0][i][0][0]+Left_rectified.shape[1]))
         y_r = int(round(imagePointsR[0][i][0][1]))
         cv.circle(vis, (x_r, y_r), 7, (0,255,255), -1)
         slope = (y_l-y_r)/(x_r-x_l)
         slopes.append(slope)
         cv.line(vis, (x_l,y_l), (x_r,y_r), (0,255,255), 2)

     avg = sum(slopes)/len(slopes)
     cv.putText(vis, 'Average slope '+str(avg),(vis.shape[1]//3, (vis.shape[0]//5)*4), font, fontScale, (0, 255, 255), 2, cv.LINE_AA)
     cv.imshow('Rectification check - remapped images', vis)
    
    cv.waitKey(0)
    cv.destroyAllWindows()

This is the result所以在我看来,一个或两个图像的旋转有问题
当我分别对两个图像进行去扭曲并连接六个对应点时,我似乎得到了比校正过程更好的行对齐视图,因此这可能证明cameraMatrix、newCamMatrix和distCoeffs是好的?!
感谢您的帮助!!

编辑:This是另一种视觉表示,在两个对应点之间使用水平线而不是连接线。
编辑2:Prove that MATLAB detected all corners

Prove that openCV detected all corners

sxissh06

sxissh061#

你可能搞砸了旋转矩阵。我用my code做了一些测试,得到了同样的结果(达到了数值精度)。
结果表明,您可能在校准过程中反转了旋转矩阵。

Rot = np.array([[0.9999, 0.0109, 0.0068],[-0.0111, 0.9998, 0.0178],[-0.0066, -0.0179, 0.9998]])

其转置(=旋转的逆)为:

Rot = np.array([[0.9999, 0.0109, 0.0068],[-0.0111, 0.9998, 0.0178],[-0.0066, -0.0179, 0.9998]]).T

您将获得预期的结果。

相关问题