I am using stereoRectifyUncalibrated() method which generates two homography matrices H1 and H2 for image rectification. I have checked both H1 and H2 by displaying the results of the rectified images and seems good. However, I now need to compute the new rectified P1 and P2 matrices and thereafter the new Q matrix for 3D projection.
Here comes the issue: the rectification matrices R1 and R2 derived from R = invK * H * K is not a true rotation (R.T *R != Identity AND det R =1)
I am not able to compute the new projection centres pertaining to the Q matrix based on the computed homographies.

To put it very simple, how to compute the new projection matrices P1, P2 and Q given camera intrinsics K1 and K2 , homographies H1, H2 computed above? I do also have R|t , the rototransformation from the stereo setup. The idea is to absolutely use uncalibrated stereo and NOT the stereoRectify function.

enter image description here