getting the same transformation matrix, matplotlib vs pytorch3d

I’ve been working on a problem where numerous 2d projections of a 3d plot are created based on spherical coordinates around the 3d plot. I have been using matplotlib to successfully generate these 2d projections. Now I would like to switch from matplotlib to pytorch3d to generate the exact same 2d projections, as with the pytorch3d library it is possible to use the functions for backpropagation.

I have the following matplotlib code with a minimal example:

    
import matplotlib.pyplot as plt
import numpy as np
    
# random coords
coords_3d = np.random.rand(10, 3)
# viewpoint where viewpoint[0] is elevation, viewpoint[1] is azimuth and viewpoint[2] is distance
viewpoints_spherical = np.array([ -0., -90.,   1.])

print(coords_3d)
print(viewpoints_spherical)

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

ax.scatter(coords_3d[:, 0], coords_3d[:, 1], coords_3d[:, 2], s = 10)

ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')

# ensure equal aspect ratio:
max = 1
ax.set_xlim(-max, max)
ax.set_ylim(-max, max)
ax.set_zlim(-max, max)

# rotate the axes and update
ax.set_box_aspect((4, 4, 4), zoom=1.2)

# set the elevation and azimuth
ax.view_init(viewpoints_spherical[0], viewpoints_spherical[1])
# get the transformation matrix
TM = np.linalg.inv(ax.get_proj())
print(TM)

The transformation matrix I receive here has the following values:

[[ 1.57792186e+00  0.00000000e+00 -1.05110711e-16 -1.05110711e-16]
 [ 1.05110711e-16  0.00000000e+00  1.57792186e+00  1.57792186e+00]
 [ 0.00000000e+00  1.57792186e+00  0.00000000e+00  0.00000000e+00]
 [-0.00000000e+00 -0.00000000e+00 -1.00000000e-01 -0.00000000e+00]]

Now, I would like to acquire the exact same transformation matrix using the pytorch3d library.

from pytorch3d.renderer import (look_at_view_transform, get_world_to_view_transform,)
distance = 1
elevation = -0
azimuth = -90
device = 'cpu'

R, T = look_at_view_transform(distance, elevation, azimuth, device=device)
res = get_world_to_view_transform(R, T).get_matrix()[0]
print(res)

The transformation matrix I get with pytorch3d has the following values:

tensor([[ 4.3711e-08,  0.0000e+00,  1.0000e+00,  0.0000e+00],
        [ 0.0000e+00,  1.0000e+00,  0.0000e+00,  0.0000e+00],
        [-1.0000e+00,  0.0000e+00,  4.3711e-08,  0.0000e+00],
        [ 0.0000e+00,  0.0000e+00,  1.0000e+00,  1.0000e+00]])

There is a difference between these projection matrices which will lead to different 2d projections from the original 3d coordinates.

I have been inspecting the get_proj() function from matplotlib (https://github.com/matplotlib/matplotlib/blob/v3.8.0/lib/mpl_toolkits/mplot3d/axes3d.py#L896-L958)
and I have replicated each step within the function. It seems that here the projection matrix computation takes into account the box aspect. I suspect that this may lead to some differences between the get_proj() function from matplotlib and the functions from pytorch3d. The problem here is that I don’t want to change the code written using matplotlib, as this code has been used to generate the projection matrices already. I am simply looking for a way to replicate the transformation matrix using pytorch3d.

I could also be mistaken and that I have been misusing/misunderstanding the pytorch3d functions look_at_view_transform() and get_world_to_view_transform(). (for instance the ‘at’ and ‘up’ parameters)

Any help is appreciated.