I have a pinhole camera with known position, rotation, and horizontal/vertical FOV. A sphere with known position is placed so that it is visible in the camera’s FOV. My goal is to find the angular offset of the sphere’s centroid from the center of the image.

Here is an example of an image captured from the camera.

With this information:

• Camera position: (150, 0, 100)
• Camera rotation (deg): (120, 115) using physics convention spherical coordinates
• Sphere position: (50, 140, 8)
• Camera HFOV: 70.006
• Camera VFOV: 43
• Image dimensions: 1920×1080

I first find the sphere’s centroid with basic image masking. Then, I use some the same equations from this post to calculate the horizontal and vertical angular offset from the image center. When graphing the angular offset vector, I expect it to intersect the sphere. However, there is an error that seems to increase as I get further away from the image center.

Here is a visualization of the error.

• The blue point represents the camera.
• The blue vector is the camera’s direction vector which passes through the image center
• The gray point is the green sphere.
• The purple vector is the calculated angular offset from the image center.

The purple vector does not pass through the gray point as expected. I have ruled out lens distortion (not present in a pinhole camera model) and perspective distortion (very minimal).