I have my own designed symmetrical object (shaft) and want to get the Realtime video pose estimation and tracking to get some information to guide me about the current location and the objective or target location where I need to direct or move the object to.

I have intel RealSense camera D435 and L515. I was working on developing the Yolov5, to get the four corners point of the minimum bounding box and using the instance segmentation the video shows the corners x and y data in realvideo and update when move the object and using the point cloud also. But that also, is not working well.

Is there any other method or ways to develop it?

I have tried the Yolov5, instance segmentation and used the depth and mask images of my object and used the pcd for the object. I expected to see a flexible realtime video update when move the object and can get the pose estimation and track the object correctly without problem.