There are numbers of libraries about human pose estimation. I want to use one of them and add my points to the predict result, being drawn with the results in realtime on the image/camera.
For example, on the outside of my forearm, near the wrist, there is a point which is 1/3 of the distance from the wrist to the elbow. I want it be showed on the image (there will be many such points).
I have studied several libraries such as SMPL and FrankMocap(skinning supported), and MMPose, MediaPipe, OpenPose(skeleton only). However, I didn’t figured out how to achieve my goal. The points I want to add don’t exist in reality, so training my own model is impracticable. MediaPipe is the easiest, and I successfully used it to draw additional points. However, because it’s not fully 3D, when the inside of the arm turns towards the camera, the outside point which should be hidden is still drawn.
I guess there are two ways to achieve my goal:
-
Before image processing, edit the model and directly add the points to the model, so that they can be output together with the original bones or skin.
-
After processing, based on the vertices of the results, calculate the positions of additional points and draw them one by one
I think 1 should be better, as I need only to manage fixed positions of the points rather than facing a lot of troublesome position and occlusion calculations. But the fact is I have no clue about either method, even ChatGPT couldn’t help me.
The positions of the points need not to be very accurate, as long as they are relatively correct overall. And, in fact, I only want to display my points, the original skin and bones need not to be showed.
Hope someone can give me an example of adding at least one point.