I am involved in a project for which it is required to measure amount of the food in photograph. Especially, I would like to capture an image of a dish, e. g. cup of tea or let’s say biryani and get the approximate occupied volume of the teal liquid in milliliters (ml) and biryani in gm. From this perspective, I understand that there must be difficulties such as perspective and the use of a reference object to address the scale issue.

At a fundamental level, I know about image processing and Computer Vision in Python and have used OpenCV and scikit-image. But, what kind of approach should I take when it comes to volume estimation from an image I do not know?

Here are the steps I am considering:Here are the steps I am considering:

For example, the “object” to identify and segment would be the cup that will appear in the images at a later stage.

In case there is no scale provided on the image, find a reference object that will help in determining its scale.

Let them put the cracker in the cup and try to estimate the dimensions of the cup and the level of liquid in the cup.

Take the given length, height and width and add the liquid level if the tank storage capacity includes a liquid component.

Can anybody explain me how to solve these steps or provide me some particular methods or heuristics to be applied for the solution? Any further kindly, sample code snippets or detailed explanation of the necessary calculations would go a long way in solving this issue.

import cv2
import numpy as np

# Load image
image = cv2.imread('cup_of_tea.jpg')

# Convert to grayscale
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Apply edge detection
edges = cv2.Canny(gray, 50, 150)

# Find contours
contours, _ = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

# Draw contours
cv2.drawContours(image, contours, -1, (0, 255, 0), 2)

# Display the result
cv2.imshow('Contours', image)
cv2.waitKey(0)
cv2.destroyAllWindows()

This code helps me to detect the contours of the cup, but I am unsure how to proceed with the volume estimation. Any guidance on the next steps or improvements to my approach would be very helpful.