I work with the interfaces of various programs on my computer, I find elements that I can interact with, for example buttons. At some point, I have accumulated a large number of buttons, and I need an optimal algorithm for finding them in this image.
I write in python, I tried cv2.matchTemplate – it just matches the template, and it seemed to me that this is not the optimal method, because I just have to go through the entire database, then I tried the methods of searching for key points: surf, sift, orb, their problem is that they do not always find key points for small buttons (and there are many of them in MS Word). I also tried to look for the contours of the images, and to match them, but I also could not come up with an optimal method.
The code example below cannot find the key points for the button.
enter image description here
enter image description here
img1 = cv2.imread( 'InputPhoto/6-2.jpg' , cv2.IMREAD_GRAYSCALE)
img2 = cv2.imread( "InputPhoto/6-1.jpg" , cv2.IMREAD_GRAYSCALE)
sift = cv2.SIFT_create()
keypoints1, descriptors1 = sift.detectAndCompute(img1, None )
keypoints2, descriptors2 = sift.detectAndCompute(img2, None )
bf = cv2.BFMatcher()
matches = bf.knnMatch(descriptors1, descriptors2, k= 2)
threshold = 0.55
good_matches = []
for m, n in matches:
if m.distance < threshold * n.distance:
good_matches.append(m)
img_matches = cv2.drawMatches(img1, keypoints1, img2, keypoints2, good_matches, None , flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)
plt.figure(figsize = ( 20 , 10 ))
plt.imshow(img_matches)
1