I got a large amount of images that I’ve collected over the years and I wanted to clean out the duplicates that have accumulated.
Most can be easily found by looking for visual similarity, as the differences might be just compression artefacts due to different JPEG levels.
But I also have a portion that were flipped and or rotated by 90 / 180 / 270 deg, and I currently don’t know how to easily find those.
One approach I tried was collecting histogram “fingerprints” of all images and then brute force compare all permutations of flipping and 90 deg rotations with matching pairs.
But then I ran into the problem that the various histogram algorithms CV2 offers, had varying false-positives and false-negatives of very similar images.
It would still be an option to reduce some duplicates, but obviously not ideal.
But it would not solve the remaining problem with cropped images. At this point I thought that it might be better to go for image recognition.
So I was wondering what I should look into for this. I took a look at trying out Milvus for this, but I’ve never set up something like that, so I didn’t get very far yet.
Which is why I’m trying to see if I’m even looking at the right option here, keeping in mind the requirements I have:
- it should be something that’s open source or otherwise without an upfront cost, this is just a “hobby” project after all
- it should have an interface with Python, whenever that’s 1st or 3rd party doesn’t matter to me
- it does not need to understand what it’s seeing in an image, only the following:
(“other image” meaning an image in it’s database)- “this image looks like a part of that other image”
- “this image looks like a flipped/rotated version of that other image”
I understand that this obviously requires that the recognition algorithm needs to be trained first, but how to do that so I get the results listed above is also something I need guidance for.
As in, how do I prepare it’s training images in the right way so it can learn the patterns instead of the exact images.
1