this isnt really a coding question but a conceptual one

i want to automate a few tasks in a game im playing (mundane stuff i really dont want to keep doing over and over again). which involves pressing buttons and stuff.

the main issue im facing is how to get the location of these buttons so i can move my mouse over to press them.

i tried using OCR but tesseract is absolutely horrible at OCR if the text has effects, different font, backgrounds etc. i also tried easyocr but no luck as i dont have a strong enough gpu and it’s just too slow.

next idea was to store buttons in images and try finding them on the screen and then pressing them but this comes with the issue that sometimes some stuff is randomly generated so i cant do that (also thought about a color based thing but again, randomly generated sometimes and sometimes a lot of buttons/dropdowns have the same color).

i tried this in python but i know other languages too.