I don’t know if this is the correct place to ask this question but I’ll ask it here anyway.
I don’t know much about programming (I only ever learned MATLAB!) but I wonder if there is a simple way to write a code to check a website every x minutes, say, to see if a certain thing changes and notify me when it does?
I hope this question is not too vague!
Thanks in advance for your help.
EDIT: I have computer running Windows and one running Ubuntu.
5
It is too vague. But you basically need the following things, and glue them together:
- A thing that downloads the webpage and saves it to your computer
- A thing that compares it to the version you downloaded before this one
- A thing that notifies you about the comparison if a difference was found
- A thing that runs this whole contraption every x minutes
For the last one, if you are on Linux, then cron is good for running something every x minutes. The rest can be done with many different languages, like Python, but I’m not going to write it all out.
1
If you know MATLAB, then this should be the right way for you:
- Get the page: webread
- Search the page for the HTML pattern containing your data: strfind
- Extract the data
- Compare the data to the last one recorded
- If changed, notify, else wait until timer runs out and re-run the process
First tell us your operating system. 🙂 You could use all languages. You have to read the html sourcecode of a website (better only the body). Be careful if there are current timestamp and so on.
Another solution would be (if supported) reading the RSS feed.
2
To complete on RemcoGerlich answer from a Linux perspective:
-
download a page using a command like
wgetorcurlor a library like libcurl (which has many wrapping, including for Python, Guile, etc…) -
compare two files with
difforcmp(or code the simple loop reading them and comparing byte by byte) -
to notify you about changes: you could send email (e.g. programmatically with vmime, or
popenamailorsendmailcommand, or code amailcommand with a here document….); you could also be notified on the desktop e.g. with notify-send -
to do something periodically, a crontab(5) entry.
As I commented, it could be a bad idea (so don’t run that too often, at most every 30 minutes to avoid crushing the website). You might be interested by RSS. And doing that automatically might be against the rules of the website! And it might not work if the website is using cookies or sessions, etc…
BTW, some sites offer a so-called Web API which could be useful, e.g. because they are providing the info perhaps in JSON format. Read also about web services & Web Sockets & REST & robots.txt