I have scheduled cron which runs every 15 minutes and query all active subscribers with active plan and hits the usage fetch api for each subscriber and update the usage in the database. all this activity run by a single thread so that every cron run took 45 to 70 minutes which is very high time and subscriber is unable to see the real time usage immediately. Once cron is running and another run waits for the previous run to finish.
Can you advise any better way to handle this problem to decrease the cron time and cron run has to be finished in 15 minutes for any number of subscribers?
Shiridi sai Golakoti is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Can you parallelize your processing?
You don’t really give enough information for anyone to advise. Let me guess, though, each “subscriber” could be processed in parallel, so you would do something like this (pseudo code):
ExecutorService executorService = Executors.newFixedThreadPool(10)
subscriberUsageUpdater() {
read subscribers
for each subscriber:
executorService.submit {
process subscriber // hits the usage fetch api for each subscriber and update the usage in the database
}
}