We are currently upgrading a Ruby on Rails web application deployed on Azure (as a web app), and we need to implement background processing for some of our tasks, specifically for processing large data sets asynchronously. After researching, we decided to use Azure Redis for managing background job queues due to its scalability and integration with Azure services.

However, we’ve encountered a persistent issue with Azure Redis where it frequently disconnects in Azure. It causes disruptions in our background processing tasks.

Microsoft seems to recommend WebJobs, but the processing is written in Ruby, and that is not a supported language for WebJobs.

What are the best practices for implementing background processing in Ruby on Rails applications deployed on Azure? Specifically for a Rails app running in Azure Web App.
How can we mitigate or resolve the frequent disconnection issues with Azure Redis in our setup?

Are there alternative solutions or technologies we should consider for background job management in Azure? We were looking to instead using solid_cache and solid_queue with Rails 7.1+ and leverage web-sockets connection to notify the user when the process is complete. Has anyone used this with success in Azure?

Any insights, suggestions, or experiences from fellow developers who have tackled similar challenges would be greatly appreciated. Thank you in advance for your help!