I’m working on a personal project to compare model performances between OLS model and SGDRessor model. The OLS models are not perfect, but work fine. The SGDR model predictions are way off. I checked the cost reduction over iterations. The results suggest that the learning rate is too high. However, lowering learning rate seems to worsen the cost reduction.

I’m new to machine learning andI’m lost in terms of how to improve the SGDR model. Any suggestion will be appreciated.

Here is my code: Google Colab link. You may focus on the sections “Gradient Descent Approach” and “Troubleshoot SGDR”.

Thank you in advance for your time and help!