I have trained a CNN model in a DDP manner using 4 GPU’s, batch size of 16 per GPU, lr = 1e-5 and wd = 5e-5 and got satisfactory results.
Now I want to recreate the same training process and accuracy using a single GPU. How should I scale my hyperparemeters? That is, how should I change by batch size, lr and wd.