I am using optuna for hyperparameter tuning and even though my device is set to “cuda” and it is in fact running on cuda, as it takes 40 minutes in CPU to complete 10 epochs, meanwhile, currently, it is taking only 6 minutes to complete 30 epochs. This means, my program is using GPU. However, I have inspected that the CPU is being stressed to 100% and my GPU is barely being utilized by the program.

(https://i.sstatic.net/MBGxG7rp.jpg)

Here is my specifications of hardware and software:

Hardware specifications:
Lenovo Legion 5 2022
Ryzen 7 6800H
NVIDIA RTX 3060 TDP 140W 
16 GB DDR5 RAM 4800 Mhz
1TB PCIE Gen 4 SSD
Optimus disabled (NVIDIA Dgpu only)

Software specifications:
python 3.10.9 
conda 23.3.1
optuna 3.6.0 conda-forge
optuna-dashboard 0.15.1 conda-forge

I could not figure out anything to work with this problem.