I’m facing an issue with my PyTorch model training on Windows 10 using Jupyter Notebook. I’ve written a Python script to train a LSTM model over 500 epochs. The training process sometimes uses 100% of the CPU, which is expected and desired, but other times it inexplicably drops to only ~33% usage, which significantly slows down the training.

Here are some details about my setup :

Operating System: Windows 11
IDE: Jupyter Notebook
Python Version: Python 3.12.4
PyTorch Version: 2.3.1
CPU: 12th Gen Intel(R) Core(TM) i7-12700

What I’ve Tried:

• Thread Configuration: I made sure to set the number of threads in PyTorch to the maximum available on my CPU:

import torch torch.set_num_threads(n) # Where n is the number of CPU cores

• Other Processes: Verified that no other resource-intensive processes are running concurrently.

Questions:

- What might be causing this intermittent drop in CPU usage?
- Are there additional configurations or steps I should consider to ensure consistent CPU utilization?
- Could this be an issue with Jupyter Notebook, and would running the script outside of Jupyter potentially help?

Any insights or suggestions would be greatly appreciated. Thank you!