I am using R for parallel computing on a machine with the following specs:
- 2 x AMD EPYC 7K62 CPUs
- 48 cores and 96 threads per CPU
- Total: 96 cores and 192 threads
I am trying to utilize all available threads using the following code:
no_cores <- detectCores() # 192
cl <- makeCluster(no_cores, outfile = '')
`
However, I encounter the following error:
Error in file(con, “w”) : all 128 connections are in use
When I reduce the ‘no_cores’ value to around 120, the code executes without any errors. This leads me to question whether there is a limit on the number of parallel threads that can be utilized on the Windows 10 platform.
Is it true that the maximum number of threads for parallel computing in R on Windows is around 128? If so, is there any way to fully utilize my CPU resources?
1