I’m using GPUs for the first time (hosted on a server) and was starting to work with CuPy. Everything was working great and then randomly, I stopped being able to perform matrix multiplication with CuPy, and I would get the following error:

RuntimeError: CuPy failed to load libnvrtc.so.12: 
OSError: libnvrtc.so.12: cannot open shared object file: No such file or directory

I tried uninstalling and reinstalling CuPy but now I can’t seem to perform any operations. For example, when I try to generate a random matrix:

>>> import cupy as cp
>>> cp.random.randn(5000)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/users/fdeguire/.local/lib/python3.9/site-packages/cupy/random/_sample.py", line 84, in randn
    return _distributions.normal(size=size, dtype=dtype)
  File "/home/users/fdeguire/.local/lib/python3.9/site-packages/cupy/random/_distributions.py", line 500, in normal
    rs = _generator.get_random_state()
  File "/home/users/fdeguire/.local/lib/python3.9/site-packages/cupy/random/_generator.py", line 1306, in get_random_state
    rs = RandomState(seed)
  File "/home/users/fdeguire/.local/lib/python3.9/site-packages/cupy/random/_generator.py", line 56, in __init__
    from cupy_backends.cuda.libs import curand
ImportError: libcurand.so.10: cannot open shared object file: No such file or directory
>>> exit()
Exception ignored in: <function RandomState.__del__ at 0x7f94a2b4ee50>
Traceback (most recent call last):
  File "/home/users/fdeguire/.local/lib/python3.9/site-packages/cupy/random/_generator.py", line 65, in __del__
    from cupy_backends.cuda.libs import curand
ImportError: libcurand.so.10: cannot open shared object file: No such file or directory

The strange thing is that I am still able to run some CuPy commands (such as importing or converting from NumPy arrays), but I can’t run many basic operations due to an OSError or ImportError related to some missing lib file.

I am also able to run PyTorch operations without issue.

For reference, here is my CuPy config:

>>> cp.show_config()
OS                           : Linux-3.10.0-1160.108.1.el7.x86_64-x86_64-with-glibc2.17
Python Version               : 3.9.0
CuPy Version                 : 13.1.0
CuPy Platform                : NVIDIA CUDA
NumPy Version                : 1.26.4
SciPy Version                : None
Cython Build Version         : 0.29.36
Cython Runtime Version       : None
CUDA Root                    : None
nvcc PATH                    : None
CUDA Build Version           : 12040
CUDA Driver Version          : 12030
CUDA Runtime Version         : 12040 (linked to CuPy) / RuntimeError('CuPy failed to load libcudart.so.12: OSError: libcudart.so.12: cannot open shared object file: No such file or directory') (locally installed)
cuBLAS Version               : (available)
cuFFT Version                : None
cuRAND Version               : ImportError('libcurand.so.10: cannot open shared object file: No such file or directory')
cuSOLVER Version             : ImportError('libcusolver.so.11: cannot open shared object file: No such file or directory')
cuSPARSE Version             : (available)
NVRTC Version                : RuntimeError('CuPy failed to load libnvrtc.so.12: OSError: libnvrtc.so.12: cannot open shared object file: No such file or directory')
Thrust Version               : 200200
CUB Build Version            : 200200
Jitify Build Version         : None
cuDNN Build Version          : (not loaded; try `import cupy.cuda.cudnn` first)
cuDNN Version                : (not loaded; try `import cupy.cuda.cudnn` first)
NCCL Build Version           : None
NCCL Runtime Version         : None
cuTENSOR Version             : None
cuSPARSELt Build Version     : None
Device 0 Name                : NVIDIA A30 MIG 1g.6gb
Device 0 Compute Capability  : 80
Device 0 PCI Bus ID          : 0000:06:00.0