I’m trying to get tensorrt working with Tensorflow 2.17, yet after trying all of the official and unofficial instructions several times I still can’t get it to work and I’m at the edge of sanity.
I’ve have installed CUDA 12.5 (as that’s reported as the last version supported by tensorrt)
$ sudo apt-get install tensorrt libnvinfer10 [16:24:34]
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libnvinfer10 is already the newest version (10.2.0.19-1+cuda12.5).
libnvinfer10 set to manually installed.
tensorrt is already the newest version (10.2.0.19-1+cuda12.5).
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
$ pip install -U tensorrt [16:24:26]
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: tensorrt in ./miniconda3/envs/tf/lib/python3.12/site-packages (10.2.0.post1)
Requirement already satisfied: tensorrt-cu12==10.2.0.post1 in ./miniconda3/envs/tf/lib/python3.12/site-packages (from tensorrt) (10.2.0.post1)
$ nvidia-smi [16:26:49]
Tue Aug 6 16:26:53 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.147.05 Driver Version: 560.76 CUDA Version: 12.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:01:00.0 On | Off |
| 0% 39C P8 15W / 450W | 1514MiB / 24564MiB | 1% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+
$ python -c "import tensorflow as tf; print(tf.__version__)" [15:31:54]
2024-08-06 16:22:46.458404: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-08-06 16:22:46.596055: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-08-06 16:22:46.644047: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-08-06 16:22:46.662195: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-08-06 16:22:46.758216: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-08-06 16:22:47.524911: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2.17.0
$ ls /usr/lib/tensorrt/bin [16:24:49]
ls: cannot access '/usr/lib/tensorrt/bin': No such file or directory
No matter what installation method I try, the /usr/lib/tensorrt/bin path is always missing.
Running Ubuntu 22.04 on WSL2, Windows 11 host.
Any help appreciated.