I am looking to run a local LLM (Large Language Model) on an Nvidia Jetson AGX Orin over the GPU CUDA Cores . Could anyone provide guidance or share resources on how to achieve this?
Thank you in advance for your help!
I was able to run a local LLM (.gguf model) over the CPU
but unable to utilize the GPU.
New contributor
Mausam Jain is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1