I am developing an application which uses Ctransformers library from langchain framework to instantiate the LLM and then using it for deriving queries from the database. I tried using a local gguf model without passing the model parameter to the Ctransformers library which needs to know the HuggingFace repository and it doesn’t work.
Strangely, similar implementation with llama cpp python gives me absurd results using the same LLM so my option is to go ahead with Ctransformers library
I wanted to know whether it is suitable to use sensitive data on Ctransformers library and whether it exposes the database data through HF API.

Tried using code llama gguf llm
with langchain Ctransformers and llama cpp python implementation and got accurate results with Ctransformers.