I am building a document QA system, under which I have a config.yml as below
Config.yml
- RETURN_SOURCE_DOCUMENTS: True
- VECTOR_COUNT: 2
- CHUNK_SIZE: 500
- CHUNK_OVERLAP: 50
- DATA_PATH: ‘data/’
- DB_FAISS_PATH: ‘vectorstore/db_faiss’
- MODEL_TYPE: ‘llama’
- MODEL_BIN_PATH: ‘modelsMeta-Llama-3-8B-Instruct.Q6_K.gguf’
- MAX_NEW_TOKENS: 256
- TEMPERATURE: 0.0
But when I run the code, I am unable to load the model and getting the below error
ERROR: byte not found in vocab: ‘
‘
CUDA error 9 at D:actransformersctransformersmodelsggmlggml-cuda.cu:6045: invalid configuration argument