I want to do LLaVA inference in ollama, so I need to convert it in gguf file format.
My model has the file format safetensors.(trained with lora)
It seems that ollama supports only llama, but not llava as shown here,
https://github.com/ollama/ollama/blob/main/docs/import.md

I followed the instruction of llama.cpp, and used the code convert_lora_to_gguf.py here,
https://github.com/ggerganov/llama.cpp/blob/master/convert_lora_to_gguf.py

But I get an error like,

ERROR:lora-to-gguf:Model LlavaLlamaForCausalLM is not supported

If I write llama model in config.json of model file and run following code, then I got another error.

model_instance.gguf_writer.add_string(gguf.Keys.General.TYPE, gguf.GGUFType.ADAPTER)
model_instance.gguf_writer.add_string(gguf.Keys.Adapter.TYPE, "lora")
model_instance.gguf_writer.add_float32(gguf.Keys.Adapter.LORA_ALPHA, float(alpha))
model_instance.gguf_writer.add_quantization_version(gguf.GGML_QUANT_VERSION)
logger.info("Exporting model...")
model_instance.write()
logger.info(f"Model successfully exported to {model_instance.fname_out}")

Traceback (most recent call last):
  File "C:Usersjjjy2OneDriveDesktopVLM_FastAPIollamaconvert_lora_to_gguf.py", line 373, in <module>
    model_instance.gguf_writer.add_string(gguf.Keys.General.FILE_TYPE, gguf.GGUFType.ADAPTER)
AttributeError: module 'gguf' has no attribute 'GGUFType'

It seems that all codes and gguf package don’t support llava, but llama only. I have to convert my own trained model into gguf. I cannot use gguf llava model from hugging face for inference.

Is there a way to convert it?