PrivateGpt application can successfully be launched with mistral version of llama model. The environment being used is Windows 11 IOT VM and application is being launched within a conda venv.

It can be seen that in the yaml settings that different ollama models can be used by changing the api_base.

ollama:
  # llm_model: mistral
  llm_model: llama3
  embedding_model: nomic-embed-text
  api_base: http://localhost:11434
  embedding_api_base: http://localhost:11434  # change if your embedding model runs on another ollama
  keep_alive: 5m

Changing the name seems to be only change the name in the frontend gui. I have pulled llama3 using ollama pull llama3, this is confirmed to work as checking `~/.ollama/models’ contains both mistral and llama3.

Running ollama serve -h only shows that there are no flags but environment variables that can be set, particularly the port variable, but when it comes to models, it seems to only be the path to the models directory.

I was wondering if there is a way to specify the launching of different llama models on different ports so I can swap between them in privateGPT application.

Changing the model in ollama settings file only appears to change the name that it shows on the gui. As when the model was asked, it was mistral. Additionally to running multiple models (on separate instances), is there any way else to confirm that the model swapped is successful?