I’m developing a FastAPI application that uses Langchain with Ollama. The application is containerized using Docker, and I’m trying to connect to a separate Ollama container. However, I’m encountering a connection refused error when attempting to use the Ollama service.

I have already setup the Ollama in my docker container.

Here’s my setup: docker-compose.development.yml

services:
  web:
    build: .
    command: uvicorn main:app --host 0.0.0.0 --port 8000 --reload
    container_name: my-project-api
    ports:
      - "8000:8000"
    volumes:
      - .:/app
    env_file:
      - .env.development
    networks:
      - ollama-network
    external_links:
      - ollama:ollama

networks:
  ollama-network:
    external: true

In my FastAPI code, I’m using Langchain with Ollama like this:

from langchain_ollama.llms import OllamaLLM
from langchain_core.prompts import ChatPromptTemplate

ollama = OllamaLLM(model="llama2:7b")

template = """{question}"""
prompt = ChatPromptTemplate.from_template(template)
question_variation_chain = prompt | ollama

@router.get("/ask")
async def ask(question: str):
    try:
        result = question_variation_chain.invoke(input={"question": question})
        return result
    except Exception as e:
        logging.error(f"Exception in analyze_question: {e}", exc_info=True)
        return f"An error occurred while processing your request. Please try again later."

When I try to make a request to the /ask endpoint, I get the following error:

httpx.ConnectError: [Errno 111] Connection refused

The full error traceback shows that the error occurs when trying to generate a response from Ollama.
What am I missing in my Docker setup or Langchain configuration to properly connect my FastAPI container to the Ollama service? How can I resolve this connection refused error?