I am working on a project using the Langchain library to process and query information from a PDF document. My setup involves loading the PDF, splitting it into chunks, adding these chunks to a vector database, and then setting up a retriever and LLM (large language model) to answer questions based on the document’s content. However, when I try to invoke the chain asynchronously, it gets stuck at the await chain.invoke() step.

Problem Details:
I am using the following components from Langchain:

• UnstructuredPDFLoader to load the PDF.
• RecursiveCharacterTextSplitter to split the text into chunks.
• Chroma and OllamaEmbeddings to create a vector database from these chunks.
• ChatOllama to initialize the language model.
• MultiQueryRetriever to generate multiple versions of a user question.
• ChatPromptTemplate and PromptTemplate to set up the query prompt.

The code runs without errors up until I try to invoke the chain with a sample question. It prints all the setup completion messages but then gets stuck at the chain invocation.

What I Tried:

from langchain_community.document_loaders import UnstructuredPDFLoader
from langchain_community.embeddings import OllamaEmbeddings
from langchain.text_splitters import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import Chroma
from langchain.prompts import ChatPromptTemplate, PromptTemplate
from langchain.output_parsers import StrOutputParser
from langchain_community.chat_models import ChatOllama
from langchain.runnables import RunnablePassthrough
from langchain.retrievers.multi_query import MultiQueryRetriever
import asyncio

# Use raw string notation for the file path
local_path = r"C:/Users/User/zven/WEF_The_Global_Cooperation_Barometer_2024.pdf"

# Load local PDF file
try:
    loader = UnstructuredPDFLoader(file_path=local_path)
    data = loader.load()
    print("PDF loaded successfully.")
except Exception as e:
    print(f"Error loading PDF: {e}")
    data = None

if data:
    # Split and chunk text
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=7500, chunk_overlap=100)
    chunks = text_splitter.split_documents(data)
    print(f"Document split into {len(chunks)} chunks.")
    print(chunks[0].page_content)  # Print content of the first chunk to verify

    # Add to vector database
    try:
        vector_db = Chroma.from_documents(
            documents=chunks, 
            embedding=OllamaEmbeddings(model="nomic-embed-text", show_progress=True),
            collection_name="local-rag"
        )
        print("Chunks added to vector database.")
    except Exception as e:
        print(f"Error adding chunks to vector database: {e}")

    # Initialize LLM from Ollama
    local_model = "mistral"
    try:
        llm = ChatOllama(model=local_model)
        print("LLM initialized successfully.")
    except Exception as e:
        print(f"Error initializing LLM: {e}")

    # Define query prompt template
    QUERY_PROMPT = PromptTemplate(
        input_variables=["question"],
        template="""You are an AI language model assistant. Your task is to generate five
        different versions of the given user question to retrieve relevant documents from
        a vector database. By generating multiple perspectives on the user question, your
        goal is to help the user overcome some of the limitations of the distance-based
        similarity search. Provide these alternative questions separated by newlines.
        Original question: {question}""",
    )

    # Initialize retriever
    try:
        retriever = MultiQueryRetriever.from_llm(
            vector_db.as_retriever(), 
            llm,
            prompt=QUERY_PROMPT
        )
        print("Retriever initialized successfully.")
    except Exception as e:
        print(f"Error initializing retriever: {e}")

    # Define RAG prompt template
    template = """Answer the question based ONLY on the following context:
    {context}
    Question: {question}
    """
    prompt = ChatPromptTemplate.from_template(template)
    print("Prompt initialized.")

    chain = (
        {"context": retriever, "question": RunnablePassthrough()}
        | prompt
        | llm
        | StrOutputParser()
    )
    print("Chain setup completed.")

    async def run_chain():
        try:
            print("Invoking chain...")
            result = await chain.invoke({"question": "What are the 5 pillars of global cooperation?"})
            print("Chain invoked successfully.")
            print("Result:", result)
        except Exception as e:
            print(f"Error invoking chain: {e}")

    asyncio.run(run_chain())

Expected Outcome:
I expected the chain.invoke() to return an answer to the question based on the PDF content. However, it seems to hang indefinitely at this step.

The outcome after running:

OllamaEmbeddings: 100%|████████████████████████████████████████████████████████████████| 11/11 [10:08<00:00, 55.30s/it]
Chunks added to vector database.
LLM initialized successfully.
Retriever initialized successfully.
Template:  Answer the question based ONLY on the following context:
    {context}
    Question: {question}

Prompt:  input_variables=['context', 'question'] messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['context', 'question'], template='Answer the question based ONLY on the following context:n    {context}n    Question: {question}n    '))]
first={
  context: MultiQueryRetriever(retriever=VectorStoreRetriever(tags=['Chroma', 'OllamaEmbeddings'], vectorstore=<langchain_community.vectorstores.chroma.Chroma object at 0x00000210B48C0390>), llm_chain=LLMChain(prompt=PromptTemplate(input_variables=['question'], template='You are an AI language model assistant. Your task is to generate fiven        different versions of the given user question to retrieve relevant documents fromn        a vector database. By generating multiple perspectives on the user question, yourn        goal is to help the user overcome some of the limitations of the distance-basedn        similarity search. Provide these alternative questions separated by newlines.n        Original question: {question}'), llm=ChatOllama(model='mistral'), output_parser=LineListOutputParser())),
  question: RunnablePassthrough()
} middle=[ChatPromptTemplate(input_variables=['context', 'question'], messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['context', 'question'], template='Answer the question based ONLY on the following context:n    {context}n    Question: {question}n    '))]), ChatOllama(model='mistral')] last=StrOutputParser()
Chain setup completed.
Invoking chain...