Before going straight to the issue, I’m a begginer at using LangChain, so I wanted to incorporate it in small problems on my daily-basis so I could get the hang of it. One of them, for example, is the task of summarizing a paper into bullet points so I could decide if it’s worth taking a look or not prior to reading. The way I did was pretty straight-forward and trying to follow the documentation (basic document retrieval and feedint into an LLM on my local machine thorugh ollama):

file_path = "paper.pdf"
loader = PyPDFLoader(file_path)

docs = loader.load()

embeddings = (OllamaEmbeddings(model='llama3'))

text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
splits = text_splitter.split_documents(docs)
vectorstore = Chroma.from_documents(documents=splits, embedding=embeddings, persist_directory="emb")

retriever = as_retriever(
    embeddings=embeddings,
    chroma=vectorstore
)

prompt_template = """Based on the following information and being really specific about it's data: '{text}'.nn Here are the goals, methodology, and conclusions/achievements of the paper, written as bullet points:"""

prompt = PromptTemplate.from_template(prompt_template)

llm = Ollama(model="llama3")

chain = (
    retriever
    | prompt
    | llm
)

result = chain.invoke({})

The thing is, the way I see there are a lot of way I see people perform summarization or question answering tasks through RAG, for example. Whether to call the llm by it’s default completion object or through it’s chat variation. Whether to use a LLMChain(), a RetrievalQA.from_chain_type() or a simple chain() specifying it’s common parameters. Or even to use a chain.invoke({}) passing no arguments without the need of workarounds. The thing is a lot of it’s use cases seem to be similar to one-another. It’s not a straight-forward question, so It shouldn’t have a straight-forward answer. In a nutshell, is there a way to know you’re following the right path, besides the logic of the RAG pipeline?

Similar approaches lead to the same output, but it sometimes feels like a lot of workarounds and tweaks are necessary to achieve the goal.