Problem Description

My JSON data looks like the following (3 entries shown), but there are actually thousands of entries.

I hope that when the user inputs a keyword, they can search relevant data. For example, when they input “automotive IC,” they can find data related to automotive IC.

The difficulty here is that, for example, in the last entry below, the progress_description mentions “autonomous driving,” which is related to automotive IC. What I expect is to find this entry as well. However, using embedding methods, the similarity scores are not high enough, so they do not appear in the search results.

{
    "INDUSTRY_STD_NAME": "O-Other",
    "INDUSTRY_NAME": "J-Other",
    "OPPORTUNITY_ID": 301301.0,
    "PROGRESS_DESCRIPTION": "MIS ENDÜSTRİYEL ÇÖZÜMLER OTOMASYON SANAYİ VE DIŞ TİCARET ANONİM ŞİRKETİ shows a clear interest in ordering USB cables and has arranged for payment and shipping. The first meeting discussed the order details and contacting the courier company account, and the second meeting confirmed the need to view product pictures before remittance and shipment. The customer stated that they are an authorized supplier of Toyota, indicating a potentially stable relationship. There are no obvious objections or refusals."
},

{
    "INDUSTRY_STD_NAME": "E-Electronics Industry",
    "INDUSTRY_NAME": "A-Assembly/Plug-in/Fixation",
    "OPPORTUNITY_ID": 301234.0,
    "PROGRESS_DESCRIPTION": "During the meeting, it was mentioned that the 3D Scanner previously purchased by the customer was not used and there is a need for software updates and inspections for AccuPick, while expressing a demand for SolVision to detect automotive IC flaws. The customer did not make a clear demand for the upcoming product but hinted at attention to the current solution. The customer showed interest in the specific application and software update of the product, but the demand was not clear, implying potential future cooperation."
}

{
    "INDUSTRY_STD_NAME": "General Industrial Machinery Manufacturing",
    "INDUSTRY_NAME": "Machine Tools",
    "OPPORTUNITY_ID": 275220.0,
    "PROGRESS_DESCRIPTION": "The focus of this business meeting was on the distribution rights of META-aivi. Faurecia showed satisfaction with the R&D technology of the FORVIA group, including intelligent autonomous driving and sustainable mobility solutions to address future challenges in the automotive industry. However, the meeting minutes did not show any clear signals of transaction opportunities, nor did they mention the customer's specific interest or purchase intention for META-aivi. Follow-up actions were planned as phone contact and email communication, indicating that at the end of this meeting, the case was still in the early exploration stage."
}

Python Code

Below is my code. I tried replacing different embedding models and prompts but could not get the desired results.

from config import OPENAI_API_KEY
from langchain.embeddings import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import FAISS
from langchain_community.document_loaders import JSONLoader
from openai import OpenAI

client = OpenAI(api_key=OPENAI_API_KEY)

def metadata_func(record: dict, metadata: dict) -> dict:
    metadata["OPPORTUNITY_ID"] = record.get("OPPORTUNITY_ID")
    metadata["OPPORTUNITY"] = record.get("OPPORTUNITY")
    metadata["INDUSTRY_STD_NAME"] = record.get("INDUSTRY_STD_NAME")

    return metadata

loader = JSONLoader(
    file_path='test.json',
    content_key='PROGRESS_DESCRIPTION',
    metadata_func=metadata_func,
)
documents = loader.load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=100)
documents = text_splitter.split_documents(documents)

embeddings = OpenAIEmbeddings(api_key=OPENAI_API_KEY)
vectorstore = FAISS.from_documents(documents, embeddings)

query = (
    "Use your knowledge base to best respond to questions. "
    f"Based on the provided keywords, search for all relevant data in the 'INDUSTRY_NAME', 'OPPORTUNITY', and 'PROGRESS_DESCRIPTION' field descriptions from the business report data, and return the values of the 'OPPORTUNITY_ID' field. Here are the keyword: "
)
keyword = 'automotive IC'
docs = vectorstore.similarity_search_with_score(query+keyword, k=5)
print(docs)

Question

1. Are there any other optimizations I can make?
2. Is this scenario not suitable for the RAG solution?
3. Does adjusting the prompt (query) affect the search results? My understanding is that the vector similarity calculation occurs between the texts themselves, so the query in the code will not affect the search results and might even cause poor results. Is my understanding correct?

Hope someone can give some suggestion, thanks!