I am using this example as is
and trying to read the the Json, using JSONLoader
https://python.langchain.com/v0.1/docs/use_cases/question_answering/chat_history/
It works quite well. reading the json, maintaingin the history , But For example i have 20 json object in my json file (int the trips[] array, with title”s and Id’s and other information.
If i ask the llm, How many trips we have , It says “we have 4 trips”
and sometimes its giving different trip object , some time different , but not having more than 4.
Is there a limitation on the size of the Json file , when we using JsonLoader
like this :
loader = JSONLoader(file_path=”experiencesFullBk.json”, jq_schema=”.trips[]”, content_key=”.attributes.title”,
is_content_key_jq_parsable=True)
docs = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
splits = text_splitter.split_documents(docs)
vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())
retriever = vectorstore.as_retriever()
If i print splits, I see in console all the trips.