This is an architectural question.Is it okay use AI search for storing chunks of 120000 pdf documents. I want to use hybrid search with documents chunks and use them with a RAG pipeline. Also I have the option of getting summary of each chunks and storing on AI search vector store.
In current method I’m getting pdf chunks using textsplitter from langchian.
But I have to deal with images as well. Are there any other suggested methods.