As indicated in my question, I would like to index a lot of documents (110,000) that I have uploaded to create a rag with LlamaIndex:
documents = SimpleDirectoryReader(‘data’,required_exts=required_exts,recursive=True).load_data()
But once I’ve indexed them, it obviously takes too long… and I can’t find any functions in the documentation to help me:
index = VectorStoreIndex.from_documents(documents,embed_model=embed_model,show_progress=True)
Do you have an idea?
1