I am working on a project for a publishing house that involves implementing semantic search across an archive of approximately 50,000 articles, each averaging 15 pages in length. My understanding is that I need to use a Retrieval Augmented Generation (RAG) approach to achieve this.
Here are my specific requirements:

I am unable to perform semantic search using SelfQuery Retriever. This performs filters effectively but when it comes to semantic search it get fails to give correct responses. How could i improve the working of this retrieval.