I am using Azure AI Search for storage of a RAG solution. Almost everything works fine, except for a border case regarding a line break. I have loaded (using KernelMemory) a simple table which consists of a column of codes (numbers) and a column of the corresponding text.
If I search for the text, everything works fine and the RAG brings the desired results. But if I search for the SAME CODE that was provided in the search by term, the result is empty.
We made a lot of troubleshooting in Azure Search Explorer. If we search for the code, NO MATCH. If we prepend “n” to the code, the results are as expected.
In Search Explorer, the text in the Index looks like this: “…L.)n50009450 CAPACITOR 150uF,16V(TANTAL.)n50009451 CAPACITORS ELEC.100uF/63V.n50009452 CAPACITOR 22…”. As you can see, there is a line break just before the codes.
For example, in Azure Search Explorer, searching the term 50009451 brings no results. But if we search for n50009451, we do get the expected results.
I think trying to bring this “workaround” to the RAG solution, is not an option as we mostly use vectorial search. We do use Hybrid Search, but Azure AI Search should be able to find the term exactly as entered, not with a “hack.
Am I missing something? Any ideas will be GREATLY appreciated.