I can generate scores for FAISS nearest neighbours, but getting the inner product doesn’t seem to work with regards to normalizing. I can only assume that the normalization doesn’t work.
faiss.normalize_L2(np.array(embedding_c["embeddings"]).astype(dtype=np.float32))
embedding_c.add_faiss_index(column="embeddings", metric_type=faiss.METRIC_INNER_PRODUCT)
scores, samples = embedding_c.get_nearest_examples(
"embeddings", np_embedding, k=10
)
array([41.96955 , 24.942688, 23.119305, 22.931778, 22.90448 , 22.66772 ,
22.657745, 22.409449, 22.363243, 22.353025], dtype=float32)
The scores output doesn’t end up being normalized.