I have deployed Milvus on GKE using the Milvus Operator with the image milvusdb/milvus:v2.4.4-gpu. My setup includes two node pools: one with 4 e2-highmem-8 nodes and another with 2 n1-highmem-8 nodes equipped with T4 GPUs (2 per node). My Milvus configuration specifies one replica each for indexNode and queryNode with defined CPU, GPU, and memory limits.
After ingesting the Cohere 10M Search Performance Dataset, I encounter errors while trying to load the collection. The error messages indicate failure in deserializing the index due to a ‘raft inner error’. Is this issue related to the Milvus Operator’s inability to configure initMemSize and maxMemSize for GPU memory control? Where can I specify these configurations?
Eureka is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.