I have with 10 million embeddings which are 1536 dims long. I’m fairly confident this means I need less than 128 GB of memory. I’m running Milvus using EKS on AWS and have 2 m6i.4xlarge machines which have 64 GB of memory each, meaning I should have plenty of memory.

Now, as I understand it, a collection must be partitioned such that it can be loaded onto multiple machines at once (https://milvus.io/docs/manage-partitions.md). Therefore, I introduced a partition key (https://milvus.io/docs/use-partition-key.md):

fields = [
            FieldSchema(name="id", dtype=DataType.VARCHAR, max_length=24, is_primary=True),
            FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=VECTOR_DIMENSION),
            FieldSchema(name="document", dtype=DataType.VARCHAR, max_length=30000),
        ]

        schema = CollectionSchema(
                fields,
                description="Vector store collection for arxiv",
                partition_key_field="document",
                n_partitions=16
        )

        milvus_collection = Collection(name=MILVUS_COLLECTION_NAME, schema=schema)

Then I added entries and indexed

index_params = {
        "index_type": "IVF_FLAT",
        "metric_type": "L2",
        "params": {"nlist": 1024}
    }
    milvus_collection.create_index(field_name="embedding", index_params=index_params)

and tried to load the collection:

milvus_collection.load()

However, it only makes it to 38% when I get the following error:

pymilvus.exceptions.MilvusException: <MilvusException: (code=65535, message=show collection failed: load segment failed, OOM if load, maxSegmentSize = 1576.6993761062622 MB, memUsage = 55411.66015625 MB, predictMemUsage = 56988.35953235626 MB, totalMem = 63273.46875 MB thresholdFactor = 0.900000)>

Why am I only able to use one machines RAM? Is there a way to check where each partition is stored and how big it is? Trieed looking on github but couldn’t find anything.