I was trying to perform data ingestion with my fake dataset to Elastic Search . My fake dataset contains a lot of 0 vectors (0,0, … , ) and got the error about ingesting zero vectors.
{'errors': True, 'took': 64, 'items': [{'create': {'_index': 'bdp-cus-feature-store-1.1-vector-p1', '_id': '5aeefc2e656b6f77c60af0d90f1280b52e298441d29a80c69321155aa9700438', 'status': 400, 'error': {'type': 'document_parsing_exception', 'reason': '[1:3931] failed to parse: The [cosine] similarity does not support vectors with zero magnitude. Preview of invalid vector: [0.0, 0.0, 0.0, 0.0, 0.0, ...]', 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'The [cosine] similarity does not support vectors with zero magnitude. Preview of invalid vector: [0.0, 0.0, 0.0, 0.0, 0.0, ...]'}}}}
I thought Elastic Search is merely a database and I am confused that what ingesting 0 vectors is not allowed. I tried to look for solution and search online but couldn’t find any post about it or how to solve it. Anyone has any idea? Many thanks.