I use Elastic 7.14v
In order to save storage in elastic I would like to index several documents that share common fields into one document. The fields that are different will be indexed into one nested field with array of values.
For example I have two documents with nested fields:
{
{
"uid" : "aaaa",
"table": "tableA",
"compressedFields": [
{
"date": "28/03/2023 09:50:00",
"classification": "1",
"uid": "12344"
},
{
"date": "28/03/2023 10:00:00",
"classification": "2",
"uid": "55555"
},
{
"date": "28/03/2023 10:10:00",
"classification": "2",
"uid": "6666"
}
]
},
{
"uid" : "bbbbb",
"table": "tableB",
"compressedFields": [
{
"date": "28/03/2023 09:40:00",
"classification": "1",
"uid": "1111"
},
{
"date": "28/03/2023 09:55:00",
"classification": "2",
"uid": "2222"
},
{
"date": "28/03/2023 10:05:00",
"classification": "2",
"uid": "3333"
}
]
}
}
When I retrieve the documents I want them to be sorted by one of the nested fields for example date or classification.
For example if I sort it by a date the expected results would be:
{
{
"uid" : "bbbbb",
"table": "tableB",
"date": "28/03/2023 09:40:00",
"classification": "1",
"uid": "1111"
},
{
"uid" : "aaaa",
"table": "tableA",
"date": "28/03/2023 09:50:00",
"classification": "1",
"uid": "12344"
},
{
"uid" : "bbbbb",
"table": "tableB",
"date": "28/03/2023 09:55:00",
"classification": "2",
"uid": "2222"
},
{
"uid" : "aaaa",
"table": "tableA",
"date": "28/03/2023 10:00:00",
"classification": "2",
"uid": "55555"
},
{
"uid" : "bbbbb",
"table": "tableB",
"date": "28/03/2023 10:05:00",
"classification": "2",
"uid": "3333"
},
{
"uid" : "aaaa",
"table": "tableA",
"date": "28/03/2023 10:10:00",
"classification": "2",
"uid": "6666"
}
}
I found a clumsy way to do it with aggregation query:
GET my_index/_search
{
"aggs":
"aggs1": {
"nested" : {
"path": "compressedFields"
},
"aggs": {
"agg2": {
"terms": {
"field": "compressedFields.date",
"order": {
"_key": "asc"
},
size: 10000
}
"aggs": {
"agg3": {
"reverse_nested": {}
"aggs": {
"agg4": {
"top_hits" {}
}
}
}
}
}
}
}
}
}
After I query I can match each nested field to the origin document, but there is a problem. I am not able to do pagination.
Do you have any better ideas?
Thanks in advance
צחי is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.