I’m using Elasticsearch 7.17.
I’m collecting logs using the AWS Cloudwatch integration. We use Fleet for agent management.
The logs being collected are unstructured. I’ve created an ingest pipeline using various processors to parse the logs for fields we’re interested in. I have tested the pipeline ok using the pipeline simulation in Dev Tools.
Question: How do I apply this pipeline to these logs, without impacting other AWS cloudwatch integrations? The other cloudwatch integrations use the same Agent Policy. I would like these logs to use thier own index. (current index is generic).
I have started looking at the reindex API, however I’m unsure if this is the right way to achieve the outcome.