I’ve been scratching my head for a while on this one. I have a variable called metrics_config containing prometheus-like configuration, which looks as follows:

- debug: var=metrics_config 

ok: [hostname01.local] => 
  metrics_config:
    configs:
    - name: default
      remote_write:
      - basic_auth:
          password_file: /var/lib/prom/pass.key
          username: user01
        headers:
          X-Scope-OrgID: user01
        url: http://nginx.local:9009/api/v1/push
      scrape_configs:
      - job_name: job1
        static_configs:
        - targets:
          - 172.16.20.110:3100
    - name: default
      scrape_configs:
      - job_name: job2
        static_configs:
        - targets:
          - 127.0.0.1:12345
    global:
      external_labels:
        hostfqdn: hostname01.local
      scrape_interval: 1m
      scrape_timeout: 10s
    wal_cleanup_age: 12h
    wal_directory: /var/lib/prom/wal

I would like to merge and pop all duplicate entries so that ‘job1’ and ‘job2’ are listed under the same unique scrape_configs. In other words I want the end result to look like this:

- debug: var=metrics_config

ok: [hostname01.local] => 
  metrics_config:
    configs:
    - name: default
      remote_write:
      - basic_auth:
          password_file: /var/lib/prom/pass.key
          username: user01
        headers:
          X-Scope-OrgID: user01
        url: http://nginx.local:9009/api/v1/push
      scrape_configs:
      - job_name: job1
        static_configs:
        - targets:
          - 172.16.20.110:3100
      - job_name: job2
        static_configs:
        - targets:
          - 127.0.0.1:12345
    global:
      external_labels:
        hostfqdn: hostname01.local
      scrape_interval: 1m
      scrape_timeout: 10s
    wal_cleanup_age: 12h
    wal_directory: /var/lib/prom/wal

How would I remove the redundant

- name: default
  scrape_configs:

and merge them so that unique key/value pairs are kept?

I’ve tried | unique without success. I’ve also tried

- set_fact:
    metrics_config: "{{ metrics_config | combine(metrics_config, recursive=true, list_merge='append_rp') }}"

Tried the following modules without success:

community.general.lists_mergeby
community.general.merge_variables
ansible.builtin.zip

Any help would be much appreciated, thank you!