I’m somewhat new to terraform. I’m having trouble reading in a nested irregular YAML and then iterating over it to create a uniform structure I can use to create rules for AWS Cost Categories. The original order in the YAML has to be preserved.

**Sample yaml:**
services:
  - service_names: [EC2]
    tags:
      - tag_names: [tag1]
      - tag_names: [tag2, tag3, tag4]
        label: ["EC2 Some text"]
  - sevice_names: [VPC]
    tags:
      - tag_names: [tag8]
  - sevice_names: [ARC]
  - service_names: [Service1, Service2, Service3, Service4]
    label: ["My Services"]

What I’d like to end up with is something like this:

rule_sets = [
        {
          tags      = ["tag1"]
          label     = "EC2" # if no label defined use join(" ", service_names) 
          services  = ["EC2"]
    },
    {
          tags      = [
                "K8SRL",
                "K8SCC",
                "K8SND",
            ]
          label     = "EC2 Some text"
          services  = ["EC2"]
        },
    { # because there were tag_names, I need 1 extra map like this at the end
          tags      = []
          label    = "EC2 Other"        
          services  = ["EC2"]
        },
        {
          tags      = ["tag8"]
          label    = "VPC"
          services  = ["VPC]
        },
        { # because there was 1 tag, I need 1 extra map like this at the end
          tags      = []
          label    = "VPC Other"
          services  = ["VPC"]
        },
        {
           tags      = []
           label    = "ARC"
           services  = ["ARC"]
        },
        {
            tags      = []
            label    = "My Services"
            services  = [
                "Service1",
                "Service2",
                "Service3",
                "Service4",
            ]
        },
        { # after processing the entire yaml, I need one more map like this
            tags      = []
            label    = "Other"
            services  = []
        },
    ]   
    

What I have that is oh so close but so far away is:

    cost_categories = yamldecode( file( "${path.module}/cost_category_rules.yaml" )) #with the yaml above


    rule_sets = flatten([
      for k, service_def in local.cost_categories.services : {
            services = service_def.service_names
            tag_names = can(service_def.tag_names) ? service_def.tag_names : []
            labels    = can(service_def.label) ? join(" ", service_def.label) : join(" ", service_def.service_names)
  }
])

Aand it gives me:

rule_sets = [
{
tags = [ # this should be the value in tag_names below not a map
{
tag_names = [“SQL”]
},
]
labels = “EC2”
services = [“EC2”]
},
{
tags = [ # this should be the value in tag_names below not a map
{
label = [“EC2 Some text”] #this should be moved
tag_names = [
“K8SRL”,
“K8SCC”,
“K8SND”,
]
},
]
labels = “EC2” #This should be the value from the inner label above
services = [“EC2”]
}, #I’m missing the extra one described in my ideal way above
{
tags = [
{
tag_names = [“tag8”]
},
]
labels = “VPC”
services = [“VPC”]
}, #I’m missing the extra one described in my ideal way above
{
tags = []
labels = “ARC”
services = [“ARC”]
},
{
tags = []
labels = “My Services”
services = [
“Service1”,
“Service2”,
“Service3”,
“Service4”,
]
} #I’m missing the extra one described in my ideal way above
]

I also tried doing this:

cost_with_codes = {for idx, cat in local.cost_categories.services: idx=>cat if can(cat.app_codes) }

cost_without_codes = {for idx, cat in local.cost_categories.services: idx=>cat if can(cat.app_codes)==false}

Which separates them into two maps, with the keys being the sequence from the original file.

cost_with_tag_name  = {"0" = {tag_names = [the inner stuff], service_names = [EC2]}, "1" = {tag_names = [the inner stuff], service_names = [VPC]}}

cost_without_tag_name = {"2" = {service_names ="ARC"}, "3" = {label= "My Services", service_names = ["Service1"...]}}

But I'm still left with the problem of iterating over these and normalizing the format of the maps.  Joining them back seems simple enough, I'm assuming because of the index, but it is a string, so maybe I'm being optimistic about that.

You might be thinking, just normalize the YAML!! Except there will be 15 to 20 service_names with various numbers of tab_names under many of the service_names from 0 to who knows 4 or 5 isn't hard to imagine. Then there are 4 or 5 YAML. So asking people to construct and maintain all of them is problematic. If this were Python, I'd have been done in the time it took me to write this post, much less the hours I've spent trying to solve it in terraform.