I need to retrieve all file paths that are derived from the software_artifact_location variable.
These paths may appear in both Jinja2 templates and YAML files.
The script should start by knowing only the value of software_artifact_location and then recursively resolve other variables that reference it, either directly or indirectly, across different files.
The final output should list all resolved file paths. So finally, we want to know which files are used under the /artifacts by a project.
We need to skip the comment rows.

Some examples:

templates/setting.j2

software_load_specs:
  deploy_software:
    app1_deploy_software_name: App_1.1.1.zip
    app2_deploy_software_name: App_2.1.1.zip
    deploy_software_location: {{artifacts}}/MYAPP
configure_software_specs:
  cert_util: cert_1.zip
  cert_util_dest: /home/user

roles/inst_app/tasks/main.yml

  - name: Set up APP base variables
    set_fact:
      cert_util: "{{this.software_load_specs.deploy_software.deploy_software_location}}/{{configure_software_specs.cert_util}}"
      cert_util_dest: "/home/user"
      cert_dest: "/opt/applications/APP/certs"
      app1_software_zip: "{{this.software_load_specs.deploy_software.deploy_software_location}}/{{hostvars[inventory_hostname].software_load_specs.deploy_software.app1_deploy_software_name}}"
      app2_software_zip: "{{this.software_load_specs.deploy_software.deploy_software_location}}/{{hostvars[inventory_hostname].software_load_specs.deploy_software.app2_deploy_software_name}}"

I tried:

#!/bin/bash
 
# Recursive function to resolve variable values
resolve_var() {
    local var_name=$1
    local file=$2
 
    echo "Resolving variable: $var_name in file: $file"
 
    # Search for the variable value in the YAML file
    value=$(grep -m 1 "^ *$var_name:" "$file" | awk -F': ' '{print $2}' | tr -d ' ')
 
    echo "Initial value found: $value"
 
    # If the value contains another variable reference, resolve it recursively
    while echo "$value" | grep -q '{{'; do
        nested_var=$(echo "$value" | awk -F'{{' '{print $2}' | awk -F'}}' '{print $1}')
        echo "Nested variable found: $nested_var"
        # Recursive call to resolve the nested variable
        value=$(resolve_var "$nested_var" "$file")
        echo "Updated value: $value"
    done
 
    # Return the resolved value
    echo "$value"
}
 
# Function to find all variable references in files
find_references() {
    local search_value=$1
 
    echo "Searching for references to $search_value in files..."
 
    # Construct the find command
    find_cmd="find . -type f ! -path './app/aec-azure-ansible-common/*' ! -name 'extra_vars.yaml' -exec grep -H '${search_value}' {} ;"
 
    # Print the find command for debugging
    echo "Executing command: $find_cmd"
 
    # Execute the find command
    eval "$find_cmd" | 
    awk -v val="${search_value}" '{gsub(/^[ t]+|[ t]+$/, "", $0); if ($0 ~ val) {sub(".*" val, val); print val substr($0, length(val)+1)}}' | sort | uniq
 
    echo "Search completed."
}
 
# Initial variable name and file
var_name="software_artifact_location"
file="/opt/app/extra_vars.yaml"
 
echo "Starting resolution for $var_name in file $file"
 
# Resolve the initial variable
resolved_value=$(resolve_var "$var_name" "$file")
echo "Resolved value of $var_name: $resolved_value"
 
# Find references to the resolved value
find_references "$resolved_value"

But it’s wrong:

/tmp/r.sh
Starting resolution for software_artifact_location in file /opt/app/extra_vars.yaml
Resolved value of software_artifact_location: Resolving variable: software_artifact_location in file: /opt/app/extra_vars.yaml
Initial value found: /artifacts
/artifacts
Searching for references to Resolving variable: software_artifact_location in file: /opt/app/extra_vars.yaml
Initial value found: /artifacts
/artifacts in files...
Executing command: find . -type f ! -path './app/app1/*' ! -name 'extra_vars.yaml' -exec grep -H 'Resolving variable: software_artifact_location in file: /opt/app/extra_vars.yaml
Initial value found: /artifacts
/artifacts' {} ;
Search completed.

Thank you for your help.