I have output which looks like this: (for readability i’ve condensed newlines into spaces, eg after each comma)

##[group]filename.json
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-02T00:00:00Z", "scope": "scope"
##[command]==========================================================================
##[endgroup]
##[group]filename2.json
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-01T00:00:00Z", "scope": "scope"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-02T00:00:00Z", "scope": "scope"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-03T00:00:00Z", "scope": "scope"
##[command]==========================================================================
##[endgroup]
##[group]filename3.json
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-01T00:00:00Z", "scope": "scope"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-02T00:00:00Z", "scope": "scope"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-03T00:00:00Z", "scope": "scope"
##[command]==========================================================================
##[endgroup]

and I want to sort them and group them by date
eg

##[group]expiresOn "2024-06-01T00:00:00Z"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-01T00:00:00Z", "scope": "scope"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-01T00:00:00Z", "scope": "scope"
##[command]==========================================================================
##[endgroup]
##[group]expiresOn "2024-06-02T00:00:00Z"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-02T00:00:00Z", "scope": "scope"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-02T00:00:00Z", "scope": "scope"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-02T00:00:00Z", "scope": "scope"
##[command]==========================================================================
##[endgroup]
##[group]expiresOn "2024-06-03T00:00:00Z"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-03T00:00:00Z", "scope": "scope"
"name": "name", "displayName": "displayName", "description": "description", "Category": "waiver", "expiresOn": "2024-06-03T00:00:00Z", "scope": "scope"
##[command]==========================================================================
##[endgroup]

The ##[ notation is used in Azure pipelines output so ideally should remain.

My code so far is:

1. check each JSON file in a directory for dates that are less than $expiry where $expiry is the date of the execution of the script + userinput days
2. clean the data to manipulate, eg eliminate quotes
3. find the beginning and end of the block where our date is
4. use sed to strip data from the file as a complete block
5. use grep -A/B to highlight key names and colour them using GREP_COLORS and append the block to the output.
   Note: this output is not an array

howManyDays=${1} # How many days in advance to check for, read from user input
cutoff=$(date --date "+${howManyDays} days" +%s) # Date in epoch seconds
for file in $(ls path/to/file)
do                 
    while read -r line key expiry
    do 
      # Strip off quotes and , from date found
      stripDate=`echo "$expiry" | tr -d "","`
      age=$(date -d "${stripDate}" +%s)
      # Work out the section of the file we want to report on from the line numbers 
      # Line number containing our expiry date
      lineStrip=`echo "$line" | tr -d ":"`
      # Line number before the expiry date where the record begins
      lineBefore=$(($lineStrip - 4))
      # Line number after the expiry date where the record ends
      lineAfter=$(($lineStrip + 4))
      # Use sed to extract the line numbers for one record, then grep to highlight the fields
      if ((${age} < ${cutoff}));then 
        GREP_COLORS= sed -n "$lineBefore","$lineAfter"p $file | 
        GREP_COLORS='mt=03;31' grep --color=always -A4 -B4 -P $expiry | 
        GREP_COLORS='mt=01;34' grep --color=always -A3 -B4 -P 'expiresOn|name|Category|scope|description' | 
        GREP_COLORS='mt=01;32' grep --color=always -A7 -B5 -P 'displayName'
        echo "##[command]=========================================================================="
        fi
    done < <(grep -n expiresOn path/to/file/$file)
done