My Cloudwatch logs from Django app deployed via Zappa to AWS Lambdas are of the following format:

  {
    taskName: foo.bar,
    @requestId: 98ccca67-46db-4793-9a62-412f9dc60ed5  
  }

  {
    @billedDuration: 24723.0, 
    @duration: 24722.88,
    @maxMemoryUsed: 2.49E8,
    @requestId: 98ccca67-46db-4793-9a62-412f9dc60ed5,
    @type: REPORT
  }

Now I want to create a cloudwatch dashboard or maybe just cloudwatch log insights query, so that I can find how long each task took over time (bin by mins/hours/days/months etc.)

example1: foo.bar - avg duration over last 1 hour is 30 seconds
example2: foo.baz - avg duration over last 1 hour is 20 seconds

and so on. The key here is reportId which is the next log and has the duration, original message has the requestId but not the duration. So we need to group by or find neighbouring reportId.

I tried GPT and other resources but couldn’t get it working, thanks in advance 🙂