I have written a web scraping tool that runs each night – it is written in Go, and runs on Fargate ECS.

Most nights it runs fine, but i’ve noticed lately, it seems to crash before writing the result file to S3 (~800MB).

The last log entry I see is where it says it will proceed to write results to S3. The file doesn’t appear in S3 and I don’t get the subsequent log entry to say “successfully written”.

While I have my suspicions .. how I can find out if/why the task crashed. Note that the entry has disappeared from the task list in nodes, when I have woken up next day.