I’ve been trying to understand why my executor tab is showing me only 3.1TiB input, when in fact my job reads at least 11TB (s3 total sizes).
I have multiple jobs and just in just one of my DAGs I can see it reading 6.3TiB (already more than executors tab).
Is it not accounting for filtered out records? Or is it a bug?
EMR job reading from S3.
Any ideas?
Thanks.