I have auto scaling enabled and I have G.8x with 100 workers, and my job ran for 1 hour. From the job metrics, I can see that the maximum number of allocated executors is 20. In G.8x 1 workers maps to 1 executor. Each worker as 8 DPUs. So 100 workers means a total of […]