I’m using Dynamic Task Mapping
feature with Task Groups
to generate n number of tasks for my input list. DAG gets generated but at execution I’m seeing odd behaviour.
Mapped Tasks from second Task are getting triggered simultaneously when Mapped Tasks from first Task are still running. This is reflected in MWAA’s logs:
This non-deterministic run causes DAG to fail.
Tried both depends_on_past=True, wait_for_downstream=True
parameters to no luck. Kind thanks for any help…