I have two AWS lambda functions that run in docker containers that I trigger using Airflow, one at a time. I currently am triggering the data pipeline using my local MWAA for testing purposes. I want each lambda to only be invoked once for every DAG run.
For some reason, when either of my lambda functions exceeds around one minute, another concurrent lambda runs at the same time, and sometimes two extra lambdas run concurrently to the original. This is not what I want. I want no concurrency at all.