I am starting to learn how to use Airflow for pipelines.
I have the simplest DAG in
from __future__ import annotations
import textwrap
from datetime import datetime, timedelta
# The DAG object; we'll need this to instantiate a DAG
from airflow.models.dag import DAG
# Operators; we need this to operate!
from airflow.operators.bash import BashOperator
with DAG(
dag_id="simplest_tutorial",
# [START default_args]
# These args will get passed on to each operator
# You can override them on a per-task basis during operator initialization
default_args={
"depends_on_past": False,
"email": ["[email protected]"],
"email_on_failure": False,
"email_on_retry": False,
"retries": 1,
"retry_delay": timedelta(minutes=5),
},
description="An even simpler tutorial DAG",
schedule=timedelta(days=1),
start_date=datetime(2021, 1, 1),
catchup=False,
tags=["example"],
) as dag:
t1 = BashOperator(
task_id="print_date",
bash_command="date",
)
t2 = BashOperator(
task_id="sleep",
depends_on_past=False,
bash_command="sleep 5",
retries=3,
)
t1 >> t2
When I run this DAG I get this in Airflow
Airflow DAG screen
so it seems the DAG has been ran twice!.
Also if I click on the green bars to the left I can see that
one run was not externally triggered and the other one was.
The audit log is also difficult to understand
Audit log
Why is this DAG being run twice?
New contributor
Kansai Hitoruna is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.