I’m trying to build my first airflow DAG in k8s running on custom image.
I did the following example code:
with DAG(
'test-1',
default_args=default_args,
schedule_interval=timedelta(minutes=100)
) as dag:
test1 = KubernetesPodOperator(
image="demo111:1.91",
cmds=["/bin/bash", "-c"],
arguments=["./test-1.sh"],
......
)
test2 = KubernetesPodOperator(
image="demo111:1.91",
cmds=["/bin/bash", "-c"],
arguments=["./test-2.sh"],
......
)
test1 >> test2
However this obviously creates 2 pods and I would like to run the tasks in the same pod.
How can I do that easily ?
Can you point me to some example DAG(s) ( maybe pod_override API or KubernetesPodOperator) where 2 scripts are executed as 2 tasks in the same pod?