I have a dag that calls on a function that returns a PythonOperator
. I want to get the results of this task, so that I can pass it to another task. Any ideas if this is possible?
Here is roughly what my code looks like:
def DAG(self):
args = {
"owner": "airflow",
"depends_on_past": False,
"end_date": None, # runs forever
"retries": self.retries,
"retry_delay": self.retry_delay,
"start_date": self.start_date,
}
with DAG(dag_name, default_args=args, schedule_interval=dag_cron,...) as dag:
self.parsing_tasks()
if self.factset_cbbo_us:
all_us_equity_mics = grab_all_mic() <----- here is where a PythonOperator is returned
generate_external_sensor_tasks(all_us_equity_mics)
return dag
def grab_all_mic():
something = PythonOperator(
task_id="blah_blah",
python_callable=blah_blah_blah,
op_args=("some_task", self.parsing_day_delta),
retries=10,
on_failure_callback=on_exhausted_retries_failure_callback,
)
return something
Do let me know if more information is needed 🙂
I found a similar question but I couldn’t quite get what I needed from the responses: Python Airflow – Return result from PythonOperator