The question is about python metrics!!! Not task or DAG status!
What the problem is:
- There is
statsd-exporter
that expoters all airflow metrics and python ones. But airflow metrics are exported for airflow while python metircs about runningstatsd-exporter
it self. - There are some libraries like airflow-exporter but they export metrics as web page that “turns on” when airflow webserver is running. When I start airflow scheduler or worker with no web – no metrics are exported.
Is there a way to export python metrics (like python_gc_collections
, process_cpu_seconds_total
, process_resident_memory_bytes
etc.) for airflow scheduler and/or worker?