Relative Content

Tag Archive for amazon-web-serviceskubernetesairflowamazon-eks

Airflow Error: Airflow exception dag ‘test’ could not be found

I deployed airflow in EKS. At first I deployed it with sqlite db and sequential executor as configuration and it is working fine. Now i added RDS as database and also Kubernetes executor. When i used that I can see airflow dags in the UI but when i am triggering manually as well they are in queue mode. In pod logs I am having error as Airflow Error: Airflow exception dag ‘test’ could not be found,either it does not exist or failed to parse. I checked db connectivity it worked fine. But scheduler is not picking up and running it now with k8 executor. What kind of trouble shooting steps caI further check.

Airflow Init:CreateContainerError

I am trying to deploy airflow in EKS and following below article: https://aws.amazon.com/blogs/containers/running-airflow-workflow-jobs-on-amazon-eks-spot-nodes/
I am stuck with Init create container error. Pod is not able to start at all.
Below are the logs:
Defaulted container “webserver” out of: webserver, scheduler, init (init)
Error from server (BadRequest): container “webserver” in pod “airflow” is waiting to start: PodInitializing
Not sure where the error is pointing though. Is there any way I can trouble shoot it OR Is there better way to deploy airflow in EKS.