DAG task fails with an error
Stay organized with collections
Save and categorize content based on your preferences.
Problem
Your Composer DAG task fails with an error message:
Failed to execute job JOB_NUMBER for task DAG_TASK_NAME (Pod GKE_POD_NAME returned a failure.
Environment
- Cloud Composer
- DAG task operator is GKEStartPodOperator or KubernetesPodOperator
- Airflow worker logs
Solution
- Resolve the issue with the downstream GKE cluster that is being called by the task operator.
Cause
This error is emitted by the downstream GKE cluster that is called by the task operator. It indicates a problem with the downstream cluster, and this bubbles up as an error message in Composer and causes the task to fail.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-12-12 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-12-12 UTC."],[],[]]
