View Airflow logs

Cloud Composer 3 | Cloud Composer 2 | Cloud Composer 1

This page describes how to access and view Apache Airflow logs for Cloud Composer.

About log types

Cloud Composer has the following Airflow logs:

• Streaming logs. These logs contain Airflow task logs, as well as logs of Airflow components of your environment, such as Airflow schedulers and workers.

  To learn about Cloud Logging and Cloud Monitoring for your Cloud Composer environment, see Monitor environments.

• Airflow task logs in the environment's bucket. These logs are associated with single DAG tasks. If you enable saving logs to the environment's bucket, Cloud Composer saves files with Airflow task logs to the /logs folder.

• Cloud Composer generates audit logs, such as Admin Activity logs. For more information, see View audit logs.

Before you begin

• Your Google Account must have a role that can view objects in environment buckets. For more information, see Access control.

• Logs stored in environment's bucket use the policy of the bucket. Cloud Composer creates buckets with the default policy that keeps data forever. For logs stored in Cloud Logging, Cloud Composer uses _Default and User-defined logs retention periods.

• To prevent data loss, logs saved in environment's bucket are not deleted after you delete your environment. You must manually delete logs from your environment's bucket.

View streaming logs

To view streaming logs in Cloud Logging:

Console

1. In the Google Cloud console, go to the Environments page.

   Go to Environments

2. In the list of environments, click the name of your environment. The Environment details page opens.

3. Go to the Logs tab.

View streaming logs in Cloud Logging

To view streaming logs in Cloud Logging:

Console

1. In the Google Cloud console, go to the Environments page.

   Go to Environments

2. In the list of environments, locate your environment.

3. in the Logs column, click the Logs link.

View Airflow task logs in the environment's bucket

Note: You can view Airflow task logs in the Airflow UI.

To view Airflow task logs in your environment's bucket:

Console

1. In Google Cloud console, go to the Environments page.

   Go to Environments

2. In the list of environments, find a row with the name of your environment and in the DAGs folder column click the DAGs link. The Bucket details page opens.

3. In the Folder browser pane, click /logs.

   Note: If this folder is empty, then saving logs to the environment's bucket might be disabled for your environment.

What's next

• Monitor environments
• Cloud Composer audit logging