Access the Airflow database

Cloud Composer 1 | Cloud Composer 2 | Cloud Composer 3

This page explains how to connect to a Cloud SQL instance that runs the Airflow database of your Cloud Composer environment and run SQL queries.

For example, you might want to run queries directly on the Airflow database, make database backups, gather statistics based on the database content, or retrieve any other custom information from the database.

Before you begin

Run a SQL query on the Airflow database

To connect to the Airflow database:

  1. Create a DAG with one or more PostgresOperator operators. To get started, you can use the example DAG.

  2. In the sql parameter of the operator, specify your SQL query.

  3. Upload this DAG to your environment.

  4. Trigger the DAG, for example, you can do it manually or wait until it runs on a schedule.

Example DAG:

import datetime
import os

import airflow
from airflow.providers.postgres.operators.postgres import PostgresOperator

SQL_DATABASE = os.environ["SQL_DATABASE"]

with airflow.DAG(
    "airflow_db_connection_example",
    start_date=datetime.datetime(2024, 1, 1),
    schedule_interval=None,
    catchup=False) as dag:

    PostgresOperator(
        task_id="run_airflow_db_query",
        dag=dag,
        postgres_conn_id="airflow_db",
        database=SQL_DATABASE,
        sql="SELECT * FROM dag LIMIT 10;",
    )

Dump database contents and transfer them to a bucket

What's next