Connector for BigQuery

Workflows connector that defines the built-in function used to access BigQuery within a workflow.

Explore further

For detailed documentation that includes this code sample, see the following:

Code sample

YAML

# This workflow demonstrates how to use the BigQuery connector:
# Create a dataset and then insert a table with data from a public dataset
# Delete both the table and dataset
# Expected output: "SUCCESS"
- init:
    assign:
      - project_id: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
      - dataset_id: "example_dataset"
      - table_id: "example_table"
      - query: "SELECT * FROM `bigquery-public-data.usa_names.usa_1910_2013` LIMIT 5000;"
      - create_disposition: "CREATE_IF_NEEDED"  # creates table if it doesn't exist
      - write_disposition: "WRITE_TRUNCATE"  # truncates table if it already exists
- create_dataset:
    call: googleapis.bigquery.v2.datasets.insert
    args:
      projectId: ${project_id}
      body:
        datasetReference:
          datasetId: ${dataset_id}
          projectId: ${project_id}
        access[].role: "roles/bigquery.dataViewer"
        access[].specialGroup: "projectReaders"
- insert_table_into_dataset:
    call: googleapis.bigquery.v2.jobs.insert
    args:
      projectId: ${project_id}
      body:
        configuration:
          query:
            query: ${query}
            destinationTable:
              projectId: ${project_id}
              datasetId: ${dataset_id}
              tableId: ${table_id}
            create_disposition: ${create_disposition}
            write_disposition: ${write_disposition}
            allowLargeResults: true
            useLegacySql: false
- delete_table_from_dataset:
    call: googleapis.bigquery.v2.tables.delete
    args:
      projectId: ${project_id}
      datasetId: ${dataset_id}
      tableId: ${table_id}
- delete_dataset:
    call: googleapis.bigquery.v2.datasets.delete
    args:
      projectId: ${project_id}
      datasetId: ${dataset_id}
- the_end:
    return: "SUCCESS"

What's next

To search and filter code samples for other Google Cloud products, see the Google Cloud sample browser.