Enabling the BigQuery Data Transfer Service

To use the BigQuery Data Transfer Service, you must complete the following steps as a Project Owner:

  • Create a project and enable the BigQuery API
  • Enroll in the BigQuery Data Transfer Service
  • Enable the BigQuery Data Transfer Service

For more information on IAM roles, see Understanding Roles.

Creating a project and enabling the BigQuery API

Before using the BigQuery Data Transfer Service, you must create a project and enable billing. You can use an existing project with the BigQuery Data Transfer Service, or you can create a new one. If you are using an existing project, you may also need to enable the BigQuery API.

To create a project and enable the BigQuery API:

  1. Sign in to your Google account.

    If you don't already have one, sign up for a new account.

  2. Select or create a Cloud Platform project.

    Go to the Manage resources page

  3. Enable billing for your project.

    Enable billing

  4. BigQuery is automatically enabled in new projects. To activate BigQuery in an existing project, Enable the BigQuery API.

    Enable the API

Enrolling in the BigQuery Data Transfer Service

Before using the BigQuery Transfer Service, you must complete the enrollment form. To complete the form, you need the project number for the project containing your transfers.

  1. Open the Cloud Platform Console.
  2. Cloud Platform Console
  3. If your transfer project isn't selected, at the top of the window, click the project name.
  4. In the Select project dialog, select your project and then click Open.
  5. On the Dashboard tab, in the Project Info box, note your project number (below your project ID). The project number is a string of numerals.
  6. Project number
  7. Complete the BigQuery Data Transfer Service enrollment form.
  8. Enroll

The enrollment request must be completed by the BigQuery team before you can use the service. Typically, requests are reviewed and approved within 10 minutes, but in some cases approval may take up to 24 hours. When your request is approved, you receive an email confirmation. If you have questions about the status of your enrollment request, contact your Google Cloud Sales Representative.

Enabling the BigQuery Data Transfer Service

Before you can create a transfer, you must enable the BigQuery Data Transfer Service. To enable the BigQuery Data Transfer Service, you must be granted the Project Owner role for your project.

To enable the BigQuery Data Transfer Service:

  1. Go to the BigQuery web UI.

    Go to the BigQuery web UI

  2. Click Transfers.

  3. Click Enable to grant permissions to the BigQuery Data Transfer Service. The first time you enable transfers in a project, you are required to grant permissions to the Data Transfer Service. After permissions are granted, you are not prompted again.

    Enable transfers

Granting bigquery.admin access

All users that will create transfers must be granted the bigquery.admin predefined IAM role. The bigquery.admin role includes the following BigQuery Data Transfer Service permissions:

  • bigquery.transfers.update
  • bigquery.transfers.get

For more information on IAM roles in BigQuery, see Access Control.

To grant bigquery.admin access to a user or Google group:

Web UI

  1. Open the IAM page in the GCP Console

    Open the IAM page

  2. Click Select a project.

  3. Select a project and click Open.

  4. Click Add to add new members to the project and set their permissions.

  5. In the Add members dialog:

    • For Members, enter the email address of the user or group.
    • In the Select a role drop-down, click BigQuery > BigQuery Admin.
    • Click Add.

    Grant admin

Command-line

You can use the gcloud command-line tool to grant a user or group the bigquery.admin role.

To add a single binding to your project's IAM policy, type the following command. To add a user, supply the --member flag in the format user:user@example.com. To add a group, supply the --member flag in the format group:group@example.com.

gcloud projects add-iam-policy-binding [PROJECT_ID] --member [GROUP/USER]:[ADDRESS] --role roles/bigquery.admin

Where:

  • [PROJECT_ID] is your project ID.
  • [GROUP/USER] is either group or user.
  • [ADDRESS] is the user or group's email address.

For example:

gcloud projects add-iam-policy-binding myproject --member group:group@example.com --role roles/bigquery.admin

The command outputs the updated policy:

    bindings:
    - members:
      - group:group@example.com
        role: roles/bigquery.admin
    

Next steps

After enabling the BigQuery Data Transfer Service, create a transfer for your data source:

Send feedback about...