To use the BigQuery Data Transfer Service, you must complete the following steps as a Project Owner:
- Create a project and enable the BigQuery API
- Enable the BigQuery Data Transfer Service
For more information on IAM roles, see Understanding Roles.
Creating a project and enabling the BigQuery API
Before using the BigQuery Data Transfer Service, you must create a project and, in most cases, enable billing on that project. You can use an existing project with the BigQuery Data Transfer Service, or you can create a new one. If you are using an existing project, you may also need to enable the BigQuery API.
To create a project and enable the BigQuery API:
In the GCP Console, go to the Manage resources page.
Select a project, or click Create Project to create a new GCP project.
In the dialog, name your project. Make a note of your generated project ID.
Click Create to create a new project.
- You will need to enable billing on your project if you want to
transfer data from any of these sources:
- Google AdWords
- DoubleClick Campaign Manager
- DoubleClick for Publishers
- Google Play (beta)
- YouTube - Channel Reports
- YouTube - Content Owner Reports
Make sure that billing is enabled for your project.
- BigQuery is automatically enabled in new projects. To activate BigQuery in an existing project, Enable the BigQuery API.
Enabling the BigQuery Data Transfer Service
Before you can create a transfer, you must enable the BigQuery Data Transfer Service. To enable the BigQuery Data Transfer Service, you must be granted the Project Owner role for your project.
To enable the BigQuery Data Transfer Service:
Go to the BigQuery web UI.
Click Enable to grant permissions to the BigQuery Data Transfer Service. The first time you enable transfers in a project, you are required to grant permissions to the Data Transfer Service. After permissions are granted, you are not prompted again.
Granting bigquery.admin access
All users that will create transfers must be granted the
predefined IAM role. The
bigquery.admin role includes the following
BigQuery Data Transfer Service permissions:
bigquery.admin role also includes
You must have
bigquery.datasets.update permissions in order to write data to
the transfer's destination dataset.
For more information on IAM roles in BigQuery, see Access Control.
bigquery.admin access to a user or Google group:
Open the IAM page in the GCP Console
Click Select a project.
Select a project and click Open.
Click Add to add new members to the project and set their permissions.
In the Add members dialog:
- For Members, enter the email address of the user or group.
- In the Select a role drop-down, click BigQuery > BigQuery Admin.
- Click Add.
You can use the gcloud command-line tool to grant a user or group the
To add a single binding to your project's IAM policy, type the following
command. To add a user, supply the
--member flag in the format
user:firstname.lastname@example.org. To add a group, supply the
--member flag in
gcloud projects add-iam-policy-binding [PROJECT_ID] --member [GROUP/USER]:[ADDRESS] --role roles/bigquery.admin
[PROJECT_ID]is your project ID.
[ADDRESS]is the user or group's email address.
gcloud projects add-iam-policy-binding myproject --member group:email@example.com --role roles/bigquery.admin
The command outputs the updated policy:
bindings: - members: - group:firstname.lastname@example.org role: roles/bigquery.admin
After enabling the BigQuery Data Transfer Service, create a transfer for your data source: