This page provides an overview of the BigQuery Data Transfer Service.
The BigQuery Data Transfer Service automates data movement from Software as a Service (SaaS) applications such as Google Ads and Google Ad Manager on a scheduled, managed basis. Your analytics team can lay the foundation for a data warehouse without writing a single line of code.
You can access the BigQuery Data Transfer Service using the:
After you configure a data transfer, the BigQuery Data Transfer Service automatically loads data into BigQuery on a regular basis. You can also initiate data backfills to recover from any outages or gaps. Currently, you cannot use the BigQuery Data Transfer Service to transfer data out of BigQuery.
Supported data sources
Currently, the BigQuery Data Transfer Service supports loading data from the following data sources:
- Campaign Manager
- Cloud Storage (beta)
- Amazon S3 (beta)
- Google Ad Manager
- Google Ads
- Google Merchant Center (beta)
- Google Play (beta)
- YouTube - Channel Reports
- YouTube - Content Owner Reports
In addition, several third-party transfersbeta are available in the GCP Marketplace.
Like BigQuery, the BigQuery Data Transfer Service is a multi-regional resource.
Data locality is specified when you create a dataset to store your BigQuery Data Transfer Service core customer data. When you set up a transfer, the transfer configuration is set to the same locality as the dataset. The BigQuery Data Transfer Service processes and stages data in the same location as the target BigQuery dataset.
For more information on BigQuery Data Transfer Service pricing, see: BigQuery Data Transfer Service pricing.
Load jobs created by transfers are included in BigQuery's
quotas on load jobs. It’s important to consider
how many transfers you enable in each project to prevent transfers and other
load jobs from producing
You can use the following equation to estimate how many load jobs are required by your transfers:
Number of daily jobs = Number of transfers x Number of tables x
Schedule frequency x Refresh window
Number of transfersis the number of transfer configurations you enable in your project.
Number of tablesis the number of tables created by each specific transfer type. The number of tables varies by transfer type:
- Campaign Manager transfers create approximately 25 tables.
- Google Ads transfers create approximately 60 tables.
- Google Ad Manager transfers create approximately 40 tables.
- Google Play transfers create approximately 25 tables.
- YouTube transfers create approximately 50 tables.
Schedule frequencydescribes how often the transfer runs. Transfer run schedules are provided for each transfer type:
Refresh windowis the number of days to include in the data transfer. If you enter 1, there is no daily backfill.
To learn how to create a transfer, see the documentation for your data source: