The BigQuery Data Transfer Service automates data movement from Software as a Service (SaaS) applications such as Google AdWords and DoubleClick on a scheduled, managed basis. Your analytics team can lay the foundation for a data warehouse without writing a single line of code.
You can access the BigQuery Data Transfer Service using the:
After you configure a data transfer, the BigQuery Data Transfer Service automatically loads data into BigQuery on a regular basis. You can also initiate data backfills to recover from any outages or gaps.
Supported data sources
Currently, the BigQuery Data Transfer Service supports loading data from the following data sources:
- Google AdWords (beta)
- DoubleClick Campaign Manager
- DoubleClick for Publishers
- YouTube - Channel Reports (beta)
- YouTube - Content Owner Reports (beta)
Like BigQuery, the BigQuery Data Transfer Service is a multi-regional resource. The BigQuery Data Transfer Service supports the same regions as BigQuery.
Data locality is specified when you create a dataset to store your BigQuery Data Transfer Service core customer data. When you set up a transfer, the transfer configuration is set to the same locality as the dataset. The BigQuery Data Transfer Service processes and stages data in the same location as the target BigQuery dataset.
For more information on BigQuery Data Transfer Service pricing, see: BigQuery Data Transfer Service Pricing.
Load jobs created by transfers are included in BigQuery's
quotas on load jobs. It’s important to consider
how many transfers you enable in each project to prevent transfers and other
load jobs from producing
You can use the following equation to estimate how many load jobs are required by your transfers:
Number of daily jobs = Number of transfers x Number of tables x Schedule frequency x Refresh window
Number of transfersis the number of transfer configurations you enable in your project.
Number of tablesis the number of tables created by each specific transfer type. The number of tables varies by transfer type:
- Google AdWords transfers create approximately 60 tables.
- DoubleClick for Publishers transfers create approximately 40 tables.
- DoubleClick Campaign Manager transfers create approximately 25 tables.
- YouTube transfers create approximately 50 tables.
Schedule frequencydescribes how often the transfer runs. Transfer run schedules are provided for each transfer type:
Refresh windowis the number of days to incude in the data transfer. If you enter 1, there is no daily backfill.