BigQuery Data Transfer Service

The BigQuery Data Transfer Service automates data movement from Software as a Service (SaaS) applications such as Google AdWords and DoubleClick on a scheduled, managed basis. Your analytics team can lay the foundation for a data warehouse without writing a single line of code.

You can access the BigQuery Data Transfer Service using the:

After you configure a data transfer, the BigQuery Data Transfer Service automatically loads data into BigQuery on a regular basis. You can also initiate data backfills to recover from any outages or gaps.

Supported data sources

Currently, the BigQuery Data Transfer Service supports loading data from the following data sources:

Supported regions

Like BigQuery, the BigQuery Data Transfer Service is a multi-regional resource. The BigQuery Data Transfer Service supports the same regions as BigQuery.

Data locality is specified when you create a dataset to store your BigQuery Data Transfer Service core customer data. When you set up a transfer, the transfer configuration is set to the same locality as the dataset. The BigQuery Data Transfer Service processes and stages data in the same location as the target BigQuery dataset.

Transfer pricing

For more information on BigQuery Data Transfer Service pricing, see: BigQuery Data Transfer Service Pricing.

Once data is transferred to BigQuery, standard BigQuery storage and query pricing applies.

Transfer quotas

Load jobs created by transfers are included in BigQuery's quotas on load jobs. It’s important to consider how many transfers you enable in each project to prevent transfers and other load jobs from producing quotaExceeded errors.

You can use the following equation to estimate how many load jobs are required by your transfers:

Number of daily jobs = Number of transfers x Number of tables x Schedule frequency x Refresh window

Where:

  • Number of transfers is the number of transfer configurations you enable in your project.
  • Number of tables is the number of tables created by each specific transfer type. The number of tables varies by transfer type:

    • Google AdWords transfers create approximately 60 tables.
    • DoubleClick for Publishers transfers create approximately 40 tables.
    • DoubleClick Campaign Manager transfers create approximately 25 tables.
    • YouTube transfers create approximately 50 tables.
  • Schedule frequency describes how often the transfer runs. Transfer run schedules are provided for each transfer type:

  • Refresh window is the number of days to incude in the data transfer. If you enter 1, there is no daily backfill.

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...