YouTube Content Owner Transfers

The BigQuery Data Transfer Service for YouTube allows you to automatically schedule and manage recurring load jobs for YouTube Content Owner reports.

Supported Reports

The BigQuery Data Transfer Service for YouTube Content Owner reports currently supports the following reporting options:

For information on how YouTube Content Owner reports are transformed into BigQuery tables and views, see YouTube Content Owner Report Transformations.

Reporting option Support
Supported API version June 18, 2018
Schedule

Daily, around 14:45 UTC

You can configure the time of day

Refresh window

Last 1 day

Not configurable.

Maximum backfill duration

Last 180 days

YouTube retains data for up to 180 days. For more information, see Historical data in the YouTube Reporting API documentation.

Before you begin

Before you create a YouTube Content Owner transfer:

  • Verify that you have completed all actions required to enable the BigQuery Data Transfer Service.
  • Create a BigQuery dataset to store the YouTube data.
  • Verify that you have a YouTube Content Owner account. A YouTube Content Owner is not the same as a YouTube channel. Typically, you only have a YouTube Content Owner account if you manage many different channels.
  • Ensure that the person creating the transfer has the following required permissions:
    • YouTube: YouTube Content Manager or YouTube Content Owner. A Content Manager is granted rights to administer YouTube content for a Content Owner. A Content Owner is an umbrella account that owns one or more YouTube channels and the videos on those channels.
    • BigQuery: bigquery.transfers.update permissions to create the transfer and bigquery.datasets.update permissions on the target dataset. The bigquery.admin predefined, project-level IAM role includes bigquery.transfers.update and bigquery.datasets.update permissions. For more information on IAM roles in BigQuery, see Access Control.
  • If you intend to create transfers using the BigQuery web UI, allow pop-ups from bigquery.cloud.google.com so you can view the permissions window. You must allow the BigQuery Data Transfer Service permission to manage your YouTube reporting data for the transfer to succeed.
  • Transfer run notifications are currently in Alpha. If you intend to setup transfer run notifications for Cloud Pub/Sub, you must have pubsub.topics.setIamPolicy permissions. Cloud Pub/Sub permissions are not required if you just set up email notifications. For more information, see BigQuery Data Transfer Service Run Notifications.

Setting up a YouTube Content Owner transfer

Setting up a YouTube Content Owner transfer requires a:

  • Content Owner ID: Provided by YouTube. When you log in to YouTube as a Content Owner or Manager, your ID appears in the URL after o=. For example, if the URL is https://youtube.com/dashboard?0=AbCDE_8FghIjK, the Content Owner ID is AbCDE_8FghIjK. For more information on your Content Manager account, see: Configure Content Manager account settings.
  • Table Suffix: A user-friendly name for the channel provided by you when you set up the transfer. The suffix is appended to the job ID to create the table name, for example [reportTypeId]_[suffix]. The suffix is used to prevent separate transfers from writing to the same tables. The table suffix must be unique across all transfers that load data into the same dataset, and the suffix should be short to minimize the length of the resulting table name.

If you currently use the YouTube Reporting API and have existing reporting jobs, the BigQuery Data Transfer Service loads your report data. If you do not have existing reporting jobs, setting up the transfer automatically enables YouTube reporting jobs.

To set up a YouTube Content Owner transfer:

Web UI

  1. Go to the BigQuery web UI.

    Go to the BigQuery web UI

  2. Click Transfers.

  3. Click Add Transfer.

  4. On the New Transfer page:

    • For Source, choose YouTube Content Owner.
    • For Display name, enter a name for the transfer such as My Transfer. The transfer name can be any value that allows you to easily identify the transfer if you need to modify it later.
    • For Schedule, leave the default value or click Edit and change the time.

      Content Owner transfer custom schedule

    • For Destination dataset, choose the appropriate dataset.

    • For Content owner ID, enter your Content Owner ID.
    • For Table suffix, enter a suffix such as MT.
    • Check Configure jobs to allow the BigQuery Transfer Service to manage YouTube reporting jobs for you. If there are YouTube reports that don't currently exist for your account, new reporting jobs are created to enable them.

      Content Owner transfer default schedule

    • (Optional) Expand the Advanced section and configure run notifications for your transfer. Transfer run notifications are currently in Alpha.

      • For Cloud Pub/Sub topic, enter your topic name, for example, projects/myproject/topics/mytopic.
      • Check Send email notifications to allow email notifications of transfer run failures.
      • Don't check Disabled when setting up a transfer. See Working with transfers to disable existing transfers.

        Cloud Pub/Sub topic

  5. Click Add.

  6. When prompted, click Allow to give the BigQuery Data Transfer Service permission to view your YouTube reporting data and to access and manage the data in BigQuery. You must allow pop-ups from bigquery.cloud.google.com to view the permissions window.

    Allow transfer

Command-line

Enter the bq mk command and supply the transfer creation flag — --transfer_config. The following flags are also required:

  • --data_source
  • --target_dataset
  • --display_name
  • --params

    bq mk --transfer_config --project_id=[PROJECT_ID] --target_dataset=[DATASET] --display_name=[NAME] --params='[PARAMETERS]' --data_source=[DATA_SOURCE]
    

Where:

  • --project_id is your project ID.
  • --target_dataset is the target dataset for the transfer configuration.
  • --display_name is the display name for the transfer configuration. The transfer name can be any value that allows you to easily identify the transfer if you need to modify it later.
  • --params contains the parameters for the created transfer configuration in JSON format. For example: --params='{"param":"param_value"}'. For YouTube Content Owner transfers, you must supply the content_owner_id and table_suffix parameters. You may optionally set the configure_jobs parameter to true to allow the BigQuery Transfer Service to manage YouTube reporting jobs for you. If there are YouTube reports that don't currently exist for your account, new reporting jobs are created to enable them.
  • --data_source is the data source — youtube_content_owner.

You can also supply the --project_id flag to specify a particular project. If --project_id isn't specified, the default project is used.

For example, the following command creates a YouTube Content Owner transfer named My Transfer using content owner ID AbCDE_8FghIjK, table suffix MT, and target dataset mydataset. The transfer is created in the default project:

bq mk --transfer_config --target_dataset=mydataset --display_name='My Transfer' --params='{"content_owner_id":"abCDE_8FghIjK","table_suffix":"MT","configure_jobs":"true"}' --data_source=youtube_content_owner

API

Use the projects.locations.transferConfigs.create method and supply an instance of the TransferConfig resource.

Querying your data

When your data is transferred to BigQuery, the data is written to ingestion-time partitioned tables. For more information, see Partitioned Tables.

If you query your tables directly instead of using the auto-generated views, you must use the _PARTITIONTIME pseudo-column in your query. For more information, see Querying Partitioned Tables.

Troubleshooting YouTube Content Owner transfer setup

If you are having issues setting up your transfer, see YouTube transfer issues in Troubleshooting BigQuery Data Transfer Service Transfer Setup.

Was this page helpful? Let us know how we did:

Send feedback about...

Need help? Visit our support page.