Schedule Google Merchant Center Transfers

Preview

This product is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Note: To get support or provide feedback for Google Merchant Center transfers with BigQuery Data Transfer Service, contact [e-mail].

Before you begin

Before you create a Google Merchant Center data transfer:

• Verify that you have completed all actions required to enable the BigQuery Data Transfer Service.
• Create a BigQuery dataset to store the Google Merchant Center data.
  • For dataset region, we support using the default option, Multi-region, in either US or EU.
  • If you want to create a dataset in a specific region, the Merchant Center data transfer is only supported in the following regions:
  • us-east4 (Northern Virginia),
  • asia-northeast1 (Tokyo),
  • asia-southeast1 (Singapore),
  • australia-southeast1 (Sydney),
  • europe-north1 (Finland),
  • europe-west2 (London),
  • europe-west6 (Zurich).
• If you intend to setup transfer run notifications for Pub/Sub, you must have pubsub.topics.setIamPolicy permissions. Pub/Sub permissions are not required if you just set up email notifications. For more information, see BigQuery Data Transfer Service Run Notifications.

Required permissions

Ensure that the person creating the data transfer has the following required permissions:

• BigQuery:

  • bigquery.transfers.update permissions to create the data transfer
  • Both bigquery.datasets.get and bigquery.datasets.update permissions on the target dataset

  The bigquery.admin predefined IAM role includes bigquery.transfers.update, bigquery.datasets.update and bigquery.datasets.get permissions. For more information on IAM roles in BigQuery Data Transfer Service, see Access control.

• Google Merchant Center:

  • Standard access to the Google Merchant Center account that is used in the transfer configuration. You can verify access by clicking the Users section in the Google Merchant Center UI.

  • To access price competitiveness and best sellers data, you must meet the eligibility requirements for market insights.

Set up a Google Merchant Center transfer

Setting up a data transfer for Google Merchant Center reporting requires the following:

• Merchant ID or Multi-client account ID: This is the Merchant ID shown in the Google Merchant Center UI.

To create a data transfer for Google Merchant Center reporting:

Console

1. Go to the Data transfers page in the Google Cloud console.

   Go to Data transfers

2. Click Create transfer.

3. On the Create Transfer page:

   • In the Source type section, for Source, choose Google Merchant Center.

   • In the Transfer config name section, for Display name, enter a name for the data transfer such as My Transfer. The transfer name can be any value that lets you identify the transfer if you need to modify it later.

     

   • In the Schedule options section:

     • Select a Repeat frequency. If you select Hours, Days, Weeks, or Months, you must also specify a frequency. You can also select Custom to specify a custom repeat frequency. If you select On-demand, then this data transfer runs when you manually trigger the transfer.

     • If applicable, select either Start now or Start at set time and provide a start date and run time.

     • For Start date and run time, enter the date and time to start the transfer. This value should be at least 24 hours later than the current UTC time. If you chose Start now, this option is disabled.

       

       If you leave the schedule options set to Start now, the first data transfer run will start immediately, and it will fail with the following error message: No data to transfer found for the Merchant account. If you have just created this transfer, you may need to wait for up to a day before the data of your Merchant account are prepared and available for the transfer. The next scheduled run should run successfully. If the data of your Merchant account are prepared on the same date in UTC time, you can set up a backfill for today's run.

   • In the Destination settings section, for Destination dataset, choose the dataset you created to store your data.

     

   • In the Data source details section, for Merchant ID, enter your Merchant ID or MCA ID.

     Select the report(s) that you would like to transfer. See Supported Reports for more details.

     

   • In the Service Account menu, select a service account from the service accounts associated with your Google Cloud project. You can associate a service account with your data transfer instead of using your user credentials. For more information about using service accounts with data transfers, see Use service accounts.

     • If you signed in with a federated identity, then a service account is required to create a data transfer. If you signed in with a Google Account, then a service account for the transfer is optional.
     • The service account must have the required permissions.

   • (Optional) In the Notification options section:

     • Click the toggle to enable email notifications. When you enable this option, the transfer administrator receives an email notification when a transfer run fails.
     • For Select a Pub/Sub topic, choose your topic name or click Create a topic. This option configures Pub/Sub run notifications for your transfer.

4. Click Save.

bq

Enter the bq mk command and supply the transfer creation flag — --transfer_config. The following flags are also required:

• --data_source
• --target_dataset
• --display_name
• --params

You can also supply the --project_id flag to specify a particular project. If --project_id isn't specified, the default project is used.

bq mk \
--transfer_config \
--project_id=project_id \
--target_dataset=dataset \
--display_name=name \
--params='parameters' \
--data_source=data_source
--service_account_name=service_account_name

Where:

• project_id is your project ID.
• dataset is the target dataset for the transfer configuration.
• name is the display name for the transfer configuration. The transfer name can be any value that lets you identify the transfer if you need to modify it later.
• parameters contains the parameters for the created transfer configuration in JSON format. For example: --params='{"param":"param_value"}'.
  • For Google Merchant Center data transfers, you must supply the merchant_id parameter.
  • The export_products parameter specifies whether to transfer product and product issues data. This parameter is included by default, even if you don't specify the export_products parameter. Google recommends that you include this parameter explicitly and set it to "true".
  • The export_regional_inventories parameter specifies whether to transfer regional inventories data.
  • The export_local_inventories parameter specifies whether to transfer local inventories data.
  • The export_price_competitiveness parameter specifies whether to transfer price competitiveness data.
  • The export_price_insights parameter specifies whether to transfer price insights data.
  • The export_best_sellers_v2 parameter specifies whether to transfer best sellers data.
  • The export_performance parameter specifies whether to transfer product performance data.
• data_source is the data source — merchant_center.
• service_account_name is the service account name used to authenticate your data transfer. The service account should be owned by the same project_id used to create the transfer and it should have all of the required permissions.

Caution: You cannot configure notifications by using the command-line tool.

For example, the following command creates a Google Merchant Center data transfer named My Transfer using Merchant ID 1234 and target dataset mydataset. The data transfer is created in your default project.

bq mk \
--transfer_config \
--target_dataset=mydataset \
--display_name='My Transfer' \
--params='{"merchant_id":"1234","export_products":"true","export_regional_inventories":"true","export_local_inventories":"true","export_price_benchmarks":"true","export_best_sellers":"true"}' \
--data_source=merchant_center

The first time you run the command, you receive a message like the following:

[URL omitted] Please copy and paste the above URL into your web browser and follow the instructions to retrieve an authentication code.

Follow the instructions in the message and paste the authentication code on the command line.

Caution: When you create a Google Merchant Center data transfer by using the command-line tool, the transfer configuration is set up using the default values for Schedule (every 24 hours at creation time). The first transfer run will start immediately, and it will fail with the following error message: `No data to transfer found for the Merchant account.

If you have just created this data transfer, you may need to wait for up to a day before the data of your Merchant account are prepared and available for the transfer.` The next scheduled run should run successfully. If the data of your Merchant account are prepared on the same date in UTC time, you can set up a backfill for today's run.

API

Use the projects.locations.transferConfigs.create method and supply an instance of the TransferConfig resource.

Troubleshoot Google Merchant Center transfer setup

If you are having issues setting up your data transfer, see Google Merchant Center transfer issues in Troubleshooting BigQuery Data Transfer Service transfer setup.