Schedule a Salesforce Marketing Cloud transfer

The BigQuery Data Transfer Service for Salesforce Marketing Cloud lets you automatically schedule and manage recurring load jobs from Salesforce Marketing Cloud into BigQuery.

Before you begin

The following sections describe the steps that you need to take before you create a Salesforce Marketing Cloud transfer.

Salesforce Marketing Cloud prerequisites

You must have the following information when creating a Salesforce Marketing Cloud transfer:

Parameter Name Description
subdomain The API subdomain
instance The API server instance
clientId The app integration client ID
clientSecret The app integration client secret

The following steps show you how you can obtain the information needed to create a Salesforce Marketing Cloud transfer:

  1. Install a server-to-server type API integration package. Note the client ID, the client secret, and the subdomain in the Components section of the package details page.
  2. Find your API server instance in the URL after you log into the Salesforce Marketing Cloud application. The instance value includes s followed by a numeric value. For example, in the URL https://mc.s4.exacttarget.com/, the instance value is s4. For more information, see Find the stack location for a Marketing Cloud account.

BigQuery prerequisites

Required BigQuery roles

To get the permissions that you need to create a transfer, ask your administrator to grant you the BigQuery Admin (roles/bigquery.admin) IAM role. For more information about granting roles, see Manage access.

This predefined role contains the permissions required to create a transfer. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to create a transfer:

  • bigquery.transfers.update on the user
  • bigquery.datasets.get on the target dataset
  • bigquery.datasets.update on the target dataset

You might also be able to get these permissions with custom roles or other predefined roles.

Set up a Salesforce Marketing Cloud data transfer

To create a Salesforce Marketing Cloud data transfer:

Console

  1. Go to the BigQuery page in the Google Cloud console.

    Go to the BigQuery page

  2. In the navigation pane, click Data transfers > Create a transfer.

  3. In the Source type section, for Source, select Salesforce Marketing Cloud.

  4. In the Transfer config name section, for Display name, enter a name for the transfer.

  5. In the Schedule options section, do the following:

    • In the Repeat frequency list, select an option to specify how often this transfer runs. To specify a custom repeat frequency, select Custom. If you select On-demand, then this transfer runs when you manually trigger the transfer.

    • If applicable, select either Start now or Start at set time and provide a start date and run time.

  6. In the Destination settings section, for Dataset, select the dataset you created to store your data.

  7. In the Data source details section, do the following:

    • For API Subdomain, enter the API subdomain.
    • For API instance, enter the API instance value.
    • For Client ID, enter the app integration client ID.
    • For Client Secret, enter the app integration client secret.
  8. In the Service Account menu, select a service account associated with your Google Cloud project. The selected service account must have the required roles to run this transfer.

    If you signed in with a federated identity, then a service account is required to create a transfer. If you signed in with a Google Account, then a service account for the transfer is optional.

    For more information about using service accounts with data transfers, see Use service accounts.

  9. Optional: In the Notification options section, do the following:

    • To enable email notifications, click the Email notification toggle. When you enable this option, the transfer administrator receives an email notification when a transfer run fails.
    • To enable Pub/Sub transfer run notifications for this transfer, click the Pub/Sub notifications toggle. You can select your topic name, or you can click Create a topic to create one.
  10. Click Save.

When this transfer runs, the BigQuery Data Transfer Service automatically populates the following tables based on the REST interface.

  • Campaigns
  • Categories
  • EventDefinitions
  • Journeys
  • JourneyActivities
  • SendDefinitions
  • Subscriptions

bq

Enter the bq mk command and supply the transfer creation flag — --transfer_config.

bq mk \
    --transfer_config \
    --project_id=PROJECT_ID \
    --data_source=DATA_SOURCE \
    --display_name=DISPLAY_NAME \
    --target_dataset=DATASET \
    --params='PARAMETERS'

Replace the following:

  • PROJECT_ID (optional): your Google Cloud project ID. If --project_id isn't supplied to specify a particular project, the default project is used.
  • DATA_SOURCE: the data source (for example, saphana).
  • DISPLAY_NAME: the display name for the transfer configuration. The transfer name can be any value that lets you identify the transfer if you need to modify it later.
  • DATASET: the target dataset for the transfer configuration.
  • PARAMETERS: the parameters for the created transfer configuration in JSON format. For example: --params='{"param":"param_value"}'. The following are the parameters for a Salesforce Marketing Cloud transfer:
    • connector.subdomain: The API subdomain.
    • connector.instance: The API instance value.
    • connector.authentication.oauth.clientId: The app ID name for the OAuth client.
    • connector.authentication.oauth.clientSecret: The app secret for the OAuth client.

For example, the following command creates a Salesforce Marketing Cloud transfer in the default project with all the required parameters:

  bq mk \
      --transfer_config \
      --target_dataset=mydataset \
      --data_source=salesforce_marketing \
      --display_name='My Transfer' \
      --params='{"connector.subdomain": "abcd",
      "connector.instance": "x",
      "connector.authentication.oauth.clientId": "1234567890",
      "connector.authentication.oauth.clientSecret":"ABC12345"}'

API

Use the projects.locations.transferConfigs.create method and supply an instance of the TransferConfig resource.

Troubleshoot transfer setup

If you are having issues setting up a Salesforce Marketing Cloud transfer, try the following troubleshooting steps:

Pricing

There is no cost to transfer Salesforce Marketing Cloud data into BigQuery while this feature is in Preview.

What's next