Campaign Manager 转移作业

借助适用于 Campaign Manager 的 BigQuery Data Transfer Service,您可以自动安排和管理 Campaign Manager 报告数据的定期加载作业。

支持的报告

适用于 Campaign Manager(以前称为 DoubleClick Campaign Manager)的 BigQuery Data Transfer Service 目前支持以下报告选项:

如需了解 Campaign Manager 报告如何转换为 BigQuery 表和视图,请参阅 Campaign Manager 报告转换

报告选项 支持
支持的 API 版本 2017 年 6 月 26 日
时间安排

每 8 小时(基于创建时间)。

无法配置

刷新时段

过去 2 天

无法配置

最大回填时长

过去 60 天

Campaign Manager 最多可将数据传输文件保留 60 天。超过 60 天的文件会由 Campaign Manager 删除。

准备工作

在创建 Campaign Manager 转移之前,请先完成以下操作:

  • 验证您已经完成启用 BigQuery Data Transfer Service 所需的所有操作。
  • 如果您要使用经典版 BigQuery 网页界面创建转移作业,请在浏览器中允许来自 bigquery.cloud.google.com 的弹出式窗口,以便查看权限窗口。您必须向 BigQuery Data Transfer Service 授予管理转移作业的权限。
  • 创建 BigQuery 数据集,以用于存储 Campaign Manager 数据。
  • 确保您的组织有权访问 Campaign Manager Data Transfer v2 (Campaign Manager DTv2) 文件。Campaign Manager 团队会将这些文件交付到 Cloud Storage 存储分区。要获得对 Campaign Manager DTv2 文件的访问权限,您需要执行后续步骤,后续步骤取决于您与 Campaign Manager 有无直接的合约关系。在这两种情况下,都可能会产生额外扣款。

    • 如果您与 Campaign Manager 有合约关系,请联系 Campaign Manager 支持团队,以设置 Campaign Manager DTv2 文件。
    • 如果您与 Campaign Manager 没有合约关系,您的代理机构或 Campaign Manager 转销商可能会有 Campaign Manager DTv2 文件的访问权限。请联系您的代理机构或转销商获取这些文件的访问权限。

    完成此步骤后,您将收到如下 Cloud Storage 存储分区名称:

    dcdt_-dcm_account123456

  • 如果您想要为 Pub/Sub 设置转移作业运行通知,那么您必须拥有 pubsub.topics.setIamPolicy 权限。如需了解详情,请参阅 BigQuery Data Transfer Service 运行通知

所需权限

  • BigQuery:确保转移作业创建者在 BigQuery 中拥有以下权限:

    • 创建转移作业所需的 bigquery.transfers.update 权限
    • 针对目标数据集的 bigquery.datasets.update 权限

    预定义 IAM 角色 bigquery.admin 中包含 bigquery.transfers.updatebigquery.datasets.update 权限。如需详细了解 BigQuery Data Transfer Service 中的 IAM 角色,请参阅访问权限控制参考文档

  • Campaign Manager:对存储在 Cloud Storage 中的 Campaign Manager DTv2 文件的读取访问权限。访问权限由您从其接收 Cloud Storage 存储分区的实体进行管理。

设置 Campaign Manager 转移

设置 Campaign Manager 传输需要以下各项:

  • Cloud Storage 存储分区:Campaign Manager DTv2 的文件的 Cloud Storage 存储分区 URI(如准备工作中所述)。存储分区名称应类似于:

    dcdt_-dcm_account123456

  • Campaign Manager ID:您的 Campaign Manager 网络、广告客户或 Floodlight ID。网络 ID 在层次结构中为父级。

查找 Campaign Manager ID

要检索您的 Campaign Manager ID,您可以使用 Cloud Storage 控制台检查 Campaign Manager Data Transfer Cloud Storage 存储分区中的文件。Campaign Manager ID 可用于在提供的 Cloud Storage 存储分区中匹配文件。该 ID 嵌入在文件名中,而不是 Cloud Storage 存储分区名称中。

例如:

  • 在名为 dcm_account123456_activity_* 的文件中,ID 为 123456
  • 在名为 dcm_floodlight7890_activity_* 的文件中,ID 为 7890
  • 在名为 dcm_advertiser567_activity_* 的文件中,ID 为 567

(可选)查找文件名前缀

在极少数情况下,Cloud Storage 存储分区中的文件可能具有自定义的非标准文件名,这些名称是由 Google Marketing Platform 服务团队为您设置的。

例如:

  • 在名为 dcm_account123456custom_activity_* 的文件中,前缀为 dcm_account123456custom - _activity 之前的所有内容。

如需帮助,请与 Campaign Manager 支持团队联系。

为 Campaign Manager 创建数据转移

控制台

  1. 转到 Cloud Console 中的 BigQuery 页面。

    转到 BigQuery 页面

  2. 点击转移

  3. 点击创建转移作业

  4. 创建转移作业页面上:

    • 在中来源类型部分的来源中,选择 Campaign Manager

      转移作业来源

    • 转移配置名称部分的显示名中,输入转移作业的名称,例如 My Transfer。转移作业名称可以是任何容易辨识的值,方便您以后在需要修改该作业时能轻松识别。

      转移作业名称

    • 时间表选项部分的时间表中,保留默认值(立即开始)或点击在设置的时间开始

      • 重复频率部分中,选择转移作业的运行频率选项。

        • 每日一次(默认值)
        • 每周一次
        • 每月一次
        • 自定义
        • 按需

        如果您选择除“每日一次”以外的选项,则系统还会提供其他选项。例如,如果您选择“每周一次”,则系统会显示一个选项,供您选择星期几。

      • 开始日期和运行时间部分,输入转移作业的开始日期和时间。如果选择立即开始,则此选项会处于停用状态。

        转移作业时间安排

    • 目标设置部分的目标数据集中,选择您创建的用来存储数据的数据集。

      转移作业数据集

    • 数据源详细信息部分,执行以下操作:

      • 对于 Cloud Storage 存储分区,请输入或浏览存储 Data Transfer V2.0 文件的 Cloud Storage 存储分区的名称。当您输入存储分区名称时,请不要包含 gs://
      • DoubleClick ID 部分中,输入相应的 Campaign Manager ID。
      • (可选)如果文件具有类似于这些示例的标准名称,请将文件名前缀 (File name prefix) 字段留空。只有当 Cloud Storage 存储分区中的文件具有类似于此示例的自定义文件名时,才需要填写文件名前缀 (File name prefix)。

        Campaign Manager 来源详细信息

    • (可选)在通知选项部分,执行以下操作:

      • 点击切换开关以启用电子邮件通知。启用此选项后,转移作业管理员会在转移作业运行失败时收到电子邮件通知。
      • 选择 Pub/Sub 主题部分,选择您的主题名称,或点击创建主题。此选项用于为您的转移作业配置 Pub/Sub 运行通知
  5. 点击保存

经典版界面

  1. 转到 BigQuery 网页界面。

    转到 BigQuery 网页界面

  2. 点击 Transfers

  3. 点击 Add Transfer

  4. 新传输 (New Transfer) 页面上,执行以下操作:

    • 来源 (Source) 部分中,选择 Campaign Manager(以前称为 DCM)[Campaign Manager (formerly DCM)]。
    • Display name 部分,输入转移作业的名称,例如 My Transfer。转移作业名称可以是任何容易辨识的值,让您以后在需要修改时能够轻松识别。
    • 目标数据集字段,选择相应的数据集。
    • Cloud Storage 存储分区部分,输入存储数据转移 V2.0 文件的 Cloud Storage 存储分区的名称。当您输入存储分区名称时,请不要包含 gs://
    • DoubleClick ID 部分中,输入相应的 Campaign Manager ID。
    • (可选)如果文件具有类似于这些示例的标准名称,请将文件名前缀 (File name prefix) 字段留空。只有当 Cloud Storage 存储分区中的文件具有类似于此示例的自定义文件名时,才需要填写文件名前缀 (File name prefix)。

      Campaign Manager 转移

    • (可选)展开 Advanced 部分,并为转移作业配置运行通知

      • Pub/Sub topic 部分,输入您的主题名称,例如 projects/myproject/topics/mytopic
      • 勾选 Send email notifications,让系统在转移作业运行失败时发送电子邮件通知。
      • 设置转移作业时请勿勾选 Disabled。如需停用现有转移作业,请参阅处理转移作业

        Pub/Sub 主题

  5. 点击添加

  6. 出现提示时,点击 Allow 以向 BigQuery Data Transfer Service 授予相应权限,允许其访问您的 Campaign Manager 报告数据以及在 BigQuery 中管理数据。您必须允许显示来自 bigquery.cloud.google.com 的弹出式窗口,才能看到权限窗口。

    允许转移作业

bq

输入 bq mk 命令并提供转移作业创建标志 --transfer_config。此外,还必须提供以下标志:

  • --data_source
  • --target_dataset
  • --display_name
  • --params
bq mk --transfer_config \
--project_id=project_id \
--target_dataset=dataset \
--display_name=name \
--params='parameters' \
--data_source=data_source

其中:

  • project_id 是项目 ID。
  • dataset 是转移作业配置的目标数据集。
  • name 是转移作业配置的显示名。转移作业名称可以是任何容易辨识的值,方便您以后在需要修改该作业时能轻松识别。
  • parameters 包含所创建转移作业配置的参数(采用 JSON 格式),例如 --params='{"param":"param_value"}'。对于 Campaign Manager,您必须提供 bucketnetwork_id 参数。bucket 是包含 Campaign Manager DTv2 文件的 Cloud Storage 存储分区。network_id 是您的网络、Floodlight 或广告客户 ID。
  • data_source 是数据源 - dcm_dt (Campaign Manager)。

您还可以提供 --project_id 标志以指定具体项目。如果未指定 --project_id,系统会使用默认项目。

例如,以下命令使用 Campaign Manager ID 123456、Cloud Storage 存储分区 dcdt_-dcm_account123456 和目标数据集 mydataset 创建名为 My Transfer 的 Campaign Manager 转移作业。参数 file_name_prefix 为可选参数,仅用于罕见的自定义文件名。

该转移作业将在默认项目中创建:

bq mk --transfer_config \
--target_dataset=mydataset \
--display_name='My Transfer' \
--params='{"bucket": "dcdt_-dcm_account123456","network_id": "123456","file_name_prefix":"YYY"}' \
--data_source=dcm_dt

运行命令后,您会收到类似如下的消息:

[URL omitted] Please copy and paste the above URL into your web browser and follow the instructions to retrieve an authentication code.

请按照说明操作,并将身份验证代码粘贴到命令行中。

API

使用 projects.locations.transferConfigs.create 方法并提供一个 TransferConfig 资源实例。

Campaign Manager 转移设置问题排查

如果您在设置转移作业时遇到问题,请参阅排查 BigQuery Data Transfer Service 转移作业设置问题中的 Campaign Manager 转移作业问题

查询数据

当数据传输到 BigQuery 时,会被写入提取时间分区表。如需了解详情,请参阅分区表简介

如果您要直接查询表,而不是使用自动生成的视图,那么必须在查询中使用 _PARTITIONTIME 伪列。如需了解详情,请参阅查询分区表

Campaign Manager 示例查询

您可以使用以下 Campaign Manager 查询示例来分析已传输的数据。您还可以在 Google Data Studio 等可视化工具中使用查询。这些查询旨在帮助您开始使用 BigQuery 来查询 Campaign Manager 数据。如果您对这些报告的功能有其他问题,请联系您的 Campaign Manager 技术代表。

在以下每个查询中,将 dataset 等变量替换为您的值。

最新广告系列

以下查询示例会检索最新广告系列。

控制台

SELECT
  Campaign,
  Campaign_ID
FROM
  `dataset.match_table_campaigns_campaign_manager_id`
WHERE
  _DATA_DATE = _LATEST_DATE

bq

bq query --use_legacy_sql=false \
'SELECT
   Campaign,
   Campaign_ID
 FROM
   `dataset.match_table_campaigns_campaign_manager_id`
 WHERE
   _DATA_DATE = _LATEST_DATE'

按广告系列划分的展示次数和不同用户数

以下查询示例会分析过去 30 天内按广告系列划分的展示次数和不同用户数。

控制台

# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
# END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
  SELECT
    Campaign_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.impression_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN start_date
    AND end_date
  GROUP BY
    Campaign_ID,
    Date

bq

# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
# END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)

bq query --use_legacy_sql=false \
'SELECT
  Campaign_ID,
  _DATA_DATE AS Date,
  COUNT(*) AS count,
  COUNT(DISTINCT User_ID) AS du
FROM
  `dataset.impression_campaign_manager_id`
WHERE
  _DATA_DATE BETWEEN start_date
  AND end_date
GROUP BY
  Campaign_ID,
  Date'

按广告系列和日期排序的最新广告系列

以下查询示例会分析过去 30 天内按广告系列和日期排序的最新广告系列。

控制台

# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
# END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
SELECT
  Campaign,
  Campaign_ID,
  Date
FROM (
  SELECT
    Campaign,
    Campaign_ID
  FROM
    `dataset.match_table_campaigns_campaign_manager_id`
  WHERE
    _DATA_DATE = _LATEST_DATE ),
  (
  SELECT
    date AS Date
  FROM
    `bigquery-public-data.utility_us.date_greg`
  WHERE
    Date BETWEEN start_date
    AND end_date )
ORDER BY
  Campaign_ID,
  Date

bq

# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
# END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
bq query --use_legacy_sql=false \
'SELECT
  Campaign,
  Campaign_ID,
  Date
FROM (
  SELECT
    Campaign,
    Campaign_ID
  FROM
    `dataset.match_table_campaigns_campaign_manager_id`
  WHERE
    _DATA_DATE = _LATEST_DATE ),
  (
  SELECT
    date AS Date
  FROM
    `bigquery-public-data.utility_us.date_greg`
  WHERE
    Date BETWEEN start_date
    AND end_date )
ORDER BY
  Campaign_ID,
  Date'

特定日期范围内按广告系列划分的展示次数和不同用户数

以下示例查询按广告系列分析 start_dateend_date 之间的展示次数和不同用户数。

控制台

# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
# END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
SELECT
  base.*,
  imp.count AS imp_count,
  imp.du AS imp_du
FROM (
  SELECT
    *
  FROM (
    SELECT
      Campaign,
      Campaign_ID
    FROM
      `dataset.match_table_campaigns_campaign_manager_id`
    WHERE
      _DATA_DATE = _LATEST_DATE ),
    (
    SELECT
      date AS Date
    FROM
      `bigquery-public-data.utility_us.date_greg`
    WHERE
      Date BETWEEN start_date
      AND end_date ) ) AS base
LEFT JOIN (
  SELECT
    Campaign_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.impression_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN start_date
    AND end_date
  GROUP BY
    Campaign_ID,
    Date ) AS imp
ON
  base.Campaign_ID = imp.Campaign_ID
  AND base.Date = imp.Date
WHERE
  base.Campaign_ID = imp.Campaign_ID
  AND base.Date = imp.Date
ORDER BY
  base.Campaign_ID,
  base.Date

bq

# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
# END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
bq query --use_legacy_sql=false \
'SELECT
  base.*,
  imp.count AS imp_count,
  imp.du AS imp_du
FROM (
  SELECT
    *
  FROM (
    SELECT
      Campaign,
      Campaign_ID
    FROM
      `dataset.match_table_campaigns_campaign_manager_id`
    WHERE
      _DATA_DATE = _LATEST_DATE ),
    (
    SELECT
      date AS Date
    FROM
      `bigquery-public-data.utility_us.date_greg`
    WHERE
      Date BETWEEN start_date
      AND end_date ) ) AS base
LEFT JOIN (
  SELECT
    Campaign_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.impression_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN start_date
    AND end_date
  GROUP BY
    Campaign_ID,
    Date ) AS imp
ON
  base.Campaign_ID = imp.Campaign_ID
  AND base.Date = imp.Date
WHERE
  base.Campaign_ID = imp.Campaign_ID
  AND base.Date = imp.Date
ORDER BY
  base.Campaign_ID,
  base.Date'

按广告系列划分的展示次数、点击次数、活动数和不同用户数

以下查询示例会分析过去 30 天内按广告系列划分的展示次数、点击次数、活动数和不同用户数。在此查询中,将 campaign_list 等变量替换为您的值。例如,将 campaign_list 替换为查询范围内所有感兴趣的 Campaign Manager 广告系列的逗号分隔列表。

控制台

# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
# END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
SELECT
  base.*,
  imp.count AS imp_count,
  imp.du AS imp_du,
  click.count AS click_count,
  click.du AS click_du,
  activity.count AS activity_count,
  activity.du AS activity_du
FROM (
  SELECT
    *
  FROM (
    SELECT
      Campaign,
      Campaign_ID
    FROM
      `dataset.match_table_campaigns_campaign_manager_id`
    WHERE
      _DATA_DATE = _LATEST_DATE ),
    (
    SELECT
      date AS Date
    FROM
      `bigquery-public-data.utility_us.date_greg`
    WHERE
      Date BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
      AND DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY) ) ) AS base
LEFT JOIN (
  SELECT
    Campaign_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.impression_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
    AND DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
  GROUP BY
    Campaign_ID,
    Date ) AS imp
ON
  base.Campaign_ID = imp.Campaign_ID
  AND base.Date = imp.Date
LEFT JOIN (
  SELECT
    Campaign_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.click_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
    AND DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
  GROUP BY
    Campaign_ID,
    Date ) AS click
ON
  base.Campaign_ID = click.Campaign_ID
  AND base.Date = click.Date
LEFT JOIN (
  SELECT
    Campaign_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.activity_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
    AND DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
  GROUP BY
    Campaign_ID,
    Date ) AS activity
ON
  base.Campaign_ID = activity.Campaign_ID
  AND base.Date = activity.Date
WHERE
  base.Campaign_ID IN campaign_list
  AND (base.Date = imp.Date
    OR base.Date = click.Date
    OR base.Date = activity.Date)
ORDER BY
  base.Campaign_ID,
  base.Date

bq

# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
# END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
bq query --use_legacy_sql=false \
'SELECT
  base.*,
  imp.count AS imp_count,
  imp.du AS imp_du,
  click.count AS click_count,
  click.du AS click_du,
  activity.count AS activity_count,
  activity.du AS activity_du
FROM (
  SELECT
    *
  FROM (
    SELECT
      Campaign,
      Campaign_ID
    FROM
      `dataset.match_table_campaigns_campaign_manager_id`
    WHERE
      _DATA_DATE = _LATEST_DATE ),
    (
    SELECT
      date AS Date
    FROM
      `bigquery-public-data.utility_us.date_greg`
    WHERE
      Date BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
      AND DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY) ) ) AS base
LEFT JOIN (
  SELECT
    Campaign_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.impression_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
    AND DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
  GROUP BY
    Campaign_ID,
    Date ) AS imp
ON
  base.Campaign_ID = imp.Campaign_ID
  AND base.Date = imp.Date
LEFT JOIN (
  SELECT
    Campaign_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.click_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
    AND DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
  GROUP BY
    Campaign_ID,
    Date ) AS click
ON
  base.Campaign_ID = click.Campaign_ID
  AND base.Date = click.Date
LEFT JOIN (
  SELECT
    Campaign_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.activity_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
    AND DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
  GROUP BY
    Campaign_ID,
    Date ) AS activity
ON
  base.Campaign_ID = activity.Campaign_ID
  AND base.Date = activity.Date
WHERE
  base.Campaign_ID IN campaign_list
  AND (base.Date = imp.Date
    OR base.Date = click.Date
    OR base.Date = activity.Date)
ORDER BY
  base.Campaign_ID,
  base.Date'

广告系列活动

以下查询示例会分析过去 30 天的广告系列活动。在此查询中,将 campaign_list 等变量替换为您的值。例如,将 campaign_list 替换为查询范围内所有感兴趣的 Campaign Manager 广告系列的逗号分隔列表。

控制台

# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
# END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
SELECT
  base.*,
  activity.count AS activity_count,
  activity.du AS activity_du
FROM (
  SELECT
    *
  FROM (
    SELECT
      Campaign,
      Campaign_ID
    FROM
      `dataset.match_table_campaigns_campaign_manager_id`
    WHERE
      _DATA_DATE = _LATEST_DATE ),
    (
    SELECT
      mt_at.Activity_Group,
      mt_ac.Activity,
      mt_ac.Activity_Type,
      mt_ac.Activity_Sub_Type,
      mt_ac.Activity_ID,
      mt_ac.Activity_Group_ID
    FROM
      `dataset.match_table_activity_cats_campaign_manager_id` AS mt_ac
    JOIN (
      SELECT
        Activity_Group,
        Activity_Group_ID
      FROM
        `dataset.match_table_activity_types_campaign_manager_id`
      WHERE
        _DATA_DATE = _LATEST_DATE ) AS mt_at
    ON
      mt_at.Activity_Group_ID = mt_ac.Activity_Group_ID
    WHERE
      _DATA_DATE = _LATEST_DATE ),
    (
    SELECT
      date AS Date
    FROM
      `bigquery-public-data.utility_us.date_greg`
    WHERE
      Date BETWEEN start_date
      AND end_date ) ) AS base
LEFT JOIN (
  SELECT
    Campaign_ID,
    Activity_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.activity_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
    AND DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
  GROUP BY
    Campaign_ID,
    Activity_ID,
    Date ) AS activity
ON
  base.Campaign_ID = activity.Campaign_ID
  AND base.Activity_ID = activity.Activity_ID
  AND base.Date = activity.Date
WHERE
  base.Campaign_ID IN campaign_list
  AND base.Activity_ID = activity.Activity_ID
ORDER BY
  base.Campaign_ID,
  base.Activity_Group_ID,
  base.Activity_ID,
  base.Date

bq

# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
# END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
bq query --use_legacy_sql=false \
'SELECT
  base.*,
  activity.count AS activity_count,
  activity.du AS activity_du
FROM (
  SELECT
    *
  FROM (
    SELECT
      Campaign,
      Campaign_ID
    FROM
      `dataset.match_table_campaigns_campaign_manager_id`
    WHERE
      _DATA_DATE = _LATEST_DATE ),
    (
    SELECT
      mt_at.Activity_Group,
      mt_ac.Activity,
      mt_ac.Activity_Type,
      mt_ac.Activity_Sub_Type,
      mt_ac.Activity_ID,
      mt_ac.Activity_Group_ID
    FROM
      `dataset.match_table_activity_cats_campaign_manager_id` AS mt_ac
    JOIN (
      SELECT
        Activity_Group,
        Activity_Group_ID
      FROM
        `dataset.match_table_activity_types_campaign_manager_id`
      WHERE
        _DATA_DATE = _LATEST_DATE ) AS mt_at
    ON
      mt_at.Activity_Group_ID = mt_ac.Activity_Group_ID
    WHERE
      _DATA_DATE = _LATEST_DATE ),
    (
    SELECT
      date AS Date
    FROM
      `bigquery-public-data.utility_us.date_greg`
    WHERE
      Date BETWEEN start_date
      AND end_date ) ) AS base
LEFT JOIN (
  SELECT
    Campaign_ID,
    Activity_ID,
    _DATA_DATE AS Date,
    COUNT(*) AS count,
    COUNT(DISTINCT User_ID) AS du
  FROM
    `dataset.activity_campaign_manager_id`
  WHERE
    _DATA_DATE BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY)
    AND DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY)
  GROUP BY
    Campaign_ID,
    Activity_ID,
    Date ) AS activity
ON
  base.Campaign_ID = activity.Campaign_ID
  AND base.Activity_ID = activity.Activity_ID
  AND base.Date = activity.Date
WHERE
  base.Campaign_ID IN campaign_list
  AND base.Activity_ID = activity.Activity_ID
ORDER BY
  base.Campaign_ID,
  base.Activity_Group_ID,
  base.Activity_ID,
  base.Date'