Introduction to Analytics Hub
Analytics Hub is a data exchange platform that enables you to share data and insights at scale across organizational boundaries with a robust security and privacy framework. With Analytics Hub, you can discover and access a data library curated by various data providers. This data library also includes Google-provided datasets.
For example, by using Analytics Hub you can augment your analytics and ML initiatives with third-party and Google datasets.
As an Analytics Hub user, you can perform the following tasks:
As an Analytics Hub publisher, you can monetize data by sharing it with your partner network or within your own organization in real time. Listings let you share data without replicating the shared data. You can build a catalog of analytics-ready data sources with granular permissions that let you deliver data to the right audiences. You can also manage subscriptions to your listings.
As an Analytics Hub subscriber, you can discover the data that you are looking for, combine shared data with your existing data, and leverage the built-in features of BigQuery. When you subscribe to a listing, a linked dataset is created in your project.
As an Analytics Hub viewer, you can browse through the datasets that you have access to in Analytics Hub and request the publisher to access the shared data.
As an Analytics Hub administrator, you can create data exchanges that enable data sharing, and then give permissions to data publishers and subscribers to access these data exchanges.
For more information about Analytics Hub user roles, see Configure Analytics Hub roles.
Architecture
Analytics Hub is built on a publish and subscribe model of BigQuery datasets. The separation of compute and storage in BigQuery's architecture enables data publishers to share data with as many subscribers as they want without having to make multiple copies of the data. Publishers are only charged for data storage, whereas subscribers only pay for queries that run against the shared data. The publisher and subscriber workflows in Analytics Hub are explained in detail in the following sections.
Publisher workflow
The following diagram describes how publishers interact with Analytics Hub:
In figure 1, the following features are labeled: Shared dataset, Data exchange, and Listing.
- A shared dataset is a BigQuery dataset that is the unit of
data sharing in Analytics Hub. As a publisher, you create or
use an existing BigQuery dataset in your project with the
following supported objects that you want to deliver to
your subscribers:
- Authorized views
- Authorized datasets
- BigQuery ML models
- External tables
- Materialized views
- Tables
- Table snapshots
- Views
- Data exchanges
- A data exchange is a container that enables self-service data sharing. It
contains listings that reference shared datasets. With
Analytics Hub, publishers and administrators can grant access to
subscribers at the exchange and the listing level. This method helps to avoid
granting access on the underlying shared datasets explicitly. An
Analytics Hub subscriber can
browse through data exchanges, discover data that they can access, and subscribe
to shared datasets. A data exchange can be of the following types:
- Private data exchange. By default, a data exchange is private and only users or groups that have access to that exchange can view or subscribe to the data.
- Public data exchange. By default, a data exchange is private and only
users or groups that have access to that exchange can view or subscribe to its
listings. However, you can choose to make a data exchange public. Listings in
public data exchanges can be discovered
and subscribed by
Google Cloud users (
allAuthenticatedUsers). For more information about public data exchanges, see Make a data exchange public.
As an Analytics Hub administrator, you can create multiple data exchanges in Analytics Hub, and manage other Analytics Hub users.
- Listings
- A listing is a reference to a shared dataset that a publisher lists in
a data exchange. As a publisher, you can create a listing and specify the
dataset description, sample queries to run on the dataset, links to
any relevant documentation, and any additional information that can help
subscribers to use your dataset. For more information, see Manage
listings. A listing can be of the following two types based on the
Identity and Access Management (IAM) policy that is set for the listing and the type of data
exchange that contains the listing:
- Public listing. It is shared with all
Google Cloud users (
allAuthenticatedUsers). Listings in a public data exchange are public listings. These listings can be references of a free public dataset or a commercial dataset. If the listing is of a commercial dataset, subscribers can request access to the listing and the data provider contacts those subscribers directly. - Private listing. It is shared directly with individuals or groups. For example, a private listing can reference marketing metrics dataset that you share with other internal teams within your organization.
- Public listing. It is shared with all
Google Cloud users (
Subscriber workflow
The following diagram describes how subscribers interact with Analytics Hub:
In figure 2, the following Analytics Hub features are labeled: Shared dataset, Data exchange, Listing, and Linked dataset.
- Linked datasets
- A linked dataset is a read-only BigQuery dataset that serves as a symbolic link to a shared dataset. Subscribing to a listing creates a linked dataset in your project and not a copy of the dataset, so subscribers can read the data but cannot add or update objects within it. When you query objects such as tables and views through a linked dataset, the data from the shared dataset is returned. For more information about linked datasets, see View and subscribe to listings. Linked datasets are authorized to access tables and views of a shared dataset. Subscribers with linked datasets access tables and views of a shared dataset without any additional Identity and Access Management authorization.
Limitations
Analytics Hub has the following limitations:
If you create a listing for a shared dataset that uses a customer-managed encryption key, subscribers won't have access to the Cloud KMS key required to access the dataset.
A shared dataset can have a maximum of 1,000 linked datasets.
A dataset with unsupported resources cannot be selected as a shared dataset when you create a listing. For more information about the BigQuery objects that Analytics Hub supports, see Shared datasets in this document.
If you are a publisher, the following BigQuery interoperability limitations apply:
Subscribers can't query views within linked datasets that reference data from other projects. You must create authorized views to grant subscribers access to the view data without giving them access to the underlying source data.
The query plan reveals the shared view query, including project IDs, and other datasets involved in authorized views. Never include anything such as encryption keys that you consider sensitive in the shared view query.
Shared datasets are indexed in Data Catalog. Updates on a shared dataset, such as adding tables or views, are made available to subscribers without any delay. However, in certain scenarios, for example, when there are more than one hundred subscribers or tables in a shared dataset, the updates might take up to 18 hours to get indexed in Data Catalog. Due to the delay in indexing, subscribers cannot search for these updated resources in the Google Cloud console immediately.
If you have set up row-level security or data masking policies on the tables that are listed, then subscribers must be an Enterprise or Enterprise Plus customer to run the query job on linked dataset. For information about editions, see Introduction to BigQuery editions.
If you are a subscriber, the following BigQuery interoperability limitations apply:
Materialized views that refer to tables in the linked dataset are not supported.
Taking snapshots of linked dataset tables is not supported.
If linked datasets are not colocated with the shared dataset, then read operations to linked dataset tables with a query size of more than 1 TB might fail. You can contact support to resolve this issue.
You cannot use region qualifiers with
INFORMATION_SCHEMAviews to view metadata for your linked dataset.
Supported regions
Analytics Hub is supported is the following regions and multi-regions.
Regions
The following table lists the regions in the Americas where Analytics Hub is available.| Region description | Region name | Details |
|---|---|---|
| Iowa | us-central1 |
 Low CO2
|
| Las Vegas | us-west4 |
|
| Los Angeles | us-west2 |
|
| Montréal | northamerica-northeast1 |
Low CO2
|
| Northern Virginia | us-east4 |
|
| Oregon | us-west1 |
Low CO2 |
| Salt Lake City | us-west3 |
|
| São Paulo | southamerica-east1 |
Low CO2
|
| Santiago | southamerica-west1 |
|
| South Carolina | us-east1 |
|
| Toronto | northamerica-northeast2 |
|
| Region description | Region name | Details |
|---|---|---|
| Delhi | asia-south2 |
|
| Hong Kong | asia-east2 |
|
| Jakarta | asia-southeast2 |
|
| Melbourne | australia-southeast2 |
|
| Mumbai | asia-south1 |
|
| Osaka | asia-northeast2 |
|
| Seoul | asia-northeast3 |
|
| Singapore | asia-southeast1 |
|
| Sydney | australia-southeast1 |
|
| Taiwan | asia-east1 |
|
| Tokyo | asia-northeast1 |
| Region description | Region name | Details |
|---|---|---|
| Belgium | europe-west1 |
Low CO2
|
| Finland | europe-north1 |
Low CO2
|
| Frankfurt | europe-west3 |
|
| London | europe-west2 |
|
| Netherlands | europe-west4 |
|
| Warsaw | europe-central2 |
|
| Zürich | europe-west6 |
Low CO2
|
Multi-regions
The following table lists the multi-regions where Analytics Hub is available.| Multi-region description | Multi-region name |
|---|---|
| Data centers within member states of the European Union1 | EU |
| Data centers in the United States | US |
1 Data located in the EU multi-region is not
stored in the europe-west2 (London) or europe-west6 (Zürich) data
centers.
Omni regions
The following table lists the Omni where Analytics Hub is available.| Omni region description | Omni region name | |
|---|---|---|
| AWS | ||
| AWS - US East (N. Virginia) | aws-us-east-1 |
|
| Azure | ||
| Azure - East US 2 | azure-eastus2 |
|
Example use case
This section shows an example of how you can use Analytics Hub.
Suppose you are a retailer and your organization has real-time demand forecasting data in a Google Cloud project named Forecasting. You want to share this demand forecasting data with hundreds of vendors in your supply-chain system. Here's how you can share your data with vendors through Analytics Hub:
Analytics Hub administrators
As the owner of the Forecasting project, you must first enable the Analytics Hub API and then assign the Analytics Hub Admin role to a user who administers the data exchange in the project. Users with the Analytics Hub Admin role are called the Analytics Hub administrators.
An Analytics Hub administrator can perform the following tasks:
Create, update, delete, and share the data exchange in your organization's Forecasting project.
Manage other Analytics Hub administrators.
Manage publishers by granting the Analytics Hub Publisher role to your organization's employees. If you want some employees to only be able to update, delete, and share listings but not create them, then you can grant them the Analytics Hub Listing Admin role.
Manage subscribers by granting the Analytics Hub Subscriber role to a Google group consisting of all vendors. If you want some vendors to only have view access to the available exchanges and listings then you can grant them the Analytics Hub Viewer role. These vendors won't be able to subscribe to listings.
For more information, see Manage data exchanges.
Analytics Hub publishers
Publishers create the following listings for their datasets in the Forecasting project or in a different project:
- Listing A: Demand Forecast Dataset 1
- Listing B: Demand Forecast Dataset 2
- Listing C: Demand Forecast Dataset 3
For more information, see Manage listings.
Analytics Hub subscribers
Subscribers can browse through listings that they have access to in data exchanges. They can also subscribe to these listings and add these datasets to their projects by creating a linked dataset. Vendors can then run queries on these linked datasets and retrieve results in real time.
For more information, see View and subscribe to listings.
Pricing
There is no additional cost for managing data exchanges or listings. Analytics Hub publishers are charged for data storage, whereas subscribers pay for queries that run against the shared data based on either on-demand or flat-rate pricing model. For information about pricing, see BigQuery pricing.
Quotas
For information about Analytics Hub quotas, see Quotas and limits.
VPC Service Controls
You can set the ingress and egress rules needed to let publishers and subscribers access data from projects that have VPC Service Controls perimeters. For more information, see Analytics Hub VPC Service Controls rules.
What's next
- Learn how to view and subscribe to listings.
- Learn how to grant roles to Analytics Hub users.