Choosing Pub/Sub or Pub/Sub Lite

Pub/Sub and Pub/Sub Lite are both horizontally scalable and managed messaging services. Pub/Sub is usually the default solution for most application integration and analytics use cases. Pub/Sub Lite is only recommended for applications where achieving extremely low cost justifies some additional operational work.

Pub/Sub offers a broader range of features, per-message parallelism, global routing, and automatically scaling resource capacity. In terms of message replication, Pub/Sub guarantees synchronous replication.

Pub/Sub Lite is a cost-effective solution that trades off operational workload, availability, and features for cost efficiency. Pub/Sub Lite requires you to manually reserve and manage resource capacity. Within Pub/Sub Lite, you can choose either zonal or regional Lite topics. Regional Lite topics offer the same availability SLA as Pub/Sub topics. However, there are reliability differences between the two services in terms of message replication.

Pub/Sub supports synchronous data replication and uses three zones. Pub/Sub guarantees synchronous replication to at least two zones and best effort replication to a third zone. Pub/Sub Lite supports asynchronous data replication. Regional Lite topics replicate data to a secondary zone. Zonal Lite topics replicate data in only one zone.

Business requirements checklist

The following questions can help you choose the right Pub/Sub messaging service:

  • Are reliability, minimal operational cost, and ease of development your top priorities? Use Cloud Pub/Sub.

  • Do you require the lowest per-throughput cost, are willing to incur in some additional operational overhead in terms of managing the topic capacity, and can work with availability risks inherent to zonal services? Use zonal Lite topics.

  • Do you require the lowest per-throughput cost, some protection against zonal failures, and are willing to incur an additional operational overhead in terms of managing the topic capacity? Use regional Lite topics.

Sample pricing scenario

Imagine a system that publishes and subscribes 10 MiBps (equivalent to 600 MiB per min or 25 TiB per month) of data.

The following is the list of sample billing in North America for different Pub/Sub configurations. Pub/Sub costs is usage- based. Pub/Sub Lite throughput costs is based on provisioned capacity. In this example, it is assumed that the ratio of actual throughput to provisioned capacity, or utilization, is 50%. In addition, it is assumed that Pub/Sub Lite storage is configured for 24 hours.

  • For Pub/Sub, the cost is $2000 per month.
  • For Pub/Sub Lite with zonal topics, the cost is $169 per month.
  • For Pub/Sub Lite with regional topics, the cost is $608 per month.

The preceding example is a very simplified sample of a billing model. Read more about the cost differences between Pub/Sub and Pub/Sub Lite.

Features comparison table

Feature Pub/Sub Pub/Sub Lite
Subscriber API Per-message leases, no partitions Ordered partitions with offsets used to track progress
Message replication Multi-zone in single region
  • Single zone for zonal Lite topics.
  • Dual zone in a single region for regional Lite topics.
Capacity Automatically provisioned Provision before you use
Pricing Pay for what you use Pay for the capacity that you provision
Storage Unlimited 30 GiB-10 TiB per Lite topic
Retention period Up to 31 days Unlimited
Service endpoints Global and regional Regional
Resource namespace Global
  • Zonal for zonal Lite topics.
  • Regional for regional Lite topics.
Message routing Global
  • Zonal for zonal Lite topics.
  • Regional for regional Lite topics.

Message routing

Both Pub/Sub and Pub/Sub Lite can publish and subscribe from all regions, and even outside of Google Cloud.

Pub/Sub routes messages globally. Applications hosted in any region can publish messages to a single topic. The messages are processed in the nearest region permitted by your organization policies and forwarded to subscribers independently of the source.

Pub/Sub Lite routes messages within a region or a zone. Subscribers must connect to the instance of the service in the region where the topic is located. As a result, publishers and subscribers must maintain network connectivity to the same region to communication. Publishers can publish data to any topic from anywhere, but this may require connecting to a remote region, adding network latency. Subscribers that need aggregate data published to topics in different locations must connect to every relevant local service instance to read the messages.

Resource namespace

Pub/Sub topics and subscriptions are global resources. The Pub/Sub service can store messages in any region allowed by a topic storage policy. In addition, a topic and a subscription to that topic can be stored in different projects, allowing for fine-grained billing and access control. Finally, subscribers get the same data regardless of which service endpoint they use to connect to Pub/Sub or which service endpoint was used by the publisher of the data.

Zonal Lite topics and subscriptions are zonal resources. These resources must be in the same Cloud project and zone.

Regional Lite topics and subscriptions are regional resources. These resources must be in the same Cloud project and region.

For a list of regions and zones that Pub/Sub Lite supports, see Pub/Sub Lite locations.

Capacity management

Pub/Sub scales automatically. Capacity for message publishing is automatically provisioned, so you can send and receive message at any rate, up to the quotas you configure for your project.

Pub/Sub Lite is manually provisioned. You must configure the number of partitions per Lite topic. You must also provision storage and throughput capacity. Throughput capacity can be provisioned using Lite reservations.

Delivery Modes

Pub/Sub supports multiple message delivery modes. These include Pull (traditional polling), Push (HTTPS POST), StreamingPull (bidirectional RPC, for lowest latency and highest efficiency), and pull over HTTP REST and gRPC.

Pub/Sub Lite only supports streaming gRPC pull message delivery. To request messages, subscribers establish bidirectional streaming connections with each partition.

Parallelism

Pub/Sub uses per-message parallelism. This enables concurrent processing of any number of messages, from one or thousands of clients for a single subscription.

Pub/Sub Lite uses partition-based parallelism. This forces a client to process each data partition in linear order on a single client instance. This limits the maximum number of client instances to at most the number of partitions.

Streaming pipelines

Both Pub/Sub and Pub/Sub Lite are integrated with Dataflow for serverless stream processing and data integration. Pub/Sub allows Dataflow to scale more flexibly and supports Dataflow SQL.

To get started with Dataflow and Pub/Sub or Pub/Sub Lite, try these tutorials: