Choose a subscription type

This document gives an overview of how different types of subscriptions work in Pub/Sub.

Key Point: Learn about a subscription, the different types of subscriptions, and how to choose a subscription suited to your business needs.

Subscription overview

To receive messages published to a topic, you must create a subscription to that topic. Only messages published to the topic after the subscription is created are available to subscriber clients. However, you can also enable topic retention to allow a subscription attached to the topic to seek back in time and replay previously published messages. The subscriber client receives and processes the messages published to the topic. A topic can have multiple subscriptions, but a given subscription belongs to a single topic.

Subscription workflow

1. After a message is sent to a subscriber, the subscriber must acknowledge the message.

2. If a message is sent out for delivery and a subscriber is yet to acknowledge it, the message is called outstanding.

3. Pub/Sub repeatedly attempts to deliver any message that is not yet acknowledged. However, Pub/Sub tries not to deliver an outstanding message to any other subscriber on the same subscription.

4. The subscriber has a configurable, limited amount of time, known as the ackDeadline, to acknowledge the outstanding message. After the deadline passes, the message is no longer considered outstanding, and Pub/Sub attempts to redeliver the message.

Types of subscriptions

When you create a subscription, you must specify the type of message delivery. Pub/Sub offers three types of message delivery that corresponds to the following three types of subscriptions. Each type of subscription is described in brief in later sections of this document.

• Pull subscription
• Push subscription
• BigQuery subscription

You can update the type of subscription at any time.

Pull subscription

For a pull subscription, your subscriber client initiates requests to a Pub/Sub server to retrieve messages. The subscriber client uses the REST Pull API, the RPC PullRequest API, the REST StreamingPullRequest API, or the RPC StreamingPullRequest API. Most subscriber clients do not make these requests directly. Instead, the clients rely on the Google Cloud-provided high-level client library that performs streaming pull requests internally and delivers messages asynchronously. For a subscriber client that needs greater control over how messages are pulled, Pub/Sub uses a low-level and automatically generated gRPC library. This library makes pull or streaming pull requests directly. These requests can be synchronous or asynchronous.

The following two images show the workflow between a subscriber client and a pull subscription.

Figure 1. Workflow for a pull subscription

Figure 2. Workflow for a streaming pull subscription

The pull workflow is as follows and references Figure 1:

1. The subscriber client explicitly calls the pull method, which requests messages for delivery. This request is the PullRequest as shown in the image.

2. The Pub/Sub server responds with zero or more messages and acknowledgment IDs. A response with zero messages or with an error does not necessarily indicate that there are no messages available to receive. This response is the PullResponse as shown in the image.

3. The subscriber client explicitly calls the acknowledged method. The client uses the returned acknowledgment ID to acknowledge that the message is processed and need not be delivered again. This request is the AckRequest as shown in the image.

For a single streaming pull request, a subscriber client can have multiple responses returned due to the open connection. In contrast, only one response is returned for each pull request.

For more information about how a pull subscription works and examples of configuration, see Pull subscriptions.

Push subscription

In a push subscription, a Pub/Sub server initiates a request to your subscriber client to deliver messages.

The following image shows the workflow between a subscriber client and a push subscription.

Figure 3. Workflow for a push subscription

Here's a brief description of the workflow that references Figure 3:

1. The Pub/Sub server sends each message as an HTTPS request to the subscriber client at a pre-configured endpoint. This request is shown as a PushRequest in the image.

2. The endpoint acknowledges the message by returning an HTTP success status code. A non-success response indicates that Pub/Sub must resend the messages. This response is shown as a PushResponse in the image.

3. Pub/Sub dynamically adjusts the rate of push requests based on the rate at which it receives success responses.

For more information about how a push subscription works and examples of configuration, see Push subscriptions.

BigQuery subscription

A BigQuery subscription writes messages to an existing BigQuery table as they are received. You need not configure a separate subscriber client.

Without the BigQuery subscription type, you need a pull or push subscription and a subscriber (such as Dataflow) that reads messages and writes them to a BigQuery table. The overhead of running a Dataflow job is not necessary when messages do not require additional processing before storing them in a BigQuery table; you can use a BigQuery subscription instead. However, a Dataflow pipeline is still recommended for Pub/Sub systems where some data transformation is required before the data is stored in a BigQuery table. To learn how to stream data from Pub/Sub to BigQuery with transformation by using Dataflow, see Stream from Pub/Sub to BigQuery.

The following image shows the workflow between a BigQuery subscription and BigQuery.

Figure 4. Workflow for a BigQuery subscription

Here is a brief description of the workflow that references Figure 4:

1. Pub/Sub uses the BigQuery storage write API to send data to the BigQuery table.
2. The messages are sent in batches to the BigQuery table.
3. After a successful completion of a write operation, the API returns an OK response.
4. If there are any failures in the write operation, the Pub/Sub message itself is negatively acknowledged. The message is then re-sent. If the message fails enough times and there's a dead letter topic configured on the subscription, then the message is moved to the dead letter topic.

For more information about how a BigQuery subscription works, see BigQuery subscriptions.

Decide on your subscription type

The following table offers some guidance in choosing the appropriate delivery mechanism for your application:

	Pull	Push	BigQuery
Use case	Large volume of messages (GBs per second). Efficiency and throughput of message processing is critical. Environments where a public HTTPS endpoint with a non-self-signed SSL certificate is not feasible to set up.	Multiple topics that must be processed by the same webhook. App Engine Standard and Cloud Functions subscribers. Environments where Google Cloud dependencies (such as credentials and the client library) are not feasible to set up.	Large volume of messages that can scale up to multiple millions of messages per second. Messages are directly sent to BigQuery without any additional processing.
Endpoints	Any device on the internet that has authorized credentials is able to call the Pub/Sub API.	An HTTPS server with non-self-signed certificate accessible on the public web. The receiving endpoint may be decoupled from the Pub/Sub subscription, so that messages from multiple subscriptions may be sent to a single endpoint.	A BigQuery table.
Load balancing	Multiple subscribers can make pull calls to the same "shared" subscription. Each subscriber receives a subset of the messages.	The push endpoint can be a load balancer.	The Pub/Sub service automatically balances the load.
Configuration	No configuration is necessary.	No configuration is necessary for App Engine apps in the same project as the subscriber. Verification of push endpoints is not required in the Google Cloud console. Endpoints must be reachable using DNS names and have SSL certificates installed.	A BigQuery table must exist for the topic subscription.
Flow control	The subscriber client controls the rate of delivery. The subscriber can dynamically modify the acknowledgment deadline, allowing message processing to be arbitrarily long.	The Pub/Sub server automatically implements flow control. There's no need to handle message flow at the client side. However, it's possible to indicate that the client cannot handle the current message load by passing back an HTTP error.	The Pub/Sub server automatically implements flow control in order to optimize writing messages to BigQuery.
Efficiency and throughput	Achieves high throughput at low CPU and bandwidth by allowing batched delivery and acknowledgments as well as massively parallel consumption. May be inefficient if aggressive polling is used to minimize message delivery time.	Delivers one message per request and limits the maximum number of outstanding messages.	Scalability is dynamically handled by Pub/Sub servers.

Default subscription properties

By default, Pub/Sub offers at-least-once delivery with no ordering guarantees on all subscription types. Alternatively, if messages have the same ordering key and are in the same region, you can enable message ordering. After you set the message ordering property, the Pub/Sub service delivers messages with the same ordering key and in the order that the Pub/Sub service receives the messages.

Pub/Sub also supports exactly-once delivery.

In general, Pub/Sub delivers each message once and in the order in which it was published. However, messages may sometimes be delivered out of order or more than once. Pub/Sub might redeliver a message even after an acknowledgement request for the message returns successfully. This redelivery can be caused by issues such as server-side restarts or client-side issues. Thus, although rare, any message can be redelivered at any time. Accommodating more-than-once delivery requires your subscriber to be idempotent when processing messages.

Subscription expiry

By default, subscriptions expire after 31 days of subscriber inactivity or if there are no updates made to the subscription. Examples of subscriber activities include open connections, active pulls, or successful pushes. If Pub/Sub detects subscriber activity or an update to the subscription properties, the subscription deletion clock restarts. Using subscription expiration policies, you can configure the inactivity duration or make the subscription persistent regardless of activity. You can also delete a subscription manually.

Although you can create a new subscription with the same name as a deleted one, the new subscription has no relationship to the old one. Even if the deleted subscription had many unacknowledged messages, a new subscription created with the same name would have no backlog (no messages waiting for delivery) at the time it's created.

For more information about working with subscriptions, see Create and use subscriptions.

What's next

• Learn more about pull subscriptions.
• Learn more about push subscriptions.
• Learn more about BigQuery subscriptions.