Subscriber overview

This document gives an overview of how subscriptions work in Cloud Pub/Sub. For details on pull and push delivery subscriptions, see the Pull Subscriber Guide and the Push Subscriber Guide.

To receive messages published to a topic, you must create a subscription to that topic. Only messages published to the topic after the subscription is created are available to subscriber applications. The subscription connects the topic to a subscriber application that receives and processes messages published to the topic. A topic can have multiple subscriptions, but a given subscription belongs to a single topic.

To learn about creating and updating subscriptions, see Managing Topics and Subscriptions.

At-Least-Once delivery

Cloud Pub/Sub delivers each published message at least once for every subscription. There are some exceptions to this at-least-once behavior:

• By default, a message that cannot be delivered within the maximum retention time of 7 days is deleted and is no longer accessible. This typically happens when subscribers do not keep up with the flow of messages. Note that you can configure message retention duration (the range is from 10 minutes to 7 days). See Replaying & Discarding Messages for more information about the message retention setting.
• A message published before a given subscription was created will usually not be delivered. Thus, a message published to a topic with no subscription cannot be retrieved.

Once a message is sent to a subscriber, the subscriber must either acknowledge or drop the message. A message is considered outstanding once it has been sent out for delivery and before a subscriber acknowledges it. Cloud Pub/Sub will repeatedly attempt to deliver any message that has not been acknowledged or that is not outstanding. A subscriber has a configurable, limited amount of time, or ackDeadline, to acknowledge the message. Once the deadline has passed, an outstanding message becomes unacknowledged.

Typically, Cloud Pub/Sub delivers each message once and in the order in which it was published. However, messages may sometimes be delivered out of order or more than once. In general, accommodating more-than-once delivery requires your subscriber to be idempotent when processing messages. You can achieve exactly once processing of Cloud Pub/Sub message streams using Cloud Dataflow PubsubIO. PubsubIO de-duplicates messages on custom message identifiers or those assigned by Cloud Pub/Sub. You can also achieve ordered processing with Cloud Dataflow by using the standard sorting APIs of the service. Alternatively, to achieve ordering, the publisher of the topic to which you subscribe can include a sequence token in the message. See Message Ordering for more information.

Pull or push delivery

A subscription can use either the pull or push mechanism for message delivery. You can change or configure the mechanism at any time.

Pull subscription

In pull delivery, your subscriber application initiates requests to the Cloud Pub/Sub server to retrieve messages.

1. The subscribing application explicitly calls the pull method, which requests messages for delivery.
2. The Cloud Pub/Sub server responds with the message (or an error if the queue is empty) , and an ack ID.
3. The subscriber explicitly calls the acknowledge method, using the returned ack ID to acknowledge receipt.

Push subscription

In push delivery, Cloud Pub/Sub initiates requests to your subscriber application to deliver messages.

1. The Cloud Pub/Sub server sends each message as an HTTPS request to the subscriber application at a pre-configured endpoint.
2. The endpoint acknowledges the message by returning an HTTP success status code. A non-success response indicates that the message should be resent.

Cloud Pub/Sub dynamically adjusts the rate of push requests based on the rate at which it receives success responses.

The following table offers some guidance in choosing the appropriate delivery mechanism for your application:

Pull	Push
Large volume of messages (many more than 1/second). Efficiency and throughput of message processing is critical. Public HTTPS endpoint, with non-self-signed SSL certificate, is not feasible to set up.	Multiple topics that must be processed by the same webhook. App Engine Standard and Cloud Functions subscribers. Environments where Google Cloud Platform dependencies (such as credentials and the client library) are not feasible to set up.

The following table compares pull and push delivery:

	Pull	Push
Endpoints	Any device on the internet that has authorized credentials is able to call the Cloud Pub/Sub API.	An HTTPS server with non-self-signed certificate accessible on the public web. The receiving endpoint may be decoupled from the Cloud Pub/Sub subscription, so that messages from multiple subscriptions may be sent to a single endpoint.
Load balancing	Multiple subscribers can make pull calls to the same "shared" subscription. Each subscriber will receive a subset of the messages.	The push endpoint can be a load balancer.
Configuration	No configuration is necessary.	No configuration is necessary for App Engine apps in the same project as the subscriber. Configuration (and verification) of push endpoints is required in the Google Cloud Platform Console for all other endpoints. Endpoints must be reachable via DNS names and have SSL certificates installed.
Flow control	The subscriber client controls the rate of delivery. The subscriber can dynamically modify the ack deadline, allowing message processing to be arbitrarily long.	The Cloud Pub/Sub server automatically implements flow control. There is no need to handle message flow at the client side, although it is possible to indicate that the client cannot handle the current message load by passing back an HTTP error.
Efficiency and throughput	Achieves high throughput at low CPU and bandwidth by allowing batched delivery and acknowledgments as well as massively parallel consumption. May be inefficient if aggressive polling is used to minimize message delivery time.	Delivers one message per request and limits maximum number of outstanding messages.

Lifecycle of a subscription

By default, subscriptions expire after 31 days of inactivity (for instance, if there are no active connections, pull requests, or push successes). If Cloud Pub/Sub detects subscriber activity, the subscription deletion clock restarts. Using subscription expiration policies (Beta) , you can configure the inactivity duration or make the subscription persistent regardless of activity. You can also delete a subscription manually.

Note that although you can create new a subscription with the same name as a deleted one, the new subscription has no relationship to the old one. Even if the deleted subscription had a large number of unacknowledged messages, a new identically-named subscription would have no backlog (no messages waiting for delivery) at the time it is created.

Configuring subscriptions

A subscription is created for a single topic. It has several properties that can be set at creation time or updated later, including:

• Delivery method: By default, Cloud Pub/Sub subscriptions use the pull method. You can switch to push delivery by specifying a non-empty, valid HTTPS push endpoint URL. You can switch back to pull delivery by specifying an empty URL.
• An acknowledgment deadline: If your code doesn't acknowledge the message before the deadline, the message is sent again. The default is 10 seconds. The maximum custom deadline you can specify is 600 seconds (10 minutes).

Modifying subscription expiration policies

Beta

This is a beta release of Subscription expiration policies. This API might be changed in backward-incompatible ways and is not subject to any SLA or deprecation policy.

By default, subscriptions expire after 31 days of inactivity. If Cloud Pub/Sub detects subscriber activity (such as open connections, active pulls, or successful pushes), the subscription deletion clock restarts. Any messages in persisted in the deleted subscription might also be lost, regardless of the message's age.

To change this behavior, modify the subscription's expiration policy. You can set the policy duration when you create the subscription or modify it later. The minimum allowed value for an expiration policy is 1 day. There is no maximum. Use any of the following options to configure the policy:

• The duration and expiration flags available in the gcloud beta subscription commands. Examples:
  
  

      alias pubsub='gcloud beta pubsub'
      # subscription that expires after one day of inactivity
      pubsub subscriptions create mySubscription --expiration-period=1d --topic myTopic
      # make the subscription non-expiring
      pubsub subscriptions update mySubscription --expiration-period=never
      

• Support for expiration policies is available in the Cloud Pub/Sub Client Libraries.

• RPC methods:
• CreateSubscription
• UpdateSubscription

• REST method:
• projects.subscriptions.create

Retaining unacknowledged and acknowledged messages

By default, Cloud Pub/Sub ensures that subscriptions retain unacknowledged messages for 7 days from the moment of publication. After the duration specified in this property, messages may be lost regardless of their acknowledgment state. As part of the Replaying and Discarding Messages beta, you can use messageRetentionDuration to specify the amount of time that a subscription should retain messages, up to a maximum of 7 days. You can also set the subscription to retain acknowledged messages for this duration.

Note: The default subscription expiration policy is 31 days and the default message retention duration is 7 days. However, if you set or modify either of these values, the subscription's expiration policy must be greater than or equal to the message retention duration.