This page explains how to receive and acknowledge messages using exactly-once semantics.
Only Java and Python client libraries are now updated to provide full support for exactly-once delivery subscriptions.
Exactly-once delivery guarantees
Pub/Sub supports exactly-once delivery, within a cloud region, based on a Pub/Sub-defined unique message ID.
When the feature is enabled, Pub/Sub provides the following guarantees:
No redelivery occurs once the message has been successfully acknowledged.
No redelivery occurs while a message is outstanding. A message is considered outstanding until the acknowledgment deadline expires or the message is acknowledged.
In case of multiple valid deliveries, due to acknowledgment deadline expiration or client-initiated negative acknowledgment, only the latest acknowledgment ID can be used to acknowledge the message. Any requests with a previous acknowledgment ID fail.
Redelivery versus duplicate
It is important to understand the difference between expected and unexpected redeliveries.
A redelivery can happen either because of client-initiated negative acknowledgment of a message or when the client doesn't extend the acknowledgment deadline of the message before the acknowledgment deadline expires. Redeliveries are considered valid and system working as intended.
A duplicate is when a message is resent after a successful acknowledgment or before acknowledgment deadline expiration.
Pub/Sub guarantees that subscriptions with exactly-once delivery enabled don't receive duplicate deliveries.
Create subscriptions with exactly-once delivery
All subscription types support exactly-once delivery. All subscribers can receive messages from subscriptions with exactly-once delivery, including subscribers that use the StreamingPull API.
You can create a subscription with exactly-once delivery using the Google Cloud console, the Google Cloud CLI, client library, or the Pub/Sub API.
Pull subscription
Console
To create a pull subscription with exactly-once delivery, follow these steps:
In the Google Cloud console, go to the Subscriptions page.
Click Create subscription.
Enter the Subscription ID.
Choose or create a topic from the drop-down menu.
The subscription receives messages from the topic.
In the Exactly once delivery section, select Enable exactly once delivery.
Click Create.
gcloud
To create a pull subscription with exactly-once delivery, use the
gcloud pubsub subscriptions create
command with the --enable-exactly-once-delivery
flag:
gcloud pubsub subscriptions create SUBSCRIPTION_ID \ --topic=TOPIC_ID \ --enable-exactly-once-delivery
Replace the following:
- SUBSCRIPTION_ID: the ID of the subscription to create
- TOPIC_ID: the ID of the topic to attach to the subscription
REST
To create a subscription with exactly-once delivery, use the
projects.subscriptions.create
method.
PUT https://pubsub.googleapis.com/v1/projects/PROJECT_ID/subscriptions/SUBSCRIPTION_ID Authorization: Bearer $(gcloud auth print-access-token)
Replace the following:
- PROJECT_ID: the project ID for the project to create the subscription in
- SUBSCRIPTION_ID: the ID of the subscription to create
To create a pull subscription with exactly-once delivery, specify this in the request body:
{ "topic": "projects/PROJECT_ID/topics/TOPIC_ID", "enableExactlyOnceDelivery": true, }
Replace the following:
- PROJECT_ID: the project ID for the project with the topic
- TOPIC_ID: the ID of the topic to attach to the subscription
Java
Before trying this sample, follow the Java setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Java API reference documentation.
Python
Before trying this sample, follow the Python setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Python API reference documentation.
Push subscription
Console
To create a push subscription with exactly-once delivery, follow these steps:
In the Google Cloud console, go to the Subscriptions page.
Click Create subscription.
Enter the Subscription ID.
Choose or create a topic from the drop-down menu. The subscription receives messages from the topic.
In the Delivery type section, click Push.
In the Endpoint URL field, enter the URL of the push endpoint.
In the Exactly once delivery section, select Enable exactly once delivery.
Click Create.
gcloud
To create a push subscription with exactly-once delivery, use the
gcloud pubsub subscriptions create
command with the --push-endpoint
and --enable-exactly-once-delivery
flags:
gcloud pubsub subscriptions create SUBSCRIPTION_ID \ --topic=TOPIC_ID \ --push-endpoint=PUSH_ENDPOINT \ --enable-exactly-once-delivery
Replace the following:
- SUBSCRIPTION_ID: the ID of the subscription to create
- TOPIC_ID: the ID of the topic to attach to the subscription
- PUSH_ENDPOINT: the URL of the server that the push subscriber runs on
REST
To create a push subscription with exactly-once delivery, use the
projects.subscriptions.create
method.
PUT https://pubsub.googleapis.com/v1/projects/PROJECT_ID/subscriptions/SUBSCRIPTION_ID Authorization: Bearer $(gcloud auth print-access-token)
Replace the following:
- PROJECT_ID: the project ID for the project to create the subscription in
- SUBSCRIPTION_ID: the ID of the subscription to create
To create a push subscription with exactly-once delivery, specify the push endpoint and enable exactly-once delivery in the request body:
{ "topic": "projects/PROJECT_ID/topics/TOPIC_ID", "pushConfig": { "pushEndpoint": "PUSH_ENDPOINT" }, "enableExactlyOnceDelivery": true, }
Replace the following:
- PROJECT_ID: the project ID for the project with the topic
- TOPIC_ID: the ID of the topic to attach to the subscription
- PUSH_ENDPOINT: the URL of the server that the push subscriber runs on
Java
Before trying this sample, follow the Java setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Java API reference documentation.
Python
Before trying this sample, follow the Python setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Python API reference documentation.
Monitor exactly-once delivery subscriptions
The
subscription/exactly_once_warning_count
metric records the number of events that
can lead to possible redeliveries (valid or duplicate). This metric counts the
times Pub/Sub fails to process requests associated with
acknowledgment IDs (ModifyAckDeadline or Acknowledgment request). The reasons
for the failure could be server or client-based. For example, if the persistence
layer used to maintain the exactly-once delivery information is unavailable, it
would be a server-based event. If the client tries to acknowledge a message with
an invalid acknowledgment ID, it would be a client based event.
Understand the metric
subscription/exactly_once_warning_count
captures events that may or may not
lead to actual redeliveries and can be noisy based on client behavior. For
example: repeated Acknowledgment or ModifyAckDeadline requests with invalid
acknowledgment IDs increment the metric repeatedly.
The following metrics are also useful in understanding the client behavior:
subscription/expired_ack_deadlines_count
metric shows the number of acknowledgment ID expirations. Acknowledgment ID expirations can lead to failures for both ModifyAckDeadline and acknowledgment requests.service.serviceruntime.googleapis.com/api/request_count
metric can be used to capture failures of ModifyAckDeadline or acknowledgment requests in cases where the requests reach Google Cloud but do not reach Pub/Sub. There are failures that this metric won't capture—for example, when clients are disconnected from Google Cloud.
In most cases of retryable failure events, supported client libraries retry the request automatically.
Exactly-once delivery and push subscriptions
Pub/Sub supports exactly-once delivery with push subscriptions. Push subscriptions acknowledge messages by responding to the push requests with a successful response. However, clients don't know if Pub/Sub received the response and processed it. This is different from pull subscriptions where acknowledgment requests are initiated by the clients and Pub/Sub responds if the request was successfully processed. Because of this, it is important for exactly-once delivery push subscribers to monitor their exactly-once delivery subscriptions for acknowledgment processing failures.
Support for ordering keys
Exactly-once delivery doesn't have support for ordered delivery.
Things to know
If acknowledgment deadline is not specified at CreateSubscription time, exactly-once delivery enabled subscriptions will have a default acknowledgment deadline of 60 seconds.
Longer default acknowledgment deadlines are beneficial in avoiding redelivery caused by network events, specifically for push subscriptions. Supported client libraries don't use the default subscription acknowledgment deadline.
Exactly-once delivery subscriptions have significantly higher publish-to-subscribe latency compared to regular subscriptions.
A subscription may still receive multiple copies of a message when exactly-once delivery is enabled if the message is published multiple times as these copies are considered unique and have distinct message IDs.
Failures apart from invalid acknowledgment IDs in
subscription/exactly_once_warning_count
are retryable and the supported client libraries will retry them automatically.