Exactly-once delivery

This page explains how to receive and acknowledge messages using exactly-once semantics.

Only Java and Python client libraries are now updated to provide full support for exactly-once delivery subscriptions.

Exactly-once delivery guarantees

Pub/Sub supports exactly-once delivery, within a cloud region, based on a Pub/Sub-defined unique message ID.

When the feature is enabled, Pub/Sub provides the following guarantees:

No redelivery occurs once the message has been successfully acknowledged.
No redelivery occurs while a message is outstanding. A message is considered outstanding until the acknowledgment deadline expires or the message is acknowledged.
In case of multiple valid deliveries, due to acknowledgment deadline expiration or client-initiated negative acknowledgment, only the latest acknowledgment ID can be used to acknowledge the message. Any requests with a previous acknowledgment ID fail.

Redelivery versus duplicate

It is important to understand the difference between expected and unexpected redeliveries.

A redelivery can happen either because of client-initiated negative acknowledgment of a message or when the client doesn't extend the acknowledgment deadline of the message before the acknowledgment deadline expires. Redeliveries are considered valid and system working as intended.
A duplicate is when a message is resent after a successful acknowledgment or before acknowledgment deadline expiration.

Pub/Sub guarantees that subscriptions with exactly-once delivery enabled don't receive duplicate deliveries.

Create subscriptions with exactly-once delivery

All subscription types support exactly-once delivery. All subscribers can receive messages from subscriptions with exactly-once delivery, including subscribers that use the StreamingPull API.

You can create a subscription with exactly-once delivery using the Google Cloud console, the Google Cloud CLI, client library, or the Pub/Sub API.

Pull subscription

Console

To create a pull subscription with exactly-once delivery, follow these steps:

In the Google Cloud console, go to the Subscriptions page.

Go to Subscriptions
Click Create subscription.
Enter the Subscription ID.
Choose or create a topic from the drop-down menu.

The subscription receives messages from the topic.
In the Exactly once delivery section, select Enable exactly once delivery.
Click Create.

gcloud

To create a pull subscription with exactly-once delivery, use the gcloud pubsub subscriptions create command with the --enable-exactly-once-delivery flag:

gcloud pubsub subscriptions create SUBSCRIPTION_ID \
  --topic=TOPIC_ID \
  --enable-exactly-once-delivery

Replace the following:

SUBSCRIPTION_ID: the ID of the subscription to create
TOPIC_ID: the ID of the topic to attach to the subscription

REST

To create a subscription with exactly-once delivery, use the projects.subscriptions.create method.

PUT https://pubsub.googleapis.com/v1/projects/PROJECT_ID/subscriptions/SUBSCRIPTION_ID
Authorization: Bearer $(gcloud auth print-access-token)

Replace the following:

PROJECT_ID: the project ID for the project to create the subscription in
SUBSCRIPTION_ID: the ID of the subscription to create

To create a pull subscription with exactly-once delivery, specify this in the request body:

{
  "topic": "projects/PROJECT_ID/topics/TOPIC_ID",
  "enableExactlyOnceDelivery": true,
}

Replace the following:

PROJECT_ID: the project ID for the project with the topic
TOPIC_ID: the ID of the topic to attach to the subscription

Java

Before trying this sample, follow the Java setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Java API reference documentation.

View on GitHub Feedback

import com.google.cloud.pubsub.v1.SubscriptionAdminClient;
import com.google.pubsub.v1.ProjectSubscriptionName;
import com.google.pubsub.v1.ProjectTopicName;
import com.google.pubsub.v1.Subscription;
import java.io.IOException;

public class CreateSubscriptionWithExactlyOnceDelivery {
  public static void main(String... args) throws Exception {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "your-project-id";
    String topicId = "your-topic-id";
    String subscriptionId = "your-subscription-id";

    createSubscriptionWithExactlyOnceDeliveryExample(projectId, topicId, subscriptionId);
  }

  public static void createSubscriptionWithExactlyOnceDeliveryExample(
      String projectId, String topicId, String subscriptionId) throws IOException {
    try (SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create()) {

      ProjectTopicName topicName = ProjectTopicName.of(projectId, topicId);
      ProjectSubscriptionName subscriptionName =
          ProjectSubscriptionName.of(projectId, subscriptionId);

      Subscription subscription =
          subscriptionAdminClient.createSubscription(
              Subscription.newBuilder()
                  .setName(subscriptionName.toString())
                  .setTopic(topicName.toString())
                  // Enable exactly once delivery in the subscription.
                  .setEnableExactlyOnceDelivery(true)
                  .build());

      System.out.println(
          "Created a subscription with exactly once delivery enabled: "
              + subscription.getAllFields());
    }
  }
}

Python

Before trying this sample, follow the Python setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Python API reference documentation.

View on GitHub Feedback

from google.cloud import pubsub_v1

# TODO(developer): Choose an existing topic.
# project_id = "your-project-id"
# topic_id = "your-topic-id"
# subscription_id = "your-subscription-id"

publisher = pubsub_v1.PublisherClient()
subscriber = pubsub_v1.SubscriberClient()
topic_path = publisher.topic_path(project_id, topic_id)
subscription_path = subscriber.subscription_path(project_id, subscription_id)

with subscriber:
    subscription = subscriber.create_subscription(
        request={
            "name": subscription_path,
            "topic": topic_path,
            "enable_exactly_once_delivery": True,
        }
    )
    print(
        f"Created subscription with exactly once delivery enabled: {subscription}"
    )

Push subscription

Console

To create a push subscription with exactly-once delivery, follow these steps:

In the Google Cloud console, go to the Subscriptions page.

Go to Subscriptions
Click Create subscription.
Enter the Subscription ID.
Choose or create a topic from the drop-down menu. The subscription receives messages from the topic.
In the Delivery type section, click Push.
In the Endpoint URL field, enter the URL of the push endpoint.
In the Exactly once delivery section, select Enable exactly once delivery.
Click Create.

gcloud

To create a push subscription with exactly-once delivery, use the gcloud pubsub subscriptions create command with the --push-endpoint and --enable-exactly-once-delivery flags:

gcloud pubsub subscriptions create SUBSCRIPTION_ID \
  --topic=TOPIC_ID \
  --push-endpoint=PUSH_ENDPOINT \
  --enable-exactly-once-delivery

Replace the following:

SUBSCRIPTION_ID: the ID of the subscription to create
TOPIC_ID: the ID of the topic to attach to the subscription
PUSH_ENDPOINT: the URL of the server that the push subscriber runs on

REST

To create a push subscription with exactly-once delivery, use the projects.subscriptions.create method.

PUT https://pubsub.googleapis.com/v1/projects/PROJECT_ID/subscriptions/SUBSCRIPTION_ID
Authorization: Bearer $(gcloud auth print-access-token)

Replace the following:

PROJECT_ID: the project ID for the project to create the subscription in
SUBSCRIPTION_ID: the ID of the subscription to create

To create a push subscription with exactly-once delivery, specify the push endpoint and enable exactly-once delivery in the request body:

{
  "topic": "projects/PROJECT_ID/topics/TOPIC_ID",
  "pushConfig": {
    "pushEndpoint": "PUSH_ENDPOINT"
  },
  "enableExactlyOnceDelivery": true,
}

Replace the following:

PROJECT_ID: the project ID for the project with the topic
TOPIC_ID: the ID of the topic to attach to the subscription
PUSH_ENDPOINT: the URL of the server that the push subscriber runs on

Java

Before trying this sample, follow the Java setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Java API reference documentation.

View on GitHub Feedback

import com.google.cloud.pubsub.v1.SubscriptionAdminClient;
import com.google.pubsub.v1.ProjectSubscriptionName;
import com.google.pubsub.v1.ProjectTopicName;
import com.google.pubsub.v1.Subscription;
import java.io.IOException;

public class CreateSubscriptionWithExactlyOnceDelivery {
  public static void main(String... args) throws Exception {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "your-project-id";
    String topicId = "your-topic-id";
    String subscriptionId = "your-subscription-id";

    createSubscriptionWithExactlyOnceDeliveryExample(projectId, topicId, subscriptionId);
  }

  public static void createSubscriptionWithExactlyOnceDeliveryExample(
      String projectId, String topicId, String subscriptionId) throws IOException {
    try (SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create()) {

      ProjectTopicName topicName = ProjectTopicName.of(projectId, topicId);
      ProjectSubscriptionName subscriptionName =
          ProjectSubscriptionName.of(projectId, subscriptionId);

      Subscription subscription =
          subscriptionAdminClient.createSubscription(
              Subscription.newBuilder()
                  .setName(subscriptionName.toString())
                  .setTopic(topicName.toString())
                  // Enable exactly once delivery in the subscription.
                  .setEnableExactlyOnceDelivery(true)
                  .build());

      System.out.println(
          "Created a subscription with exactly once delivery enabled: "
              + subscription.getAllFields());
    }
  }
}

Python

Before trying this sample, follow the Python setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Python API reference documentation.

View on GitHub Feedback

from google.cloud import pubsub_v1

# TODO(developer): Choose an existing topic.
# project_id = "your-project-id"
# topic_id = "your-topic-id"
# subscription_id = "your-subscription-id"

publisher = pubsub_v1.PublisherClient()
subscriber = pubsub_v1.SubscriberClient()
topic_path = publisher.topic_path(project_id, topic_id)
subscription_path = subscriber.subscription_path(project_id, subscription_id)

with subscriber:
    subscription = subscriber.create_subscription(
        request={
            "name": subscription_path,
            "topic": topic_path,
            "enable_exactly_once_delivery": True,
        }
    )
    print(
        f"Created subscription with exactly once delivery enabled: {subscription}"
    )

Monitor exactly-once delivery subscriptions

The subscription/exactly_once_warning_count metric records the number of events that can lead to possible redeliveries (valid or duplicate). This metric counts the times Pub/Sub fails to process requests associated with acknowledgment IDs (ModifyAckDeadline or Acknowledgment request). The reasons for the failure could be server or client-based. For example, if the persistence layer used to maintain the exactly-once delivery information is unavailable, it would be a server-based event. If the client tries to acknowledge a message with an invalid acknowledgment ID, it would be a client based event.

Understand the metric

subscription/exactly_once_warning_count captures events that may or may not lead to actual redeliveries and can be noisy based on client behavior. For example: repeated Acknowledgment or ModifyAckDeadline requests with invalid acknowledgment IDs increment the metric repeatedly.

The following metrics are also useful in understanding the client behavior:

subscription/expired_ack_deadlines_count metric shows the number of acknowledgment ID expirations. Acknowledgment ID expirations can lead to failures for both ModifyAckDeadline and acknowledgment requests.
service.serviceruntime.googleapis.com/api/request_count metric can be used to capture failures of ModifyAckDeadline or acknowledgment requests in cases where the requests reach Google Cloud but do not reach Pub/Sub. There are failures that this metric won't capture—for example, when clients are disconnected from Google Cloud.

In most cases of retryable failure events, supported client libraries retry the request automatically.

Exactly-once delivery and push subscriptions

Pub/Sub supports exactly-once delivery with push subscriptions. Push subscriptions acknowledge messages by responding to the push requests with a successful response. However, clients don't know if Pub/Sub received the response and processed it. This is different from pull subscriptions where acknowledgment requests are initiated by the clients and Pub/Sub responds if the request was successfully processed. Because of this, it is important for exactly-once delivery push subscribers to monitor their exactly-once delivery subscriptions for acknowledgment processing failures.

Support for ordering keys

Exactly-once delivery doesn't have support for ordered delivery.

Things to know

If acknowledgment deadline is not specified at CreateSubscription time, exactly-once delivery enabled subscriptions will have a default acknowledgment deadline of 60 seconds.
Longer default acknowledgment deadlines are beneficial in avoiding redelivery caused by network events, specifically for push subscriptions. Supported client libraries don't use the default subscription acknowledgment deadline.
Exactly-once delivery subscriptions have significantly higher publish-to-subscribe latency compared to regular subscriptions.
A subscription may still receive multiple copies of a message when exactly-once delivery is enabled if the message is published multiple times as these copies are considered unique and have distinct message IDs.
Failures apart from invalid acknowledgment IDs in subscription/exactly_once_warning_count are retryable and the supported client libraries will retry them automatically.