Exactly-once delivery

Stay organized with collections Save and categorize content based on your preferences.

This page explains how to receive and acknowledge messages using exactly-once semantics. Only the pull subscription type supports exactly-once delivery, including subscribers that use the StreamingPull API.

Exactly-once delivery guarantees

Pub/Sub supports exactly-once delivery, within a cloud region, based on a Pub/Sub-defined unique message ID.

When the feature is enabled, Pub/Sub provides the following guarantees:

  • No redelivery occurs once the message has been successfully acknowledged.

  • No redelivery occurs while a message is outstanding. A message is considered outstanding until the acknowledgment deadline expires or the message is acknowledged.

  • In case of multiple valid deliveries, due to acknowledgment deadline expiration or client-initiated negative acknowledgment, only the latest acknowledgment ID can be used to acknowledge the message. Any requests with a previous acknowledgment ID fail.

Redelivery versus duplicate

It is important to understand the difference between expected and unexpected redeliveries.

  • A redelivery can happen either because of client-initiated negative acknowledgment of a message or when the client doesn't extend the acknowledgment deadline of the message before the acknowledgment deadline expires. Redeliveries are considered valid and system working as intended.

  • A duplicate is when a message is resent after a successful acknowledgment or before acknowledgment deadline expiration.

Pub/Sub guarantees that subscriptions with exactly-once delivery enabled don't receive duplicate deliveries.

Exactly-once delivery support in client libraries

  • Supported client libraries have an interface for acknowledgment with response (example: Go). You can use this interface to check if the acknowledgment request succeeded. If the acknowledgment request succeeds, the clients are guaranteed to not receive a re-delivery. If the acknowledgment request fails, the clients can expect a re-delivery.

  • Clients can also use the supported client libraries without the acknowledgment interface. However, in such cases, the acknowledgment failures can lead to silent re-deliveries of messages.

  • Supported client libraries have interfaces for setting the minimum lease extension time (example: Go). You must set the value for the minimum lease extension to a high number to avoid any network-related acknowledgment expirations. The maximum value is set at 600 seconds.

Create subscriptions with exactly-once delivery

You can create a subscription with exactly-once delivery using the Google Cloud console, the Google Cloud CLI, client library, or the Pub/Sub API.

Pull subscription

Console

To create a pull subscription with exactly-once delivery, follow these steps:

  1. In the Google Cloud console, go to the Subscriptions page.

    Go to Subscriptions

  2. Click Create subscription.

  3. Enter the Subscription ID.

  4. Choose or create a topic from the drop-down menu.

    The subscription receives messages from the topic.

  5. In the Exactly once delivery section, select Enable exactly once delivery.

  6. Click Create.

gcloud

To create a pull subscription with exactly-once delivery, use the gcloud pubsub subscriptions create command with the --enable-exactly-once-delivery flag:

gcloud pubsub subscriptions create SUBSCRIPTION_ID \
  --topic=TOPIC_ID \
  --enable-exactly-once-delivery

Replace the following:

  • SUBSCRIPTION_ID: the ID of the subscription to create
  • TOPIC_ID: the ID of the topic to attach to the subscription

REST

To create a subscription with exactly-once delivery, use the projects.subscriptions.create method.

PUT https://pubsub.googleapis.com/v1/projects/PROJECT_ID/subscriptions/SUBSCRIPTION_ID
Authorization: Bearer $(gcloud auth print-access-token)

Replace the following:

  • PROJECT_ID: the project ID for the project to create the subscription in
  • SUBSCRIPTION_ID: the ID of the subscription to create

To create a pull subscription with exactly-once delivery, specify this in the request body:

{
  "topic": "projects/PROJECT_ID/topics/TOPIC_ID",
  "enableExactlyOnceDelivery": true,
}

Replace the following:

  • PROJECT_ID: the project ID for the project with the topic
  • TOPIC_ID: the ID of the topic to attach to the subscription

C++

Before trying this sample, follow the C++ setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub C++ API reference documentation. 

namespace pubsub = ::google::cloud::pubsub;
[](pubsub::SubscriptionAdminClient client, std::string const& project_id,
   std::string const& topic_id, std::string const& subscription_id) {
  auto sub = client.CreateSubscription(
      pubsub::Topic(project_id, topic_id),
      pubsub::Subscription(project_id, subscription_id),
      pubsub::SubscriptionBuilder{}.enable_exactly_once_delivery(true));
  if (sub.status().code() == google::cloud::StatusCode::kAlreadyExists) {
    std::cout << "The subscription already exists\n";
    return;
  }
  if (!sub) throw std::runtime_error(sub.status().message());

  std::cout << "The subscription was successfully created: "
            << sub->DebugString() << "\n";
}

C#

Before trying this sample, follow the C# setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub C# API reference documentation. 


using Google.Cloud.PubSub.V1;
using Grpc.Core;

public class CreateSubscriptionWithExactlyOnceDeliverySample
{
    public Subscription CreateSubscriptionWithExactlyOnceDelivery(string projectId, string topicId, string subscriptionId)
    {
        SubscriberServiceApiClient subscriber = SubscriberServiceApiClient.Create();
        TopicName topicName = TopicName.FromProjectTopic(projectId, topicId);
        SubscriptionName subscriptionName = SubscriptionName.FromProjectSubscription(projectId, subscriptionId);

        var subscriptionRequest = new Subscription
        {
            SubscriptionName = subscriptionName,
            TopicAsTopicName = topicName,
            EnableExactlyOnceDelivery = true
        };

        Subscription subscription = null;

        try
        {
            subscription = subscriber.CreateSubscription(subscriptionRequest);
        }
        catch (RpcException e) when (e.Status.StatusCode == StatusCode.AlreadyExists)
        {
            // Already exists.  That's fine.
        }
        return subscription;
    }
}

Go

Before trying this sample, follow the Go setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Go API reference documentation. 

import (
	"context"
	"fmt"
	"io"

	"cloud.google.com/go/pubsub"
)

func createSubscriptionWithExactlyOnceDelivery(w io.Writer, projectID, subID string, topic *pubsub.Topic) error {
	// projectID := "my-project-id"
	// subID := "my-sub"
	// topic of type https://godoc.org/cloud.google.com/go/pubsub#Topic
	ctx := context.Background()
	client, err := pubsub.NewClient(ctx, projectID)
	if err != nil {
		return fmt.Errorf("pubsub.NewClient: %v", err)
	}
	defer client.Close()

	sub, err := client.CreateSubscription(ctx, subID, pubsub.SubscriptionConfig{
		Topic:                     topic,
		EnableExactlyOnceDelivery: true,
	})
	if err != nil {
		return err
	}
	fmt.Fprintf(w, "Created a subscription with exactly once delivery enabled: %v\n", sub)
	return nil
}

Java

Before trying this sample, follow the Java setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Java API reference documentation. 

import com.google.cloud.pubsub.v1.SubscriptionAdminClient;
import com.google.pubsub.v1.ProjectSubscriptionName;
import com.google.pubsub.v1.ProjectTopicName;
import com.google.pubsub.v1.Subscription;
import java.io.IOException;

public class CreateSubscriptionWithExactlyOnceDelivery {
  public static void main(String... args) throws Exception {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "your-project-id";
    String topicId = "your-topic-id";
    String subscriptionId = "your-subscription-id";

    createSubscriptionWithExactlyOnceDeliveryExample(projectId, topicId, subscriptionId);
  }

  public static void createSubscriptionWithExactlyOnceDeliveryExample(
      String projectId, String topicId, String subscriptionId) throws IOException {
    try (SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create()) {

      ProjectTopicName topicName = ProjectTopicName.of(projectId, topicId);
      ProjectSubscriptionName subscriptionName =
          ProjectSubscriptionName.of(projectId, subscriptionId);

      Subscription subscription =
          subscriptionAdminClient.createSubscription(
              Subscription.newBuilder()
                  .setName(subscriptionName.toString())
                  .setTopic(topicName.toString())
                  // Enable exactly once delivery in the subscription.
                  .setEnableExactlyOnceDelivery(true)
                  .build());

      System.out.println(
          "Created a subscription with exactly once delivery enabled: "
              + subscription.getAllFields());
    }
  }
}

Python

Before trying this sample, follow the Python setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Python API reference documentation. 

from google.cloud import pubsub_v1

# TODO(developer): Choose an existing topic.
# project_id = "your-project-id"
# topic_id = "your-topic-id"
# subscription_id = "your-subscription-id"

publisher = pubsub_v1.PublisherClient()
subscriber = pubsub_v1.SubscriberClient()
topic_path = publisher.topic_path(project_id, topic_id)
subscription_path = subscriber.subscription_path(project_id, subscription_id)

with subscriber:
    subscription = subscriber.create_subscription(
        request={
            "name": subscription_path,
            "topic": topic_path,
            "enable_exactly_once_delivery": True,
        }
    )
    print(
        f"Created subscription with exactly once delivery enabled: {subscription}"
    )

Node(JavaScript)

Before trying this sample, follow the Node.js setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Node.js API reference documentation. 

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const topicNameOrId = 'YOUR_TOPIC_NAME_OR_ID';
// const subscriptionNameOrId = 'YOUR_SUBSCRIPTION_NAME_OR_ID';

// Imports the Google Cloud client library
const {PubSub} = require('@google-cloud/pubsub');

// Creates a client; cache this for further use
const pubSubClient = new PubSub();

async function createSubscriptionWithExactlyOnceDelivery(
  topicNameOrId,
  subscriptionNameOrId
) {
  // Creates a new subscription
  await pubSubClient
    .topic(topicNameOrId)
    .createSubscription(subscriptionNameOrId, {
      enableExactlyOnceDelivery: true,
    });
  console.log(
    `Created subscription ${subscriptionNameOrId} with exactly-once delivery.`
  );
  console.log(
    'To process messages, remember to check the return value of ackWithResponse().'
  );
}

Node(TypeScript)

Before trying this sample, follow the Node.js setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Node.js API reference documentation. 

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const topicNameOrId = 'YOUR_TOPIC_NAME_OR_ID';
// const subscriptionNameOrId = 'YOUR_SUBSCRIPTION_NAME_OR_ID';

// Imports the Google Cloud client library
import {PubSub} from '@google-cloud/pubsub';

// Creates a client; cache this for further use
const pubSubClient = new PubSub();

async function createSubscriptionWithExactlyOnceDelivery(
  topicNameOrId: string,
  subscriptionNameOrId: string
) {
  // Creates a new subscription
  await pubSubClient
    .topic(topicNameOrId)
    .createSubscription(subscriptionNameOrId, {
      enableExactlyOnceDelivery: true,
    });
  console.log(
    `Created subscription ${subscriptionNameOrId} with exactly-once delivery.`
  );
  console.log(
    'To process messages, remember to check the return value of ackWithResponse().'
  );
}

Ruby

Before trying this sample, follow the Ruby setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Ruby API reference documentation. 

require "google/cloud/pubsub"

# Shows how to create a new subscription with exactly once delivery enabled
class PubsubCreateSubscriptionWithExactlyOnceDelivery
  def create_subscription_with_exactly_once_delivery project_id:, topic_id:, subscription_id:
    pubsub = Google::Cloud::Pubsub.new project_id: project_id
    topic = pubsub.topic topic_id
    subscription = topic.subscribe subscription_id, enable_exactly_once_delivery: true
    puts "Created subscription with exactly once delivery enabled: #{subscription_id}"
  end

  def self.run
    # TODO(developer): Replace these variables before running the sample.
    project_id = "your-project-id"
    topic_id = "your-topic-id"
    subscription_id = "id-for-new-subcription"
    pubsub_create_subscription_with_exactly_once_delivery = PubsubCreateSubscriptionWithExactlyOnceDelivery.new
    pubsub_create_subscription_with_exactly_once_delivery.create_subscription_with_exactly_once_delivery(
      project_id: project_id,
      topic_id: topic_id,
      subscription_id: subscription_id
    )
  end
end

if $PROGRAM_NAME == __FILE__
  PubsubCreateSubscriptionWithExactlyOnceDelivery.run
end

PHP

Before trying this sample, follow the PHP setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub PHP API reference documentation. 

use Google\Cloud\PubSub\PubSubClient;

/**
 * Creates a Pub/Sub subscription with `Exactly Once Delivery` enabled.
 *
 * @param string $projectId  The Google project ID.
 * @param string $topicName  The Pub/Sub topic name.
 * @param string $subscriptionName  The Pub/Sub subscription name.
 */
function create_subscription_with_exactly_once_delivery(
    string $projectId,
    string $topicName,
    string $subscriptionName
): void {
    $pubsub = new PubSubClient([
        'projectId' => $projectId,
    ]);
    $topic = $pubsub->topic($topicName);
    $subscription = $topic->subscription($subscriptionName);
    $subscription->create([
        'enableExactlyOnceDelivery' => true
    ]);

    // Exactly Once Delivery status for the subscription
    $status = $subscription->info()['enableExactlyOnceDelivery'];

    printf('Subscription created with exactly once delivery status: %s' . PHP_EOL, $status ? 'true' : 'false');
}

Monitor exactly-once delivery subscriptions

The subscription/exactly_once_warning_count metric records the number of events that can lead to possible redeliveries (valid or duplicate). This metric counts the times Pub/Sub fails to process requests associated with acknowledgment IDs (ModifyAckDeadline or Acknowledgment request). The reasons for the failure could be server or client-based. For example, if the persistence layer used to maintain the exactly-once delivery information is unavailable, it would be a server-based event. If the client tries to acknowledge a message with an invalid acknowledgment ID, it would be a client based event.

Understand the metric

subscription/exactly_once_warning_count captures events that may or may not lead to actual redeliveries and can be noisy based on client behavior. For example: repeated Acknowledgment or ModifyAckDeadline requests with invalid acknowledgment IDs increment the metric repeatedly.

The following metrics are also useful in understanding the client behavior:

  • subscription/expired_ack_deadlines_count metric shows the number of acknowledgment ID expirations. Acknowledgment ID expirations can lead to failures for both ModifyAckDeadline and acknowledgment requests.

  • service.serviceruntime.googleapis.com/api/request_count metric can be used to capture failures of ModifyAckDeadline or acknowledgment requests in cases where the requests reach Google Cloud but do not reach Pub/Sub. There are failures that this metric won't capture—for example, when clients are disconnected from Google Cloud.

In most cases of retryable failure events, supported client libraries retry the request automatically.

Quotas

Exactly-once delivery subscriptions are subjected to additional quota requirements. These quota are enforced on:

  • Number of messages consumed from subscriptions with exactly-once delivery enabled per region.
  • Number of messages acknowledged or whose deadline is extended when using subscriptions with exactly-once delivery enabled per region.

For more information regarding these quotas, see the table in the Quotas topic.

Exactly-once delivery and push subscriptions

Pub/Sub supports exactly-once delivery only with Pull subscriptions.

Push subscriptions acknowledge messages by responding to the push requests with a successful response. However, clients don't know if Pub/Sub received the response and processed it. This is different from pull subscriptions where acknowledgment requests are initiated by the clients and Pub/Sub responds if the request was successfully processed. Because of this, exactly-once delivery sematics do not align well with push subscriptions.

Support for ordering keys

Exactly-once delivery doesn't have support for ordered delivery.

Things to know

  • If acknowledgment deadline is not specified at CreateSubscription time, exactly-once delivery enabled subscriptions will have a default acknowledgment deadline of 60 seconds.

  • Longer default acknowledgment deadlines are beneficial in avoiding redelivery caused by network events. Supported client libraries don't use the default subscription acknowledgment deadline.

  • Exactly-once delivery subscriptions have significantly higher publish-to-subscribe latency compared to regular subscriptions.

  • A subscription might receive multiple copies of the same message due to publish side duplicates, even with exactly-once delivery enabled. Publish-side duplicates can be due to multiple unique publish retries by the publishing client or the Pub/Sub service. Multiple unique publishes by the publishing client, across retries, leads to redeliveries with different message IDs. Multiple unique publishes by the Pub/Sub service, to respond to a client publish request, leads to redeliveries with the same message IDs.

  • Failures apart from invalid acknowledgment IDs in subscription/exactly_once_warning_count are retryable and the supported client libraries will retry them automatically.