Replay a message in Pub/Sub by seeking to a snapshot or timestamp
Pub/Sub cannot retrieve the messages after you have acknowledged them. However, sometimes you might find it necessary to replay the acknowledged messages, for example, if you performed an erroneous acknowledgment. Then you can use the Seek feature to mark previously acknowledged messages as unacknowledged, and force Pub/Sub to redeliver those messages. You can also use seek to delete the unacknowledged messages by changing their state to acknowledged.
Seek to a snapshot or seek to a timestamp to replay the messages in a subscription. This guide shows examples of how to replay previously acknowledged Pub/Sub messages using seek.
To follow step-by-step guidance for this task directly in the Google Cloud console, click Guide me:
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
Set up a Google Cloud console project.
Click to:
- Create or select a project.
- Enable the Pub/Sub API for that project.
You can view and manage these resources at any time in the Google Cloud console.
- Install the Google Cloud CLI.
-
To initialize the gcloud CLI, run the following command:
gcloud init -
Set up a Google Cloud console project.
Click to:
- Create or select a project.
- Enable the Pub/Sub API for that project.
You can view and manage these resources at any time in the Google Cloud console.
- Install the Google Cloud CLI.
-
To initialize the gcloud CLI, run the following command:
gcloud init
Using the gcloud Command-Line Interface
The easiest way to use gcloud is in the Cloud Shell . You can also use the command line tool that comes with the Google Cloud CLI. Setup prerequisites are met automatically for the gcloud CLI if you create a brand new Compute Engine machine instance with access scopes including all Cloud APIs and the default Debian image. The simplest way to create it is via the Compute Engine UI. For the rest of this document, we will assume that you are using one of these command line tools.
For more information about using gcloud, see the gcloud quickstart.
Also see the gcloud pubsub section in the
gcloud reference
for a complete list of Pub/Sub gcloud commands.
Create and Seek to Snapshots
Create a topic and a subscription. For better interactivity, use a short acknowledgement deadline:
gcloud pubsub topics create seek-demo-topic gcloud pubsub subscriptions create seek-demo-sub --topic=seek-demo-topic --ack-deadline=10
So far, no messages in the subscription have been acknowledged. Capture this state by creating a snapshot:
gcloud pubsub snapshots create my-snapshot --subscription=seek-demo-sub gcloud pubsub snapshots list
Now publish, pull, and acknowledge a message:
gcloud pubsub topics publish seek-demo-topic --message 'hello, world' gcloud pubsub subscriptions pull --auto-ack seek-demo-sub
Note that subsequent pulls yield no new messages (some transient duplicates are possible):
gcloud pubsub subscriptions pull --auto-ack seek-demo-sub
You can recover the message by seeking the subscription to the snapshot. Note that the message may not be delivered instantly: Pub/Sub does not guarantee message delivery latency. However, what is guaranteed is that the messages retained in the snapshot will be eventually delivered at least once:
gcloud pubsub subscriptions seek seek-demo-sub --snapshot=my-snapshot gcloud pubsub subscriptions pull seek-demo-sub --auto-ack
Note that you can seek other subscriptions to the same snapshot, as long as the subscription's topic is the same as the snapshot's topic:
gcloud pubsub subscriptions create seek-demo-sub2 --topic=seek-demo-topic --ack-deadline=10 gcloud pubsub subscriptions seek seek-demo-sub2 --snapshot=my-snapshot
Pulling from seek-demo-sub2 will now yield messages which were published before
seek-demo-sub2 was created:
gcloud pubsub subscriptions pull seek-demo-sub2 --auto-ack
When you are done with the snapshot, clean up:
gcloud pubsub snapshots delete my-snapshot
Seek to a timestamp
Another way to replay messages that have been acknowledged is to seek to a
timestamp. To seek to a timestamp, you must first configure the subscription to
retain acknowledged messages using retain-acked-messages. If
retain-acked-messages is set, Pub/Sub retains acknowledged
messages for 7 days.
You only need to do this step if you intend to seek to a timestamp, not to a snapshot.
gcloud pubsub subscriptions update seek-demo-sub --retain-acked-messages
Now publish, pull, and acknowledge a message:
gcloud pubsub topics publish seek-demo-topic --message 'hello, world' gcloud pubsub subscriptions pull --auto-ack seek-demo-sub
The next pull should return no messages:
gcloud pubsub subscriptions pull seek-demo-sub
Now seek the subscription back in time to recover the message. Note that the date arithmetic flags differ between the GNU and BSD (MacOS) versions of the date command:
export TS_FORMAT=%Y-%m-%dT%H:%M:%SZ gcloud pubsub subscriptions seek seek-demo-sub --time=$(date -u -d '-10 min' +$TS_FORMAT)
For more information about supported time formats, see gcloud topic datetimes.
A subsequent pull should yield the message again:
gcloud pubsub subscriptions pull --auto-ack seek-demo-sub
You can also use seek to skip delivery of all messages published before some point in time, such as the current time. To do this, seek the subscription to the current time to mark as acknowledged all past messages:
gcloud pubsub topics publish seek-demo-topic --message 'hello, world' gcloud pubsub subscriptions seek seek-demo-sub --time=$(date -u +$TS_FORMAT)
After some propagation delay (~60 seconds), the message will no longer be delivered. You can check this by repeatedly executing the pull command without acknowledging the message:
gcloud pubsub subscriptions pull seek-demo-sub
Using the Cloud console
You can create snapshots, or seek to existing snapshots, in the subscription detail page of the Cloud console. For snapshot naming rules, see Resource names.
Creating a snapshot
In the Google Cloud console, go to the Pub/Sub subscriptions page.
Select a subscription from the list.
Click Create snapshot.
Enter an ID for the snapshot, then click Create to save.
Snapshots can be viewed on the snapshots page in the Google Cloud console. You can also manage snapshots in the details page of an individual topic.
Replaying a subscription
The Replay messages dialog lets you seek to a previous time point or a saved snapshot. To open the dialog:
In the Google Cloud console, go to the Pub/Sub subscriptions page.
Select a subscription from the list.
Click Replay messages.
Under Seek, select either To a previous time point or To a snapshot, then select the time point or snapshot to seek to.
Click Seek to replay messages.
How did it go?
Note that additional resources and links are available on the Pub/Sub support page.