Replay and purge messages with seek

After you acknowledge messages in Pub/Sub, these messages become inaccessible to the subscriber client. In addition, subscriber clients must process every message in a subscription even if only a subset is needed.

The seek feature extends subscriber capabilities by allowing you to alter the acknowledgement state of messages in bulk. For example, you can replay previously acknowledged messages or purge messages in bulk. In addition, you can copy the acknowledgement state of one subscription to another by using seek in combination with a snapshot.

For a quick demonstration of how these features work, see Replay a message in Pub/Sub by seeking to a snapshot or timestamp.

Snapshot and seek overview

A Pub/Sub snapshot is a durable, consistent, and reliable point-in-time view of the message acknowledgment (ack) state of a subscription. A snapshot records the acknowledgment state of all the messages within a subscription at the moment of its creation. A snapshot retains the unacknowledged messages of the source subscription at the time of creation of the snapshot, and any messages published to the topic after the snapshot is created.

The lifetime of a snapshot is determined by the existing backlog of the source subscription. The lifetime equals 7 days minus the age of the oldest unacknowledged message in the subscription. For example, consider a snapshot of a subscription with a backlog where the oldest unacknowledged message is 1 day old. The snapshot expires after 6 days. This timeline is necessary for snapshots to offer strong at-least-once delivery guarantee.

The maximum possible lifetime of a snapshot is seven days. You cannot create a snapshot that would expire in less than 1 hour after the time of its creation.

The seek feature lets you seek to a specific snapshot or timestamp for a subscription. This feature lets you control how Pub/Sub can deliver messages from a specific point in time or from a specific snapshot.

To seek to a time in the past and replay previously-acknowledged messages, you must first configure message retention on the topic or configure the subscription to retain acknowledged messages.

Eventual consistency of seek operations

Seek operations are strictly consistent in regard to the message delivery guarantee. This means that any message that is to become unacknowledged based on the seek condition is guaranteed to be eventually delivered after the seek operation succeeds. However, delivered messages don't instantly become consistent with the seek operation. So a message that was published before the seek timestamp or that is acknowledged in a snapshot might be delivered after the seek operation. In a sense, message delivery operates as an eventually consistent system with respect to the seek operation; it might take as long as a minute for the operation to take full effect.

Use cases for seek operations

• Update subscriber code safely. A concern with deploying new subscriber code is that the new executable may erroneously acknowledge messages, leading to message loss. Incorporating snapshots into your deployment process gives you a way to recover from bugs in new subscriber code.
• Recover from unexpected subscriber problems. In cases where subscriber problems are not associated with a specific deployment event, you might not have a relevant snapshot. In this case, if you have enabled acknowledged message retention for a subscription, seeking to a time in the past gives you a way to recover from the error.
• Save processing time and cost. Perform a bulk acknowledgement on a large backlog of messages that are no longer relevant.
• Test subscriber code on known data. When testing subscriber code for performance and consistency, it is useful to use the same data in every run. Snapshots enable consistent data with strong semantics. In addition, snapshots can be applied to any subscription on a given topic, including a newly created one.

Configure message retention

You can configure message retention on a topic and configure any of its subscriptions to retain acknowledged messages. You may want to configure topic message retention if you want messages to be retained for replay for a duration longer than the message retention configured on the subscription. In this situation, the topic's project and the subscription's project are charged for message storage according to their respective message retention settings.

If topic message retention is not configured, an unacknowledged message is deleted from the subscription when its age exceeds the subscription's message_retention_duration property. On the other hand, if topic message retention is configured, the unacknowledged message is deleted from the subscription only when its age exceeds the maximum of the topic's and the subscription's message_retention_durations.

Configure topic message retention

By default, a Pub/Sub topic discards messages as soon as they are acknowledged by all subscriptions attached to the topic. Configuring a topic with message retention gives you more flexibility, allowing any subscription attached to the topic to seek back in time and replay previously acknowledged messages up to the topic's message_retention_duration. Topic message retention also allows a subscription to replay messages that are published before you created a subscription.

A topic can retain published messages for a maximum of 31 days (configurable by the topic's message_retention_duration property) even after they have been acknowledged by all attached subscriptions. In cases where the topic's message_retention_duration is greater than the subscription's message_retention_duration, Pub/Sub discards a message only when its age exceeds the topic's message_retention_duration.

If topic message retention is enabled, storage costs for the messages retained by the topic are billed to the topic's project.

Note: You can use Cloud Monitoring to view the following topic message retention metrics:

• topic/retained_bytes to see the total bytes stored by topic retention
• topic/num_retained_messages to see the number of retained messages
• topic/oldest_retained_message_age to see the age of the oldest retained message

For more information about these metrics, see Pub/Sub metrics.

Console

To create a topic with message retention enabled, follow these steps:

1. In the Google Cloud console, go to the Pub/Sub topics page.

   Go to the topics page

2. Click Create topic.

3. In the Topic ID field, enter an ID for your topic.

4. Enable Set message retention duration.

   Leave the other options in their default settings.

5. Use the drop-down menu Message retention duration to select the number of days, hours, and minutes to retain messages.

6. Click Create topic to save the topic.

To update a topic's message retention settings:

1. Select your topic from the Pub/Sub topics page.

   Go to the topics page

2. Click Edit at the top of the topic details page.

3. Adjust the retention time or enable or disable message retention by checking or unchecking the option Enable message retention.

4. Click Update to save changes to the topic.

gcloud

To create a topic with a message retention duration of 7 days, use the following gcloud pubsub topics create command:

gcloud pubsub topics create TOPIC_ID --message-retention-duration=7d

You can update this setting by using gcloud pubsub topics update . This also lets you enable message retention for an existing topic:

gcloud pubsub topics update TOPIC_ID --message-retention-duration=1d

You can also disable message retention for a topic with the update command:

gcloud pubsub topics update TOPIC_ID --clear-message-retention-duration

Configure subscription message retention

Pub/Sub begins retaining messages on behalf of a subscription when the subscription is created. By default, Pub/Sub discards a message from a subscription as soon as the message is acknowledged. Unacknowledged messages are retained for a default of 7 days (configurable by the subscription's message_retention_duration property).

Configuring a subscription to retain acknowledged messages (using the retain_acked_messages property) lets you replay previously-acked messages retained by the subscription. You can configure messages to be retained for a maximum of 31 days in a subscription. This configuration applies to both acknowledged and unacknowledged messages.

If a subscription is configured to retain acknowledged messages, storage costs for the acknowledged messages retained by the subscription are billed to the subscription's project.

Note: When you modify either the message retention duration or subscription expiration policy, the expiration period must be set to a value greater than the message retention duration. The default message retention duration is 7 days and the default expiration period is 31 days.

Console

To create a subscription with retention of acked messages enabled, follow these steps:

1. In the Google Cloud console, go to the Pub/Sub subscriptions page.

   Go to the subscriptions page

2. Click Create subscription.

3. In the Subscription ID field, enter an ID for your subscription.

4. Use the drop-down menu Message retention duration to select the number of days, hours, and minutes to retain messages.

5. Enable Retain acknowledged messages. Leave the other options in their default settings.

6. Click Create subscription to save the subscription.

To update a subscription's message retention settings:

1. Select your subscription from the Pub/Sub subscriptions page.

   Go to the subscriptions page

2. Click Edit at the top of the subscription details page.

3. Adjust the message retention duration, or enable or disable retention of acked messages by checking or unchecking the field labeled Retain acknowledged messages.

4. Click Update to save changes to the subscription.

gcloud

To create a subscription with retention of acked messages enabled, use the following gcloud pubsub subscriptions create command:

gcloud pubsub subscriptions create SUBSCRIPTION_ID
    --retain-acked-messages
    --message-retention-duration=5d

Note: If you omit the --message-retention-duration parameter, acked and unacked messages will be retained for the default duration of 7 days.

You can update this setting by using gcloud pubsub subscriptions update . This also lets you enable retention of acked messages for an existing subscription:

gcloud pubsub subscriptions update SUBSCRIPTION_ID --message-retention-duration=1d

You can also disable retention of acked messages for a subscription with the update command:

gcloud pubsub subscriptions update SUBSCRIPTION_ID --no-retain-acked-messages

Create a snapshot

You can create a snapshot using the console, the Google APIs, or the Google Cloud CLI CLI.

Console

To create a snapshot, follow these steps:

1. In the Google Cloud console, go to the Snapshots page.

   Go to the snapshots page

2. Click Create snapshot.

3. For Select a Pub/Sub subscription, select a subscription.

4. For Snapshot ID, enter a name for the snapshot.

   For more information on how to name Pub/Sub resources, see Guidelines to name a topic, subscription, schema, or snapshot.

5. Click Create to create the snapshot.

You can also create a snapshot from the Subscriptions page. If you create a snapshot immediately after creating a subscription, you might get an error due to the propagation delay for the newly-created subscription.

gcloud

To create a snapshot, use the following gcloud pubsub snapshots create command:

gcloud pubsub snapshots create \
    --project=PROJECT_ID \
    --subscription=SUBSCRIPTION_ID \
    SNAPSHOT_ID

Replace the following:

• PROJECT_ID. Specifies the ID of the project.

• SUBSCRIPTION_ID. Specifies the ID of the subscription.

• SNAPSHOT_ID. Specifies the ID of the snapshot.

Seek to a timestamp

Seeking to a time marks every message received by Pub/Sub before the time as acknowledged, and all messages received after the time as unacknowledged.

You can perform the following types of seek operations based on timestamps:

• To purge all messages, you can seek to a time in the future.

• To replay and reprocess previously acknowledged messages, seek to a time in the past.

The message publication time is generated by the Pub/Sub servers (see publishTime in the API reference). This approach is imprecise due to the following reasons:

• Possible clock skew among Pub/Sub servers.

• The fact that Pub/Sub has to work with the arrival time of the publish request rather than when an event occurred in the source system.

You can seek to a timestamp using the console, the Google APIs, or the Google Cloud CLI CLI. Before you seek to a timestamp on a subscription, ensure that message retention is enabled on the subscription.

Console

To seek to a timestamp, follow these steps:

1. In the Google Cloud console, go to the Subscription page.

   Go to Subscription

2. Click a subscription which has message retention enabled.

3. In the subscription details page, click Replay messages.

4. For Seek, click To a previous time point.

5. Select an appropriate date and time and then click Seek.

gcloud

To seek to a timestamp, use the following gcloud pubsub subscriptions seek command:

gcloud pubsub subscriptions seek SUBSCRIPTION_ID \
    --time=TIME \

Replace the following:

• TIME: the time to which you want to perform the seek operation.
• SUBSCRIPTION_ID: the ID of the subscription.

For more information about supported time formats, see gcloud topic datetimes.

Seek to a snapshot

You can replay unacknowledged messages by using a snapshot to seek to any of the topic's subscriptions.

Unlike with seeking to a time, you don't need to perform any special subscription configuration to seek to a snapshot. You just need to create the snapshot ahead of time. For example, you might create a snapshot when deploying new subscriber code, in case you need to recover from unexpected or erroneous acknowledgements.

If the backlog in the subscription is too old and the resulting snapshot would expire in less than 1 hour, the seek operation fails.

You can seek to a snapshot using the console, the Google APIs, or the Google Cloud CLI CLI.

Console

To seek to a snapshot, follow these steps:

1. In the Google Cloud console, go to the Subscription page.

   Go to Subscription

2. Click a subscription.

3. In the subscription details page, click Replay messages.

4. For Seek, click To a snapshot.

5. Select an appropriate snapshot and then click Seek.

gcloud

To seek to a snapshot, use the following gcloud pubsub subscriptions seek command:

gcloud pubsub subscriptions seek SUBSCRIPTION_ID \
    --snapshot=SNAPSHOT_ID

Replace the following:

• SNAPSHOT_ID: the ID of a snapshot. The snapshot's topic must be the same as that of the subscription.
• SUBSCRIPTION_ID: the ID of the subscription.

Seek with filters

You can replay messages from subscriptions with filters. If you seek to a timestamp using a subscription with a filter, the Pub/Sub service only redelivers the messages that match the filter.

A snapshot of a subscription with a filter contains the following messages:

• All messages that are newer than the snapshot, including messages that don't match the filter.
• Unacknowledged messages that are older than the snapshot.

Note: The snapshot might contain messages that are older than the snapshot and don't match the filter.

If you seek to a snapshot using a subscription with a filter, the Pub/Sub service only redelivers the messages in the snapshot that match the filter of the subscription making the seek request.

For more information about filters, see Filtering messages.

Seek with dead-letter topics

If you seek messages in a subscription with a dead-letter topic, Pub/Sub sets the delivery attempts to 0. The messages that you receive from these subscriptions have a field that tallies the number of delivery attempts.

For more information about dead-letter topics, see Forwarding to dead-letter topics.

Seek with retry policies

If you seek messages in a subscription with a retry policy, Pub/Sub resets the delay between the following:

1. The acknowledgement deadline expiring or the subscriber sending a negative acknowledgment.
2. Pub/Sub resending the message.

For more information about retry policies, see Using retry policies.

Seek with exactly once delivery

If you seek messages in a subscription with exactly once delivery, Pub/Sub resends the previously acknowledged messages that are eligible for delivery. Any acknowledgements, for a delivery made before the seek operation, will fail. Seek operations are eventually consistent.

For more information about retry policies, see Exactly once delivery.