Why is Google shipping a new publish-subscribe service?
Although messaging middleware is a mature market, there is no single dominant product or protocol, especially in cloud environments. We believe that Cloud Pub/Sub can provide a unique combination of capabilities for Google Cloud Platform customers. Furthermore, Cloud Pub/Sub marks a new generation of efforts by Google to use the same technology internally and externally: Google also uses the Cloud Pub/Sub API for its own products. Nevertheless, we aim to support open standards, and are actively exploring ways to make Cloud Pub/Sub easy to use for developers familiar with related protocols and APIs.
Is this related to Google Cloud Messaging (GCM)?
Yes and no. Both are systems to deliver messages, but Google Cloud Messaging is used to deliver messages to and from end-user devices, while Cloud Pub/Sub is used to communicate between servers. Google Cloud Messaging is designed to scale to a very large number of delivery end points, but has low throughput (messages per second per channel). Cloud Pub/Sub does not have limits on throughput and has a more generic API.
Is this related to PubSubHubbub?
No. While Googlers were closely involved in originating PubSubHubbub, its strengths in RSS and content syndication generally are not use cases that Cloud Pub/Sub is designed to address. Aside from the name, they have very little in common.
Can I use Cloud Pub/Sub to communicate between different App Engine modules?
Yes. You can use Cloud Pub/Sub to send and receive messages between different modules in a App Engine application, and even between different applications in the same project, without any special ACL configuration.
Can I use Cloud Pub/Sub with platforms other than App Engine?
Yes. You can use Cloud Pub/Sub with applications hosted on Google Compute Engine or even on non-Google platforms. All you need is a Google Cloud Platform Console project to get started.
How can I connect App Engine and Compute Engine applications?
How can I monitor Cloud Pub/Sub metrics?Please refer to this article.
Is Cloud Pub/Sub integrated with Cloud KMS?
No. By design Cloud Pub/Sub is a message delivery system with a temporary queue for storage. While Cloud Pub/Sub is not integrated with Cloud Key Management Service, a client could implement encryption of their messages on their own (using Cloud KMS or other key management service). A client also can control the lifetime of the message.
Are messages delivered in order?
Cloud Pub/Sub does not guarantee in-order, or FIFO (first-in-first-out) delivery. Messages are delivered as fast as possible, with preference given to delivering older messages first, but this is not guaranteed. Strict ordering is at odds with Cloud Pub/Sub's availability and scalability guarantees; see Message Ordering for a detailed discussion of this topic.
How are messages sent to push endpoints secured?
Messages sent to push endpoints are sent encrypted over HTTPS. Your push endpoint must be configured with SSL certificates.
If you additionally would like to verify that the messages originated from Cloud Pub/Sub, you could configure your endpoint to only accept messages that are accompanied by a secret token argument, for example,
What happens if I publish to a topic without subscriptions?
The publish operation succeeds, but the messages are dropped because there is no subscription interested in them.
Are messages durable or persistent if the subscriber is not present?
Yes. The Cloud Pub/Sub system guarantees that subscriptions retain unacknowledged
messages in persistent storage for 7 days from the moment they were published.
In addition, note that subscriptions whose client presence has not been detected
for 30 days may be automatically deleted. For example, if messages are pulled
within 30 days of the last
Pull operation, this has the effect of
restarting the 30 day clock for the subscription deletion; however, any messages
published before 7 days may be lost regardless of their acknowledgment state.
Why am I getting quota errors when my usage is well below the limits?
Quotas are enforced against the project associated with the authenticated client, as explained
here. Make sure that you are using a service account or
API_KEY associated with a project that has sufficient quotas. The
command line in particular uses a shared project, and the quotas are very limited.
Why are there too many duplicate messages?
Cloud Pub/Sub guarantees at-least-once message delivery, which means that occasional duplicates
are to be expected. However, a high rate of duplicates may indicate that the client is not
acknowledging messages within the configured
ack_deadline_seconds, and Cloud Pub/Sub
is retrying the message delivery. This can be observed in the
for pull subscriptions, and
push subscriptions. Look for elevated
Another possibility is that the subscriber is not
acknowledging some messages because the code path processing those specific messages fails, and the
Acknowledge call is never made; or the push endpoint never responds or responds
with an error.
How do I detect duplicate messages?
Cloud Pub/Sub assigns a unique `message_id` to each message, which can be used to detect duplicate messages received by the subscriber. This will not, however, allow you to detect duplicates resulting from multiple publish requests on the same data. Detecting those will require a unique message identifier to be provided by the publisher. See this for further discussion.