Why is Google shipping a new publish-subscribe service?
Although messaging middleware is a mature market, there is no single dominant product or protocol, especially in cloud environments. We believe that Cloud Pub/Sub can provide a unique combination of capabilities for Google Cloud Platform customers. Furthermore, Cloud Pub/Sub marks a new generation of efforts by Google to use the same technology internally and externally: Google also uses the Cloud Pub/Sub API for its own products. Nevertheless, we aim to support open standards, and are actively exploring ways to make Cloud Pub/Sub easy to use for developers familiar with related protocols and APIs.
Is this related to Firebase Cloud Messaging (FCM) or to the deprecated Google Cloud Messaging (GCM)?
Yes and no. These systems are each used to deliver messages, but FCM is used to deliver messages to and from end-user devices while Cloud Pub/Sub is used to communicate between servers. FCM is designed to scale to a very large number of delivery end points, but has low throughput (messages per second per channel). Cloud Pub/Sub does not have limits on throughput and has a more generic API.
Is this related to PubSubHubbub?
No. While Googlers were closely involved in originating PubSubHubbub, its strengths in RSS and content syndication generally are not use cases that Cloud Pub/Sub is designed to address. Aside from the name, they have very little in common.
Can I use Cloud Pub/Sub to communicate between different App Engine modules?
Yes. You can use Cloud Pub/Sub to send and receive messages between different modules in a App Engine application, and even between different applications in the same project, without any special ACL configuration.
Can I use Cloud Pub/Sub with platforms other than App Engine?
Yes. You can use Cloud Pub/Sub with applications hosted on Google Compute Engine or even on non-Google platforms. All you need is a Google Cloud Platform Console project to get started.
How can I connect App Engine and Compute Engine applications?
How can I monitor Cloud Pub/Sub metrics?Please refer to this article.
Is Cloud Pub/Sub integrated with Cloud KMS?
No. By design Cloud Pub/Sub is a message delivery system with a temporary queue for storage. While Cloud Pub/Sub is not integrated with Cloud Key Management Service, a client could implement encryption of their messages on their own (using Cloud KMS or other key management service). A client also can control the lifetime of the message.
Are messages delivered in order?
Cloud Pub/Sub does not guarantee in-order, or FIFO (first-in-first-out) delivery. Messages are delivered as fast as possible, with preference given to delivering older messages first, but this is not guaranteed. Strict ordering is at odds with Cloud Pub/Sub's availability and scalability guarantees; see Message Ordering for a detailed discussion of this topic.
Why am I getting quota errors when my usage is well below the limits?
Quotas are enforced against the project associated with the authenticated client, as explained in
Quotas and Limits. Make sure that you are using a service account or
API_KEY associated with a project that has sufficient quotas. The
command line in particular uses a shared project, and the quotas are very limited.
Why are there too many duplicate messages?
Cloud Pub/Sub guarantees at-least-once message delivery, which means that occasional duplicates
are to be expected. However, a high rate of duplicates may indicate that the client is not
acknowledging messages within the configured
ack_deadline_seconds, and Cloud Pub/Sub
is retrying the message delivery. This can be observed in the
for pull subscriptions, and
push subscriptions. Look for elevated
/response_code. This is particularly likely if there are many small messages,
since Cloud Pub/Sub may batch messages internally and a partially acknowledged batch will be
Another possibility is that the subscriber is not
acknowledging some messages because the code path processing those specific messages fails, and the
Acknowledge call is never made; or the push endpoint never responds or responds
with an error.
How do I detect duplicate messages?
Cloud Pub/Sub assigns a unique `message_id` to each message, which can be used to detect duplicate messages received by the subscriber. This will not, however, allow you to detect duplicates resulting from multiple publish requests on the same data. Detecting those will require a unique message identifier to be provided by the publisher. See Cloud Pub/Sub I/O for further discussion.