Pub/Sub supports both push and pull message delivery. For an overview and comparison of pull and push subscriptions, see the Subscriber Overview. This document describes push delivery. For a discussion of pull delivery, see the Pull Subscriber Guide.
If a subscription uses push delivery, the Pub/Sub service delivers messages to a push endpoint. The push endpoint must be a publicly accessible HTTPS address. The server for the push endpoint must have a valid SSL certificate signed by a certificate authority.
The Pub/Sub service delivers messages to push endpoints from the same Google Cloud region that the Pub/Sub service stores the messages. The Pub/Sub service delivers messages from the same Google Cloud region on a best-effort basis.
In addition, push subscriptions can be configured to provide an authorization header to allow the endpoints to authenticate the requests. Automatic authentication and authorization mechanisms are available for App Engine Standard and Cloud Functions endpoints hosted in the same project as the subscription.
Receiving messages
When Pub/Sub delivers a message to a push endpoint,
Pub/Sub sends the message in the body of a POST
request. The
body of the request is a JSON object and the message data is in the
message.data
field. The message data is base64-encoded.
The following example is the body of a POST
request to a push endpoint:
{ "message": { "attributes": { "key": "value" }, "data": "SGVsbG8gQ2xvdWQgUHViL1N1YiEgSGVyZSBpcyBteSBtZXNzYWdlIQ==", "messageId": "2070443601311540", "message_id": "2070443601311540", "publishTime": "2021-02-26T19:13:55.749Z", "publish_time": "2021-02-26T19:13:55.749Z", }, "subscription": "projects/myproject/subscriptions/mysubscription" }
To receive messages from push subscriptions, use a webhook and process the
POST
requests that Pub/Sub sends to the push endpoint. For more
information about processing these POST
requests in App Engine, see
Writing and responding to Pub/Sub messages.
After you receive a push request, return an HTTP status code. To acknowledge the message, return one of the following status codes:
102
200
201
202
204
To send a negative acknowledgement for the message, return any other status code. If you send a negative acknowledgement or the acknowledgement deadline expires, Pub/Sub resends the message. You can't modify the acknowledgement deadline of individual messages that you receive from push subscriptions.
Authentication and authorization
If a push subscription uses authentication, the Pub/Sub service signs a JSON Web Token (JWT) and sends the JWT in the authorization header of the push request. The JWT includes claims and a signature.
Subscribers can decode the JWT and verify the following:
- The claims are accurate.
- The Pub/Sub service signed the claims.
If subscribers use a firewall, they can't receive push requests. To receive push requests, you must turn off the firewall and verify the JWT.
JWT format
The JWT is an OpenIDConnect JWT that consists of a header, claim set, and signature. The Pub/Sub service encodes the JWT as a base64 string with period delimiters.
For example, the following authorization header includes an encoded JWT:
"Authorization" : "Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IjdkNjgwZDhjNzBkNDRlOTQ3MTMzY2JkNDk5ZWJjMWE2MWMzZDVh YmMiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJodHRwczovL2V4YW1wbGUuY29tIiwiYXpwIjoiMTEzNzc0M jY0NDYzMDM4MzIxOTY0IiwiZW1haWwiOiJnYWUtZ2NwQGFwcHNwb3QuZ3NlcnZpY2VhY2NvdW50LmNvb SIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJleHAiOjE1NTAxODU5MzUsImlhdCI6MTU1MDE4MjMzNSwia XNzIjoiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tIiwic3ViIjoiMTEzNzc0MjY0NDYzMDM4MzIxO TY0In0.QVjyqpmadTyDZmlX2u3jWd1kJ68YkdwsRZDo-QxSPbxjug4ucLBwAs2QePrcgZ6hhkvdc4UHY 4YF3fz9g7XHULNVIzX5xh02qXEH8dK6PgGndIWcZQzjSYfgO-q-R2oo2hNM5HBBsQN4ARtGK_acG-NGG WM3CQfahbEjZPAJe_B8M7HfIu_G5jOLZCw2EUcGo8BvEwGcLWB2WqEgRM0-xt5-UPzoa3-FpSPG7DHk7 z9zRUeq6eB__ldb-2o4RciJmjVwHgnYqn3VvlX9oVKEgXpNFhKuYA-mWh5o7BCwhujSMmFoBOh6mbIXF cyf5UiVqKjpqEbqPGo_AvKvIQ9VTQ"
The header and claim set are JSON strings. Once decoded, they take the following form:
{"alg":"RS256","kid":"7d680d8c70d44e947133cbd499ebc1a61c3d5abc","typ":"JWT"} { "aud":"https://example.com", "azp":"113774264463038321964", "email":"gae-gcp@appspot.gserviceaccount.com", "sub":"113774264463038321964", "email_verified":true, "exp":1550185935, "iat":1550182335, "iss":"https://accounts.google.com" }
The tokens attached to requests sent to push endpoints may be up to an hour old.
Setting up Pub/Sub for push authentication
Authentication configuration for a subscription consists of two parameters:
- Service account: The GCP service account associated with the push
subscription. Push requests carry the identity of this service account.
As an example, a push subscription configured with a service account that has
the role
roles/run.invoker
and is bound to a particular Cloud Run (fully managed) service can invoke that Cloud Run (fully managed) service. - Token audience (optional): A single, case-insensitive string that can be used by the webhook to validate the intended audience of this particular token.
In addition to configuring these fields, you must also grant
Pub/Sub the permissions needed to create tokens for your service
account. Pub/Sub creates and maintains a special service account
for your project: service-PROJECT_NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com.
This service account needs the Service Account Token Creator role. If you
use the Cloud Console to set up the subscription for push authentication, the
role is granted automatically. Otherwise, you must explicitly grant the role to the account.
COMMAND-LINE
# grant Cloud Pub/Sub the permission to create tokens PUBSUB_SERVICE_ACCOUNT="service-${PROJECT_NUMBER}@gcp-sa-pubsub.iam.gserviceaccount.com" gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member="serviceAccount:${PUBSUB_SERVICE_ACCOUNT}"\ --role='roles/iam.serviceAccountTokenCreator' # configure the subscription push identity gcloud pubsub subscriptions (create|update|modify-push-config) ${SUBSCRIPTION} \ --topic=${TOPIC} \ --push-endpoint=${PUSH_ENDPOINT_URI} \ --push-auth-service-account=${SERVICE_ACCOUNT_EMAIL} \ --push-auth-token-audience=${OPTIONAL_AUDIENCE_OVERRIDE}
console
Go to the Pub/Sub Topics page.
Click a topic name.
Create or update a subscription.
Enter an identity and (optionally) an audience.
Authentication and authorization by the push endpoint
Claims
The JWT can be used to validate that the claims -- including email
and aud
claims -- are signed by Google. For more information about how Google's OAuth
2.0 APIs can be used for both authentication and authorization, see
OpenID Connect.
There are two mechanisms that make these claims meaningful. First,
Pub/Sub requires that the user or service account used to
associate a service account identity with a push subscription have
the Service Account User role (roles/iam.serviceAccountUser
) for the
project or the service account.
Second, access to the certificates used to sign the tokens is tightly
controlled. To create the token, Pub/Sub must call an internal
Google service using a separate signing service account identity.
The signing service account must be authorized to create tokens for
the claimed service account or the project containing the account.
This is done using the iam.serviceAccounts.getOpenIdToken
permission or
a Service Account Token Creator role
(roles/iam.serviceAccountTokenCreator
).
This role or permission can be granted to any account. However, you can use the IAM service to ensure the Pub/Sub signing account is the one with this permission. Specifically, Pub/Sub uses a service account like this one:
service-{project_number}@gcp-sa-pubsub.iam.gserviceaccount.com
{project_number}
: the GCP project that contains the subscription.gcp-sa-pubsub
: the Google-owned project which contains the signing service account.
Validating tokens
The following example illustrates how to authenticate a push request to a App Engine application.
protocol
Request:
GET https://oauth2.googleapis.com/tokeninfo?id_token={BEARER_TOKEN}
Response:
200 OK
{ "alg": "RS256", "aud": "example.com", "azp": "104176025330667568672", "email": "{SERVICE_ACCOUNT_NAME}@{YOUR_PROJECT_NAME}.iam.gserviceaccount.com", "email_verified": "true", "exp": "1555463097", "iat": "1555459497", "iss": "https://accounts.google.com", "kid": "3782d3f0bc89008d9d2c01730f765cfb19d3b70e", "sub": "104176025330667568672", "typ": "JWT" }
Go
Java
Node.js
Python
Ruby
You will find additional examples of how to validate the bearer JWT in this Guide for Google Sign-in for Websites. A broader overview of OpenID tokens is available in the OpenID Connect Guide, including a list of client libraries that help validate JWTs.
Cloud Run and App Engine
Cloud Run and App Engine automatically authenticate HTTP calls by verifying Pub/Sub-generated tokens. The only configuration required of the user is that the necessary IAM roles be granted to the caller account. For example, you can authorize or revoke permission to call a particular Cloud Run endpoint for an account. For details, see the following tutorials:
Stopping and resuming delivery
To temporarily stop Pub/Sub from sending requests to the push endpoint, change the subscription to pull. Note that it can take several minutes for this changeover to take effect.
To resume push delivery, set the URL to a valid endpoint again. To permanently stop delivery, delete the subscription.
Quotas, limits and delivery rate
Note that push subscriptions are subject to a set of quotas and resource limits.
Push backoff
If a push subscriber sends negative acknowledgements, Pub/Sub might deliver messages using a push backoff. When Pub/Sub uses a push backoff, it stops delivering messages for 100 milliseconds to 60 seconds and then starts delivering messages again.
The push backoff is an exponential backoff that prevents a push subscriber from receiving messages that it can't process. The amount of time that Pub/Sub stops delivering messages depends on the number of negative acknowledgments that push subscribers send.
For example, if a push subscriber receives five messages per second and sends one negative acknowledgment per second, Pub/Sub delivers messages approximately every 500 milliseconds. If the push subscriber sends five negative acknowledgment per second, Pub/Sub delivers messages every 30-60 seconds.
Delivery rate
Pub/Sub adjusts the number of concurrent push requests using a slow-start algorithm. The maximum allowed number of concurrent push requests is the push window. The push window increases on any successful delivery and decreases on any failure. The system starts with a small window: 3 times N, where N is the number of publish regions.
When a subscriber acknowledges messages, the window increases exponentially up to 3,000 times N outstanding messages. For subscriptions where subscribers acknowledge greater than 99% of messages and average less than one second of push request latency, the push window increases up to 30,000 times N outstanding messages.
The push request latency includes the following:
- The round-trip network latency between Pub/Sub servers and the push endpoint
- The processing time of the subscriber
After 3,000 outstanding messages, the window increases linearly to prevent the push endpoint from receiving too many messages. If the average latency exceeds one second or the subscriber acknowledges less than 99% of requests, the window decreases to the lower limit of 3,000 outstanding messages.
For more information about the metrics you can use to monitor push delivery, see Monitoring push subscriptions.