PubSubSubscription


Property Value
Google Cloud Service Name Cloud Pub/Sub
Google Cloud Service Documentation /pubsub/docs/
Google Cloud REST Resource Name v1.projects.subscriptions
Google Cloud REST Resource Documentation /pubsub/docs/reference/rest/v1/projects.subscriptions
Config Connector Resource Short Names gcppubsubsubscription
gcppubsubsubscriptions
pubsubsubscription
Config Connector Service Name pubsub.googleapis.com
Config Connector Resource Fully Qualified Name pubsubsubscriptions.pubsub.cnrm.cloud.google.com
Can Be Referenced by IAMPolicy/IAMPolicyMember Yes
Supports IAM Conditions No
Supports IAM Audit Configs No
IAM External Reference Format

projects/{{project}}/subscriptions/{{name}}

Config Connector Default Average Reconcile Interval In Seconds 600

Custom Resource Definition Properties

Annotations

Fields
cnrm.cloud.google.com/project-id
cnrm.cloud.google.com/state-into-spec

Spec

Schema

ackDeadlineSeconds: integer
bigqueryConfig:
  dropUnknownFields: boolean
  tableRef:
    external: string
    name: string
    namespace: string
  useTopicSchema: boolean
  writeMetadata: boolean
cloudStorageConfig:
  avroConfig:
    writeMetadata: boolean
  bucketRef:
    external: string
    name: string
    namespace: string
  filenamePrefix: string
  filenameSuffix: string
  maxBytes: integer
  maxDuration: string
  state: string
deadLetterPolicy:
  deadLetterTopicRef:
    external: string
    name: string
    namespace: string
  maxDeliveryAttempts: integer
enableExactlyOnceDelivery: boolean
enableMessageOrdering: boolean
expirationPolicy:
  ttl: string
filter: string
messageRetentionDuration: string
pushConfig:
  attributes:
    string: string
  noWrapper:
    writeMetadata: boolean
  oidcToken:
    audience: string
    serviceAccountEmail: string
  pushEndpoint: string
resourceID: string
retainAckedMessages: boolean
retryPolicy:
  maximumBackoff: string
  minimumBackoff: string
topicRef:
  external: string
  name: string
  namespace: string
Fields

ackDeadlineSeconds

Optional

integer

This value is the maximum time after a subscriber receives a message before the subscriber should acknowledge the message. After message delivery but before the ack deadline expires and before the message is acknowledged, it is an outstanding message and will not be delivered again during that time (on a best-effort basis). For pull subscriptions, this value is used as the initial value for the ack deadline. To override this value for a given message, call subscriptions.modifyAckDeadline with the corresponding ackId if using pull. The minimum custom deadline you can specify is 10 seconds. The maximum custom deadline you can specify is 600 seconds (10 minutes). If this parameter is 0, a default value of 10 seconds is used. For push delivery, this value is also used to set the request timeout for the call to the push endpoint. If the subscriber never acknowledges the message, the Pub/Sub system will eventually redeliver the message.

bigqueryConfig

Optional

object

If delivery to BigQuery is used with this subscription, this field is used to configure it. Either pushConfig, bigQueryConfig or cloudStorageConfig can be set, but not combined. If all three are empty, then the subscriber will pull and ack messages using API methods.

bigqueryConfig.dropUnknownFields

Optional

boolean

When true and useTopicSchema is true, any fields that are a part of the topic schema that are not part of the BigQuery table schema are dropped when writing to BigQuery. Otherwise, the schemas must be kept in sync and any messages with extra fields are not written and remain in the subscription's backlog.

bigqueryConfig.tableRef

Required*

object

The name of the table to which to write data.

bigqueryConfig.tableRef.external

Optional

string

Allowed value: string of the format `{{project}}.{{dataset_id}}.{{value}}`, where {{value}} is the `name` field of a `BigQueryTable` resource.

bigqueryConfig.tableRef.name

Optional

string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

bigqueryConfig.tableRef.namespace

Optional

string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

bigqueryConfig.useTopicSchema

Optional

boolean

When true, use the topic's schema as the columns to write to in BigQuery, if it exists.

bigqueryConfig.writeMetadata

Optional

boolean

When true, write the subscription name, messageId, publishTime, attributes, and orderingKey to additional columns in the table. The subscription name, messageId, and publishTime fields are put in their own columns while all other message properties (other than data) are written to a JSON object in the attributes column.

cloudStorageConfig

Optional

object

If delivery to Cloud Storage is used with this subscription, this field is used to configure it. Either pushConfig, bigQueryConfig or cloudStorageConfig can be set, but not combined. If all three are empty, then the subscriber will pull and ack messages using API methods.

cloudStorageConfig.avroConfig

Optional

object

If set, message data will be written to Cloud Storage in Avro format.

cloudStorageConfig.avroConfig.writeMetadata

Optional

boolean

When true, write the subscription name, messageId, publishTime, attributes, and orderingKey as additional fields in the output.

cloudStorageConfig.bucketRef

Required*

object

User-provided name for the Cloud Storage bucket. The bucket must be created by the user. The bucket name must be without any prefix like "gs://".

cloudStorageConfig.bucketRef.external

Optional

string

Allowed value: The `name` field of a `StorageBucket` resource.

cloudStorageConfig.bucketRef.name

Optional

string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

cloudStorageConfig.bucketRef.namespace

Optional

string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

cloudStorageConfig.filenamePrefix

Optional

string

User-provided prefix for Cloud Storage filename.

cloudStorageConfig.filenameSuffix

Optional

string

User-provided suffix for Cloud Storage filename. Must not end in "/".

cloudStorageConfig.maxBytes

Optional

integer

The maximum bytes that can be written to a Cloud Storage file before a new file is created. Min 1 KB, max 10 GiB. The maxBytes limit may be exceeded in cases where messages are larger than the limit.

cloudStorageConfig.maxDuration

Optional

string

The maximum duration that can elapse before a new Cloud Storage file is created. Min 1 minute, max 10 minutes, default 5 minutes. May not exceed the subscription's acknowledgement deadline. A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".

cloudStorageConfig.state

Optional

string

An output-only field that indicates whether or not the subscription can receive messages.

deadLetterPolicy

Optional

object

A policy that specifies the conditions for dead lettering messages in this subscription. If dead_letter_policy is not set, dead lettering is disabled. The Cloud Pub/Sub service account associated with this subscription's parent project (i.e., service-{project_number}@gcp-sa-pubsub.iam.gserviceaccount.com) must have permission to Acknowledge() messages on this subscription.

deadLetterPolicy.deadLetterTopicRef

Optional

object

deadLetterPolicy.deadLetterTopicRef.external

Optional

string

Allowed value: string of the format `projects/{{project}}/topics/{{value}}`, where {{value}} is the `name` field of a `PubSubTopic` resource.

deadLetterPolicy.deadLetterTopicRef.name

Optional

string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

deadLetterPolicy.deadLetterTopicRef.namespace

Optional

string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

deadLetterPolicy.maxDeliveryAttempts

Optional

integer

The maximum number of delivery attempts for any message. The value must be between 5 and 100. The number of delivery attempts is defined as 1 + (the sum of number of NACKs and number of times the acknowledgement deadline has been exceeded for the message). A NACK is any call to ModifyAckDeadline with a 0 deadline. Note that client libraries may automatically extend ack_deadlines. This field will be honored on a best effort basis. If this parameter is 0, a default value of 5 is used.

enableExactlyOnceDelivery

Optional

boolean

If 'true', Pub/Sub provides the following guarantees for the delivery of a message with a given value of messageId on this Subscriptions': - The message sent to a subscriber is guaranteed not to be resent before the message's acknowledgement deadline expires. - An acknowledged message will not be resent to a subscriber. Note that subscribers may still receive multiple copies of a message when 'enable_exactly_once_delivery' is true if the message was published multiple times by a publisher client. These copies are considered distinct by Pub/Sub and have distinct messageId values.

enableMessageOrdering

Optional

boolean

Immutable. If 'true', messages published with the same orderingKey in PubsubMessage will be delivered to the subscribers in the order in which they are received by the Pub/Sub system. Otherwise, they may be delivered in any order.

expirationPolicy

Optional

object

A policy that specifies the conditions for this subscription's expiration. A subscription is considered active as long as any connected subscriber is successfully consuming messages from the subscription or is issuing operations on the subscription. If expirationPolicy is not set, a default policy with ttl of 31 days will be used. If it is set but ttl is "", the resource never expires. The minimum allowed value for expirationPolicy.ttl is 1 day.

expirationPolicy.ttl

Required*

string

Specifies the "time-to-live" duration for an associated resource. The resource expires if it is not active for a period of ttl. If ttl is set to "", the associated resource never expires. A duration in seconds with up to nine fractional digits, terminated by 's'. Example - "3.5s".

filter

Optional

string

Immutable. The subscription only delivers the messages that match the filter. Pub/Sub automatically acknowledges the messages that don't match the filter. You can filter messages by their attributes. The maximum length of a filter is 256 bytes. After creating the subscription, you can't modify the filter.

messageRetentionDuration

Optional

string

How long to retain unacknowledged messages in the subscription's backlog, from the moment a message is published. If retain_acked_messages is true, then this also configures the retention of acknowledged messages, and thus configures how far back in time a subscriptions.seek can be done. Defaults to 7 days. Cannot be more than 7 days ('"604800s"') or less than 10 minutes ('"600s"'). A duration in seconds with up to nine fractional digits, terminated by 's'. Example: '"600.5s"'.

pushConfig

Optional

object

If push delivery is used with this subscription, this field is used to configure it. An empty pushConfig signifies that the subscriber will pull and ack messages using API methods.

pushConfig.attributes

Optional

map (key: string, value: string)

Endpoint configuration attributes. Every endpoint has a set of API supported attributes that can be used to control different aspects of the message delivery. The currently supported attribute is x-goog-version, which you can use to change the format of the pushed message. This attribute indicates the version of the data expected by the endpoint. This controls the shape of the pushed message (i.e., its fields and metadata). The endpoint version is based on the version of the Pub/Sub API. If not present during the subscriptions.create call, it will default to the version of the API used to make such call. If not present during a subscriptions.modifyPushConfig call, its value will not be changed. subscriptions.get calls will always return a valid version, even if the subscription was created without this attribute. The possible values for this attribute are: - v1beta1: uses the push format defined in the v1beta1 Pub/Sub API. - v1 or v1beta2: uses the push format defined in the v1 Pub/Sub API.

pushConfig.noWrapper

Optional

object

When set, the payload to the push endpoint is not wrapped.Sets the 'data' field as the HTTP body for delivery.

pushConfig.noWrapper.writeMetadata

Required*

boolean

When true, writes the Pub/Sub message metadata to 'x-goog-pubsub-:' headers of the HTTP request. Writes the Pub/Sub message attributes to ':' headers of the HTTP request.

pushConfig.oidcToken

Optional

object

If specified, Pub/Sub will generate and attach an OIDC JWT token as an Authorization header in the HTTP request for every pushed message.

pushConfig.oidcToken.audience

Optional

string

Audience to be used when generating OIDC token. The audience claim identifies the recipients that the JWT is intended for. The audience value is a single case-sensitive string. Having multiple values (array) for the audience field is not supported. More info about the OIDC JWT token audience here: https://tools.ietf.org/html/rfc7519#section-4.1.3 Note: if not specified, the Push endpoint URL will be used.

pushConfig.oidcToken.serviceAccountEmail

Required*

string

Service account email to be used for generating the OIDC token. The caller (for subscriptions.create, subscriptions.patch, and subscriptions.modifyPushConfig RPCs) must have the iam.serviceAccounts.actAs permission for the service account.

pushConfig.pushEndpoint

Required*

string

A URL locating the endpoint to which messages should be pushed. For example, a Webhook endpoint might use "https://example.com/push".

resourceID

Optional

string

Immutable. Optional. The name of the resource. Used for creation and acquisition. When unset, the value of `metadata.name` is used as the default.

retainAckedMessages

Optional

boolean

Indicates whether to retain acknowledged messages. If 'true', then messages are not expunged from the subscription's backlog, even if they are acknowledged, until they fall out of the messageRetentionDuration window.

retryPolicy

Optional

object

A policy that specifies how Pub/Sub retries message delivery for this subscription. If not set, the default retry policy is applied. This generally implies that messages will be retried as soon as possible for healthy subscribers. RetryPolicy will be triggered on NACKs or acknowledgement deadline exceeded events for a given message.

retryPolicy.maximumBackoff

Optional

string

The maximum delay between consecutive deliveries of a given message. Value should be between 0 and 600 seconds. Defaults to 600 seconds. A duration in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s".

retryPolicy.minimumBackoff

Optional

string

The minimum delay between consecutive deliveries of a given message. Value should be between 0 and 600 seconds. Defaults to 10 seconds. A duration in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s".

topicRef

Required

object

Reference to a PubSubTopic.

topicRef.external

Optional

string

Allowed value: string of the format `projects/{{project}}/topics/{{value}}`, where {{value}} is the `name` field of a `PubSubTopic` resource.

topicRef.name

Optional

string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

topicRef.namespace

Optional

string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

* Field is required when parent field is specified

Status

Schema

conditions:
- lastTransitionTime: string
  message: string
  reason: string
  status: string
  type: string
observedGeneration: integer
Fields
conditions

list (object)

Conditions represent the latest available observation of the resource's current state.

conditions[]

object

conditions[].lastTransitionTime

string

Last time the condition transitioned from one status to another.

conditions[].message

string

Human-readable message indicating details about last transition.

conditions[].reason

string

Unique, one-word, CamelCase reason for the condition's last transition.

conditions[].status

string

Status is the status of the condition. Can be True, False, Unknown.

conditions[].type

string

Type is the type of the condition.

observedGeneration

integer

ObservedGeneration is the generation of the resource that was most recently observed by the Config Connector controller. If this is equal to metadata.generation, then that means that the current reported status reflects the most recent desired state of the resource.

Sample YAML(s)

Basic Pubsub Subscription

# Copyright 2020 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubSubscription
metadata:
  labels:
    label-one: "value-one"
  name: pubsubsubscription-sample-basic
spec:
  ackDeadlineSeconds: 15
  messageRetentionDuration: 86400s
  retainAckedMessages: false
  topicRef:
    name: pubsubsubscription-dep1-basic
  deadLetterPolicy:
    deadLetterTopicRef:
      name: pubsubsubscription-dep2-basic
---
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubTopic
metadata:
  name: pubsubsubscription-dep1-basic
---
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubTopic
metadata:
  name: pubsubsubscription-dep2-basic

BigQuery Pubsub Subscription

# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Replace ${PROJECT_ID?} below with your desired project ID.
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubSubscription
metadata:
  name: pubsubsubscription-sample-bigquery
  annotations:
    # Replace ${PROJECT_ID?} with your project ID
    cnrm.cloud.google.com/project-id: ${PROJECT_ID?}
spec:
  bigqueryConfig:
    tableRef:
      name: pubsubsubscription-dep-bigquery
  topicRef:
    name: pubsubsubscription-dep-bigquery
---
# Replace ${PROJECT_ID?} below with your desired project ID.
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
kind: BigQueryDataset
metadata:
  name: pubsubsubscription-dep-bigquery
  annotations:
    # Replace ${PROJECT_ID?} with your project ID
    cnrm.cloud.google.com/project-id: ${PROJECT_ID?}
spec:
  resourceID: pubsubsubscriptiondepbigquery
---
# Replace ${PROJECT_ID?} below with your desired project ID.
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
kind: BigQueryTable
metadata:
  name: pubsubsubscription-dep-bigquery
  annotations:
    # Replace ${PROJECT_ID?} with your project ID
    cnrm.cloud.google.com/project-id: ${PROJECT_ID?}
spec:
  resourceID: pubsubsubscriptiondepbigquery
  friendlyName: pubsubsubscription-dep-bigquery
  datasetRef:
    name: pubsubsubscription-dep-bigquery
  schema: >
    [
      {
        "name": "data",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "The data"
      }
    ]
---
# Replace ${PROJECT_ID?} and ${PROJECT_NUMBER?} below with your desired project
# ID and project number.
apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMPolicyMember
metadata:
  name: pubsubsubscription-dep1-bigquery
spec:
  member: serviceAccount:service-${PROJECT_NUMBER?}@gcp-sa-pubsub.iam.gserviceaccount.com
  role: roles/bigquery.metadataViewer
  resourceRef:
    apiVersion: resourcemanager.cnrm.cloud.google.com/v1beta1
    kind: Project
    external: projects/${PROJECT_ID?}
---
apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMPolicyMember
metadata:
  name: pubsubsubscription-dep2-bigquery
spec:
  member: serviceAccount:service-${PROJECT_NUMBER?}@gcp-sa-pubsub.iam.gserviceaccount.com
  role: roles/bigquery.dataEditor
  resourceRef:
    apiVersion: resourcemanager.cnrm.cloud.google.com/v1beta1
    kind: Project
    external: projects/${PROJECT_ID?}
---
# Replace ${PROJECT_ID?} below with your desired project ID.
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubTopic
metadata:
  name: pubsubsubscription-dep-bigquery
  annotations:
    # Replace ${PROJECT_ID?} with your project ID
    cnrm.cloud.google.com/project-id: ${PROJECT_ID?}