Class CloudStorage (2.27.0)

CloudStorage(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Ingestion settings for Cloud Storage.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Attributes
Name	Description
`state :noindex:`	`google.pubsub_v1.types.IngestionDataSourceSettings.CloudStorage.State` Output only. An output-only field that indicates the state of the Cloud Storage ingestion source.
`bucket :noindex:`	`str` Optional. Cloud Storage bucket. The bucket name must be without any prefix like "gs://". See the [bucket naming requirements] (https://cloud.google.com/storage/docs/buckets#naming).
`text_format :noindex:`	`google.pubsub_v1.types.IngestionDataSourceSettings.CloudStorage.TextFormat` Optional. Data from Cloud Storage will be interpreted as text. This field is a member of `oneof`_ `input_format`.
`avro_format :noindex:`	`google.pubsub_v1.types.IngestionDataSourceSettings.CloudStorage.AvroFormat` Optional. Data from Cloud Storage will be interpreted in Avro format. This field is a member of `oneof`_ `input_format`.
`pubsub_avro_format :noindex:`	`google.pubsub_v1.types.IngestionDataSourceSettings.CloudStorage.PubSubAvroFormat` Optional. It will be assumed data from Cloud Storage was written via `Cloud Storage subscriptions
`minimum_object_create_time :noindex:`	`google.protobuf.timestamp_pb2.Timestamp` Optional. Only objects with a larger or equal creation timestamp will be ingested.
`match_glob :noindex:`	`str` Optional. Glob pattern used to match objects that will be ingested. If unset, all objects will be ingested. See the `supported patterns

Classes

AvroFormat

AvroFormat(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Configuration for reading Cloud Storage data in Avro binary format. The bytes of each object will be set to the data field of a Pub/Sub message.

PubSubAvroFormat

PubSubAvroFormat(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Configuration for reading Cloud Storage data written via Cloud Storage subscriptions <https://cloud.google.com/pubsub/docs/cloudstorage>__. The data and attributes fields of the originally exported Pub/Sub message will be restored when publishing.

State

State(value)

Possible states for ingestion from Cloud Storage.

    -  storage.objects.list: to list the objects in a bucket.
    -  storage.objects.get: to read the objects in a bucket.
    -  storage.buckets.get: to verify the bucket exists.
PUBLISH_PERMISSION_DENIED (3):
    Permission denied encountered while publishing to the topic.
    This can happen if the Pub/Sub SA has not been granted the
    `appropriate publish
    permissions <https://cloud.google.com/pubsub/docs/access-control#pubsub.publisher>`__
BUCKET_NOT_FOUND (4):
    The provided Cloud Storage bucket doesn't
    exist.
TOO_MANY_OBJECTS (5):
    The Cloud Storage bucket has too many
    objects, ingestion will be paused.

TextFormat

TextFormat(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Configuration for reading Cloud Storage data in text format. Each line of text as specified by the delimiter will be set to the data field of a Pub/Sub message.