REST Resource: projects.locations.streams

Resource: Stream

JSON representation 
{
  "name": string,
  "createTime": string,
  "updateTime": string,
  "labels": {
    string: string,
    ...
  },
  "displayName": string,
  "sourceConfig": {
    object (SourceConfig)
  },
  "destinationConfig": {
    object (DestinationConfig)
  },
  "state": enum (State),
  "errors": [
    {
      object (Error)
    }
  ],

  // Union field backfill_strategy can be only one of the following:
  "backfillAll": {
    object (BackfillAllStrategy)
  },
  "backfillNone": {
    object (BackfillNoneStrategy)
  }
  // End of list of possible types for union field backfill_strategy.
  "customerManagedEncryptionKey": string
}
Fields
name

string

Output only. The stream's name.
createTime

string (Timestamp format)

Output only. The creation time of the stream.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
updateTime

string (Timestamp format)

Output only. The last update time of the stream.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
labels

map (key: string, value: string)

Labels.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
displayName

string

Required. Display name.
sourceConfig

object (SourceConfig)

Required. Source connection profile configuration.
destinationConfig

object (DestinationConfig)

Required. Destination connection profile configuration.
state

enum (State)

The state of the stream.
errors[]

object (Error)

Output only. Errors on the Stream.
Union field backfill_strategy. Stream backfill strategy. backfill_strategy can be only one of the following:
backfillAll

object (BackfillAllStrategy)

Automatically backfill objects included in the stream source configuration. Specific objects can be excluded.
backfillNone

object (BackfillNoneStrategy)

Do not automatically backfill any objects.
customerManagedEncryptionKey

string

Immutable. A reference to a KMS encryption key. If provided, it will be used to encrypt the data. If left blank, data will be encrypted using an internal Stream-specific encryption key provisioned through KMS.

SourceConfig

The configuration of the stream source.

JSON representation 
{
  "sourceConnectionProfileName": string,

  // Union field source_stream_config can be only one of the following:
  "oracleSourceConfig": {
    object (OracleSourceConfig)
  },
  "mysqlSourceConfig": {
    object (MysqlSourceConfig)
  }
  // End of list of possible types for union field source_stream_config.
}
Fields
sourceConnectionProfileName

string

Required. Source connection profile identifier.
Union field source_stream_config. Stream configuration that is specific to the data source type. source_stream_config can be only one of the following:
oracleSourceConfig

object (OracleSourceConfig)

Oracle data source configuration
mysqlSourceConfig

object (MysqlSourceConfig)

MySQL data source configuration

OracleSourceConfig

Oracle data source configuration

JSON representation 
{
  "allowlist": {
    object (OracleRdbms)
  },
  "rejectlist": {
    object (OracleRdbms)
  },

  // Union field large_objects_handling can be only one of the following:
  "dropLargeObjects": {
    object (DropLargeObjects)
  }
  // End of list of possible types for union field large_objects_handling.
}
Fields
allowlist

object (OracleRdbms)

Oracle objects to include in the stream.
rejectlist

object (OracleRdbms)

Oracle objects to exclude from the stream.
Union field large_objects_handling. The configuration for handle Oracle large objects. large_objects_handling can be only one of the following:
dropLargeObjects

object (DropLargeObjects)

Drop large object values.

OracleRdbms

Oracle database structure.

JSON representation 
{
  "oracleSchemas": [
    {
      object (OracleSchema)
    }
  ]
}
Fields
oracleSchemas[]

object (OracleSchema)

Oracle schemas/databases in the database server.

OracleSchema

Oracle schema.

JSON representation 
{
  "schemaName": string,
  "oracleTables": [
    {
      object (OracleTable)
    }
  ]
}
Fields
schemaName

string

Schema name.
oracleTables[]

object (OracleTable)

Tables in the schema.

OracleTable

Oracle table.

JSON representation 
{
  "tableName": string,
  "oracleColumns": [
    {
      object (OracleColumn)
    }
  ]
}
Fields
tableName

string

Table name.
oracleColumns[]

object (OracleColumn)

Oracle columns in the schema. When unspecified as part of inclue/exclude lists, includes/excludes everything.

OracleColumn

Oracle Column.

JSON representation 
{
  "columnName": string,
  "dataType": string,
  "length": integer,
  "precision": integer,
  "scale": integer,
  "encoding": string,
  "primaryKey": boolean,
  "nullable": boolean,
  "ordinalPosition": integer
}
Fields
columnName

string

Column name.
dataType

string

The Oracle data type.
length

integer

Column length.
precision

integer

Column precision.
scale

integer

Column scale.
encoding

string

Column encoding.
primaryKey

boolean

Whether or not the column represents a primary key.
nullable

boolean

Whether or not the column can accept a null value.
ordinalPosition

integer

The ordinal position of the column in the table.

DropLargeObjects

Configuration to drop large object values.

MysqlSourceConfig

MySQL source configuration

JSON representation 
{
  "allowlist": {
    object (MysqlRdbms)
  },
  "rejectlist": {
    object (MysqlRdbms)
  }
}
Fields
allowlist

object (MysqlRdbms)

MySQL objects to retrieve from the source.
rejectlist

object (MysqlRdbms)

MySQL objects to exclude from the stream.

MysqlRdbms

MySQL database structure

JSON representation 
{
  "mysqlDatabases": [
    {
      object (MysqlDatabase)
    }
  ]
}
Fields
mysqlDatabases[]

object (MysqlDatabase)

Mysql databases on the server

MysqlDatabase

MySQL database.

JSON representation 
{
  "databaseName": string,
  "mysqlTables": [
    {
      object (MysqlTable)
    }
  ]
}
Fields
databaseName

string

Database name.
mysqlTables[]

object (MysqlTable)

Tables in the database.

MysqlTable

MySQL table.

JSON representation 
{
  "tableName": string,
  "mysqlColumns": [
    {
      object (MysqlColumn)
    }
  ]
}
Fields
tableName

string

Table name.
mysqlColumns[]

object (MysqlColumn)

MySQL columns in the database. When unspecified as part of include/exclude lists, includes/excludes everything.

MysqlColumn

MySQL Column.

JSON representation 
{
  "columnName": string,
  "dataType": string,
  "length": integer,
  "collation": string,
  "primaryKey": boolean,
  "nullable": boolean,
  "ordinalPosition": integer
}
Fields
columnName

string

Column name.
dataType

string

The MySQL data type. Full data types list can be found here: https://dev.mysql.com/doc/refman/8.0/en/data-types.html
length

integer

Column length.
collation

string

Column collation.
primaryKey

boolean

Whether or not the column represents a primary key.
nullable

boolean

Whether or not the column can accept a null value.
ordinalPosition

integer

The ordinal position of the column in the table.

DestinationConfig

The configuration of the stream destination.

JSON representation 
{
  "destinationConnectionProfileName": string,

  // Union field destination_stream_config can be only one of the following:
  "gcsDestinationConfig": {
    object (GcsDestinationConfig)
  }
  // End of list of possible types for union field destination_stream_config.
}
Fields
destinationConnectionProfileName

string

Required. Destination connection profile identifier.
Union field destination_stream_config. Stream configuration that is specific to the data destination type. destination_stream_config can be only one of the following:
gcsDestinationConfig

object (GcsDestinationConfig)

GCS destination configuration.

GcsDestinationConfig

Google Cloud Storage destination configuration

JSON representation 
{
  "path": string,
  "gcsFileFormat": enum (GcsFileFormat),
  "fileRotationMb": integer,
  "fileRotationInterval": string,

  // Union field file_format can be only one of the following:
  "avroFileFormat": {
    object (AvroFileFormat)
  },
  "jsonFileFormat": {
    object (JsonFileFormat)
  }
  // End of list of possible types for union field file_format.
}
Fields
path

string

Path inside the Cloud Storage bucket to write data to.
gcsFileFormat
(deprecated)

enum (GcsFileFormat)

File format that data should be written in. Deprecated field (b/169501737) - use file_format instead.
fileRotationMb

integer

The maximum file size to be saved in the bucket.
fileRotationInterval

string (Duration format)

The maximum duration for which new events are added before a file is closed and a new file is created.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".
Union field file_format. File Format that the data should be written in. file_format can be only one of the following:
avroFileFormat

object (AvroFileFormat)

AVRO file format configuration.
jsonFileFormat

object (JsonFileFormat)

JSON file format configuration.

GcsFileFormat

File format in Cloud Storage.

Enums
GCS_FILE_FORMAT_UNSPECIFIED Unspecified Cloud Storage file format.
AVRO Avro file format

AvroFileFormat

AVRO file format configuration.

JsonFileFormat

JSON file format configuration.

JSON representation 
{
  "schemaFileFormat": enum (SchemaFileFormat),
  "compression": enum (JsonCompression)
}
Fields
schemaFileFormat

enum (SchemaFileFormat)

The schema file format along JSON data files.
compression

enum (JsonCompression)

Compression of the loaded JSON file.

SchemaFileFormat

Schema file format.

Enums
SCHEMA_FILE_FORMAT_UNSPECIFIED Unspecified schema file format.
NO_SCHEMA_FILE Do not attach schema file.
AVRO_SCHEMA_FILE Avro schema format.

JsonCompression

Json file compression.

Enums
JSON_COMPRESSION_UNSPECIFIED Unspecified json file compression.
NO_COMPRESSION Do not compress JSON file.
GZIP Gzip compression.

State

Stream state.

Enums
STATE_UNSPECIFIED Unspecified stream state.
CREATED The stream has been created.
RUNNING The stream is running.
PAUSED The stream is paused.
MAINTENANCE

The stream is in maintenance mode.

Updates are rejected on the resource in this state.
FAILED The stream is experiencing an error that is preventing data from being streamed.
FAILED_PERMANENTLY The stream has experienced a terminal failure.
STARTING The stream is starting, but not yet running.
DRAINING The Stream is no longer reading new events, but still writing events in the buffer.

BackfillAllStrategy

Backfill strategy to automatically backfill the Stream's objects. Specific objects can be excluded.

JSON representation 
{

  // Union field excluded_objects can be only one of the following:
  "oracleExcludedObjects": {
    object (OracleRdbms)
  },
  "mysqlExcludedObjects": {
    object (MysqlRdbms)
  }
  // End of list of possible types for union field excluded_objects.
}
Fields
Union field excluded_objects. List of objects to exclude. excluded_objects can be only one of the following:
oracleExcludedObjects

object (OracleRdbms)

Oracle data source objects to avoid backfilling.
mysqlExcludedObjects

object (MysqlRdbms)

MySQL data source objects to avoid backfilling.

BackfillNoneStrategy

Backfill strategy to disable automatic backfill for the Stream's objects.

Methods

create
(deprecated)

 Use this method to create a stream.

delete
(deprecated)

 Use this method to delete a stream.

fetchErrors
(deprecated)

 Use this method to fetch any errors associated with a stream.

get
(deprecated)

 Use this method to get details about a stream.

list
(deprecated)

 Use this method to list streams in a project and location.

patch
(deprecated)

 Use this method to update the configuration of a stream.