Types overview

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

CheckValidCredsResponse

A response indicating whether the credentials exist and are valid.

Fields
hasValidCreds	boolean If set to true, the credentials exist and are valid.

DataSource

Defines the properties and custom parameters for a data source.

Fields
authorizationType	enum Indicates the type of authorization.
Enum type. Can be one of the following:
AUTHORIZATION_TYPE_UNSPECIFIED	Type unspecified.
AUTHORIZATION_CODE	Use OAuth 2 authorization codes that can be exchanged for a refresh token on the backend.
GOOGLE_PLUS_AUTHORIZATION_CODE	Return an authorization code for a given Google+ page that can then be exchanged for a refresh token on the backend.
FIRST_PARTY_OAUTH	Use First Party OAuth.
clientId	string Data source client id which should be used to receive refresh token.
dataRefreshType	enum Specifies whether the data source supports automatic data refresh for the past few days, and how it's supported. For some data sources, data might not be complete until a few days later, so it's useful to refresh data automatically.
Enum type. Can be one of the following:
DATA_REFRESH_TYPE_UNSPECIFIED	The data source won't support data auto refresh, which is default value.
SLIDING_WINDOW	The data source supports data auto refresh, and runs will be scheduled for the past few days. Does not allow custom values to be set for each transfer config.
CUSTOM_SLIDING_WINDOW	The data source supports data auto refresh, and runs will be scheduled for the past few days. Allows custom values to be set for each transfer config.
dataSourceId	string Data source id.
defaultDataRefreshWindowDays	integer (int32 format) Default data refresh window on days. Only meaningful when data_refresh_type = SLIDING_WINDOW.
defaultSchedule	string Default data transfer schedule. Examples of valid schedules include: 1st,3rd monday of month 15:30, every wed,fri of jan,jun 13:15, and first sunday of quarter 00:00.
description	string User friendly data source description string.
displayName	string User friendly data source name.
helpUrl	string Url for the help document for this data source.
manualRunsDisabled	boolean Disables backfilling and manual run scheduling for the data source.
minimumScheduleInterval	string (Duration format) The minimum interval for scheduler to schedule runs.
name	string Output only. Data source resource name.
parameters[]	object (DataSourceParameter) Data source parameters.
scopes[]	string Api auth scopes for which refresh token needs to be obtained. These are scopes needed by a data source to prepare data and ingest them into BigQuery, e.g., https://www.googleapis.com/auth/bigquery
supportsCustomSchedule	boolean Specifies whether the data source supports a user defined schedule, or operates on the default schedule. When set to true, user can override default schedule.
supportsMultipleTransfers	boolean Deprecated. This field has no effect.
transferType	enum Deprecated. This field has no effect.
Enum type. Can be one of the following:
TRANSFER_TYPE_UNSPECIFIED	Invalid or Unknown transfer type placeholder.
BATCH	Batch data transfer.
STREAMING	Streaming data transfer. Streaming data source currently doesn't support multiple transfer configs per project.
updateDeadlineSeconds	integer (int32 format) The number of seconds to wait for an update from the data source before the Data Transfer Service marks the transfer as FAILED.

DataSourceParameter

A parameter used to define custom fields in a data source definition.

Fields
allowedValues[]	string All possible values for the parameter.
deprecated	boolean If true, it should not be used in new transfers, and it should not be visible to users.
description	string Parameter description.
displayName	string Parameter display name in the user interface.
fields[]	object (DataSourceParameter) Deprecated. This field has no effect.
immutable	boolean Cannot be changed after initial creation.
maxValue	number (double format) For integer and double values specifies maximum allowed value.
minValue	number (double format) For integer and double values specifies minimum allowed value.
paramId	string Parameter identifier.
recurse	boolean Deprecated. This field has no effect.
repeated	boolean Deprecated. This field has no effect.
required	boolean Is parameter required.
type	enum Parameter type.
Enum type. Can be one of the following:
TYPE_UNSPECIFIED	Type unspecified.
STRING	String parameter.
INTEGER	Integer parameter (64-bits). Will be serialized to json as string.
DOUBLE	Double precision floating point parameter.
BOOLEAN	Boolean parameter.
RECORD	Deprecated. This field has no effect.
PLUS_PAGE	Page ID for a Google+ Page.
validationDescription	string Description of the requirements for this field, in case the user input does not fulfill the regex pattern or min/max values.
validationHelpUrl	string URL to a help document to further explain the naming requirements.
validationRegex	string Regular expression which can be used for parameter validation.

EmailPreferences

Represents preferences for sending email notifications for transfer run events.

Fields
enableFailureEmail	boolean If true, email notifications will be sent on transfer run failures.

EnrollDataSourcesRequest

A request to enroll a set of data sources so they are visible in the BigQuery UI's Transfer tab.

Fields
dataSourceIds[]	string Data sources that are enrolled. It is required to provide at least one data source id.

ListDataSourcesResponse

Returns list of supported data sources and their metadata.

Fields
dataSources[]	object (DataSource) List of supported data sources and their transfer settings.
nextPageToken	string Output only. The next-pagination token. For multiple-page list results, this token can be used as the ListDataSourcesRequest.page_token to request the next page of list results.

ListLocationsResponse

The response message for Locations.ListLocations.

Fields
locations[]	object (Location) A list of locations that matches the specified filter in the request.
nextPageToken	string The standard List next-page token.

ListTransferConfigsResponse

The returned list of pipelines in the project.

Fields
nextPageToken	string Output only. The next-pagination token. For multiple-page list results, this token can be used as the ListTransferConfigsRequest.page_token to request the next page of list results.
transferConfigs[]	object (TransferConfig) Output only. The stored pipeline transfer configurations.

ListTransferLogsResponse

The returned list transfer run messages.

Fields
nextPageToken	string Output only. The next-pagination token. For multiple-page list results, this token can be used as the GetTransferRunLogRequest.page_token to request the next page of list results.
transferMessages[]	object (TransferMessage) Output only. The stored pipeline transfer messages.

ListTransferRunsResponse

The returned list of pipelines in the project.

Fields
nextPageToken	string Output only. The next-pagination token. For multiple-page list results, this token can be used as the ListTransferRunsRequest.page_token to request the next page of list results.
transferRuns[]	object (TransferRun) Output only. The stored pipeline transfer runs.

Location

A resource that represents Google Cloud Platform location.

Fields
displayName	string The friendly name for this location, typically a nearby city name. For example, "Tokyo".
labels	map (key: string, value: string) Cross-service attributes for the location. For example {"cloud.googleapis.com/region": "us-east1"}
locationId	string The canonical id for this location. For example: "us-east1".
metadata	map (key: string, value: any) Service-specific metadata. For example the available capacity at the given location.
name	string Resource name for the location, which may vary between implementations. For example: "projects/example-project/locations/us-east1"

ScheduleOptions

Options customizing the data transfer schedule.

Fields
disableAutoScheduling	boolean If true, automatic scheduling of data transfer runs for this configuration will be disabled. The runs can be started on ad-hoc basis using StartManualTransferRuns API. When automatic scheduling is disabled, the TransferConfig.schedule field will be ignored.
endTime	string (Timestamp format) Defines time to stop scheduling transfer runs. A transfer run cannot be scheduled at or after the end time. The end time can be changed at any moment. The time when a data transfer can be triggered manually is not limited by this option.
startTime	string (Timestamp format) Specifies time to start scheduling transfer runs. The first run will be scheduled at or after the start time according to a recurrence pattern defined in the schedule string. The start time can be changed at any moment. The time when a data transfer can be triggered manually is not limited by this option.

ScheduleTransferRunsRequest

A request to schedule transfer runs for a time range.

Fields
endTime	string (Timestamp format) Required. End time of the range of transfer runs. For example, "2017-05-30T00:00:00+00:00".
startTime	string (Timestamp format) Required. Start time of the range of transfer runs. For example, "2017-05-25T00:00:00+00:00".

ScheduleTransferRunsResponse

A response to schedule transfer runs for a time range.

Fields
runs[]	object (TransferRun) The transfer runs that were scheduled.

StartManualTransferRunsRequest

A request to start manual transfer runs.

Fields
requestedRunTime	string (Timestamp format) Specific run_time for a transfer run to be started. The requested_run_time must not be in the future.
requestedTimeRange	object (TimeRange) Time range for the transfer runs that should be started.

StartManualTransferRunsResponse

A response to start manual transfer runs.

Fields
runs[]	object (TransferRun) The transfer runs that were created.

Status

The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC. Each Status message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the API Design Guide.

Fields
code	integer (int32 format) The status code, which should be an enum value of google.rpc.Code.
details[]	object A list of messages that carry the error details. There is a common set of message types for APIs to use.
message	string A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.

TimeRange

A specification for a time range, this will request transfer runs with run_time between start_time (inclusive) and end_time (exclusive).

Fields
endTime	string (Timestamp format) End time of the range of transfer runs. For example, "2017-05-30T00:00:00+00:00". The end_time must not be in the future. Creates transfer runs where run_time is in the range between start_time (inclusive) and end_time (exclusive).
startTime	string (Timestamp format) Start time of the range of transfer runs. For example, "2017-05-25T00:00:00+00:00". The start_time must be strictly less than the end_time. Creates transfer runs where run_time is in the range between start_time (inclusive) and end_time (exclusive).

TransferConfig

Represents a data transfer configuration. A transfer configuration contains all metadata needed to perform a data transfer. For example, destination_dataset_id specifies where data should be stored. When a new transfer configuration is created, the specified destination_dataset_id is created when needed and shared with the appropriate data source service account.

Fields
dataRefreshWindowDays	integer (int32 format) The number of days to look back to automatically refresh the data. For example, if data_refresh_window_days = 10, then every day BigQuery reingests data for [today-10, today-1], rather than ingesting data for just [today-1]. Only valid if the data source supports the feature. Set the value to 0 to use the default value.
dataSourceId	string Data source id. Cannot be changed once data transfer is created.
datasetRegion	string Output only. Region in which BigQuery dataset is located.
destinationDatasetId	string The BigQuery target dataset id.
disabled	boolean Is this config disabled. When set to true, no runs are scheduled for a given transfer.
displayName	string User specified display name for the data transfer.
emailPreferences	object (EmailPreferences) Email notifications will be sent according to these preferences to the email address of the user who owns this transfer config.
name	string The resource name of the transfer config. Transfer config names have the form projects/{project_id}/locations/{region}/transferConfigs/{config_id}. Where config_id is usually a uuid, even though it is not guaranteed or required. The name is ignored when creating a transfer config.
nextRunTime	string (Timestamp format) Output only. Next time when data transfer will run.
notificationPubsubTopic	string Pub/Sub topic where notifications will be sent after transfer runs associated with this transfer config finish. The format for specifying a pubsub topic is: projects/{project}/topics/{topic}
ownerInfo	object (UserInfo) Output only. Information about the user whose credentials are used to transfer data. Populated only for transferConfigs.get requests. In case the user information is not available, this field will not be populated.
params	map (key: string, value: any) Parameters specific to each data source. For more information see the bq tab in the 'Setting up a data transfer' section for each data source. For example the parameters for Cloud Storage transfers are listed here: https://cloud.google.com/bigquery-transfer/docs/cloud-storage-transfer#bq
schedule	string Data transfer schedule. If the data source does not support a custom schedule, this should be empty. If it is empty, the default value for the data source will be used. The specified times are in UTC. Examples of valid format: 1st,3rd monday of month 15:30, every wed,fri of jan,jun 13:15, and first sunday of quarter 00:00. See more explanation about the format here: https://cloud.google.com/appengine/docs/flexible/python/scheduling-jobs-with-cron-yaml#the_schedule_format NOTE: The minimum interval time between recurring transfers depends on the data source; refer to the documentation for your data source.
scheduleOptions	object (ScheduleOptions) Options customizing the data transfer schedule.
state	enum Output only. State of the most recently updated transfer run.
Enum type. Can be one of the following:
TRANSFER_STATE_UNSPECIFIED	State placeholder (0).
PENDING	Data transfer is scheduled and is waiting to be picked up by data transfer backend (2).
RUNNING	Data transfer is in progress (3).
SUCCEEDED	Data transfer completed successfully (4).
FAILED	Data transfer failed (5).
CANCELLED	Data transfer is cancelled (6).
updateTime	string (Timestamp format) Output only. Data transfer modification time. Ignored by server on input.
userId	string (int64 format) Deprecated. Unique ID of the user on whose behalf transfer is done.

TransferMessage

Represents a user facing message for a particular data transfer run.

Fields
messageText	string Message text.
messageTime	string (Timestamp format) Time when message was logged.
severity	enum Message severity.
Enum type. Can be one of the following:
MESSAGE_SEVERITY_UNSPECIFIED	No severity specified.
INFO	Informational message.
WARNING	Warning message.
ERROR	Error message.

TransferRun

Represents a data transfer run.

Fields
dataSourceId	string Output only. Data source id.
destinationDatasetId	string Output only. The BigQuery target dataset id.
emailPreferences	object (EmailPreferences) Output only. Email notifications will be sent according to these preferences to the email address of the user who owns the transfer config this run was derived from.
endTime	string (Timestamp format) Output only. Time when transfer run ended. Parameter ignored by server for input requests.
errorStatus	object (Status) Status of the transfer run.
name	string The resource name of the transfer run. Transfer run names have the form projects/{project_id}/locations/{location}/transferConfigs/{config_id}/runs/{run_id}. The name is ignored when creating a transfer run.
notificationPubsubTopic	string Output only. Pub/Sub topic where a notification will be sent after this transfer run finishes. The format for specifying a pubsub topic is: projects/{project}/topics/{topic}
params	map (key: string, value: any) Output only. Parameters specific to each data source. For more information see the bq tab in the 'Setting up a data transfer' section for each data source. For example the parameters for Cloud Storage transfers are listed here: https://cloud.google.com/bigquery-transfer/docs/cloud-storage-transfer#bq
runTime	string (Timestamp format) For batch transfer runs, specifies the date and time of the data should be ingested.
schedule	string Output only. Describes the schedule of this transfer run if it was created as part of a regular schedule. For batch transfer runs that are scheduled manually, this is empty. NOTE: the system might choose to delay the schedule depending on the current load, so schedule_time doesn't always match this.
scheduleTime	string (Timestamp format) Minimum time after which a transfer run can be started.
startTime	string (Timestamp format) Output only. Time when transfer run was started. Parameter ignored by server for input requests.
state	enum Data transfer run state. Ignored for input requests.
Enum type. Can be one of the following:
TRANSFER_STATE_UNSPECIFIED	State placeholder (0).
PENDING	Data transfer is scheduled and is waiting to be picked up by data transfer backend (2).
RUNNING	Data transfer is in progress (3).
SUCCEEDED	Data transfer completed successfully (4).
FAILED	Data transfer failed (5).
CANCELLED	Data transfer is cancelled (6).
updateTime	string (Timestamp format) Output only. Last time the data transfer run state was updated.
userId	string (int64 format) Deprecated. Unique ID of the user on whose behalf transfer is done.

UserInfo

Information about a user.

Fields
email	string E-mail address of the user.