Class DataTransferServiceClient (0.4.1)

DataTransferServiceClient(
    transport=None,
    channel=None,
    credentials=None,
    client_config=None,
    client_info=None,
    client_options=None,
)

The Google BigQuery Data Transfer Service API enables BigQuery users to configure the transfer of their data from other Google Products into BigQuery. This service contains methods that are end user exposed. It backs up the frontend.

Inheritance

builtins.object > google.cloud.bigquery_datatransfer_v1.gapic.data_transfer_service_client.DataTransferServiceClient > DataTransferServiceClient

Methods

DataTransferServiceClient

DataTransferServiceClient(
    transport=None,
    channel=None,
    credentials=None,
    client_config=None,
    client_info=None,
    client_options=None,
)

Constructor.

Parameters
NameDescription
channel grpc.Channel

DEPRECATED. A Channel instance through which to make calls. This argument is mutually exclusive with credentials; providing both will raise an exception.

credentials google.auth.credentials.Credentials

The authorization credentials to attach to requests. These credentials identify this application to the service. If none are specified, the client will attempt to ascertain the credentials from the environment. This argument is mutually exclusive with providing a transport instance to transport; doing so will raise an exception.

client_config dict

DEPRECATED. A dictionary of call options for each method. If not specified, the default configuration is used.

client_info google.api_core.gapic_v1.client_info.ClientInfo

The client info used to send a user-agent string along with API requests. If None, then default info will be used. Generally, you only need to set this if you're developing your own client library.

client_options Union[dict, google.api_core.client_options.ClientOptions]

Client options used to set user options on the client. API Endpoint should be set through client_options.

check_valid_creds

check_valid_creds(name, retry=<object object>, timeout=<object object>, metadata=None)

Returns true if valid credentials exist for the given data source and requesting user. Some data sources doesn't support service account, so we need to talk to them on behalf of the end user. This API just checks whether we have OAuth token for the particular user, which is a pre-requisite before user can create a transfer config.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

name = client.project_data_source_path('[PROJECT]', '[DATA_SOURCE]')

response = client.check_valid_creds(name)

Parameters
NameDescription
name str

The data source in the form: projects/{project_id}/dataSources/{data_source_id}

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

create_transfer_config

create_transfer_config(parent, transfer_config, authorization_code=None, version_info=None, retry=<object object>, timeout=<object object>, metadata=None)

Creates a new data transfer configuration.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

parent = client.project_path('[PROJECT]')

TODO: Initialize transfer_config:

transfer_config = {}

response = client.create_transfer_config(parent, transfer_config)

Parameters
NameDescription
parent str

The BigQuery project id where the transfer configuration should be created. Must be in the format projects/{project_id}/locations/{location_id} If specified location and location of the destination bigquery dataset do not match - the request will fail.

transfer_config Union[dict, TransferConfig]

Data transfer configuration to create. If a dict is provided, it must be of the same form as the protobuf message TransferConfig

authorization_code str

Optional OAuth2 authorization code to use with this transfer configuration. This is required if new credentials are needed, as indicated by CheckValidCreds. In order to obtain authorization_code, please make a request to https://www.gstatic.com/bigquerydatatransfer/oauthz/auth?client_id=&scope=<data_source_scopes>&redirect_uri=<redirect_uri> - client_id should be OAuth client_id of BigQuery DTS API for the given data source returned by ListDataSources method. - data_source_scopes are the scopes returned by ListDataSources method. - redirect_uri is an optional parameter. If not specified, then authorization code is posted to the opener of authorization flow window. Otherwise it will be sent to the redirect uri. A special value of urn:ietf:wg:oauth:2.0:oob means that authorization code should be returned in the title bar of the browser, with the page text prompting the user to copy the code and paste it in the application.

version_info str

Optional version info. If users want to find a very recent access token, that is, immediately after approving access, users have to set the version_info claim in the token request. To obtain the version_info, users must use the "none+gsession" response type. which be return a version_info back in the authorization response which be be put in a JWT claim in the token request.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

delete_transfer_config

delete_transfer_config(name, retry=<object object>, timeout=<object object>, metadata=None)

Deletes a data transfer configuration, including any associated transfer runs and logs.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

name = client.project_transfer_config_path('[PROJECT]', '[TRANSFER_CONFIG]')

client.delete_transfer_config(name)

Parameters
NameDescription
name str

The field will contain name of the resource requested, for example: projects/{project_id}/transferConfigs/{config_id}

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

delete_transfer_run

delete_transfer_run(name, retry=<object object>, timeout=<object object>, metadata=None)

Deletes the specified transfer run.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

name = client.project_run_path('[PROJECT]', '[TRANSFER_CONFIG]', '[RUN]')

client.delete_transfer_run(name)

Parameters
NameDescription
name str

The field will contain name of the resource requested, for example: projects/{project_id}/transferConfigs/{config_id}/runs/{run_id}

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

from_service_account_file

from_service_account_file(filename, *args, **kwargs)

Creates an instance of this client using the provided credentials file.

Parameter
NameDescription
filename str

The path to the service account private key json file.

Returns
TypeDescription
DataTransferServiceClientThe constructed client.

from_service_account_json

from_service_account_json(filename, *args, **kwargs)

Creates an instance of this client using the provided credentials file.

Parameter
NameDescription
filename str

The path to the service account private key json file.

Returns
TypeDescription
DataTransferServiceClientThe constructed client.

get_data_source

get_data_source(name, retry=<object object>, timeout=<object object>, metadata=None)

Retrieves a supported data source and returns its settings, which can be used for UI rendering.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

name = client.project_data_source_path('[PROJECT]', '[DATA_SOURCE]')

response = client.get_data_source(name)

Parameters
NameDescription
name str

The field will contain name of the resource requested, for example: projects/{project_id}/dataSources/{data_source_id}

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

get_transfer_config

get_transfer_config(name, retry=<object object>, timeout=<object object>, metadata=None)

Returns information about a data transfer config.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

name = client.project_transfer_config_path('[PROJECT]', '[TRANSFER_CONFIG]')

response = client.get_transfer_config(name)

Parameters
NameDescription
name str

The field will contain name of the resource requested, for example: projects/{project_id}/transferConfigs/{config_id}

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

get_transfer_run

get_transfer_run(name, retry=<object object>, timeout=<object object>, metadata=None)

Returns information about the particular transfer run.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

name = client.project_run_path('[PROJECT]', '[TRANSFER_CONFIG]', '[RUN]')

response = client.get_transfer_run(name)

Parameters
NameDescription
name str

The field will contain name of the resource requested, for example: projects/{project_id}/transferConfigs/{config_id}/runs/{run_id}

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

list_data_sources

list_data_sources(parent, page_size=None, retry=<object object>, timeout=<object object>, metadata=None)

Lists supported data sources and returns their settings, which can be used for UI rendering.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

parent = client.project_path('[PROJECT]')

Iterate over all results

for element in client.list_data_sources(parent): ... # process element ... pass

Alternatively:

Iterate over results one page at a time

for page in client.list_data_sources(parent).pages: ... for element in page: ... # process element ... pass

Parameters
NameDescription
parent str

The BigQuery project id for which data sources should be returned. Must be in the form: projects/{project_id}

page_size int

The maximum number of resources contained in the underlying API response. If page streaming is performed per- resource, this parameter does not affect the return value. If page streaming is performed per-page, this determines the maximum number of resources in a page.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

list_transfer_configs

list_transfer_configs(parent, data_source_ids=None, page_size=None, retry=<object object>, timeout=<object object>, metadata=None)

Returns information about all data transfers in the project.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

parent = client.project_path('[PROJECT]')

Iterate over all results

for element in client.list_transfer_configs(parent): ... # process element ... pass

Alternatively:

Iterate over results one page at a time

for page in client.list_transfer_configs(parent).pages: ... for element in page: ... # process element ... pass

Parameters
NameDescription
parent str

The BigQuery project id for which data sources should be returned: projects/{project_id}.

data_source_ids list[str]

When specified, only configurations of requested data sources are returned.

page_size int

The maximum number of resources contained in the underlying API response. If page streaming is performed per- resource, this parameter does not affect the return value. If page streaming is performed per-page, this determines the maximum number of resources in a page.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

list_transfer_logs

list_transfer_logs(parent, page_size=None, message_types=None, retry=<object object>, timeout=<object object>, metadata=None)

Returns user facing log messages for the data transfer run.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

parent = client.project_run_path('[PROJECT]', '[TRANSFER_CONFIG]', '[RUN]')

Iterate over all results

for element in client.list_transfer_logs(parent): ... # process element ... pass

Alternatively:

Iterate over results one page at a time

for page in client.list_transfer_logs(parent).pages: ... for element in page: ... # process element ... pass

Parameters
NameDescription
parent str

Transfer run name in the form: projects/{project_id}/transferConfigs/{config_Id}/runs/{run_id}.

page_size int

The maximum number of resources contained in the underlying API response. If page streaming is performed per- resource, this parameter does not affect the return value. If page streaming is performed per-page, this determines the maximum number of resources in a page.

message_types list[MessageSeverity]

Message types to return. If not populated - INFO, WARNING and ERROR messages are returned.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

list_transfer_runs

list_transfer_runs(parent, states=None, page_size=None, run_attempt=None, retry=<object object>, timeout=<object object>, metadata=None)

Returns information about running and completed jobs.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

parent = client.project_transfer_config_path('[PROJECT]', '[TRANSFER_CONFIG]')

Iterate over all results

for element in client.list_transfer_runs(parent): ... # process element ... pass

Alternatively:

Iterate over results one page at a time

for page in client.list_transfer_runs(parent).pages: ... for element in page: ... # process element ... pass

Parameters
NameDescription
parent str

Name of transfer configuration for which transfer runs should be retrieved. Format of transfer configuration resource name is: projects/{project_id}/transferConfigs/{config_id}.

states list[TransferState]

When specified, only transfer runs with requested states are returned.

page_size int

The maximum number of resources contained in the underlying API response. If page streaming is performed per- resource, this parameter does not affect the return value. If page streaming is performed per-page, this determines the maximum number of resources in a page.

run_attempt RunAttempt

Indicates how run attempts are to be pulled.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

location_data_source_path

location_data_source_path(project, location, data_source)

Return a fully-qualified location_data_source string.

location_path

location_path(project, location)

Return a fully-qualified location string.

location_run_path

location_run_path(project, location, transfer_config, run)

Return a fully-qualified location_run string.

location_transfer_config_path

location_transfer_config_path(project, location, transfer_config)

Return a fully-qualified location_transfer_config string.

project_data_source_path

project_data_source_path(project, data_source)

Return a fully-qualified project_data_source string.

project_path

project_path(project)

Return a fully-qualified project string.

project_run_path

project_run_path(project, transfer_config, run)

Return a fully-qualified project_run string.

project_transfer_config_path

project_transfer_config_path(project, transfer_config)

Return a fully-qualified project_transfer_config string.

schedule_transfer_runs

schedule_transfer_runs(parent, start_time, end_time, retry=<object object>, timeout=<object object>, metadata=None)

Creates transfer runs for a time range [start_time, end_time]. For each date - or whatever granularity the data source supports - in the range, one transfer run is created. Note that runs are created per UTC time in the time range. DEPRECATED: use StartManualTransferRuns instead.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

parent = client.project_transfer_config_path('[PROJECT]', '[TRANSFER_CONFIG]')

TODO: Initialize start_time:

start_time = {}

TODO: Initialize end_time:

end_time = {}

response = client.schedule_transfer_runs(parent, start_time, end_time)

Parameters
NameDescription
parent str

Transfer configuration name in the form: projects/{project_id}/transferConfigs/{config_id}.

start_time Union[dict, Timestamp]

Start time of the range of transfer runs. For example, "2017-05-25T00:00:00+00:00". If a dict is provided, it must be of the same form as the protobuf message Timestamp

end_time Union[dict, Timestamp]

End time of the range of transfer runs. For example, "2017-05-30T00:00:00+00:00". If a dict is provided, it must be of the same form as the protobuf message Timestamp

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

start_manual_transfer_runs

start_manual_transfer_runs(parent=None, requested_time_range=None, requested_run_time=None, retry=<object object>, timeout=<object object>, metadata=None)

Start manual transfer runs to be executed now with schedule_time equal to current time. The transfer runs can be created for a time range where the run_time is between start_time (inclusive) and end_time (exclusive), or for a specific run_time.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

response = client.start_manual_transfer_runs()

Parameters
NameDescription
parent str

Transfer configuration name in the form: projects/{project_id}/transferConfigs/{config_id}.

requested_time_range Union[dict, TimeRange]

Time range for the transfer runs that should be started. If a dict is provided, it must be of the same form as the protobuf message TimeRange

requested_run_time Union[dict, Timestamp]

Specific run_time for a transfer run to be started. The requested_run_time must not be in the future. If a dict is provided, it must be of the same form as the protobuf message Timestamp

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.

update_transfer_config

update_transfer_config(transfer_config, update_mask, authorization_code=None, version_info=None, retry=<object object>, timeout=<object object>, metadata=None)

Updates a data transfer configuration. All fields must be set, even if they are not updated.

.. rubric:: Example

from google.cloud import bigquery_datatransfer_v1

client = bigquery_datatransfer_v1.DataTransferServiceClient()

TODO: Initialize transfer_config:

transfer_config = {}

TODO: Initialize update_mask:

update_mask = {}

response = client.update_transfer_config(transfer_config, update_mask)

Parameters
NameDescription
transfer_config Union[dict, TransferConfig]

Data transfer configuration to create. If a dict is provided, it must be of the same form as the protobuf message TransferConfig

update_mask Union[dict, FieldMask]

Required list of fields to be updated in this request. If a dict is provided, it must be of the same form as the protobuf message FieldMask

authorization_code str

Optional OAuth2 authorization code to use with this transfer configuration. If it is provided, the transfer configuration will be associated with the authorizing user. In order to obtain authorization_code, please make a request to https://www.gstatic.com/bigquerydatatransfer/oauthz/auth?client_id=&scope=<data_source_scopes>&redirect_uri=<redirect_uri> - client_id should be OAuth client_id of BigQuery DTS API for the given data source returned by ListDataSources method. - data_source_scopes are the scopes returned by ListDataSources method. - redirect_uri is an optional parameter. If not specified, then authorization code is posted to the opener of authorization flow window. Otherwise it will be sent to the redirect uri. A special value of urn:ietf:wg:oauth:2.0:oob means that authorization code should be returned in the title bar of the browser, with the page text prompting the user to copy the code and paste it in the application.

version_info str

Optional version info. If users want to find a very recent access token, that is, immediately after approving access, users have to set the version_info claim in the token request. To obtain the version_info, users must use the "none+gsession" response type. which be return a version_info back in the authorization response which be be put in a JWT claim in the token request.

retry Optional[google.api_core.retry.Retry]

A retry object used to retry requests. If None is specified, requests will not be retried.

timeout Optional[float]

The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

metadata Optional[Sequence[Tuple[str, str]]]

Additional metadata that is provided to the method.

Exceptions
TypeDescription
google.api_core.exceptions.GoogleAPICallErrorIf the request failed for any reason.
google.api_core.exceptions.RetryErrorIf the request failed due to a retryable error and retry attempts failed.
ValueErrorIf the parameters are invalid.