BatchControllerClient(*, credentials: Optional[google.auth.credentials.Credentials] = None, transport: Optional[Union[str, google.cloud.dataproc_v1.services.batch_controller.transports.base.BatchControllerTransport]] = None, client_options: Optional[google.api_core.client_options.ClientOptions] = None, client_info: google.api_core.gapic_v1.client_info.ClientInfo = <google.api_core.gapic_v1.client_info.ClientInfo object>)The BatchController provides methods to manage batch workloads.
Inheritance
builtins.object > BatchControllerClientProperties
transport
Returns the transport used by the client instance.
| Type | Description |
| BatchControllerTransport | The transport used by the client instance. |
Methods
BatchControllerClient
BatchControllerClient(*, credentials: Optional[google.auth.credentials.Credentials] = None, transport: Optional[Union[str, google.cloud.dataproc_v1.services.batch_controller.transports.base.BatchControllerTransport]] = None, client_options: Optional[google.api_core.client_options.ClientOptions] = None, client_info: google.api_core.gapic_v1.client_info.ClientInfo = <google.api_core.gapic_v1.client_info.ClientInfo object>)Instantiates the batch controller client.
| Name | Description |
| credentials |
Optional[google.auth.credentials.Credentials]
The authorization credentials to attach to requests. These credentials identify the application to the service; if none are specified, the client will attempt to ascertain the credentials from the environment. |
| transport |
Union[str, BatchControllerTransport]
The transport to use. If set to None, a transport is chosen automatically. |
| client_options |
google.api_core.client_options.ClientOptions
Custom options for the client. It won't take effect if a |
| client_info |
google.api_core.gapic_v1.client_info.ClientInfo
The client info used to send a user-agent string along with API requests. If |
| Type | Description |
| google.auth.exceptions.MutualTLSChannelError | If mutual TLS transport creation failed for any reason. |
__exit__
__exit__(type, value, traceback)Releases underlying transport's resources.
.. warning:: ONLY use as a context manager if the transport is NOT shared with other clients! Exiting the with block will CLOSE the transport and may cause errors in other clients!
batch_path
batch_path(project: str, location: str, batch: str)Returns a fully-qualified batch string.
common_billing_account_path
common_billing_account_path(billing_account: str)Returns a fully-qualified billing_account string.
common_folder_path
common_folder_path(folder: str)Returns a fully-qualified folder string.
common_location_path
common_location_path(project: str, location: str)Returns a fully-qualified location string.
common_organization_path
common_organization_path(organization: str)Returns a fully-qualified organization string.
common_project_path
common_project_path(project: str)Returns a fully-qualified project string.
create_batch
create_batch(request: Optional[Union[google.cloud.dataproc_v1.types.batches.CreateBatchRequest, dict]] = None, *, parent: Optional[str] = None, batch: Optional[google.cloud.dataproc_v1.types.batches.Batch] = None, batch_id: Optional[str] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Creates a batch workload that executes asynchronously.
from google.cloud import dataproc_v1
def sample_create_batch():
# Create a client
client = dataproc_v1.BatchControllerClient()
# Initialize request argument(s)
batch = dataproc_v1.Batch()
batch.pyspark_batch.main_python_file_uri = "main_python_file_uri_value"
request = dataproc_v1.CreateBatchRequest(
parent="parent_value",
batch=batch,
)
# Make the request
operation = client.create_batch(request=request)
print("Waiting for operation to complete...")
response = operation.result()
# Handle the response
print(response)
| Name | Description |
| request |
Union[google.cloud.dataproc_v1.types.CreateBatchRequest, dict]
The request object. A request to create a batch workload. |
| parent |
str
Required. The parent resource where this batch will be created. This corresponds to the |
| batch |
google.cloud.dataproc_v1.types.Batch
Required. The batch to create. This corresponds to the |
| batch_id |
str
Optional. The ID to use for the batch, which will become the final component of the batch's resource name. This value must be 4-63 characters. Valid characters are |
| retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
| timeout |
float
The timeout for this request. |
| metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
| Type | Description |
| google.api_core.operation.Operation | An object representing a long-running operation. The result type for the operation will be Batch A representation of a batch workload in the service. |
delete_batch
delete_batch(request: Optional[Union[google.cloud.dataproc_v1.types.batches.DeleteBatchRequest, dict]] = None, *, name: Optional[str] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Deletes the batch workload resource. If the batch is not in
terminal state, the delete fails and the response returns
FAILED_PRECONDITION.
from google.cloud import dataproc_v1
def sample_delete_batch():
# Create a client
client = dataproc_v1.BatchControllerClient()
# Initialize request argument(s)
request = dataproc_v1.DeleteBatchRequest(
name="name_value",
)
# Make the request
client.delete_batch(request=request)
| Name | Description |
| request |
Union[google.cloud.dataproc_v1.types.DeleteBatchRequest, dict]
The request object. A request to delete a batch workload. |
| name |
str
Required. The name of the batch resource to delete. This corresponds to the |
| retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
| timeout |
float
The timeout for this request. |
| metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
from_service_account_file
from_service_account_file(filename: str, *args, **kwargs)Creates an instance of this client using the provided credentials file.
| Name | Description |
| filename |
str
The path to the service account private key json file. |
| Type | Description |
| BatchControllerClient | The constructed client. |
from_service_account_info
from_service_account_info(info: dict, *args, **kwargs)Creates an instance of this client using the provided credentials info.
| Name | Description |
| info |
dict
The service account private key info. |
| Type | Description |
| BatchControllerClient | The constructed client. |
from_service_account_json
from_service_account_json(filename: str, *args, **kwargs)Creates an instance of this client using the provided credentials file.
| Name | Description |
| filename |
str
The path to the service account private key json file. |
| Type | Description |
| BatchControllerClient | The constructed client. |
get_batch
get_batch(request: Optional[Union[google.cloud.dataproc_v1.types.batches.GetBatchRequest, dict]] = None, *, name: Optional[str] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Gets the batch workload resource representation.
from google.cloud import dataproc_v1
def sample_get_batch():
# Create a client
client = dataproc_v1.BatchControllerClient()
# Initialize request argument(s)
request = dataproc_v1.GetBatchRequest(
name="name_value",
)
# Make the request
response = client.get_batch(request=request)
# Handle the response
print(response)
| Name | Description |
| request |
Union[google.cloud.dataproc_v1.types.GetBatchRequest, dict]
The request object. A request to get the resource representation for a batch workload. |
| name |
str
Required. The name of the batch to retrieve. This corresponds to the |
| retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
| timeout |
float
The timeout for this request. |
| metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
| Type | Description |
| google.cloud.dataproc_v1.types.Batch | A representation of a batch workload in the service. |
get_mtls_endpoint_and_cert_source
get_mtls_endpoint_and_cert_source(
client_options: Optional[google.api_core.client_options.ClientOptions] = None,
)Return the API endpoint and client cert source for mutual TLS.
The client cert source is determined in the following order:
(1) if GOOGLE_API_USE_CLIENT_CERTIFICATE environment variable is not "true", the
client cert source is None.
(2) if client_options.client_cert_source is provided, use the provided one; if the
default client cert source exists, use the default one; otherwise the client cert
source is None.
The API endpoint is determined in the following order:
(1) if client_options.api_endpoint if provided, use the provided one.
(2) if GOOGLE_API_USE_CLIENT_CERTIFICATE environment variable is "always", use the
default mTLS endpoint; if the environment variabel is "never", use the default API
endpoint; otherwise if client cert source exists, use the default mTLS endpoint, otherwise
use the default API endpoint.
More details can be found at https://google.aip.dev/auth/4114.
| Name | Description |
| client_options |
google.api_core.client_options.ClientOptions
Custom options for the client. Only the |
| Type | Description |
| google.auth.exceptions.MutualTLSChannelError | If any errors happen. |
| Type | Description |
| Tuple[str, Callable[[], Tuple[bytes, bytes]]] | returns the API endpoint and the client cert source to use. |
list_batches
list_batches(request: Optional[Union[google.cloud.dataproc_v1.types.batches.ListBatchesRequest, dict]] = None, *, parent: Optional[str] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Lists batch workloads.
from google.cloud import dataproc_v1
def sample_list_batches():
# Create a client
client = dataproc_v1.BatchControllerClient()
# Initialize request argument(s)
request = dataproc_v1.ListBatchesRequest(
parent="parent_value",
)
# Make the request
page_result = client.list_batches(request=request)
# Handle the response
for response in page_result:
print(response)
| Name | Description |
| request |
Union[google.cloud.dataproc_v1.types.ListBatchesRequest, dict]
The request object. A request to list batch workloads in a project. |
| parent |
str
Required. The parent, which owns this collection of batches. This corresponds to the |
| retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
| timeout |
float
The timeout for this request. |
| metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
| Type | Description |
| google.cloud.dataproc_v1.services.batch_controller.pagers.ListBatchesPager | A list of batch workloads. Iterating over this object will yield results and resolve additional pages automatically. |
parse_batch_path
parse_batch_path(path: str)Parses a batch path into its component segments.
parse_common_billing_account_path
parse_common_billing_account_path(path: str)Parse a billing_account path into its component segments.
parse_common_folder_path
parse_common_folder_path(path: str)Parse a folder path into its component segments.
parse_common_location_path
parse_common_location_path(path: str)Parse a location path into its component segments.
parse_common_organization_path
parse_common_organization_path(path: str)Parse a organization path into its component segments.
parse_common_project_path
parse_common_project_path(path: str)Parse a project path into its component segments.