Class JobControllerClient (1.0.1)

Version 1.0.1

• 5.16.0 (latest)
• 5.15.1
• 5.14.0
• 5.13.0
• 5.12.0
• 5.10.2
• 5.9.3
• 5.8.0
• 5.7.0
• 5.6.0
• 5.5.1
• 5.4.3
• 5.3.0
• 5.2.0
• 5.1.0
• 5.0.3
• 4.0.3
• 3.3.2
• 3.2.0
• 3.1.1
• 3.0.0
• 2.6.2
• 2.5.0
• 2.4.0
• 2.3.1
• 2.2.0
• 2.0.2
• 1.1.3
• 1.0.1
• 0.8.2
• 0.7.0
• 0.6.1
• 0.5.0

JobControllerClient(
    transport=None,
    channel=None,
    credentials=None,
    client_config=None,
    client_info=None,
    client_options=None,
)

The JobController provides methods to manage jobs.

Methods

JobControllerClient

JobControllerClient(
    transport=None,
    channel=None,
    credentials=None,
    client_config=None,
    client_info=None,
    client_options=None,
)

Constructor.

Parameters
Name	Description
channel	grpc.Channel DEPRECATED. A Channel instance through which to make calls. This argument is mutually exclusive with credentials; providing both will raise an exception.
credentials	google.auth.credentials.Credentials The authorization credentials to attach to requests. These credentials identify this application to the service. If none are specified, the client will attempt to ascertain the credentials from the environment. This argument is mutually exclusive with providing a transport instance to transport; doing so will raise an exception.
client_config	dict DEPRECATED. A dictionary of call options for each method. If not specified, the default configuration is used.
client_info	google.api_core.gapic_v1.client_info.ClientInfo The client info used to send a user-agent string along with API requests. If None, then default info will be used. Generally, you only need to set this if you're developing your own client library.
client_options	Union[dict, google.api_core.client_options.ClientOptions] Client options used to set user options on the client. API Endpoint should be set through client_options.

cancel_job

cancel_job(project_id, region, job_id, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Starts a job cancellation request. To access the job resource after cancellation, call regions/{region}/jobs.list <https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.jobs/list> or regions/{region}/jobs.get <https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.jobs/get>.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job_id:

job_id = ''

response = client.cancel_job(project_id, region, job_id)

Parameters
Name	Description
project_id	str Required. The ID of the Google Cloud Platform project that the job belongs to.
region	str Required. The Dataproc region in which to handle the request.
job_id	str Required. The job ID.
retry	Optional[google.api_core.retry.Retry] A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.
timeout	Optional[float] The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata	Optional[Sequence[Tuple[str, str]]] Additional metadata that is provided to the method.

Exceptions
Type	Description
google.api_core.exceptions.GoogleAPICallError	If the request failed for any reason.
google.api_core.exceptions.RetryError	If the request failed due to a retryable error and retry attempts failed.
ValueError	If the parameters are invalid.

delete_job

delete_job(project_id, region, job_id, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Deletes the job from the project. If the job is active, the delete fails, and the response returns FAILED_PRECONDITION.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job_id:

job_id = ''

client.delete_job(project_id, region, job_id)

Parameters
Name	Description
project_id	str Required. The ID of the Google Cloud Platform project that the job belongs to.
region	str Required. The Dataproc region in which to handle the request.
job_id	str Required. The job ID.
retry	Optional[google.api_core.retry.Retry] A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.
timeout	Optional[float] The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata	Optional[Sequence[Tuple[str, str]]] Additional metadata that is provided to the method.

Exceptions
Type	Description
google.api_core.exceptions.GoogleAPICallError	If the request failed for any reason.
google.api_core.exceptions.RetryError	If the request failed due to a retryable error and retry attempts failed.
ValueError	If the parameters are invalid.

from_service_account_file

from_service_account_file(filename, *args, **kwargs)

Creates an instance of this client using the provided credentials file.

Parameter
Name	Description
filename	str The path to the service account private key json file.

Returns
Type	Description
JobControllerClient	The constructed client.

from_service_account_json

from_service_account_json(filename, *args, **kwargs)

Creates an instance of this client using the provided credentials file.

Parameter
Name	Description
filename	str The path to the service account private key json file.

Returns
Type	Description
JobControllerClient	The constructed client.

get_job

get_job(project_id, region, job_id, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Gets the resource representation for a job in a project.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job_id:

job_id = ''

response = client.get_job(project_id, region, job_id)

Parameters
Name	Description
project_id	str Required. The ID of the Google Cloud Platform project that the job belongs to.
region	str Required. The Dataproc region in which to handle the request.
job_id	str Required. The job ID.
retry	Optional[google.api_core.retry.Retry] A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.
timeout	Optional[float] The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata	Optional[Sequence[Tuple[str, str]]] Additional metadata that is provided to the method.

Exceptions
Type	Description
google.api_core.exceptions.GoogleAPICallError	If the request failed for any reason.
google.api_core.exceptions.RetryError	If the request failed due to a retryable error and retry attempts failed.
ValueError	If the parameters are invalid.

list_jobs

list_jobs(project_id, region, page_size=None, cluster_name=None, job_state_matcher=None, filter_=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Lists regions/{region}/jobs in a project.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

Iterate over all results

for element in client.list_jobs(project_id, region): ... # process element ... pass

Alternatively:

Iterate over results one page at a time

for page in client.list_jobs(project_id, region).pages: ... for element in page: ... # process element ... pass

Parameters
Name	Description
project_id	str Required. The ID of the Google Cloud Platform project that the job belongs to.
region	str Required. The Dataproc region in which to handle the request.
page_size	int The maximum number of resources contained in the underlying API response. If page streaming is performed per- resource, this parameter does not affect the return value. If page streaming is performed per-page, this determines the maximum number of resources in a page.
cluster_name	str Optional. If set, the returned jobs list includes only jobs that were submitted to the named cluster.
job_state_matcher	JobStateMatcher Optional. Specifies enumerated categories of jobs to list. (default = match ALL jobs). If filter is provided, jobStateMatcher will be ignored.
filter_	str Optional. A filter constraining the jobs to list. Filters are case-sensitive and have the following syntax: [field = value] AND [field [= value]] ... where field is status.state or labels.[KEY], and [KEY] is a label key. value can be * to match all values. status.state can be either ACTIVE or NON_ACTIVE. Only the logical AND operator is supported; space-separated items are treated as having an implicit AND operator. Example filter: status.state = ACTIVE AND labels.env = staging AND labels.starred = *
retry	Optional[google.api_core.retry.Retry] A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.
timeout	Optional[float] The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata	Optional[Sequence[Tuple[str, str]]] Additional metadata that is provided to the method.

Exceptions
Type	Description
google.api_core.exceptions.GoogleAPICallError	If the request failed for any reason.
google.api_core.exceptions.RetryError	If the request failed due to a retryable error and retry attempts failed.
ValueError	If the parameters are invalid.

submit_job

submit_job(project_id, region, job, request_id=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Submits a job to a cluster.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job:

job = {}

response = client.submit_job(project_id, region, job)

Parameters
Name	Description
project_id	str Required. The ID of the Google Cloud Platform project that the job belongs to.
region	str Required. The Dataproc region in which to handle the request.
job	Union[dict, Job] Required. The job resource. If a dict is provided, it must be of the same form as the protobuf message Job
request_id	str Optional. A unique id used to identify the request. If the server receives two SubmitJobRequest requests with the same id, then the second request will be ignored and the first Job created and stored in the backend is returned. It is recommended to always set this value to a UUID https://en.wikipedia.org/wiki/Universally_unique_identifier_. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (), and hyphens (-). The maximum length is 40 characters.
retry	Optional[google.api_core.retry.Retry] A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.
timeout	Optional[float] The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata	Optional[Sequence[Tuple[str, str]]] Additional metadata that is provided to the method.

Exceptions
Type	Description
google.api_core.exceptions.GoogleAPICallError	If the request failed for any reason.
google.api_core.exceptions.RetryError	If the request failed due to a retryable error and retry attempts failed.
ValueError	If the parameters are invalid.

submit_job_as_operation

submit_job_as_operation(project_id, region, job, request_id=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Submits job to a cluster.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job:

job = {}

response = client.submit_job_as_operation(project_id, region, job)

def callback(operation_future): ... # Handle result. ... result = operation_future.result()

response.add_done_callback(callback)

Handle metadata.

metadata = response.metadata()

Parameters
Name	Description
project_id	str Required. The ID of the Google Cloud Platform project that the job belongs to.
region	str Required. The Dataproc region in which to handle the request.
job	Union[dict, Job] Required. The job resource. If a dict is provided, it must be of the same form as the protobuf message Job
request_id	str Optional. A unique id used to identify the request. If the server receives two SubmitJobRequest requests with the same id, then the second request will be ignored and the first Job created and stored in the backend is returned. It is recommended to always set this value to a UUID https://en.wikipedia.org/wiki/Universally_unique_identifier_. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (), and hyphens (-). The maximum length is 40 characters.
retry	Optional[google.api_core.retry.Retry] A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.
timeout	Optional[float] The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata	Optional[Sequence[Tuple[str, str]]] Additional metadata that is provided to the method.

Exceptions
Type	Description
google.api_core.exceptions.GoogleAPICallError	If the request failed for any reason.
google.api_core.exceptions.RetryError	If the request failed due to a retryable error and retry attempts failed.
ValueError	If the parameters are invalid.

update_job

update_job(project_id, region, job_id, job, update_mask, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Updates a job in a project.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job_id:

job_id = ''

TODO: Initialize job:

job = {}

TODO: Initialize update_mask:

update_mask = {}

response = client.update_job(project_id, region, job_id, job, update_mask)

Parameters
Name	Description
project_id	str Required. The ID of the Google Cloud Platform project that the job belongs to.
region	str Required. The Dataproc region in which to handle the request.
job_id	str Required. The job ID.
job	Union[dict, Job] Required. The changes to the job. If a dict is provided, it must be of the same form as the protobuf message Job
update_mask	Union[dict, FieldMask] Required. Specifies the path, relative to Job, of the field to update. For example, to update the labels of a Job the update_mask parameter would be specified as labels, and the PATCH request body would specify the new value. Note: Currently, labels is the only field that can be updated. If a dict is provided, it must be of the same form as the protobuf message FieldMask
retry	Optional[google.api_core.retry.Retry] A retry object used to retry requests. If None is specified, requests will be retried using a default configuration.
timeout	Optional[float] The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata	Optional[Sequence[Tuple[str, str]]] Additional metadata that is provided to the method.

Exceptions
Type	Description
google.api_core.exceptions.GoogleAPICallError	If the request failed for any reason.
google.api_core.exceptions.RetryError	If the request failed due to a retryable error and retry attempts failed.
ValueError	If the parameters are invalid.