WorkflowTemplateServiceClient(*, credentials: Optional[google.auth.credentials.Credentials] = None, transport: Optional[Union[str, google.cloud.dataproc_v1.services.workflow_template_service.transports.base.WorkflowTemplateServiceTransport]] = None, client_options: Optional[google.api_core.client_options.ClientOptions] = None, client_info: google.api_core.gapic_v1.client_info.ClientInfo = <google.api_core.gapic_v1.client_info.ClientInfo object>)The API interface for managing Workflow Templates in the Dataproc API.
Properties
transport
Returns the transport used by the client instance.
| Returns | |
|---|---|
| Type | Description |
WorkflowTemplateServiceTransport | The transport used by the client instance. |
Methods
WorkflowTemplateServiceClient
WorkflowTemplateServiceClient(*, credentials: Optional[google.auth.credentials.Credentials] = None, transport: Optional[Union[str, google.cloud.dataproc_v1.services.workflow_template_service.transports.base.WorkflowTemplateServiceTransport]] = None, client_options: Optional[google.api_core.client_options.ClientOptions] = None, client_info: google.api_core.gapic_v1.client_info.ClientInfo = <google.api_core.gapic_v1.client_info.ClientInfo object>)Instantiates the workflow template service client.
| Parameters | |
|---|---|
| Name | Description |
credentials |
Optional[google.auth.credentials.Credentials]
The authorization credentials to attach to requests. These credentials identify the application to the service; if none are specified, the client will attempt to ascertain the credentials from the environment. |
transport |
Union[str, WorkflowTemplateServiceTransport]
The transport to use. If set to None, a transport is chosen automatically. |
client_options |
google.api_core.client_options.ClientOptions
Custom options for the client. It won't take effect if a |
client_info |
google.api_core.gapic_v1.client_info.ClientInfo
The client info used to send a user-agent string along with API requests. If |
| Exceptions | |
|---|---|
| Type | Description |
google.auth.exceptions.MutualTLSChannelError | If mutual TLS transport creation failed for any reason. |
__exit__
__exit__(type, value, traceback)Releases underlying transport's resources.
cluster_path
cluster_path(project: str, location: str, cluster: str)Returns a fully-qualified cluster string.
common_billing_account_path
common_billing_account_path(billing_account: str)Returns a fully-qualified billing_account string.
common_folder_path
common_folder_path(folder: str)Returns a fully-qualified folder string.
common_location_path
common_location_path(project: str, location: str)Returns a fully-qualified location string.
common_organization_path
common_organization_path(organization: str)Returns a fully-qualified organization string.
common_project_path
common_project_path(project: str)Returns a fully-qualified project string.
create_workflow_template
create_workflow_template(request: Optional[Union[google.cloud.dataproc_v1.types.workflow_templates.CreateWorkflowTemplateRequest, dict]] = None, *, parent: Optional[str] = None, template: Optional[google.cloud.dataproc_v1.types.workflow_templates.WorkflowTemplate] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Creates new workflow template.
from google.cloud import dataproc_v1
def sample_create_workflow_template():
# Create a client
client = dataproc_v1.WorkflowTemplateServiceClient()
# Initialize request argument(s)
template = dataproc_v1.WorkflowTemplate()
template.id = "id_value"
template.placement.managed_cluster.cluster_name = "cluster_name_value"
template.jobs.hadoop_job.main_jar_file_uri = "main_jar_file_uri_value"
template.jobs.step_id = "step_id_value"
request = dataproc_v1.CreateWorkflowTemplateRequest(
parent="parent_value",
template=template,
)
# Make the request
response = client.create_workflow_template(request=request)
# Handle the response
print(response)
| Parameters | |
|---|---|
| Name | Description |
request |
Union[google.cloud.dataproc_v1.types.CreateWorkflowTemplateRequest, dict]
The request object. A request to create a workflow template. |
parent |
str
Required. The resource name of the region or location, as described in https://cloud.google.com/apis/design/resource_names. - For |
template |
google.cloud.dataproc_v1.types.WorkflowTemplate
Required. The Dataproc workflow template to create. This corresponds to the |
retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
timeout |
float
The timeout for this request. |
metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
| Returns | |
|---|---|
| Type | Description |
google.cloud.dataproc_v1.types.WorkflowTemplate | A Dataproc workflow template resource. |
delete_workflow_template
delete_workflow_template(request: Optional[Union[google.cloud.dataproc_v1.types.workflow_templates.DeleteWorkflowTemplateRequest, dict]] = None, *, name: Optional[str] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Deletes a workflow template. It does not cancel in-progress workflows.
from google.cloud import dataproc_v1
def sample_delete_workflow_template():
# Create a client
client = dataproc_v1.WorkflowTemplateServiceClient()
# Initialize request argument(s)
request = dataproc_v1.DeleteWorkflowTemplateRequest(
name="name_value",
)
# Make the request
client.delete_workflow_template(request=request)
| Parameters | |
|---|---|
| Name | Description |
request |
Union[google.cloud.dataproc_v1.types.DeleteWorkflowTemplateRequest, dict]
The request object. A request to delete a workflow template. Currently started workflows will remain running. |
name |
str
Required. The resource name of the workflow template, as described in https://cloud.google.com/apis/design/resource_names. - For |
retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
timeout |
float
The timeout for this request. |
metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
from_service_account_file
from_service_account_file(filename: str, *args, **kwargs)Creates an instance of this client using the provided credentials file.
| Parameter | |
|---|---|
| Name | Description |
filename |
str
The path to the service account private key json file. |
| Returns | |
|---|---|
| Type | Description |
WorkflowTemplateServiceClient | The constructed client. |
from_service_account_info
from_service_account_info(info: dict, *args, **kwargs)Creates an instance of this client using the provided credentials info.
| Parameter | |
|---|---|
| Name | Description |
info |
dict
The service account private key info. |
| Returns | |
|---|---|
| Type | Description |
WorkflowTemplateServiceClient | The constructed client. |
from_service_account_json
from_service_account_json(filename: str, *args, **kwargs)Creates an instance of this client using the provided credentials file.
| Parameter | |
|---|---|
| Name | Description |
filename |
str
The path to the service account private key json file. |
| Returns | |
|---|---|
| Type | Description |
WorkflowTemplateServiceClient | The constructed client. |
get_mtls_endpoint_and_cert_source
get_mtls_endpoint_and_cert_source(
client_options: Optional[google.api_core.client_options.ClientOptions] = None,
)Return the API endpoint and client cert source for mutual TLS.
The client cert source is determined in the following order:
(1) if GOOGLE_API_USE_CLIENT_CERTIFICATE environment variable is not "true", the
client cert source is None.
(2) if client_options.client_cert_source is provided, use the provided one; if the
default client cert source exists, use the default one; otherwise the client cert
source is None.
The API endpoint is determined in the following order:
(1) if client_options.api_endpoint if provided, use the provided one.
(2) if GOOGLE_API_USE_CLIENT_CERTIFICATE environment variable is "always", use the
default mTLS endpoint; if the environment variabel is "never", use the default API
endpoint; otherwise if client cert source exists, use the default mTLS endpoint, otherwise
use the default API endpoint.
More details can be found at https://google.aip.dev/auth/4114.
| Parameter | |
|---|---|
| Name | Description |
client_options |
google.api_core.client_options.ClientOptions
Custom options for the client. Only the |
| Exceptions | |
|---|---|
| Type | Description |
google.auth.exceptions.MutualTLSChannelError | If any errors happen. |
| Returns | |
|---|---|
| Type | Description |
Tuple[str, Callable[[], Tuple[bytes, bytes]]] | returns the API endpoint and the client cert source to use. |
get_workflow_template
get_workflow_template(request: Optional[Union[google.cloud.dataproc_v1.types.workflow_templates.GetWorkflowTemplateRequest, dict]] = None, *, name: Optional[str] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Retrieves the latest workflow template. Can retrieve previously instantiated template by specifying optional version parameter.
from google.cloud import dataproc_v1
def sample_get_workflow_template():
# Create a client
client = dataproc_v1.WorkflowTemplateServiceClient()
# Initialize request argument(s)
request = dataproc_v1.GetWorkflowTemplateRequest(
name="name_value",
)
# Make the request
response = client.get_workflow_template(request=request)
# Handle the response
print(response)
| Parameters | |
|---|---|
| Name | Description |
request |
Union[google.cloud.dataproc_v1.types.GetWorkflowTemplateRequest, dict]
The request object. A request to fetch a workflow template. |
name |
str
Required. The resource name of the workflow template, as described in https://cloud.google.com/apis/design/resource_names. - For |
retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
timeout |
float
The timeout for this request. |
metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
| Returns | |
|---|---|
| Type | Description |
google.cloud.dataproc_v1.types.WorkflowTemplate | A Dataproc workflow template resource. |
instantiate_inline_workflow_template
instantiate_inline_workflow_template(request: Optional[Union[google.cloud.dataproc_v1.types.workflow_templates.InstantiateInlineWorkflowTemplateRequest, dict]] = None, *, parent: Optional[str] = None, template: Optional[google.cloud.dataproc_v1.types.workflow_templates.WorkflowTemplate] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Instantiates a template and begins execution.
This method is equivalent to executing the sequence xref_CreateWorkflowTemplate, xref_InstantiateWorkflowTemplate, xref_DeleteWorkflowTemplate.
The returned Operation can be used to track execution of
workflow by polling
operations.get][google.longrunning.Operations.GetOperation].
The Operation will complete when entire workflow is finished.
The running workflow can be aborted via
operations.cancel][google.longrunning.Operations.CancelOperation].
This will cause any inflight jobs to be cancelled and
workflow-owned clusters to be deleted.
The Operation.metadata][google.longrunning.Operation.metadata]
will be
WorkflowMetadata <https://cloud.google.com/dataproc/docs/reference/rpc/google.cloud.dataproc.v1#workflowmetadata>.
Also see Using
WorkflowMetadata <https://cloud.google.com/dataproc/docs/concepts/workflows/debugging#using_workflowmetadata>.
On successful completion,
Operation.response][google.longrunning.Operation.response] will
be Empty][google.protobuf.Empty].
from google.cloud import dataproc_v1
def sample_instantiate_inline_workflow_template():
# Create a client
client = dataproc_v1.WorkflowTemplateServiceClient()
# Initialize request argument(s)
template = dataproc_v1.WorkflowTemplate()
template.id = "id_value"
template.placement.managed_cluster.cluster_name = "cluster_name_value"
template.jobs.hadoop_job.main_jar_file_uri = "main_jar_file_uri_value"
template.jobs.step_id = "step_id_value"
request = dataproc_v1.InstantiateInlineWorkflowTemplateRequest(
parent="parent_value",
template=template,
)
# Make the request
operation = client.instantiate_inline_workflow_template(request=request)
print("Waiting for operation to complete...")
response = operation.result()
# Handle the response
print(response)
| Parameters | |
|---|---|
| Name | Description |
request |
Union[google.cloud.dataproc_v1.types.InstantiateInlineWorkflowTemplateRequest, dict]
The request object. A request to instantiate an inline workflow template. |
parent |
str
Required. The resource name of the region or location, as described in https://cloud.google.com/apis/design/resource_names. - For |
template |
google.cloud.dataproc_v1.types.WorkflowTemplate
Required. The workflow template to instantiate. This corresponds to the |
retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
timeout |
float
The timeout for this request. |
metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
| Returns | |
|---|---|
| Type | Description |
google.api_core.operation.Operation | An object representing a long-running operation. The result type for the operation will be google.protobuf.empty_pb2.Empty A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); } The JSON representation for Empty is empty JSON object {}. |
instantiate_workflow_template
instantiate_workflow_template(request: Optional[Union[google.cloud.dataproc_v1.types.workflow_templates.InstantiateWorkflowTemplateRequest, dict]] = None, *, name: Optional[str] = None, parameters: Optional[Sequence[google.cloud.dataproc_v1.types.workflow_templates.InstantiateWorkflowTemplateRequest.ParametersEntry]] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Instantiates a template and begins execution.
The returned Operation can be used to track execution of
workflow by polling
operations.get][google.longrunning.Operations.GetOperation].
The Operation will complete when entire workflow is finished.
The running workflow can be aborted via
operations.cancel][google.longrunning.Operations.CancelOperation].
This will cause any inflight jobs to be cancelled and
workflow-owned clusters to be deleted.
The Operation.metadata][google.longrunning.Operation.metadata]
will be
WorkflowMetadata <https://cloud.google.com/dataproc/docs/reference/rpc/google.cloud.dataproc.v1#workflowmetadata>.
Also see Using
WorkflowMetadata <https://cloud.google.com/dataproc/docs/concepts/workflows/debugging#using_workflowmetadata>.
On successful completion,
Operation.response][google.longrunning.Operation.response] will
be Empty][google.protobuf.Empty].
from google.cloud import dataproc_v1
def sample_instantiate_workflow_template():
# Create a client
client = dataproc_v1.WorkflowTemplateServiceClient()
# Initialize request argument(s)
request = dataproc_v1.InstantiateWorkflowTemplateRequest(
name="name_value",
)
# Make the request
operation = client.instantiate_workflow_template(request=request)
print("Waiting for operation to complete...")
response = operation.result()
# Handle the response
print(response)
| Parameters | |
|---|---|
| Name | Description |
request |
Union[google.cloud.dataproc_v1.types.InstantiateWorkflowTemplateRequest, dict]
The request object. A request to instantiate a workflow template. |
name |
str
Required. The resource name of the workflow template, as described in https://cloud.google.com/apis/design/resource_names. - For |
parameters |
Sequence[google.cloud.dataproc_v1.types.InstantiateWorkflowTemplateRequest.ParametersEntry]
Optional. Map from parameter names to values that should be used for those parameters. Values may not exceed 1000 characters. This corresponds to the |
retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
timeout |
float
The timeout for this request. |
metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
| Returns | |
|---|---|
| Type | Description |
google.api_core.operation.Operation | An object representing a long-running operation. The result type for the operation will be google.protobuf.empty_pb2.Empty A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); } The JSON representation for Empty is empty JSON object {}. |
list_workflow_templates
list_workflow_templates(request: Optional[Union[google.cloud.dataproc_v1.types.workflow_templates.ListWorkflowTemplatesRequest, dict]] = None, *, parent: Optional[str] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Lists workflows that match the specified filter in the request.
from google.cloud import dataproc_v1
def sample_list_workflow_templates():
# Create a client
client = dataproc_v1.WorkflowTemplateServiceClient()
# Initialize request argument(s)
request = dataproc_v1.ListWorkflowTemplatesRequest(
parent="parent_value",
)
# Make the request
page_result = client.list_workflow_templates(request=request)
# Handle the response
for response in page_result:
print(response)
| Parameters | |
|---|---|
| Name | Description |
request |
Union[google.cloud.dataproc_v1.types.ListWorkflowTemplatesRequest, dict]
The request object. A request to list workflow templates in a project. |
parent |
str
Required. The resource name of the region or location, as described in https://cloud.google.com/apis/design/resource_names. - For |
retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
timeout |
float
The timeout for this request. |
metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
| Returns | |
|---|---|
| Type | Description |
google.cloud.dataproc_v1.services.workflow_template_service.pagers.ListWorkflowTemplatesPager | A response to a request to list workflow templates in a project. Iterating over this object will yield results and resolve additional pages automatically. |
parse_cluster_path
parse_cluster_path(path: str)Parses a cluster path into its component segments.
parse_common_billing_account_path
parse_common_billing_account_path(path: str)Parse a billing_account path into its component segments.
parse_common_folder_path
parse_common_folder_path(path: str)Parse a folder path into its component segments.
parse_common_location_path
parse_common_location_path(path: str)Parse a location path into its component segments.
parse_common_organization_path
parse_common_organization_path(path: str)Parse a organization path into its component segments.
parse_common_project_path
parse_common_project_path(path: str)Parse a project path into its component segments.
parse_service_path
parse_service_path(path: str)Parses a service path into its component segments.
parse_workflow_template_path
parse_workflow_template_path(path: str)Parses a workflow_template path into its component segments.
service_path
service_path(project: str, location: str, service: str)Returns a fully-qualified service string.
update_workflow_template
update_workflow_template(request: Optional[Union[google.cloud.dataproc_v1.types.workflow_templates.UpdateWorkflowTemplateRequest, dict]] = None, *, template: Optional[google.cloud.dataproc_v1.types.workflow_templates.WorkflowTemplate] = None, retry: Union[google.api_core.retry.Retry, google.api_core.gapic_v1.method._MethodDefault] = <_MethodDefault._DEFAULT_VALUE: <object object>>, timeout: Optional[float] = None, metadata: Sequence[Tuple[str, str]] = ())Updates (replaces) workflow template. The updated template must contain version that matches the current server version.
from google.cloud import dataproc_v1
def sample_update_workflow_template():
# Create a client
client = dataproc_v1.WorkflowTemplateServiceClient()
# Initialize request argument(s)
template = dataproc_v1.WorkflowTemplate()
template.id = "id_value"
template.placement.managed_cluster.cluster_name = "cluster_name_value"
template.jobs.hadoop_job.main_jar_file_uri = "main_jar_file_uri_value"
template.jobs.step_id = "step_id_value"
request = dataproc_v1.UpdateWorkflowTemplateRequest(
template=template,
)
# Make the request
response = client.update_workflow_template(request=request)
# Handle the response
print(response)
| Parameters | |
|---|---|
| Name | Description |
request |
Union[google.cloud.dataproc_v1.types.UpdateWorkflowTemplateRequest, dict]
The request object. A request to update a workflow template. |
template |
google.cloud.dataproc_v1.types.WorkflowTemplate
Required. The updated workflow template. The |
retry |
google.api_core.retry.Retry
Designation of what errors, if any, should be retried. |
timeout |
float
The timeout for this request. |
metadata |
Sequence[Tuple[str, str]]
Strings which should be sent along with the request as metadata. |
| Returns | |
|---|---|
| Type | Description |
google.cloud.dataproc_v1.types.WorkflowTemplate | A Dataproc workflow template resource. |
workflow_template_path
workflow_template_path(project: str, region: str, workflow_template: str)Returns a fully-qualified workflow_template string.