Package Classes (5.13.0)

Summary of entries of Classes for dataproc.

Classes

AutoscalingPolicyServiceAsyncClient

The API interface for managing autoscaling policies in the Dataproc API.

AutoscalingPolicyServiceClient

The API interface for managing autoscaling policies in the Dataproc API.

ListAutoscalingPoliciesAsyncPager

A pager for iterating through list_autoscaling_policies requests.

This class thinly wraps an initial ListAutoscalingPoliciesResponse object, and provides an __aiter__ method to iterate through its policies field.

If there are more pages, the __aiter__ method will make additional ListAutoscalingPolicies requests and continue to iterate through the policies field on the corresponding responses.

All the usual ListAutoscalingPoliciesResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

ListAutoscalingPoliciesPager

A pager for iterating through list_autoscaling_policies requests.

This class thinly wraps an initial ListAutoscalingPoliciesResponse object, and provides an __iter__ method to iterate through its policies field.

If there are more pages, the __iter__ method will make additional ListAutoscalingPolicies requests and continue to iterate through the policies field on the corresponding responses.

All the usual ListAutoscalingPoliciesResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

BatchControllerAsyncClient

The BatchController provides methods to manage batch workloads.

BatchControllerClient

The BatchController provides methods to manage batch workloads.

ListBatchesAsyncPager

A pager for iterating through list_batches requests.

This class thinly wraps an initial ListBatchesResponse object, and provides an __aiter__ method to iterate through its batches field.

If there are more pages, the __aiter__ method will make additional ListBatches requests and continue to iterate through the batches field on the corresponding responses.

All the usual ListBatchesResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

ListBatchesPager

A pager for iterating through list_batches requests.

This class thinly wraps an initial ListBatchesResponse object, and provides an __iter__ method to iterate through its batches field.

If there are more pages, the __iter__ method will make additional ListBatches requests and continue to iterate through the batches field on the corresponding responses.

All the usual ListBatchesResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

ClusterControllerAsyncClient

The ClusterControllerService provides methods to manage clusters of Compute Engine instances.

ClusterControllerClient

The ClusterControllerService provides methods to manage clusters of Compute Engine instances.

ListClustersAsyncPager

A pager for iterating through list_clusters requests.

This class thinly wraps an initial ListClustersResponse object, and provides an __aiter__ method to iterate through its clusters field.

If there are more pages, the __aiter__ method will make additional ListClusters requests and continue to iterate through the clusters field on the corresponding responses.

All the usual ListClustersResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

ListClustersPager

A pager for iterating through list_clusters requests.

This class thinly wraps an initial ListClustersResponse object, and provides an __iter__ method to iterate through its clusters field.

If there are more pages, the __iter__ method will make additional ListClusters requests and continue to iterate through the clusters field on the corresponding responses.

All the usual ListClustersResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

JobControllerAsyncClient

The JobController provides methods to manage jobs.

JobControllerClient

The JobController provides methods to manage jobs.

ListJobsAsyncPager

A pager for iterating through list_jobs requests.

This class thinly wraps an initial ListJobsResponse object, and provides an __aiter__ method to iterate through its jobs field.

If there are more pages, the __aiter__ method will make additional ListJobs requests and continue to iterate through the jobs field on the corresponding responses.

All the usual ListJobsResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

ListJobsPager

A pager for iterating through list_jobs requests.

This class thinly wraps an initial ListJobsResponse object, and provides an __iter__ method to iterate through its jobs field.

If there are more pages, the __iter__ method will make additional ListJobs requests and continue to iterate through the jobs field on the corresponding responses.

All the usual ListJobsResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

NodeGroupControllerAsyncClient

The NodeGroupControllerService provides methods to manage node groups of Compute Engine managed instances.

NodeGroupControllerClient

The NodeGroupControllerService provides methods to manage node groups of Compute Engine managed instances.

SessionControllerAsyncClient

The SessionController provides methods to manage interactive sessions.

SessionControllerClient

The SessionController provides methods to manage interactive sessions.

ListSessionsAsyncPager

A pager for iterating through list_sessions requests.

This class thinly wraps an initial ListSessionsResponse object, and provides an __aiter__ method to iterate through its sessions field.

If there are more pages, the __aiter__ method will make additional ListSessions requests and continue to iterate through the sessions field on the corresponding responses.

All the usual ListSessionsResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

ListSessionsPager

A pager for iterating through list_sessions requests.

This class thinly wraps an initial ListSessionsResponse object, and provides an __iter__ method to iterate through its sessions field.

If there are more pages, the __iter__ method will make additional ListSessions requests and continue to iterate through the sessions field on the corresponding responses.

All the usual ListSessionsResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

SessionTemplateControllerAsyncClient

The SessionTemplateController provides methods to manage session templates.

SessionTemplateControllerClient

The SessionTemplateController provides methods to manage session templates.

ListSessionTemplatesAsyncPager

A pager for iterating through list_session_templates requests.

This class thinly wraps an initial ListSessionTemplatesResponse object, and provides an __aiter__ method to iterate through its session_templates field.

If there are more pages, the __aiter__ method will make additional ListSessionTemplates requests and continue to iterate through the session_templates field on the corresponding responses.

All the usual ListSessionTemplatesResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

ListSessionTemplatesPager

A pager for iterating through list_session_templates requests.

This class thinly wraps an initial ListSessionTemplatesResponse object, and provides an __iter__ method to iterate through its session_templates field.

If there are more pages, the __iter__ method will make additional ListSessionTemplates requests and continue to iterate through the session_templates field on the corresponding responses.

All the usual ListSessionTemplatesResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

WorkflowTemplateServiceAsyncClient

The API interface for managing Workflow Templates in the Dataproc API.

WorkflowTemplateServiceClient

The API interface for managing Workflow Templates in the Dataproc API.

ListWorkflowTemplatesAsyncPager

A pager for iterating through list_workflow_templates requests.

This class thinly wraps an initial ListWorkflowTemplatesResponse object, and provides an __aiter__ method to iterate through its templates field.

If there are more pages, the __aiter__ method will make additional ListWorkflowTemplates requests and continue to iterate through the templates field on the corresponding responses.

All the usual ListWorkflowTemplatesResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

ListWorkflowTemplatesPager

A pager for iterating through list_workflow_templates requests.

This class thinly wraps an initial ListWorkflowTemplatesResponse object, and provides an __iter__ method to iterate through its templates field.

If there are more pages, the __iter__ method will make additional ListWorkflowTemplates requests and continue to iterate through the templates field on the corresponding responses.

All the usual ListWorkflowTemplatesResponse attributes are available on the pager. If multiple requests are made, only the most recent response is retained, and thus used for attribute lookup.

AcceleratorConfig

Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine <https://cloud.google.com/compute/docs/gpus/>__.

AutoscalingConfig

Autoscaling Policy config associated with the cluster.

AutoscalingPolicy

Describes an autoscaling policy for Dataproc cluster autoscaler.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

AutotuningConfig

Autotuning configuration of the workload.

Scenario

Scenario represents a specific goal that autotuning will attempt to achieve by modifying workloads.

AuxiliaryNodeGroup

Node group identification and configuration information.

AuxiliaryServicesConfig

Auxiliary services configuration for a Cluster.

BasicAutoscalingAlgorithm

Basic algorithm for autoscaling.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

BasicYarnAutoscalingConfig

Basic autoscaling configurations for YARN.

Batch

A representation of a batch workload in the service.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

LabelsEntry

The abstract base class for a message.

State

The batch state.

StateHistory

Historical state information.

BatchOperationMetadata

Metadata describing the Batch operation.

BatchOperationType

Operation type for Batch resources

LabelsEntry

The abstract base class for a message.

CancelJobRequest

A request to cancel a job.

Cluster

Describes the identifying information, config, and status of a Dataproc cluster

LabelsEntry

The abstract base class for a message.

ClusterConfig

The cluster config.

ClusterMetrics

Contains cluster daemon metrics, such as HDFS and YARN stats.

Beta Feature: This report is available for testing purposes only. It may be changed before final release.

HdfsMetricsEntry

The abstract base class for a message.

YarnMetricsEntry

The abstract base class for a message.

ClusterOperation

The cluster operation triggered by a workflow.

ClusterOperationMetadata

Metadata describing the operation.

LabelsEntry

The abstract base class for a message.

ClusterOperationStatus

The status of the operation.

State

The operation state.

ClusterSelector

A selector that chooses target cluster for jobs based on metadata.

ClusterLabelsEntry

The abstract base class for a message.

ClusterStatus

The status of a cluster and its instances.

State

The cluster state.

    **Note:** The cluster state changes from "creating" to
    "running" status after the master node(s), first two primary
    worker nodes (and the last primary worker node if primary
    workers > 2) are running.
ERROR (3):
    The cluster encountered an error. It is not
    ready for use.
ERROR_DUE_TO_UPDATE (9):
    The cluster has encountered an error while
    being updated. Jobs can be submitted to the
    cluster, but the cluster cannot be updated.
DELETING (4):
    The cluster is being deleted. It cannot be
    used.
UPDATING (5):
    The cluster is being updated. It continues to
    accept and process jobs.
STOPPING (6):
    The cluster is being stopped. It cannot be
    used.
STOPPED (7):
    The cluster is currently stopped. It is not
    ready for use.
STARTING (8):
    The cluster is being started. It is not ready
    for use.
REPAIRING (10):
    The cluster is being repaired. It is not
    ready for use.

Substate

The cluster substate.

    Applies to RUNNING state.
STALE_STATUS (2):
    The agent-reported status is out of date (may
    occur if Dataproc loses communication with
    Agent).

    Applies to RUNNING state.

Component

Cluster components that can be activated.

ConfidentialInstanceConfig

Confidential Instance Config for clusters using Confidential VMs <https://cloud.google.com/compute/confidential-vm/docs>__

CreateAutoscalingPolicyRequest

A request to create an autoscaling policy.

CreateBatchRequest

A request to create a batch workload.

CreateClusterRequest

A request to create a cluster.

CreateNodeGroupRequest

A request to create a node group.

CreateSessionRequest

A request to create a session.

CreateSessionTemplateRequest

A request to create a session template.

CreateWorkflowTemplateRequest

A request to create a workflow template.

DataprocMetricConfig

Dataproc metric config.

Metric

A Dataproc custom metric.

MetricSource

A source for the collection of Dataproc custom metrics (see Custom metrics).

DeleteAutoscalingPolicyRequest

A request to delete an autoscaling policy.

Autoscaling policies in use by one or more clusters will not be deleted.

DeleteBatchRequest

A request to delete a batch workload.

DeleteClusterRequest

A request to delete a cluster.

DeleteJobRequest

A request to delete a job.

DeleteSessionRequest

A request to delete a session.

DeleteSessionTemplateRequest

A request to delete a session template.

DeleteWorkflowTemplateRequest

A request to delete a workflow template.

Currently started workflows will remain running.

DiagnoseClusterRequest

A request to collect cluster diagnostic information.

TarballAccess

Defines who has access to the diagnostic tarball

DiagnoseClusterResults

The location of diagnostic output.

DiskConfig

Specifies the config of disk options for a group of VM instances.

DriverSchedulingConfig

Driver scheduling configuration.

EncryptionConfig

Encryption settings for the cluster.

EndpointConfig

Endpoint config for this cluster

HttpPortsEntry

The abstract base class for a message.

EnvironmentConfig

Environment configuration for a workload.

ExecutionConfig

Execution configuration for a workload.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

FailureAction

Actions in response to failure of a resource associated with a cluster.

FlinkJob

A Dataproc job for running Apache Flink applications on YARN.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

PropertiesEntry

The abstract base class for a message.

GceClusterConfig

Common config settings for resources of Compute Engine cluster instances, applicable to all instances in the cluster.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

MetadataEntry

The abstract base class for a message.

PrivateIpv6GoogleAccess

PrivateIpv6GoogleAccess controls whether and how Dataproc cluster nodes can communicate with Google Services through gRPC over IPv6. These values are directly mapped to corresponding values in the Compute Engine Instance fields <https://cloud.google.com/compute/docs/reference/rest/v1/instances>__.

GetAutoscalingPolicyRequest

A request to fetch an autoscaling policy.

GetBatchRequest

A request to get the resource representation for a batch workload.

GetClusterRequest

Request to get the resource representation for a cluster in a project.

GetJobRequest

A request to get the resource representation for a job in a project.

GetNodeGroupRequest

A request to get a node group .

GetSessionRequest

A request to get the resource representation for a session.

GetSessionTemplateRequest

A request to get the resource representation for a session template.

GetWorkflowTemplateRequest

A request to fetch a workflow template.

GkeClusterConfig

The cluster's GKE config.

GkeNodePoolConfig

The configuration of a GKE node pool used by a Dataproc-on-GKE cluster <https://cloud.google.com/dataproc/docs/concepts/jobs/dataproc-gke#create-a-dataproc-on-gke-cluster>__.

GkeNodeConfig

Parameters that describe cluster nodes.

GkeNodePoolAcceleratorConfig

A GkeNodeConfigAcceleratorConfig represents a Hardware Accelerator request for a node pool.

GkeNodePoolAutoscalingConfig

GkeNodePoolAutoscaling contains information the cluster autoscaler needs to adjust the size of the node pool to the current cluster usage.

GkeNodePoolTarget

GKE node pools that Dataproc workloads run on.

Role

Role specifies the tasks that will run on the node pool. Roles can be specific to workloads. Exactly one GkeNodePoolTarget within the virtual cluster must have the DEFAULT role, which is used to run all workloads that are not associated with a node pool.

HadoopJob

A Dataproc job for running Apache Hadoop MapReduce <https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html> jobs on Apache Hadoop YARN <https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html>.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

PropertiesEntry

The abstract base class for a message.

HiveJob

A Dataproc job for running Apache Hive <https://hive.apache.org/>__ queries on YARN.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

PropertiesEntry

The abstract base class for a message.

ScriptVariablesEntry

The abstract base class for a message.

IdentityConfig

Identity related configuration, including service account based secure multi-tenancy user mappings.

UserServiceAccountMappingEntry

The abstract base class for a message.

InstanceFlexibilityPolicy

Instance flexibility Policy allowing a mixture of VM shapes and provisioning models.

InstanceSelection

Defines machines types and a rank to which the machines types belong.

InstanceSelectionResult

Defines a mapping from machine types to the number of VMs that are created with each machine type.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

InstanceGroupAutoscalingPolicyConfig

Configuration for the size bounds of an instance group, including its proportional size to other groups.

InstanceGroupConfig

The config settings for Compute Engine resources in an instance group, such as a master or worker group.

Preemptibility

Controls the use of preemptible instances within the group.

    This option is allowed for all instance groups
    and is the only valid value for Master and
    Worker instance groups.
PREEMPTIBLE (2):
    Instances are [preemptible]
    (https://cloud.google.com/compute/docs/instances/preemptible).

    This option is allowed only for [secondary worker]
    (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms)
    groups.
SPOT (3):
    Instances are [Spot VMs]
    (https://cloud.google.com/compute/docs/instances/spot).

    This option is allowed only for [secondary worker]
    (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms)
    groups. Spot VMs are the latest version of [preemptible VMs]
    (https://cloud.google.com/compute/docs/instances/preemptible),
    and provide additional features.

InstanceReference

A reference to a Compute Engine instance.

InstantiateInlineWorkflowTemplateRequest

A request to instantiate an inline workflow template.

InstantiateWorkflowTemplateRequest

A request to instantiate a workflow template.

ParametersEntry

The abstract base class for a message.

Job

A Dataproc job resource.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

LabelsEntry

The abstract base class for a message.

JobMetadata

Job Operation metadata.

JobPlacement

Dataproc job config.

ClusterLabelsEntry

The abstract base class for a message.

JobReference

Encapsulates the full scoping used to reference a job.

JobScheduling

Job scheduling options.

JobStatus

Dataproc job status.

State

The job state.

    Applies to restartable jobs only.

Substate

The job substate.

    Applies to RUNNING state.
QUEUED (2):
    The Job has been received and is awaiting
    execution (it might be waiting for a condition
    to be met). See the "details" field for the
    reason for the delay.

    Applies to RUNNING state.
STALE_STATUS (3):
    The agent-reported status is out of date,
    which can be caused by a loss of communication
    between the agent and Dataproc. If the agent
    does not send a timely update, the job will
    fail.

    Applies to RUNNING state.

JupyterConfig

Jupyter configuration for an interactive session.

Kernel

Jupyter kernel types.

KerberosConfig

Specifies Kerberos related configuration.

KubernetesClusterConfig

The configuration for running the Dataproc cluster on Kubernetes.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

KubernetesSoftwareConfig

The software configuration for this Dataproc cluster running on Kubernetes.

ComponentVersionEntry

The abstract base class for a message.

PropertiesEntry

The abstract base class for a message.

LifecycleConfig

Specifies the cluster auto-delete schedule configuration.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

ListAutoscalingPoliciesRequest

A request to list autoscaling policies in a project.

ListAutoscalingPoliciesResponse

A response to a request to list autoscaling policies in a project.

ListBatchesRequest

A request to list batch workloads in a project.

ListBatchesResponse

A list of batch workloads.

ListClustersRequest

A request to list the clusters in a project.

ListClustersResponse

The list of all clusters in a project.

ListJobsRequest

A request to list jobs in a project.

JobStateMatcher

A matcher that specifies categories of job states.

ListJobsResponse

A list of jobs in a project.

ListSessionTemplatesRequest

A request to list session templates in a project.

ListSessionTemplatesResponse

A list of session templates.

ListSessionsRequest

A request to list sessions in a project.

ListSessionsResponse

A list of interactive sessions.

ListWorkflowTemplatesRequest

A request to list workflow templates in a project.

ListWorkflowTemplatesResponse

A response to a request to list workflow templates in a project.

LoggingConfig

The runtime logging config of the job.

DriverLogLevelsEntry

The abstract base class for a message.

Level

The Log4j level for job execution. When running an Apache Hive <https://hive.apache.org/>__ job, Cloud Dataproc configures the Hive client to an equivalent verbosity level.

ManagedCluster

Cluster that is managed by the workflow.

LabelsEntry

The abstract base class for a message.

ManagedGroupConfig

Specifies the resources used to actively manage an instance group.

MetastoreConfig

Specifies a Metastore configuration.

NodeGroup

Dataproc Node Group. The Dataproc NodeGroup resource is not related to the Dataproc NodeGroupAffinity resource.

LabelsEntry

The abstract base class for a message.

Role

Node pool roles.

NodeGroupAffinity

Node Group Affinity for clusters using sole-tenant node groups. The Dataproc NodeGroupAffinity resource is not related to the Dataproc NodeGroup resource.

NodeGroupOperationMetadata

Metadata describing the node group operation.

LabelsEntry

The abstract base class for a message.

NodeGroupOperationType

Operation type for node group resources.

NodeInitializationAction

Specifies an executable to run on a fully configured node and a timeout period for executable completion.

OrderedJob

A job executed by the workflow.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

LabelsEntry

The abstract base class for a message.

ParameterValidation

Configuration for parameter validation.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

PeripheralsConfig

Auxiliary services configuration for a workload.

PigJob

A Dataproc job for running Apache Pig <https://pig.apache.org/>__ queries on YARN.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

PropertiesEntry

The abstract base class for a message.

ScriptVariablesEntry

The abstract base class for a message.

PrestoJob

A Dataproc job for running Presto <https://prestosql.io/> queries. IMPORTANT: The Dataproc Presto Optional Component <https://cloud.google.com/dataproc/docs/concepts/components/presto> must be enabled when the cluster is created to submit a Presto job to the cluster.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

PropertiesEntry

The abstract base class for a message.

PyPiRepositoryConfig

Configuration for PyPi repository

PySparkBatch

A configuration for running an Apache PySpark <https://spark.apache.org/docs/latest/api/python/getting_started/quickstart.html>__ batch workload.

PySparkJob

A Dataproc job for running Apache PySpark <https://spark.apache.org/docs/0.9.0/python-programming-guide.html>__ applications on YARN.

PropertiesEntry

The abstract base class for a message.

QueryList

A list of queries to run on a cluster.

RegexValidation

Validation based on regular expressions.

RepositoryConfig

Configuration for dependency repositories

ReservationAffinity

Reservation Affinity for consuming Zonal reservation.

Type

Indicates whether to consume capacity from an reservation or not.

ResizeNodeGroupRequest

A request to resize a node group.

RuntimeConfig

Runtime configuration for a workload.

PropertiesEntry

The abstract base class for a message.

RuntimeInfo

Runtime information about workload execution.

EndpointsEntry

The abstract base class for a message.

SecurityConfig

Security related configuration, including encryption, Kerberos, etc.

Session

A representation of a session.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

LabelsEntry

The abstract base class for a message.

SessionStateHistory

Historical state information.

State

The session state.

SessionOperationMetadata

Metadata describing the Session operation.

LabelsEntry

The abstract base class for a message.

SessionOperationType

Operation type for Session resources

SessionTemplate

A representation of a session template.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

LabelsEntry

The abstract base class for a message.

ShieldedInstanceConfig

Shielded Instance Config for clusters using Compute Engine Shielded VMs <https://cloud.google.com/security/shielded-cloud/shielded-vm>__.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

SoftwareConfig

Specifies the selection and config of software inside the cluster.

PropertiesEntry

The abstract base class for a message.

SparkBatch

A configuration for running an Apache Spark <https://spark.apache.org/>__ batch workload.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

SparkConnectConfig

Spark Connect configuration for an interactive session.

SparkHistoryServerConfig

Spark History Server configuration for the workload.

SparkJob

A Dataproc job for running Apache Spark <https://spark.apache.org/>__ applications on YARN.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

PropertiesEntry

The abstract base class for a message.

SparkRBatch

A configuration for running an Apache SparkR <https://spark.apache.org/docs/latest/sparkr.html>__ batch workload.

SparkRJob

A Dataproc job for running Apache SparkR <https://spark.apache.org/docs/latest/sparkr.html>__ applications on YARN.

PropertiesEntry

The abstract base class for a message.

SparkSqlBatch

A configuration for running Apache Spark SQL <https://spark.apache.org/sql/>__ queries as a batch workload.

QueryVariablesEntry

The abstract base class for a message.

SparkSqlJob

A Dataproc job for running Apache Spark SQL <https://spark.apache.org/sql/>__ queries.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

PropertiesEntry

The abstract base class for a message.

ScriptVariablesEntry

The abstract base class for a message.

StartClusterRequest

A request to start a cluster.

StartupConfig

Configuration to handle the startup of instances during cluster create and update process.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

StopClusterRequest

A request to stop a cluster.

SubmitJobRequest

A request to submit a job.

TemplateParameter

A configurable parameter that replaces one or more fields in the template. Parameterizable fields:

Labels
File uris
Job properties
Job arguments
Script variables
Main class (in HadoopJob and SparkJob)
Zone (in ClusterSelector)

TerminateSessionRequest

A request to terminate an interactive session.

TrinoJob

A Dataproc job for running Trino <https://trino.io/> queries. IMPORTANT: The Dataproc Trino Optional Component <https://cloud.google.com/dataproc/docs/concepts/components/trino> must be enabled when the cluster is created to submit a Trino job to the cluster.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

PropertiesEntry

The abstract base class for a message.

UpdateAutoscalingPolicyRequest

A request to update an autoscaling policy.

UpdateClusterRequest

A request to update a cluster.

UpdateJobRequest

A request to update a job.

UpdateSessionTemplateRequest

A request to update a session template.

UpdateWorkflowTemplateRequest

A request to update a workflow template.

UsageMetrics

Usage metrics represent approximate total resources consumed by a workload.

UsageSnapshot

The usage snapshot represents the resources consumed by a workload at a specified time.

ValueValidation

Validation based on a list of allowed values.

VirtualClusterConfig

The Dataproc cluster config for a cluster that does not directly control the underlying compute resources, such as a Dataproc-on-GKE cluster <https://cloud.google.com/dataproc/docs/guides/dpgke/dataproc-gke-overview>__.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields