Google Cloud Dataproc v1 API - Class PySparkBatch (5.5.0)

public sealed class PySparkBatch : IMessage<PySparkBatch>, IEquatable<PySparkBatch>, IDeepCloneable<PySparkBatch>, IBufferMessage, IMessage

Reference documentation and code samples for the Google Cloud Dataproc v1 API class PySparkBatch.

A configuration for running an Apache PySpark batch workload.

Inheritance

object > PySparkBatch

Implements

IMessage PySparkBatch, IEquatable PySparkBatch, IDeepCloneable PySparkBatch, IBufferMessage, IMessage

Inherited Members

object.GetHashCode()

object.GetType()

object.ToString()

Namespace

Google CloudGoogle.Cloud.DataprocV1

Assembly

Google.Cloud.Dataproc.V1.dll

Constructors

PySparkBatch()

public PySparkBatch()

PySparkBatch(PySparkBatch)

public PySparkBatch(PySparkBatch other)

Parameter
Name	Description
`other`	`PySparkBatch`

Properties

ArchiveUris

public RepeatedField<string> ArchiveUris { get; }

Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Property Value
Type	Description
`RepeatedFieldstring`

Args

public RepeatedField<string> Args { get; }

Optional. The arguments to pass to the driver. Do not include arguments that can be set as batch properties, such as --conf, since a collision can occur that causes an incorrect batch submission.

Property Value
Type	Description
`RepeatedFieldstring`

FileUris

public RepeatedField<string> FileUris { get; }

Optional. HCFS URIs of files to be placed in the working directory of each executor.

Property Value
Type	Description
`RepeatedFieldstring`

JarFileUris

public RepeatedField<string> JarFileUris { get; }

Optional. HCFS URIs of jar files to add to the classpath of the Spark driver and tasks.

Property Value
Type	Description
`RepeatedFieldstring`

MainPythonFileUri

public string MainPythonFileUri { get; set; }

Required. The HCFS URI of the main Python file to use as the Spark driver. Must be a .py file.

Property Value
Type	Description
`string`

PythonFileUris

public RepeatedField<string> PythonFileUris { get; }

Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.

Property Value
Type	Description
`RepeatedFieldstring`