Class Task.Types.SparkTaskConfig (2.1.0)

public sealed class SparkTaskConfig : IMessage<Task.Types.SparkTaskConfig>, IEquatable<Task.Types.SparkTaskConfig>, IDeepCloneable<Task.Types.SparkTaskConfig>, IBufferMessage, IMessage

User-specified config for running a Spark task.

Inheritance

Object > Task.Types.SparkTaskConfig

Implements

IMessage<Task.Types.SparkTaskConfig>, IEquatable<Task.Types.SparkTaskConfig>, IDeepCloneable<Task.Types.SparkTaskConfig>, IBufferMessage, IMessage

Inherited Members

Object.GetHashCode()

Object.GetType()

Object.MemberwiseClone()

Object.ToString()

Namespace

Google.Cloud.Dataplex.V1

Assembly

Google.Cloud.Dataplex.V1.dll

Constructors

SparkTaskConfig()

public SparkTaskConfig()

SparkTaskConfig(Task.Types.SparkTaskConfig)

public SparkTaskConfig(Task.Types.SparkTaskConfig other)

Parameter
Name	Description
`other`	`Task.Types.SparkTaskConfig`

Properties

ArchiveUris

public RepeatedField<string> ArchiveUris { get; }

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Property Value
Type	Description
`RepeatedField<String>`

DriverCase

public Task.Types.SparkTaskConfig.DriverOneofCase DriverCase { get; }

Property Value
Type	Description
`Task.Types.SparkTaskConfig.DriverOneofCase`

FileUris

public RepeatedField<string> FileUris { get; }

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Property Value
Type	Description
`RepeatedField<String>`

InfrastructureSpec

public Task.Types.InfrastructureSpec InfrastructureSpec { get; set; }

Optional. Infrastructure specification for the execution.

Property Value
Type	Description
`Task.Types.InfrastructureSpec`

MainClass

public string MainClass { get; set; }

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris. The execution args are passed in as a sequence of named process arguments (--key=value).

Property Value
Type	Description
`String`

MainJarFileUri

public string MainJarFileUri { get; set; }

The Cloud Storage URI of the jar file that contains the main class. The execution args are passed in as a sequence of named process arguments (--key=value).

Property Value
Type	Description
`String`

PythonScriptFile

public string PythonScriptFile { get; set; }

The Gcloud Storage URI of the main Python file to use as the driver. Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Property Value
Type	Description
`String`

SqlScript

public string SqlScript { get; set; }

The query text. The execution args are used to declare a set of script variables (set key="value";).

Property Value
Type	Description
`String`

SqlScriptFile

public string SqlScriptFile { get; set; }

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Property Value
Type	Description
`String`