Cloud Dataplex v1 API - Class Task.Types.SparkTaskConfig (2.3.0)

Version 2.3.0

• 3.4.0 (latest)
• 3.3.0
• 3.2.0
• 3.1.0
• 3.0.0
• 2.15.0
• 2.14.0
• 2.13.0
• 2.12.0
• 2.11.0
• 2.10.0
• 2.9.0
• 2.8.0
• 2.7.0
• 2.6.0
• 2.5.0
• 2.4.0
• 2.3.0
• 2.2.0
• 2.1.0
• 2.0.0
• 1.0.0

public sealed class Task.Types.SparkTaskConfig : IMessage<Task.Types.SparkTaskConfig>, IEquatable<Task.Types.SparkTaskConfig>, IDeepCloneable<Task.Types.SparkTaskConfig>, IBufferMessage, IMessage

Reference documentation and code samples for the Cloud Dataplex v1 API class Task.Types.SparkTaskConfig.

User-specified config for running a Spark task.

Inheritance

object > Task.Types.SparkTaskConfig

Implements

IMessageTaskTypesSparkTaskConfig, IEquatableTaskTypesSparkTaskConfig, IDeepCloneableTaskTypesSparkTaskConfig, IBufferMessage, IMessage

Inherited Members

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ToString()

Namespace

GoogleCloudGoogle.Cloud.DataplexV1

Assembly

Google.Cloud.Dataplex.V1.dll

Constructors

SparkTaskConfig()

public SparkTaskConfig()

SparkTaskConfig(SparkTaskConfig)

public SparkTaskConfig(Task.Types.SparkTaskConfig other)

Parameter
Name	Description
other	TaskTypesSparkTaskConfig

Properties

ArchiveUris

public RepeatedField<string> ArchiveUris { get; }

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Property Value
Type	Description
RepeatedFieldstring

DriverCase

public Task.Types.SparkTaskConfig.DriverOneofCase DriverCase { get; }

Property Value
Type	Description
TaskTypesSparkTaskConfigDriverOneofCase

FileUris

public RepeatedField<string> FileUris { get; }

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Property Value
Type	Description
RepeatedFieldstring

InfrastructureSpec

public Task.Types.InfrastructureSpec InfrastructureSpec { get; set; }

Optional. Infrastructure specification for the execution.

Property Value
Type	Description
TaskTypesInfrastructureSpec

MainClass

public string MainClass { get; set; }

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris. The execution args are passed in as a sequence of named process arguments (--key=value).

Property Value
Type	Description
string

MainJarFileUri

public string MainJarFileUri { get; set; }

The Cloud Storage URI of the jar file that contains the main class. The execution args are passed in as a sequence of named process arguments (--key=value).

Property Value
Type	Description
string

PythonScriptFile

public string PythonScriptFile { get; set; }

The Gcloud Storage URI of the main Python file to use as the driver. Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Property Value
Type	Description
string

SqlScript

public string SqlScript { get; set; }

The query text. The execution args are used to declare a set of script variables (set key="value";).

Property Value
Type	Description
string

SqlScriptFile

public string SqlScriptFile { get; set; }

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Property Value
Type	Description
string