Cloud Dataplex v1 API - Class Task.Types.SparkTaskConfig (2.11.0)

public sealed class Task.Types.SparkTaskConfig : IMessage<Task.Types.SparkTaskConfig>, IEquatable<Task.Types.SparkTaskConfig>, IDeepCloneable<Task.Types.SparkTaskConfig>, IBufferMessage, IMessage

Reference documentation and code samples for the Cloud Dataplex v1 API class Task.Types.SparkTaskConfig.

User-specified config for running a Spark task.

Inheritance

object > Task.Types.SparkTaskConfig

Implements

IMessage Task Types SparkTaskConfig, IEquatable Task Types SparkTaskConfig, IDeepCloneable Task Types SparkTaskConfig, IBufferMessage, IMessage

Inherited Members

object.GetHashCode()

object.GetType()

object.ToString()

Namespace

Google.Cloud.Dataplex.V1

Assembly

Google.Cloud.Dataplex.V1.dll

Constructors

SparkTaskConfig()

public SparkTaskConfig()

SparkTaskConfig(SparkTaskConfig)

public SparkTaskConfig(Task.Types.SparkTaskConfig other)

Parameter
Name	Description
`other`	`TaskTypesSparkTaskConfig`

Properties

ArchiveUris

public RepeatedField<string> ArchiveUris { get; }

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Property Value
Type	Description
`RepeatedFieldstring`

DriverCase

public Task.Types.SparkTaskConfig.DriverOneofCase DriverCase { get; }

Property Value
Type	Description
`TaskTypesSparkTaskConfigDriverOneofCase`

FileUris

public RepeatedField<string> FileUris { get; }

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Property Value
Type	Description
`RepeatedFieldstring`

HasMainClass

public bool HasMainClass { get; }

Gets whether the "main_class" field is set

Property Value
Type	Description
`bool`

HasMainJarFileUri

public bool HasMainJarFileUri { get; }

Gets whether the "main_jar_file_uri" field is set

Property Value
Type	Description
`bool`

HasPythonScriptFile

public bool HasPythonScriptFile { get; }

Gets whether the "python_script_file" field is set

Property Value
Type	Description
`bool`

HasSqlScript

public bool HasSqlScript { get; }

Gets whether the "sql_script" field is set

Property Value
Type	Description
`bool`

HasSqlScriptFile

public bool HasSqlScriptFile { get; }

Gets whether the "sql_script_file" field is set

Property Value
Type	Description
`bool`

InfrastructureSpec

public Task.Types.InfrastructureSpec InfrastructureSpec { get; set; }

Optional. Infrastructure specification for the execution.

Property Value
Type	Description
`TaskTypesInfrastructureSpec`

MainClass

public string MainClass { get; set; }

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris. The execution args are passed in as a sequence of named process arguments (--key=value).

Property Value
Type	Description
`string`

MainJarFileUri

public string MainJarFileUri { get; set; }

The Cloud Storage URI of the jar file that contains the main class. The execution args are passed in as a sequence of named process arguments (--key=value).

Property Value
Type	Description
`string`

PythonScriptFile

public string PythonScriptFile { get; set; }

The Gcloud Storage URI of the main Python file to use as the driver. Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Property Value
Type	Description
`string`

SqlScript

public string SqlScript { get; set; }

The query text. The execution args are used to declare a set of script variables (set key="value";).

Property Value
Type	Description
`string`

SqlScriptFile

public string SqlScriptFile { get; set; }

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Property Value
Type	Description
`string`