Google Cloud Dataplex V1 Client - Class SparkTaskConfig (0.5.1)

Reference documentation and code samples for the Google Cloud Dataplex V1 Client class SparkTaskConfig.

User-specified config for running a Spark task.

Generated from protobuf message google.cloud.dataplex.v1.Task.SparkTaskConfig

Namespace

Google \ Cloud \ Dataplex \ V1 \ Task

Methods

__construct

Constructor.

Parameters
NameDescription
data array

Optional. Data for populating the Message object.

↳ main_jar_file_uri string

The Cloud Storage URI of the jar file that contains the main class. The execution args are passed in as a sequence of named process arguments (--key=value).

↳ main_class string

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris. The execution args are passed in as a sequence of named process arguments (--key=value).

↳ python_script_file string

The Gcloud Storage URI of the main Python file to use as the driver. Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

↳ sql_script_file string

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

↳ sql_script string

The query text. The execution args are used to declare a set of script variables (set key="value";).

↳ file_uris array

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

↳ archive_uris array

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

↳ infrastructure_spec Google\Cloud\Dataplex\V1\Task\InfrastructureSpec

Optional. Infrastructure specification for the execution.

getMainJarFileUri

The Cloud Storage URI of the jar file that contains the main class.

The execution args are passed in as a sequence of named process arguments (--key=value).

Returns
TypeDescription
string

hasMainJarFileUri

setMainJarFileUri

The Cloud Storage URI of the jar file that contains the main class.

The execution args are passed in as a sequence of named process arguments (--key=value).

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getMainClass

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.

The execution args are passed in as a sequence of named process arguments (--key=value).

Returns
TypeDescription
string

hasMainClass

setMainClass

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.

The execution args are passed in as a sequence of named process arguments (--key=value).

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getPythonScriptFile

The Gcloud Storage URI of the main Python file to use as the driver.

Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Returns
TypeDescription
string

hasPythonScriptFile

setPythonScriptFile

The Gcloud Storage URI of the main Python file to use as the driver.

Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getSqlScriptFile

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Returns
TypeDescription
string

hasSqlScriptFile

setSqlScriptFile

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getSqlScript

The query text.

The execution args are used to declare a set of script variables (set key="value";).

Returns
TypeDescription
string

hasSqlScript

setSqlScript

The query text.

The execution args are used to declare a set of script variables (set key="value";).

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getFileUris

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Returns
TypeDescription
Google\Protobuf\Internal\RepeatedField

setFileUris

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Parameter
NameDescription
var string[]
Returns
TypeDescription
$this

getArchiveUris

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Returns
TypeDescription
Google\Protobuf\Internal\RepeatedField

setArchiveUris

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Parameter
NameDescription
var string[]
Returns
TypeDescription
$this

getInfrastructureSpec

Optional. Infrastructure specification for the execution.

Returns
TypeDescription
Google\Cloud\Dataplex\V1\Task\InfrastructureSpec|null

hasInfrastructureSpec

clearInfrastructureSpec

setInfrastructureSpec

Optional. Infrastructure specification for the execution.

Parameter
NameDescription
var Google\Cloud\Dataplex\V1\Task\InfrastructureSpec
Returns
TypeDescription
$this

getDriver

Returns
TypeDescription
string