Google Cloud Dataplex V1 Client - Class SparkTaskConfig (0.14.0)

Version 0.14.0

• 1.1.0 (latest)
• 1.0.1
• 0.16.1
• 0.15.1
• 0.14.0
• 0.13.0
• 0.12.0
• 0.11.1
• 0.10.0
• 0.9.0
• 0.8.0
• 0.7.0
• 0.6.0
• 0.5.1
• 0.4.0
• 0.3.0
• 0.2.3
• 0.1.6

Reference documentation and code samples for the Google Cloud Dataplex V1 Client class SparkTaskConfig.

User-specified config for running a Spark task.

Generated from protobuf message google.cloud.dataplex.v1.Task.SparkTaskConfig

Namespace

Google \ Cloud \ Dataplex \ V1 \ Task

Methods

__construct

Constructor.

Parameters
Name	Description
data	array Optional. Data for populating the Message object.
↳ main_jar_file_uri	string The Cloud Storage URI of the jar file that contains the main class. The execution args are passed in as a sequence of named process arguments (--key=value).
↳ main_class	string The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris. The execution args are passed in as a sequence of named process arguments (--key=value).
↳ python_script_file	string The Gcloud Storage URI of the main Python file to use as the driver. Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).
↳ sql_script_file	string A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).
↳ sql_script	string The query text. The execution args are used to declare a set of script variables (set key="value";).
↳ file_uris	array Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.
↳ archive_uris	array Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
↳ infrastructure_spec	Google\Cloud\Dataplex\V1\Task\InfrastructureSpec Optional. Infrastructure specification for the execution.

getMainJarFileUri

The Cloud Storage URI of the jar file that contains the main class.

The execution args are passed in as a sequence of named process arguments (--key=value).

Returns
Type	Description
string

hasMainJarFileUri

setMainJarFileUri

The Cloud Storage URI of the jar file that contains the main class.

The execution args are passed in as a sequence of named process arguments (--key=value).

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getMainClass

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.

The execution args are passed in as a sequence of named process arguments (--key=value).

Returns
Type	Description
string

hasMainClass

setMainClass

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.

The execution args are passed in as a sequence of named process arguments (--key=value).

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getPythonScriptFile

The Gcloud Storage URI of the main Python file to use as the driver.

Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Returns
Type	Description
string

hasPythonScriptFile

setPythonScriptFile

The Gcloud Storage URI of the main Python file to use as the driver.

Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getSqlScriptFile

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Returns
Type	Description
string

hasSqlScriptFile

setSqlScriptFile

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getSqlScript

The query text.

The execution args are used to declare a set of script variables (set key="value";).

Returns
Type	Description
string

hasSqlScript

setSqlScript

The query text.

The execution args are used to declare a set of script variables (set key="value";).

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getFileUris

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Returns
Type	Description
Google\Protobuf\Internal\RepeatedField

setFileUris

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Parameter
Name	Description
var	string[]

Returns
Type	Description
$this

getArchiveUris

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Returns
Type	Description
Google\Protobuf\Internal\RepeatedField

setArchiveUris

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Parameter
Name	Description
var	string[]

Returns
Type	Description
$this

getInfrastructureSpec

Optional. Infrastructure specification for the execution.

Returns
Type	Description
Google\Cloud\Dataplex\V1\Task\InfrastructureSpec|null

hasInfrastructureSpec

clearInfrastructureSpec

setInfrastructureSpec

Optional. Infrastructure specification for the execution.

Parameter
Name	Description
var	Google\Cloud\Dataplex\V1\Task\InfrastructureSpec

Returns
Type	Description
$this

getDriver

Returns
Type	Description
string