Google Cloud Dataplex V1 Client - Class SparkTaskConfig (1.0.0-RC1)

Reference documentation and code samples for the Google Cloud Dataplex V1 Client class SparkTaskConfig.

User-specified config for running a Spark task.

Generated from protobuf message google.cloud.dataplex.v1.Task.SparkTaskConfig

Namespace

Google \ Cloud \ Dataplex \ V1 \ Task

Methods

__construct

Constructor.

Parameters
Name Description
data array

Optional. Data for populating the Message object.

↳ main_jar_file_uri string

The Cloud Storage URI of the jar file that contains the main class. The execution args are passed in as a sequence of named process arguments (--key=value).

↳ main_class string

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris. The execution args are passed in as a sequence of named process arguments (--key=value).

↳ python_script_file string

The Gcloud Storage URI of the main Python file to use as the driver. Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

↳ sql_script_file string

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

↳ sql_script string

The query text. The execution args are used to declare a set of script variables (set key="value";).

↳ file_uris array

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

↳ archive_uris array

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

↳ infrastructure_spec Google\Cloud\Dataplex\V1\Task\InfrastructureSpec

Optional. Infrastructure specification for the execution.

getMainJarFileUri

The Cloud Storage URI of the jar file that contains the main class.

The execution args are passed in as a sequence of named process arguments (--key=value).

Returns
Type Description
string

hasMainJarFileUri

setMainJarFileUri

The Cloud Storage URI of the jar file that contains the main class.

The execution args are passed in as a sequence of named process arguments (--key=value).

Parameter
Name Description
var string
Returns
Type Description
$this

getMainClass

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.

The execution args are passed in as a sequence of named process arguments (--key=value).

Returns
Type Description
string

hasMainClass

setMainClass

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.

The execution args are passed in as a sequence of named process arguments (--key=value).

Parameter
Name Description
var string
Returns
Type Description
$this

getPythonScriptFile

The Gcloud Storage URI of the main Python file to use as the driver.

Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Returns
Type Description
string

hasPythonScriptFile

setPythonScriptFile

The Gcloud Storage URI of the main Python file to use as the driver.

Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Parameter
Name Description
var string
Returns
Type Description
$this

getSqlScriptFile

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Returns
Type Description
string

hasSqlScriptFile

setSqlScriptFile

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Parameter
Name Description
var string
Returns
Type Description
$this

getSqlScript

The query text.

The execution args are used to declare a set of script variables (set key="value";).

Returns
Type Description
string

hasSqlScript

setSqlScript

The query text.

The execution args are used to declare a set of script variables (set key="value";).

Parameter
Name Description
var string
Returns
Type Description
$this

getFileUris

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Returns
Type Description
Google\Protobuf\Internal\RepeatedField

setFileUris

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Parameter
Name Description
var string[]
Returns
Type Description
$this

getArchiveUris

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Returns
Type Description
Google\Protobuf\Internal\RepeatedField

setArchiveUris

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Parameter
Name Description
var string[]
Returns
Type Description
$this

getInfrastructureSpec

Optional. Infrastructure specification for the execution.

Returns
Type Description
Google\Cloud\Dataplex\V1\Task\InfrastructureSpec|null

hasInfrastructureSpec

clearInfrastructureSpec

setInfrastructureSpec

Optional. Infrastructure specification for the execution.

Parameter
Name Description
var Google\Cloud\Dataplex\V1\Task\InfrastructureSpec
Returns
Type Description
$this

getDriver

Returns
Type Description
string