Google Cloud Dataplex V1 Client - Class SparkTaskConfig (0.1.6)

Reference documentation and code samples for the Google Cloud Dataplex V1 Client class SparkTaskConfig.

User-specified config for running a Spark task.

Generated from protobuf message google.cloud.dataplex.v1.Task.SparkTaskConfig

Methods

__construct

Constructor.

Parameters
NameDescription
data array

Optional. Data for populating the Message object.

↳ main_jar_file_uri string

The Cloud Storage URI of the jar file that contains the main class. The execution args are passed in as a sequence of named process arguments (--key=value).

↳ main_class string

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris. The execution args are passed in as a sequence of named process arguments (--key=value).

↳ python_script_file string

The Gcloud Storage URI of the main Python file to use as the driver. Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

↳ sql_script_file string

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

↳ sql_script string

The query text. The execution args are used to declare a set of script variables (set key="value";).

↳ file_uris string[]

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

↳ archive_uris string[]

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

↳ infrastructure_spec Google\Cloud\Dataplex\V1\Task\InfrastructureSpec

Optional. Infrastructure specification for the execution.

getMainJarFileUri

The Cloud Storage URI of the jar file that contains the main class.

The execution args are passed in as a sequence of named process arguments (--key=value).

Generated from protobuf field string main_jar_file_uri = 100;

Returns
TypeDescription
string

hasMainJarFileUri

setMainJarFileUri

The Cloud Storage URI of the jar file that contains the main class.

The execution args are passed in as a sequence of named process arguments (--key=value).

Generated from protobuf field string main_jar_file_uri = 100;

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getMainClass

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.

The execution args are passed in as a sequence of named process arguments (--key=value).

Generated from protobuf field string main_class = 101;

Returns
TypeDescription
string

hasMainClass

setMainClass

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.

The execution args are passed in as a sequence of named process arguments (--key=value).

Generated from protobuf field string main_class = 101;

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getPythonScriptFile

The Gcloud Storage URI of the main Python file to use as the driver.

Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Generated from protobuf field string python_script_file = 102;

Returns
TypeDescription
string

hasPythonScriptFile

setPythonScriptFile

The Gcloud Storage URI of the main Python file to use as the driver.

Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

Generated from protobuf field string python_script_file = 102;

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getSqlScriptFile

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Generated from protobuf field string sql_script_file = 104;

Returns
TypeDescription
string

hasSqlScriptFile

setSqlScriptFile

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

Generated from protobuf field string sql_script_file = 104;

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getSqlScript

The query text.

The execution args are used to declare a set of script variables (set key="value";).

Generated from protobuf field string sql_script = 105;

Returns
TypeDescription
string

hasSqlScript

setSqlScript

The query text.

The execution args are used to declare a set of script variables (set key="value";).

Generated from protobuf field string sql_script = 105;

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getFileUris

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Generated from protobuf field repeated string file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];

Returns
TypeDescription
Google\Protobuf\Internal\RepeatedField

setFileUris

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

Generated from protobuf field repeated string file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];

Parameter
NameDescription
var string[]
Returns
TypeDescription
$this

getArchiveUris

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Generated from protobuf field repeated string archive_uris = 4 [(.google.api.field_behavior) = OPTIONAL];

Returns
TypeDescription
Google\Protobuf\Internal\RepeatedField

setArchiveUris

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

Generated from protobuf field repeated string archive_uris = 4 [(.google.api.field_behavior) = OPTIONAL];

Parameter
NameDescription
var string[]
Returns
TypeDescription
$this

getInfrastructureSpec

Optional. Infrastructure specification for the execution.

Generated from protobuf field .google.cloud.dataplex.v1.Task.InfrastructureSpec infrastructure_spec = 6 [(.google.api.field_behavior) = OPTIONAL];

Returns
TypeDescription
Google\Cloud\Dataplex\V1\Task\InfrastructureSpec|null

hasInfrastructureSpec

clearInfrastructureSpec

setInfrastructureSpec

Optional. Infrastructure specification for the execution.

Generated from protobuf field .google.cloud.dataplex.v1.Task.InfrastructureSpec infrastructure_spec = 6 [(.google.api.field_behavior) = OPTIONAL];

Parameter
NameDescription
var Google\Cloud\Dataplex\V1\Task\InfrastructureSpec
Returns
TypeDescription
$this

getDriver

Returns
TypeDescription
string