PigJob

A Cloud Dataproc job for running Apache Pig queries on YARN.

JSON representation
{
  "continueOnFailure": boolean,
  "scriptVariables": {
    string: string,
    ...
  },
  "properties": {
    string: string,
    ...
  },
  "jarFileUris": [
    string
  ],
  "loggingConfig": {
    object(LoggingConfig)
  },

  // Union field queries can be only one of the following:
  "queryFileUri": string,
  "queryList": {
    object(QueryList)
  }
  // End of list of possible types for union field queries.
}
Fields
continueOnFailure

boolean

Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.

scriptVariables

map (key: string, value: string)

Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.

properties

map (key: string, value: string)

Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.

jarFileUris[]

string

Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.

loggingConfig

object(LoggingConfig)

Optional. The runtime log config for job execution.

Union field queries. Required. The sequence of Pig queries to execute, specified as an HCFS file URI or a list of queries. queries can be only one of the following:
queryFileUri

string

The HCFS URI of the script that contains the Pig queries.

queryList

object(QueryList)

A list of queries.

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Dataproc