public sealed class HadoopJob : IMessage<HadoopJob>, IEquatable<HadoopJob>, IDeepCloneable<HadoopJob>, IBufferMessage, IMessage
Reference documentation and code samples for the Google Cloud Dataproc v1 API class HadoopJob.
A Dataproc job for running Apache Hadoop MapReduce jobs on Apache Hadoop YARN.
Implements
IMessageHadoopJob, IEquatableHadoopJob, IDeepCloneableHadoopJob, IBufferMessage, IMessageNamespace
Google.Cloud.Dataproc.V1Assembly
Google.Cloud.Dataproc.V1.dll
Constructors
HadoopJob()
public HadoopJob()
HadoopJob(HadoopJob)
public HadoopJob(HadoopJob other)
Parameter | |
---|---|
Name | Description |
other | HadoopJob |
Properties
ArchiveUris
public RepeatedField<string> ArchiveUris { get; }
Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
Property Value | |
---|---|
Type | Description |
RepeatedFieldstring |
Args
public RepeatedField<string> Args { get; }
Optional. The arguments to pass to the driver. Do not
include arguments, such as -libjars
or -Dfoo=bar
, that can be set as
job properties, since a collision may occur that causes an incorrect job
submission.
Property Value | |
---|---|
Type | Description |
RepeatedFieldstring |
DriverCase
public HadoopJob.DriverOneofCase DriverCase { get; }
Property Value | |
---|---|
Type | Description |
HadoopJobDriverOneofCase |
FileUris
public RepeatedField<string> FileUris { get; }
Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
Property Value | |
---|---|
Type | Description |
RepeatedFieldstring |
HasMainClass
public bool HasMainClass { get; }
Gets whether the "main_class" field is set
Property Value | |
---|---|
Type | Description |
bool |
HasMainJarFileUri
public bool HasMainJarFileUri { get; }
Gets whether the "main_jar_file_uri" field is set
Property Value | |
---|---|
Type | Description |
bool |
JarFileUris
public RepeatedField<string> JarFileUris { get; }
Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
Property Value | |
---|---|
Type | Description |
RepeatedFieldstring |
LoggingConfig
public LoggingConfig LoggingConfig { get; set; }
Optional. The runtime log config for job execution.
Property Value | |
---|---|
Type | Description |
LoggingConfig |
MainClass
public string MainClass { get; set; }
The name of the driver's main class. The jar file containing the class
must be in the default CLASSPATH or specified in jar_file_uris
.
Property Value | |
---|---|
Type | Description |
string |
MainJarFileUri
public string MainJarFileUri { get; set; }
The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
Property Value | |
---|---|
Type | Description |
string |
Properties
public MapField<string, string> Properties { get; }
Optional. A mapping of property names to values, used to configure Hadoop.
Properties that conflict with values set by the Dataproc API may be
overwritten. Can include properties set in /etc/hadoop/conf/*-site
and
classes in user code.
Property Value | |
---|---|
Type | Description |
MapFieldstringstring |