Google Cloud Dataflow SDK for Java, version 1.9.1
Class IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn<InputT,OutputT>
- java.lang.Object
-
- com.google.cloud.dataflow.sdk.transforms.DoFn<InputT,OutputT>
-
- com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn<InputT,OutputT>
-
- Type Parameters:
InputT
- the type of the (main) input elementsOutputT
- the type of the (main) output elements
- All Implemented Interfaces:
- HasDisplayData, Serializable
- Enclosing class:
- IntraBundleParallelization
public static class IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn<InputT,OutputT> extends DoFn<InputT,OutputT>
A multi-threadedDoFn
wrapper.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.transforms.DoFn
DoFn.Context, DoFn.ProcessContext, DoFn.RequiresWindowAccess
-
-
Constructor Summary
Constructors Constructor and Description MultiThreadedIntraBundleProcessingDoFn(DoFn<InputT,OutputT> doFn, int maxParallelism)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method and Description void
finishBundle(DoFn.Context c)
Finishes processing this batch of elements.protected TypeDescriptor<InputT>
getInputTypeDescriptor()
Returns aTypeDescriptor
capturing what is known statically about the input type of thisDoFn
instance's most-derived class.protected TypeDescriptor<OutputT>
getOutputTypeDescriptor()
Returns aTypeDescriptor
capturing what is known statically about the output type of thisDoFn
instance's most-derived class.void
processElement(DoFn.ProcessContext c)
Processes one input element.void
startBundle(DoFn.Context c)
Prepares thisDoFn
instance for processing a batch of elements.-
Methods inherited from class com.google.cloud.dataflow.sdk.transforms.DoFn
createAggregator, createAggregator, getAllowedTimestampSkew, populateDisplayData
-
-
-
-
Method Detail
-
startBundle
public void startBundle(DoFn.Context c) throws Exception
Description copied from class:DoFn
Prepares thisDoFn
instance for processing a batch of elements.By default, does nothing.
- Overrides:
startBundle
in classDoFn<InputT,OutputT>
- Throws:
Exception
-
processElement
public void processElement(DoFn.ProcessContext c) throws Exception
Description copied from class:DoFn
Processes one input element.The current element of the input
PCollection
is returned byc.element()
. It should be considered immutable. The Dataflow runtime will not mutate the element, so it is safe to cache, etc. The element should not be mutated by any of theDoFn
methods, because it may be cached elsewhere, retained by the Dataflow runtime, or used in other unspecified ways.A value is added to the main output
PCollection
byDoFn.Context.output(OutputT)
. Once passed tooutput
the element should be considered immutable and not be modified in any way. It may be cached elsewhere, retained by the Dataflow runtime, or used in other unspecified ways.- Specified by:
processElement
in classDoFn<InputT,OutputT>
- Throws:
Exception
- See Also:
DoFn.ProcessContext
-
finishBundle
public void finishBundle(DoFn.Context c) throws Exception
Description copied from class:DoFn
Finishes processing this batch of elements.By default, does nothing.
- Overrides:
finishBundle
in classDoFn<InputT,OutputT>
- Throws:
Exception
-
getInputTypeDescriptor
protected TypeDescriptor<InputT> getInputTypeDescriptor()
Description copied from class:DoFn
Returns aTypeDescriptor
capturing what is known statically about the input type of thisDoFn
instance's most-derived class.See
DoFn.getOutputTypeDescriptor()
for more discussion.- Overrides:
getInputTypeDescriptor
in classDoFn<InputT,OutputT>
-
getOutputTypeDescriptor
protected TypeDescriptor<OutputT> getOutputTypeDescriptor()
Description copied from class:DoFn
Returns aTypeDescriptor
capturing what is known statically about the output type of thisDoFn
instance's most-derived class.In the normal case of a concrete
DoFn
subclass with no generic type parameters of its own (including anonymous inner classes), this will be a complete non-generic type, which is good for choosing a default outputCoder<OutputT>
for the outputPCollection<OutputT>
.- Overrides:
getOutputTypeDescriptor
in classDoFn<InputT,OutputT>
-
-