Google Cloud Dataflow SDK for Java, version 1.9.1
Class CountingInput.UnboundedCountingInput
- java.lang.Object
-
- com.google.cloud.dataflow.sdk.transforms.PTransform<PBegin,PCollection<Long>>
-
- com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput
-
- All Implemented Interfaces:
- HasDisplayData, Serializable
- Enclosing class:
- CountingInput
public static class CountingInput.UnboundedCountingInput extends PTransform<PBegin,PCollection<Long>>
APTransform
that will produce numbers starting from0
up toLong.MAX_VALUE
.After
Long.MAX_VALUE
, the transform never produces more output. (In practice, this limit should never be reached.)Elements in the resulting
PCollection<Long>
will by default have timestamps corresponding to processing time at element generation, provided byInstant.now()
. Use the transform returned bywithTimestampFn(SerializableFunction)
to control the output timestamps.- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform
name
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method and Description PCollection<Long>
apply(PBegin begin)
Applies thisPTransform
on the givenInputT
, and returns itsOutput
.void
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.CountingInput.UnboundedCountingInput
withMaxNumRecords(long maxRecords)
Returns anCountingInput.UnboundedCountingInput
like this one, but that will read at most the specified number of elements.CountingInput.UnboundedCountingInput
withMaxReadTime(Duration readTime)
Returns anCountingInput.UnboundedCountingInput
like this one, but that will read for at most the specified amount of time.CountingInput.UnboundedCountingInput
withRate(long numElements, Duration periodLength)
Returns anCountingInput.UnboundedCountingInput
like this one, but with output production limited to an aggregate rate of no more than the number of elements per the period length.CountingInput.UnboundedCountingInput
withTimestampFn(SerializableFunction<Long,Instant> timestampFn)
Returns anCountingInput.UnboundedCountingInput
like this one, but where output elements have the timestamp specified by the timestampFn.-
Methods inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform
getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, toString, validate
-
-
-
-
Method Detail
-
withTimestampFn
public CountingInput.UnboundedCountingInput withTimestampFn(SerializableFunction<Long,Instant> timestampFn)
Returns anCountingInput.UnboundedCountingInput
like this one, but where output elements have the timestamp specified by the timestampFn.Note that the timestamps produced by
timestampFn
may not decrease.
-
withMaxNumRecords
public CountingInput.UnboundedCountingInput withMaxNumRecords(long maxRecords)
Returns anCountingInput.UnboundedCountingInput
like this one, but that will read at most the specified number of elements.A bounded amount of elements will be produced by the result transform, and the result
PCollection
will bebounded
.
-
withRate
public CountingInput.UnboundedCountingInput withRate(long numElements, Duration periodLength)
Returns anCountingInput.UnboundedCountingInput
like this one, but with output production limited to an aggregate rate of no more than the number of elements per the period length.Note that when there are multiple splits, each split outputs independently. This may lead to elements not being produced evenly across time, though the aggregate rate will still approach the specified rate.
A duration of
Duration.ZERO
will produce output as fast as possible.
-
withMaxReadTime
public CountingInput.UnboundedCountingInput withMaxReadTime(Duration readTime)
Returns anCountingInput.UnboundedCountingInput
like this one, but that will read for at most the specified amount of time.A bounded amount of elements will be produced by the result transform, and the result
PCollection
will bebounded
.
-
apply
public PCollection<Long> apply(PBegin begin)
Description copied from class:PTransform
Applies thisPTransform
on the givenInputT
, and returns itsOutput
.Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
The default implementation throws an exception. A derived class must either implement apply, or else each runner must supply a custom implementation via
PipelineRunner.apply(com.google.cloud.dataflow.sdk.transforms.PTransform<InputT, OutputT>, InputT)
.- Overrides:
apply
in classPTransform<PBegin,PCollection<Long>>
-
populateDisplayData
public void populateDisplayData(DisplayData.Builder builder)
Description copied from class:PTransform
Register display data for the given transform or component.populateDisplayData(DisplayData.Builder)
is invoked by Pipeline runners to collect display data viaDisplayData.from(HasDisplayData)
. Implementations may callsuper.populateDisplayData(builder)
in order to register display data in the current namespace, but should otherwise usesubcomponent.populateDisplayData(builder)
to use the namespace of the subcomponent.By default, does not register any display data. Implementors may override this method to provide their own display data.
- Specified by:
populateDisplayData
in interfaceHasDisplayData
- Overrides:
populateDisplayData
in classPTransform<PBegin,PCollection<Long>>
- Parameters:
builder
- The builder to populate with display data.- See Also:
HasDisplayData
-
-