CountingInput (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

com.google.cloud.dataflow.sdk.io

Class CountingInput



  • public class CountingInput
    extends Object
    A PTransform that produces longs. When used to produce a bounded PCollection, CountingInput starts at 0 and counts up to a specified maximum. When used to produce an unbounded PCollection, it counts up to Long.MAX_VALUE and then never produces more output. (In practice, this limit should never be reached.)

    The bounded CountingInput is implemented based on OffsetBasedSource and OffsetBasedSource.OffsetBasedReader, so it performs efficient initial splitting and it supports dynamic work rebalancing.

    To produce a bounded PCollection<Long>, use upTo(long):

    
     Pipeline p = ...
     PTransform<PBegin, PCollection<Long>> producer = CountingInput.upTo(1000);
     PCollection<Long> bounded = p.apply(producer);
     

    To produce an unbounded PCollection<Long>, use unbounded(), calling CountingInput.UnboundedCountingInput.withTimestampFn(SerializableFunction) to provide values with timestamps other than Instant.now().

    
     Pipeline p = ...
    
     // To create an unbounded producer that uses processing time as the element timestamp.
     PCollection<Long> unbounded = p.apply(CountingInput.unbounded());
     // Or, to create an unbounded source that uses a provided function to set the element timestamp.
     PCollection<Long> unboundedWithTimestamps =
         p.apply(CountingInput.unbounded().withTimestampFn(someFn));
     


이 페이지가 도움이 되었나요? 평가를 부탁드립니다.

다음에 대한 의견 보내기...

도움이 필요하시나요? 지원 페이지를 방문하세요.