CountingSource (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

com.google.cloud.dataflow.sdk.io

Class CountingSource



  • public class CountingSource
    extends Object
    A source that produces longs. When used as a BoundedSource, CountingSource starts at 0 and counts up to a specified maximum. When used as an UnboundedSource, it counts up to Long.MAX_VALUE and then never produces more output. (In practice, this limit should never be reached.)

    The bounded CountingSource is implemented based on OffsetBasedSource and OffsetBasedSource.OffsetBasedReader, so it performs efficient initial splitting and it supports dynamic work rebalancing.

    To produce a bounded PCollection<Long>, use upTo(long):

    
     Pipeline p = ...
     PTransform<PBegin, PCollection<Long>> producer = CountingInput.upTo(1000);
     PCollection<Long> bounded = p.apply(producer);
     

    To produce an unbounded PCollection<Long>, use CountingInput.unbounded(), calling CountingInput.UnboundedCountingInput.withTimestampFn(SerializableFunction) to provide values with timestamps other than Instant.now().

    
     Pipeline p = ...
    
     // To create an unbounded PCollection that uses processing time as the element timestamp.
     PCollection<Long> unbounded = p.apply(CountingInput.unbounded());
     // Or, to create an unbounded source that uses a provided function to set the element timestamp.
     PCollection<Long> unboundedWithTimestamps =
         p.apply(CountingInput.unbounded().withTimestampFn(someFn));
    
     


이 페이지가 도움이 되었나요? 평가를 부탁드립니다.

다음에 대한 의견 보내기...

도움이 필요하시나요? 지원 페이지를 방문하세요.