Write (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, you can use unsupported SDKs temporarily while you upgrade your pipelines. For more information, see Using an unsupported SDK.

For information on migrating from the 1.x SDK, see Migrating from Dataflow SDK 1.x for Java.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.io

Class Write

• java.lang.Object
• • com.google.cloud.dataflow.sdk.io.Write

• Direct Known Subclasses:

  Write

  ---

  
  

  @Experimental(value=SOURCE_SINK)
  public class Write
  extends Object

  A PTransform that writes to a Sink. A write begins with a sequential global initialization of a sink, followed by a parallel write, and ends with a sequential finalization of the write. The output of a write is PDone.

  By default, every bundle in the input PCollection will be processed by a Sink.WriteOperation, so the number of outputs will vary based on runner behavior, though at least 1 output will always be produced. The exact parallelism of the write stage can be controlled using Write.Bound.withNumShards(int), typically used to control how many files are produced or to globally limit the number of workers connecting to an external service. However, this option can often hurt performance: it adds an additional GroupByKey to the pipeline.

  Write re-windows the data into the global window, so it is typically not well suited to use in streaming pipelines.

  Example usage with runner-controlled sharding:

   p.apply(Write.to(new MySink(...)));

  Example usage with a fixed number of shards:

   p.apply(Write.to(new MySink(...)).withNumShards(3));

• • Nested Class Summary

    Nested Classes 

    Modifier and Type	Class and Description
    static class	Write.Bound<T> A PTransform that writes to a Sink.

  • Constructor Summary

    Constructors 

    Constructor and Description
    Write()

  • Method Summary

    All Methods Static Methods Concrete Methods 

    Modifier and Type	Method and Description
    static <T> Write.Bound<T>	to(Sink<T> sink) Creates a Write transform that writes to the given Sink, letting the runner control how many different shards are produced.

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

• • Constructor Detail

    • Write

      public Write()

  • Method Detail

    • to

      public static <T> Write.Bound<T> to(Sink<T> sink)

      Creates a Write transform that writes to the given Sink, letting the runner control how many different shards are produced.