CloudBigtableIO.CloudBigtableMultiTableWriteFn (Apache Beam + Cloud Bigtable Connector 1.10.0 API)

com.google.cloud.bigtable.beam

Class CloudBigtableIO.CloudBigtableMultiTableWriteFn

• java.lang.Object
• • org.apache.beam.sdk.transforms.DoFn<In,Out>
  • • com.google.cloud.bigtable.beam.AbstractCloudBigtableTableDoFn<InputType,Void>
    • • com.google.cloud.bigtable.beam.CloudBigtableIO.CloudBigtableMultiTableWriteFn

• All Implemented Interfaces:

  Serializable, HasDisplayData

  Enclosing class:

  CloudBigtableIO

  ---

  
  

  public static class CloudBigtableIO.CloudBigtableMultiTableWriteFn
  extends AbstractCloudBigtableTableDoFn<InputType,Void>

  A DoFn that can write either a bounded or unbounded PCollection of KV of (String tableName, List of Mutations) to the specified table.

  NOTE: This DoFn will write Puts and Deletes, not Appends and Increments. This limitation exists because if the batch fails partway through, Appends/Increments might be re-run, causing the Mutation to be executed twice, which is never the user's intent. Re-running a Delete will not cause any differences. Re-running a Put isn't normally a problem, but might cause problems in some cases when the number of versions supported by the column family is greater than one. In a case where multiple versions could be a problem, it's best to add a timestamp to the Put.

  See Also:

  Serialized Form

• • Nested Class Summary

    • Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

      DoFn.BoundedPerElement, DoFn.Element, DoFn.FieldAccess, DoFn.FinishBundle, DoFn.FinishBundleContext, DoFn.GetInitialRestriction, DoFn.GetRestrictionCoder, DoFn.MultiOutputReceiver, DoFn.NewTracker, DoFn.OnTimer, DoFn.OnTimerContext, DoFn.OnWindowExpiration, DoFn.OutputReceiver<T>, DoFn.ProcessContext, DoFn.ProcessContinuation, DoFn.ProcessElement, DoFn.RequiresStableInput, DoFn.Setup, DoFn.SplitRestriction, DoFn.StartBundle, DoFn.StartBundleContext, DoFn.StateId, DoFn.Teardown, DoFn.TimerId, DoFn.Timestamp, DoFn.UnboundedPerElement, DoFn.WindowedContext

  • Field Summary

    • Fields inherited from class com.google.cloud.bigtable.beam.AbstractCloudBigtableTableDoFn

      config, connection, DOFN_LOG

  • Constructor Summary

    Constructors 

    Constructor and Description
    CloudBigtableMultiTableWriteFn(CloudBigtableConfiguration config)

  • Method Summary

    All Methods Instance Methods Concrete Methods 

    Modifier and Type	Method and Description
    void	finishBundle(DoFn.FinishBundleContext c)
    void	processElement(DoFn.ProcessContext context) Uses the connection to create a new Table to write the Mutations to.
    void	startBundle()

    • Methods inherited from class com.google.cloud.bigtable.beam.AbstractCloudBigtableTableDoFn

      getConfig, getConnection, logExceptions, logRetriesExhaustedWithDetailsException, populateDisplayData, rethrowException

    • Methods inherited from class org.apache.beam.sdk.transforms.DoFn

      getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, prepareForProcessing

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

• • Constructor Detail

    • CloudBigtableMultiTableWriteFn

      public CloudBigtableMultiTableWriteFn(CloudBigtableConfiguration config)

  • Method Detail

    • startBundle

      @DoFn.StartBundle
      public void startBundle()
                                         throws Exception

      Throws:

      Exception

    • processElement

      @DoFn.ProcessElement
      public void processElement(DoFn.ProcessContext context)
                                               throws Exception

      Uses the connection to create a new Table to write the Mutations to.

      NOTE: This method does not create a new table in Cloud Bigtable. The table must already exist.

      Parameters:

      context - The context for the DoFn.

      Throws:

      Exception

    • finishBundle

      @DoFn.FinishBundle
      public void finishBundle(DoFn.FinishBundleContext c)
                                           throws Exception

      Throws:

      Exception