CloudBigtableIO.CloudBigtableMultiTableWriteFn (Apache Beam + Cloud Bigtable Connector 1.3.0 API)

com.google.cloud.bigtable.beam

Class CloudBigtableIO.CloudBigtableMultiTableWriteFn

• java.lang.Object
• • org.apache.beam.sdk.transforms.DoFn<In,Out>
  • • com.google.cloud.bigtable.beam.AbstractCloudBigtableTableDoFn<InputType,Void>
    • • com.google.cloud.bigtable.beam.CloudBigtableIO.CloudBigtableMultiTableWriteFn

• All Implemented Interfaces:

  Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

  Enclosing class:

  CloudBigtableIO

  ---

  
  

  public static class CloudBigtableIO.CloudBigtableMultiTableWriteFn
  extends AbstractCloudBigtableTableDoFn<InputType,Void>

  A DoFn that can write either a bounded or unbounded PCollection of KV of (String tableName, List of Mutations) to the specified table.

  NOTE: This DoFn will write Puts and Deletes, not Appends and Increments. This limitation exists because if the batch fails partway through, Appends/Increments might be re-run, causing the Mutation to be executed twice, which is never the user's intent. Re-running a Delete will not cause any differences. Re-running a Put isn't normally a problem, but might cause problems in some cases when the number of versions supported by the column family is greater than one. In a case where multiple versions could be a problem, it's best to add a timestamp to the Put.

  See Also:

  Serialized Form

• • Nested Class Summary

    • Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

      org.apache.beam.sdk.transforms.DoFn.BoundedPerElement, org.apache.beam.sdk.transforms.DoFn.FinishBundle, org.apache.beam.sdk.transforms.DoFn.FinishBundleContext, org.apache.beam.sdk.transforms.DoFn.GetInitialRestriction, org.apache.beam.sdk.transforms.DoFn.GetRestrictionCoder, org.apache.beam.sdk.transforms.DoFn.NewTracker, org.apache.beam.sdk.transforms.DoFn.OnTimer, org.apache.beam.sdk.transforms.DoFn.OnTimerContext, org.apache.beam.sdk.transforms.DoFn.OutputReceiver<T>, org.apache.beam.sdk.transforms.DoFn.ProcessContext, org.apache.beam.sdk.transforms.DoFn.ProcessContinuation, org.apache.beam.sdk.transforms.DoFn.ProcessElement, org.apache.beam.sdk.transforms.DoFn.RequiresStableInput, org.apache.beam.sdk.transforms.DoFn.Setup, org.apache.beam.sdk.transforms.DoFn.SplitRestriction, org.apache.beam.sdk.transforms.DoFn.StartBundle, org.apache.beam.sdk.transforms.DoFn.StartBundleContext, org.apache.beam.sdk.transforms.DoFn.StateId, org.apache.beam.sdk.transforms.DoFn.Teardown, org.apache.beam.sdk.transforms.DoFn.TimerId, org.apache.beam.sdk.transforms.DoFn.UnboundedPerElement, org.apache.beam.sdk.transforms.DoFn.WindowedContext

  • Field Summary

    • Fields inherited from class com.google.cloud.bigtable.beam.AbstractCloudBigtableTableDoFn

      config, connection, DOFN_LOG

  • Constructor Summary

    Constructors 

    Constructor and Description
    CloudBigtableMultiTableWriteFn(CloudBigtableConfiguration config)

  • Method Summary

    All Methods Instance Methods Concrete Methods 

    Modifier and Type	Method and Description
    void	finishBundle(org.apache.beam.sdk.transforms.DoFn.FinishBundleContext c)
    void	processElement(org.apache.beam.sdk.transforms.DoFn.ProcessContext context) Uses the connection to create a new Table to write the Mutations to.
    void	startBundle()

    • Methods inherited from class com.google.cloud.bigtable.beam.AbstractCloudBigtableTableDoFn

      getConfig, getConnection, logExceptions, logRetriesExhaustedWithDetailsException, populateDisplayData, rethrowException

    • Methods inherited from class org.apache.beam.sdk.transforms.DoFn

      getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, prepareForProcessing

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

• • Constructor Detail

    • CloudBigtableMultiTableWriteFn

      public CloudBigtableMultiTableWriteFn(CloudBigtableConfiguration config)

  • Method Detail

    • startBundle

      @DoFn.StartBundle
      public void startBundle()
                                         throws Exception

      Throws:

      Exception

    • processElement

      @DoFn.ProcessElement
      public void processElement(org.apache.beam.sdk.transforms.DoFn.ProcessContext context)
                                               throws Exception

      Uses the connection to create a new Table to write the Mutations to.

      NOTE: This method does not create a new table in Cloud Bigtable. The table must already exist.

      Parameters:

      context - The context for the DoFn.

      Throws:

      Exception

    • finishBundle

      @DoFn.FinishBundle
      public void finishBundle(org.apache.beam.sdk.transforms.DoFn.FinishBundleContext c)
                                           throws Exception

      Throws:

      Exception