CloudBigtableIO.CloudBigtableMultiTableWriteFn (Apache Beam + Cloud Bigtable Connector 1.0.0-pre3 API)

com.google.cloud.bigtable.beam

Class CloudBigtableIO.CloudBigtableMultiTableWriteFn

  • All Implemented Interfaces:
    Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    Enclosing class:
    CloudBigtableIO


    public static class CloudBigtableIO.CloudBigtableMultiTableWriteFn
    extends AbstractCloudBigtableTableDoFn<InputType,Void>
    A DoFn that can write either a bounded or unbounded PCollection of KV of (String tableName, List of Mutations) to the specified table.

    NOTE: This DoFn will write Puts and Deletes, not Appends and Increments. This limitation exists because if the batch fails partway through, Appends/Increments might be re-run, causing the Mutation to be executed twice, which is never the user's intent. Re-running a Delete will not cause any differences. Re-running a Put isn't normally a problem, but might cause problems in some cases when the number of versions supported by the column family is greater than one. In a case where multiple versions could be a problem, it's best to add a timestamp to the Put.

    See Also:
    Serialized Form
    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

        org.apache.beam.sdk.transforms.DoFn.BoundedPerElement, org.apache.beam.sdk.transforms.DoFn.FinishBundle, org.apache.beam.sdk.transforms.DoFn.FinishBundleContext, org.apache.beam.sdk.transforms.DoFn.GetInitialRestriction, org.apache.beam.sdk.transforms.DoFn.GetRestrictionCoder, org.apache.beam.sdk.transforms.DoFn.NewTracker, org.apache.beam.sdk.transforms.DoFn.OnTimer, org.apache.beam.sdk.transforms.DoFn.OnTimerContext, org.apache.beam.sdk.transforms.DoFn.OutputReceiver<T>, org.apache.beam.sdk.transforms.DoFn.ProcessContext, org.apache.beam.sdk.transforms.DoFn.ProcessContinuation, org.apache.beam.sdk.transforms.DoFn.ProcessElement, org.apache.beam.sdk.transforms.DoFn.Setup, org.apache.beam.sdk.transforms.DoFn.SplitRestriction, org.apache.beam.sdk.transforms.DoFn.StartBundle, org.apache.beam.sdk.transforms.DoFn.StartBundleContext, org.apache.beam.sdk.transforms.DoFn.StateId, org.apache.beam.sdk.transforms.DoFn.Teardown, org.apache.beam.sdk.transforms.DoFn.TimerId, org.apache.beam.sdk.transforms.DoFn.UnboundedPerElement, org.apache.beam.sdk.transforms.DoFn.WindowedContext
    • Method Detail

      • startBundle

        @DoFn.StartBundle
        public void startBundle()
                                           throws Exception
        Throws:
        Exception
      • processElement

        @DoFn.ProcessElement
        public void processElement(org.apache.beam.sdk.transforms.DoFn.ProcessContext context)
                                                 throws Exception
        Uses the connection to create a new Table to write the Mutations to.

        NOTE: This method does not create a new table in Cloud Bigtable. The table must already exist.

        Parameters:
        context - The context for the DoFn.
        Throws:
        Exception
      • finishBundle

        @DoFn.FinishBundle
        public void finishBundle(org.apache.beam.sdk.transforms.DoFn.FinishBundleContext c)
                                             throws Exception
        Throws:
        Exception


Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Cloud Bigtable Documentation