Index (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. See Migrating from Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

A B C D E F G H I J K L M N O P R S T U V W X Y 

A

AccumulatingCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn

 

accumulatingFiredPanes() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window

Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that accumulates elements in a pane after they are triggered.

accumulatingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound

Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that accumulates elements in a pane after they are triggered.

accumulatingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound

Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that accumulates elements in a pane after they are triggered.

add(Iterable<WindowedValue<ElemT>>) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.InProcessPipelineRunner.PCollectionViewWriter

 

add(WindowedValue<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.InProcessPipelineRunner.UncommittedBundle

Outputs an element to this bundle.

add(Long) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique

Adds a value to the heap, returning whether the value is (large enough to be) in the heap.

add(DisplayData.Item<?>) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder

Register the given display item.

addCollectionToSingletonOutput(String, PValue, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an output with the given name to the previously added CollectionToSingleton Dataflow step, consuming the specified input PValue and producing the specified output PValue.

addComposite(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode

Adds a composite operation to the transform node.

addElements(T, T...) - Method in class com.google.cloud.dataflow.sdk.testing.TestStream.Builder

Adds the specified elements to the source with timestamp equal to the current watermark.

addElements(TimestampedValue<T>, TimestampedValue<T>...) - Method in class com.google.cloud.dataflow.sdk.testing.TestStream.Builder

Adds the specified elements to the source with the provided timestamps.

addEncodingInput(Coder<?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Sets the encoding for the current Dataflow step.

addExtensionsFrom(Class<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder

Deprecated.

use Proto2Coder.withExtensionsFrom(java.lang.Iterable<java.lang.Class<?>>)

addExtensionsFrom(Iterable<Class<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder

Deprecated.

use Proto2Coder.withExtensionsFrom(java.lang.Iterable<java.lang.Class<?>>)

addIfNotDefault(DisplayData.Item<T>, T) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder

Register the given display item if the value is different than the specified default.

addIfNotNull(DisplayData.Item<?>) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder

Register the given display item if the value is not null.

addInput(String, Boolean) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input with the given name and value to the current Dataflow step.

addInput(String, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input with the given name and value to the current Dataflow step.

addInput(String, Long) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input with the given name and value to the current Dataflow step.

addInput(String, PInput) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input with the given name to the previously added Dataflow step, coming from the specified input PValue.

addInput(String, Map<String, Object>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input that is a dictionary of strings to objects.

addInput(String, List<? extends Map<String, Object>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input that is a list of objects.

addInput(TransformTreeNode, PInput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy

Adds an input to the given node.

addInput(ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique, T) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn

 

addInput(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn.Accumulator

Adds the given input value to this accumulator, modifying this accumulator.

addInput(AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn

 

addInput(double[], Double) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn

 

addInput(Combine.Holder<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn

 

addInput(int[], Integer) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn

 

addInput(long[], Long) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn

 

addInput(AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

Adds the given input value to the given accumulator, returning the new accumulator value.

addInput(List<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn

 

addInput(K, AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn

Adds the given input value to the given accumulator, returning the new accumulator value.

addInput(Object[], DataT) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn

 

addInput(Object[], DataT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext

 

addInput(K, Object[], DataT) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn

 

addInput(K, Object[], DataT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext

 

addInput(AccumT, InputT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext

Adds the given input value to the given accumulator, returning the new accumulator value.

addInput(K, AccumT, InputT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext

Adds the given input value to the given accumulator, returning the new accumulator value.

addInput(Top.BoundedHeap<KV<Integer, T>, SerializableComparator<KV<Integer, T>>>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn

 

addInputProducer(PValue, TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode

Adds an input to the transform node.

addInputsRandomly(PerKeyCombineFnRunner<? super K, ? super InputT, AccumT, ?>, K, Iterable<InputT>, Random, DoFn<?, ?>.ProcessContext) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn

Create a random list of accumulators from the given list of values.

addOutput(String, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.

addOutput(InProcessPipelineRunner.UncommittedBundle<?>, InProcessPipelineRunner.UncommittedBundle<?>...) - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.StepTransformResult.Builder

 

addOutput(Collection<InProcessPipelineRunner.UncommittedBundle<?>>) - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.StepTransformResult.Builder

 

addOverrideForClass(Class<?>, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides

Overrides the default log level for the passed in class.

addOverrideForName(String, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides

Overrides the default log level for the passed in name.

addOverrideForPackage(Package, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides

Overrides the default log level for the passed in package.

addStep(PTransform<?, ?>, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds a step to the Dataflow workflow for the given transform, with the given Dataflow step type.

addStep(PTransform<?, ? extends PValue>, Step) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds a pre-defined step to the Dataflow workflow.

addTraceFor(AbstractGoogleClient, String) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer

Creates a GoogleApiDebugOptions.GoogleApiTracer that sets the trace destination on all calls that match the given client type.

addTraceFor(AbstractGoogleClientRequest<?>, String) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer

Creates a GoogleApiDebugOptions.GoogleApiTracer that sets the trace traceDestination on all calls that match for the given request type.

addUnprocessedElements(Iterable<? extends WindowedValue<?>>) - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.StepTransformResult.Builder

 

addValue(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator

Adds a new value into the Aggregator.

addValueInternal(PValue) - Method in class com.google.cloud.dataflow.sdk.Pipeline

Adds the given PValue to this Pipeline.

addValueOnlyOutput(String, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.

advance() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader

Deprecated.

 

advance() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader

 

advance() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader

Advances the reader to the next valid record.

advance() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader

Advances the reader to the next valid record.

advanceImpl() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader

 

advanceImpl() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader

Advances to the next record and returns true, or returns false if there is no next record.

advanceProcessingTime(Duration) - Method in class com.google.cloud.dataflow.sdk.testing.TestStream.Builder

Advance the processing time by the specified amount.

advanceWatermarkTo(Instant) - Method in class com.google.cloud.dataflow.sdk.testing.TestStream.Builder

Advance the watermark of this source to the specified instant.

advanceWatermarkToInfinity() - Method in class com.google.cloud.dataflow.sdk.testing.TestStream.Builder

Advance the watermark to infinity, completing this TestStream.

AfterAll<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

Create a Trigger that fires and finishes once after all of its sub-triggers have fired.

AfterDelayFromFirstElement<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A base class for triggers that happen after a processing time delay from the arrival of the first element in a pane.

AfterDelayFromFirstElement(TimeDomain, List<SerializableFunction<Instant, Instant>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement

 

AfterEach<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A composite Trigger that executes its sub-triggers in order.

afterElement(WindowedValue<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.ModelEnforcement

Called after a call to TransformEvaluator.processElement(WindowedValue) on the provided WindowedValue.

afterFinish(InProcessPipelineRunner.CommittedBundle<T>, InProcessTransformResult, Iterable<? extends InProcessPipelineRunner.CommittedBundle<?>>) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.ModelEnforcement

Called after a bundle has been completed and TransformEvaluator.finishBundle() has been called, producing the provided InProcessTransformResult and output bundles.

AfterFirst<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

Create a composite Trigger that fires once after at least one of its sub-triggers have fired.

AfterPane<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

Triggers that fire based on properties of the elements in the current pane.

AfterProcessingTime<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

AfterProcessingTime triggers fire based on the current processing time.

AfterWatermark<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

AfterWatermark triggers fire based on progress of the system watermark.

AfterWatermark.AfterWatermarkEarly<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing

Interface for building an AfterWatermarkTrigger with early firings already filled in.

AfterWatermark.AfterWatermarkLate<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing

Interface for building an AfterWatermarkTrigger with late firings already filled in.

AfterWatermark.FromEndOfWindow<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A watermark trigger targeted relative to the end of the window.

Aggregator<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms

An Aggregator<InputT> enables monitoring of values of type InputT, to be combined across all bundles.

AggregatorPipelineExtractor - Class in com.google.cloud.dataflow.sdk.runners

Retrieves Aggregators at each ParDo and returns a Map of Aggregator to the PTransforms in which it is present.

AggregatorPipelineExtractor(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.runners.AggregatorPipelineExtractor

Creates an AggregatorPipelineExtractor for the given Pipeline.

AggregatorRetrievalException - Exception in com.google.cloud.dataflow.sdk.runners

Signals that an exception has occurred while retrieving Aggregators.

AggregatorRetrievalException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.AggregatorRetrievalException

Constructs a new AggregatorRetrievalException with the specified detail message and cause.

AggregatorRetriever - Class in com.google.cloud.dataflow.sdk.transforms

An internal class for extracting Aggregators from DoFns.

AggregatorValues<T> - Class in com.google.cloud.dataflow.sdk.runners

A collection of values associated with an Aggregator.

AggregatorValues() - Constructor for class com.google.cloud.dataflow.sdk.runners.AggregatorValues

 

alignedTo(Duration, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement

Aligns timestamps to the smallest multiple of size since the offset greater than the timestamp.

alignedTo(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement

Aligns the time to be the smallest multiple of size greater than the timestamp since the epoch.

ALL_CONTEXTS - Static variable in class com.google.cloud.dataflow.sdk.testing.CoderProperties

All the contexts, for use in test cases.

ALL_KEYS - Static variable in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange

The range of all keys, with empty start and end keys.

allowsDynamicSplitting() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource

Whether this source should allow dynamic splitting of the offset ranges.

and(TupleTag<V>, List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult

Returns a new CoGbkResult based on this, with the given tag and given data added to it.

and(TupleTag<V>, PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple

Returns a new KeyedPCollectionTuple<K> that is the same as this, appended with the given PCollection.

and(PCollection.IsBounded) - Method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded

Returns the composed IsBounded property.

and(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList

Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollection appended to the end.

and(Iterable<PCollection<T>>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList

Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollections appended to the end, in order.

and(TupleTag<T>, PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple

Returns a new PCollectionTuple that has each PCollection and TupleTag of this PCollectionTuple plus the given PCollection associated with the given TupleTag.

and(TupleTag<?>) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList

Returns a new TupleTagList that has all the TupleTags of this TupleTagList plus the given TupleTag appended to the end.

and(List<TupleTag<?>>) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList

Returns a new TupleTagList that has all the TupleTags of this TupleTagList plus the given TupleTags appended to the end, in order.

any(long) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample

Sample#any(long) takes a PCollection<T> and a limit, and produces a new PCollection<T> containing up to limit elements of the input PCollection.

applicableTo(PCollection<?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey

 

ApplicationNameOptions - Interface in com.google.cloud.dataflow.sdk.options

Options that allow setting the application name.

AppliedPTransform<InputT extends PInput,OutputT extends POutput,TransformT extends PTransform<? super InputT,OutputT>> - Class in com.google.cloud.dataflow.sdk.transforms

Represents the application of a PTransform to a specific input to produce a specific output.

apply(InputT) - Method in interface com.google.cloud.dataflow.sdk.coders.DelegateCoder.CodingFunction

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound

 

apply(PCollection<TableRow>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read

 

apply(PCollection<KV<ByteString, Iterable<Mutation>>>) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.BoundedCountingInput

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.datastore.DatastoreV1.Read

Applies this PTransform on the given InputT, and returns its Output.

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubUnboundedSink

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.PubsubUnboundedSource

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.Write.Bound

 

apply(PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.Pipeline

Like Pipeline.apply(String, PTransform) but the transform node in the Pipeline graph will be named according to PTransform.getName().

apply(String, PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.Pipeline

Adds a root PTransform, such as Read or Create, to this Pipeline.

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner

 

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner

Applies the given transform to the input.

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner

 

apply(InputT) - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.ForwardingPTransform

 

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.InProcessPipelineRunner

 

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.PipelineRunner

Applies a transform to the given input, returning the output.

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.TemplatingDataflowPipelineRunner

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.GroupThenAssert

 

apply(PCollection<Iterable<T>>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.GroupThenAssertForSingleton

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.OneSideInputAssert

 

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.testing.TestStream

 

apply(double, double) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn

Applies the binary operation to the two operands, returning the result.

apply(V, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn

Applies the binary operation to the two operands, returning the result.

apply(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn

Applies the binary operation to the two operands, returning the result.

apply(long, long) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn

Applies the binary operation to the two operands, returning the result.

apply(Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

Applies this CombineFn to a collection of input values to produce a combined output value.

apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally

 

apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView

 

apply(PCollection<? extends KV<K, ? extends Iterable<InputT>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues

 

apply(K, Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn

Applies this KeyedCombineFn to a key and a collection of input values to produce a combined output value.

apply(PCollection<KV<K, InputT>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey

 

apply(PCollection<KV<K, InputT>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKeyWithHotKeyFanout

 

apply(K, Iterable<? extends InputT>, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext

Applies this KeyedCombineFnWithContext to a key and a collection of input values to produce a combined output value.

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Count.PerElement

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.TimestampedValues

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Filter

 

apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.FlatMapElements

 

apply(PCollection<? extends Iterable<T>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenIterables

 

apply(PCollectionList<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenPCollectionList

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey

 

apply(PCollection<KV<K, Iterable<WindowedValue<V>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupAlsoByWindow

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.ReifyTimestampsAndWindows

 

apply(PCollection<KV<K, Iterable<WindowedValue<V>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.SortValuesByTimestamp

 

apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Bound

 

apply(KeyedPCollectionTuple<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGroupByKey

 

apply(PTransform<KeyedPCollectionTuple<K>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple

Like KeyedPCollectionTuple.apply(String, PTransform) but defaulting to the name provided by the PTransform.

apply(String, PTransform<KeyedPCollectionTuple<K>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple

Applies the given PTransform to this input KeyedPCollectionTuple and returns its OutputT.

apply(PCollection<? extends KV<K, ?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Keys

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.KvSwap

 

apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.MapElements

 

apply(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn

 

apply(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn

 

apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound

 

apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Partition

 

apply(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform

Applies this PTransform on the given InputT, and returns its Output.

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates.WithRepresentativeValues

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.SampleAny

 

apply(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.SerializableFunction

Returns the result of invoking this function on the given input.

apply(double, double) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn

 

apply(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn

 

apply(long, long) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn

 

apply(PCollection<? extends KV<?, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Values

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsIterable

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsList

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMap

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMultimap

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton

 

apply(PCollection<ElemT>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.CreatePCollectionView

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Remerge

 

apply(PCollection<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithKeys

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithTimestamps

 

apply(PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PBegin

Like PBegin.apply(String, PTransform) but defaulting to the name of the PTransform.

apply(String, PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PBegin

Applies the given PTransform to this input PBegin, using name to identify this specific application of the transform.

apply(PTransform<? super PCollection<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection

Like PCollection.apply(String, PTransform) but defaulting to the name of the PTransform.

apply(String, PTransform<? super PCollection<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection

Applies the given PTransform to this input PCollection, using name to identify this specific application of the transform.

apply(PTransform<PCollectionList<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList

Like PCollectionList.apply(String, PTransform) but defaulting to the name of the PTransform.

apply(String, PTransform<PCollectionList<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList

Applies the given PTransform to this input PCollectionList, using name to identify this specific application of the transform.

apply(PTransform<PCollectionTuple, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple

Like PCollectionTuple.apply(String, PTransform) but defaulting to the name of the PTransform.

apply(String, PTransform<PCollectionTuple, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple

Applies the given PTransform to this input PCollectionTuple, using name to identify this specific application of the transform.

applyTransform(InputT, PTransform<? super InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.Pipeline

Like Pipeline.applyTransform(String, PInput, PTransform) but defaulting to the name provided by the PTransform.

applyTransform(String, InputT, PTransform<? super InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.Pipeline

Applies the given PTransform to this input InputT and returns its OutputT.

ApproximateQuantiles - Class in com.google.cloud.dataflow.sdk.transforms

PTransforms for getting an idea of a PCollection's data distribution using approximate N-tiles (e.g.

ApproximateQuantiles.ApproximateQuantilesCombineFn<T,ComparatorT extends Comparator<T> & Serializable> - Class in com.google.cloud.dataflow.sdk.transforms

The ApproximateQuantilesCombineFn combiner gives an idea of the distribution of a collection of values using approximate N-tiles.

ApproximateUnique - Class in com.google.cloud.dataflow.sdk.transforms

PTransforms for estimating the number of distinct elements in a PCollection, or the number of distinct values associated with each key in a PCollection of KVs.

ApproximateUnique() - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique

 

ApproximateUnique.ApproximateUniqueCombineFn<T> - Class in com.google.cloud.dataflow.sdk.transforms

CombineFn that computes an estimate of the number of distinct values that were combined.

ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique - Class in com.google.cloud.dataflow.sdk.transforms

A heap utility class to efficiently track the largest added elements.

ApproximateUniqueCombineFn(long, Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn

 

areAllSubtriggersFinished() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo

Returns true if all the sub-triggers of the current trigger are marked finished.

as(Class<T>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions

Transforms this object into an object of type <T> saving each property that has been manipulated.

as(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory

Creates and returns an object that implements <T>.

as(Class<T>) - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder

Creates and returns an object that implements <T> using the values configured on this builder during construction.

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder

 

asCloudObject() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder

Returns the CloudObject that represents this Coder.

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.CustomCoder

Returns the CloudObject that represents this Coder.

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.IterableCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder

Deprecated.

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag

 

asIterable() - Static method in class com.google.cloud.dataflow.sdk.transforms.View

Returns a View.AsIterable transform that takes a PCollection as input and produces a PCollectionView mapping each window to an Iterable of the values in that window.

asKeyedFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

 

asKeyedFn() - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn

Converts this GloballyCombineFn into an equivalent CombineFnBase.PerKeyCombineFn that ignores the keys passed to it and combines the values according to this GloballyCombineFn.

asKeyedFn() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext

 

asList() - Static method in class com.google.cloud.dataflow.sdk.transforms.View

Returns a View.AsList transform that takes a PCollection and returns a PCollectionView mapping each window to a List containing all of the elements in the window.

asMap() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData

 

asMap() - Static method in class com.google.cloud.dataflow.sdk.transforms.View

Returns a View.AsMap transform that takes a PCollection<KV<K, V>> as input and produces a PCollectionView mapping each window to a Map<K, V>.

asMultimap() - Static method in class com.google.cloud.dataflow.sdk.transforms.View

Returns a View.AsMultimap transform that takes a PCollection<KV&ltK, V>> as input and produces a PCollectionView mapping each window to its contents as a Map<K, Iterable<V>> for use as a side input.

asOutputReference(PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Encode a PValue reference as an output reference.

asPath() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription

Returns the string representation of this subscription as a path used in the Cloud Pub/Sub API.

asPath() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic

Returns the string representation of this topic as a path used in the Cloud Pub/Sub API.

assertSourcesEqualReferenceSource(BoundedSource<T>, List<? extends BoundedSource<T>>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Given a reference Source and a list of Sources, assert that the union of the records read from the list of sources is equal to the records read from the reference source.

assertSplitAtFractionBehavior(BoundedSource<T>, int, double, SourceTestUtils.ExpectedSplitOutcome, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Asserts that the source's reader either fails to splitAtFraction(fraction) after reading numItemsToReadBeforeSplit items, or succeeds in a way that is consistent according to SourceTestUtils.assertSplitAtFractionSucceedsAndConsistent(com.google.cloud.dataflow.sdk.io.BoundedSource<T>, int, double, com.google.cloud.dataflow.sdk.options.PipelineOptions).

assertSplitAtFractionExhaustive(BoundedSource<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Asserts that for each possible start position, BoundedSource.BoundedReader#splitAtFraction at every interesting fraction (halfway between two fractions that differ by at least one item) can be called successfully and the results are consistent if a split succeeds.

assertSplitAtFractionFails(BoundedSource<T>, int, double, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Asserts that the source's reader fails to splitAtFraction(fraction) after reading numItemsToReadBeforeSplit items.

assertSplitAtFractionSucceedsAndConsistent(BoundedSource<T>, int, double, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Verifies some consistency properties of BoundedSource.BoundedReader#splitAtFraction on the given source.

assertUnstartedReaderReadsSameAsItsSource(BoundedSource.BoundedReader<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Assert that a Reader returns a Source that, when read from, produces the same records as the reader.

AssignContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext

 

assignedWindows(WindowFn<T, W>, long) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils

 

assignOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn

Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.

assignOutputTime(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow

Returns the output timestamp to use for data in the specified window.

assignOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow

Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.

assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows

 

assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn

 

assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn

Returns true if this WindowFn assigns each element to a single window.

assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows

 

assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows

 

assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows

 

assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows

 

assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn

Returns the single window to which elements with this timestamp belong.

assignWindows(WindowFn<Object, GlobalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows

 

assignWindows(WindowFn<Object, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows

 

assignWindows(WindowFn<T, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn

 

assignWindows(WindowFn<Object, IntervalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions

 

assignWindows(WindowFn<Object, IntervalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows

 

assignWindows(WindowFn<T, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn

Given a timestamp and element, returns the set of windows into which it should be placed.

asSingleton() - Static method in class com.google.cloud.dataflow.sdk.transforms.View

Returns a View.AsSingleton transform that takes a PCollection with a single value per window as input and produces a PCollectionView that returns the value in the main input window when read as a side input.

asSingletonView() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally

Returns a PTransform that produces a PCollectionView whose elements are the result of combining elements per-window in the input PCollection.

asV1Beta1Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription

Deprecated.

the v1beta1 API for Cloud Pub/Sub is deprecated.

asV1Beta1Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic

Deprecated.

the v1beta1 API for Cloud Pub/Sub is deprecated.

asV1Beta2Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription

Deprecated.

the v1beta2 API for Cloud Pub/Sub is deprecated.

asV1Beta2Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic

Deprecated.

the v1beta2 API for Cloud Pub/Sub is deprecated.

AtomicCoder<T> - Class in com.google.cloud.dataflow.sdk.coders

A Coder that has no component Coders or other state.

AtomicCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.AtomicCoder

 

AtomicLongFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.PipelineOptions.AtomicLongFactory

 

AvroCoder<T> - Class in com.google.cloud.dataflow.sdk.coders

A Coder using Avro binary format.

AvroCoder(Class<T>, Schema) - Constructor for class com.google.cloud.dataflow.sdk.coders.AvroCoder

 

AvroIO - Class in com.google.cloud.dataflow.sdk.io

PTransforms for reading and writing Avro files.

AvroIO.Read - Class in com.google.cloud.dataflow.sdk.io

A root PTransform that reads from an Avro file (or multiple Avro files matching a pattern) and returns a PCollection containing the decoding of each record.

AvroIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io

A PTransform that reads from an Avro file (or multiple Avro files matching a pattern) and returns a bounded PCollection containing the decoding of each record.

AvroIO.Write - Class in com.google.cloud.dataflow.sdk.io

A root PTransform that writes a PCollection to an Avro file (or multiple Avro files matching a sharding pattern).

AvroIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io

A PTransform that writes a bounded PCollection to an Avro file (or multiple Avro files matching a sharding pattern).

AvroReader(AvroSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader

Reads Avro records of type T from the specified source.

AvroSource<T> - Class in com.google.cloud.dataflow.sdk.io

A FileBasedSource for reading Avro files.

AvroSource.AvroReader<T> - Class in com.google.cloud.dataflow.sdk.io

A BlockBasedSource.BlockBasedReader for reading blocks from Avro files.

awaitCompletion() - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.InProcessPipelineRunner.InProcessPipelineResult

Blocks until the Pipeline execution represented by this InProcessPipelineRunner.InProcessPipelineResult is complete, returning the terminal state.

B

BACKLOG_UNKNOWN - Static variable in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader

Constant representing an unknown amount of backlog.

baseOutputFilename - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink

Base filename for final output files.

baseTemporaryFilename - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation

Base filename used for temporary output files.

BATCH_WORKER_HARNESS_CONTAINER_IMAGE - Static variable in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner

 

BatchBigQueryIONativeReadTranslator() - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.BatchBigQueryIONativeReadTranslator

 

beforeElement(WindowedValue<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.ModelEnforcement

Called before a call to TransformEvaluator.processElement(WindowedValue) on the provided WindowedValue.

begin() - Method in class com.google.cloud.dataflow.sdk.Pipeline

Returns a PBegin owned by this Pipeline.

beginningOnDay(int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows

 

beginningOnDay(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows

 

BigEndianIntegerCoder - Class in com.google.cloud.dataflow.sdk.coders

A BigEndianIntegerCoder encodes Integers in 4 bytes, big-endian.

BigEndianLongCoder - Class in com.google.cloud.dataflow.sdk.coders

A BigEndianLongCoder encodes Longs in 8 bytes, big-endian.

BigQueryIO - Class in com.google.cloud.dataflow.sdk.io

PTransforms for reading and writing BigQuery tables.

BigQueryIO.Read - Class in com.google.cloud.dataflow.sdk.io

A PTransform that reads from a BigQuery table and returns a PCollection of TableRows containing each of the rows of the table.

BigQueryIO.Read.Bound - Class in com.google.cloud.dataflow.sdk.io

A PTransform that reads from a BigQuery table and returns a bounded PCollection of TableRows.

BigQueryIO.Write - Class in com.google.cloud.dataflow.sdk.io

A PTransform that writes a PCollection containing TableRows to a BigQuery table.

BigQueryIO.Write.Bound - Class in com.google.cloud.dataflow.sdk.io

A PTransform that can write either a bounded or unbounded PCollection of TableRows to a BigQuery table.

BigQueryIO.Write.CreateDisposition - Enum in com.google.cloud.dataflow.sdk.io

An enumeration type for the BigQuery create disposition strings.

BigQueryIO.Write.WriteDisposition - Enum in com.google.cloud.dataflow.sdk.io

An enumeration type for the BigQuery write disposition strings.

BigQueryOptions - Interface in com.google.cloud.dataflow.sdk.options

Properties needed when using BigQuery with the Dataflow SDK.

BigtableIO - Class in com.google.cloud.dataflow.sdk.io.bigtable

A bounded source and sink for Google Cloud Bigtable.

BigtableIO.Read - Class in com.google.cloud.dataflow.sdk.io.bigtable

A PTransform that reads from Google Cloud Bigtable.

BigtableIO.Write - Class in com.google.cloud.dataflow.sdk.io.bigtable

A PTransform that writes to Google Cloud Bigtable.

BinaryCombineDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn

 

BinaryCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn

 

BinaryCombineIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn

 

BinaryCombineLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn

 

Block() - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block

 

BlockBasedReader(BlockBasedSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader

 

BlockBasedSource<T> - Class in com.google.cloud.dataflow.sdk.io

A BlockBasedSource is a FileBasedSource where a file consists of blocks of records.

BlockBasedSource(String, long) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource

Creates a BlockBasedSource based on a file name or pattern.

BlockBasedSource(String, long, long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource

Creates a BlockBasedSource for a single file.

BlockBasedSource.Block<T> - Class in com.google.cloud.dataflow.sdk.io

A Block represents a block of records that can be read.

BlockBasedSource.BlockBasedReader<T> - Class in com.google.cloud.dataflow.sdk.io

A Reader that reads records from a BlockBasedSource.

BlockingDataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options

Options that are used to configure the BlockingDataflowPipelineRunner.

BlockingDataflowPipelineOptions.StandardOutputFactory - Class in com.google.cloud.dataflow.sdk.options

Returns a default of System.out.

BlockingDataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners

A PipelineRunner that's like DataflowPipelineRunner but that waits for the launched job to finish.

BlockingDataflowPipelineRunner(DataflowPipelineRunner, BlockingDataflowPipelineOptions) - Constructor for class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner

 

Bound() - Constructor for class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound

Deprecated.

Should be private. Instead, use one of the factory methods in BigQueryIO.Write, such as BigQueryIO.Write.to(String), to create an instance of this class.

BoundedReader() - Constructor for class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader

 

BoundedSource<T> - Class in com.google.cloud.dataflow.sdk.io

A Source that reads a finite amount of input and, because of that, supports some additional operations.

BoundedSource() - Constructor for class com.google.cloud.dataflow.sdk.io.BoundedSource

 

BoundedSource.BoundedReader<T> - Class in com.google.cloud.dataflow.sdk.io

A Reader that reads a bounded amount of input and supports some additional operations, such as progress estimation and dynamic work rebalancing.

BoundedWindow - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A BoundedWindow represents a finite grouping of elements, with an upper bound (larger timestamps represent more recent data) on the timestamps of elements that can be placed in the window.

BoundedWindow() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow

 

build() - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.StepTransformResult.Builder

 

buildTemporaryFilename(String, String) - Static method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation

Build a temporary filename using the temporary filename separator with the given prefix and suffix.

buildTrigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger

 

buildTrigger() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.TriggerBuilder

Return the Trigger built by this builder.

by(PredicateT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter

Deprecated.

use Filter.byPredicate(PredicateT), which returns a Filter transform instead of a ParDo.Bound.

byPredicate(PredicateT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter

Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that satisfy the given predicate.

ByteArrayCoder - Class in com.google.cloud.dataflow.sdk.coders

A Coder for byte[].

ByteCoder - Class in com.google.cloud.dataflow.sdk.coders

A ByteCoder encodes Byte values in 1 byte using Java serialization.

ByteKey - Class in com.google.cloud.dataflow.sdk.io.range

A class representing a key consisting of an array of bytes.

ByteKeyRange - Class in com.google.cloud.dataflow.sdk.io.range

A class representing a range of ByteKeys.

ByteKeyRangeTracker - Class in com.google.cloud.dataflow.sdk.io.range

A RangeTracker for ByteKeys in ByteKeyRanges.

ByteStringCoder - Class in com.google.cloud.dataflow.sdk.coders

A Coder for ByteString objects based on their encoded Protocol Buffer form.

C

CalendarWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A collection of WindowFns that windows values into calendar-based windows such as spans of days, months, or years.

CalendarWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows

 

CalendarWindows.DaysWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A WindowFn that windows elements into periods measured by days.

CalendarWindows.MonthsWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A WindowFn that windows elements into periods measured by months.

CalendarWindows.YearsWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A WindowFn that windows elements into periods measured by years.

cancel() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob

Cancels the job.

CannotProvideCoderException - Exception in com.google.cloud.dataflow.sdk.coders

The exception thrown when a CoderProvider cannot provide a Coder that has been requested.

CannotProvideCoderException(String) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException(String, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException(String, Throwable, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException(Throwable, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException.ReasonCode - Enum in com.google.cloud.dataflow.sdk.coders

Indicates the reason that Coder inference failed.

clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement

 

clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane

 

clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger

 

clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger

Clear any state associated with this trigger in the given window.

clearOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester

Clears the record of the elements output so far to the main output.

clearSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester

Clears the record of the elements output so far to the side output with the given tag.

Clock - Interface in com.google.cloud.dataflow.sdk.runners.inprocess

Access to the current time.

cloneAs(Class<T>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions

Makes a deep clone of this object, and transforms the cloned object into the specified type kls.

close() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader

Deprecated.

 

close() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter

Closes the channel and return the bundle result.

close() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader

Closes any ReadableByteChannel created for the current reader.

close() - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer

Finishes writing the bundle.

close() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader

Closes the reader.

CloudDebuggerOptions - Interface in com.google.cloud.dataflow.sdk.options

Options for controlling Cloud Debugger.

Coder<T> - Interface in com.google.cloud.dataflow.sdk.coders

A Coder<T> defines how to encode and decode values of type T into byte streams.

Coder.Context - Class in com.google.cloud.dataflow.sdk.coders

The context in which encoding or decoding is being done.

Coder.NonDeterministicException - Class in com.google.cloud.dataflow.sdk.coders

Exception thrown by Coder.verifyDeterministic() if the encoding is not deterministic, including details of why the encoding is not deterministic.

coderAllowsEncoding(Coder<T>, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderConsistentWithEquals(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderConsistentWithEqualsInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderDecodeEncodeContentsEqual(Coder<CollectionT>, CollectionT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in any Coder.Context.

coderDecodeEncodeContentsEqualInContext(Coder<CollectionT>, Coder.Context, CollectionT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in the given Coder.Context.

coderDecodeEncodeContentsInSameOrder(Coder<IterableT>, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in any Coder.Context.

coderDecodeEncodeContentsInSameOrderInContext(Coder<IterableT>, Coder.Context, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<Iterable<T>>, and value of type Iterable<T>, encoding followed by decoding yields an equal value of type Collection<T>, in the given Coder.Context.

coderDecodeEncodeEqual(Coder<T>, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<T>, and value of type T, encoding followed by decoding yields an equal value of type T, in any Coder.Context.

coderDecodeEncodeEqualInContext(Coder<T>, Coder.Context, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<T>, Coder.Context, and value of type T, encoding followed by decoding yields an equal value of type T.

coderDecodesBase64(Coder<T>, String, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderDecodesBase64(Coder<T>, List<String>, List<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderDecodesBase64ContentsEqual(Coder<IterableT>, String, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderDecodesBase64ContentsEqual(Coder<IterableT>, List<String>, List<IterableT>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderDeterministic(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<T>, and values of type T, if the values are equal then the encoded bytes are equal, in any Coder.Context.

coderDeterministicInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<T>, Coder.Context, and values of type T, if the values are equal then the encoded bytes are equal.

coderEncodesBase64(Coder<T>, T, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderEncodesBase64(Coder<T>, List<T>, List<String>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

CoderException - Exception in com.google.cloud.dataflow.sdk.coders

An Exception thrown if there is a problem encoding or decoding a value.

CoderException(String) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException

 

CoderException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException

 

CoderException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException

 

CoderFactories - Class in com.google.cloud.dataflow.sdk.coders

Static utility methods for creating and working with Coders.

CoderFactory - Interface in com.google.cloud.dataflow.sdk.coders

A CoderFactory creates coders and decomposes values.

coderHasEncodingId(Coder<T>, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

CoderProperties - Class in com.google.cloud.dataflow.sdk.testing

Properties for use in Coder tests.

CoderProperties() - Constructor for class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

CoderProvider - Interface in com.google.cloud.dataflow.sdk.coders

A CoderProvider may create a Coder for any concrete class.

coderProvider() - Static method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder

Deprecated.

 

coderProvider() - Static method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder

A CoderProvider that returns a ProtoCoder with an empty ExtensionRegistry.

CoderProviders - Class in com.google.cloud.dataflow.sdk.coders

Static utility methods for working with CoderProviders.

CoderRegistry - Class in com.google.cloud.dataflow.sdk.coders

A CoderRegistry allows registering the default Coder to use for a Java class, and looking up and instantiating the default Coder for a Java type.

CoderRegistry() - Constructor for class com.google.cloud.dataflow.sdk.coders.CoderRegistry

 

coderSerializable(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

CoGbkResult - Class in com.google.cloud.dataflow.sdk.transforms.join

A row result of a CoGroupByKey.

CoGbkResult(CoGbkResultSchema, Iterable<RawUnionValue>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult

A row in the PCollection resulting from a CoGroupByKey transform.

CoGbkResult(CoGbkResultSchema, Iterable<RawUnionValue>, int) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult

 

CoGbkResult.CoGbkResultCoder - Class in com.google.cloud.dataflow.sdk.transforms.join

A Coder for CoGbkResults.

CoGbkResultSchema - Class in com.google.cloud.dataflow.sdk.transforms.join

A schema for the results of a CoGroupByKey.

CoGbkResultSchema(TupleTagList) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema

Builds a schema from a tuple of TupleTag<?>s.

CoGroupByKey<K> - Class in com.google.cloud.dataflow.sdk.transforms.join

A PTransform that performs a CoGroupByKey on a tuple of tables.

CollectionCoder<T> - Class in com.google.cloud.dataflow.sdk.coders

A CollectionCoder encodes Collections in the format of IterableLikeCoder.

CollectionCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.CollectionCoder

 

com.google.cloud.dataflow.sdk - package com.google.cloud.dataflow.sdk

Provides a simple, powerful model for building both batch and streaming parallel data processing Pipelines.

com.google.cloud.dataflow.sdk.annotations - package com.google.cloud.dataflow.sdk.annotations

Defines annotations used across the SDK.

com.google.cloud.dataflow.sdk.coders - package com.google.cloud.dataflow.sdk.coders

Defines Coders to specify how data is encoded to and decoded from byte strings.

com.google.cloud.dataflow.sdk.coders.protobuf - package com.google.cloud.dataflow.sdk.coders.protobuf

Defines a Coder for Protocol Buffers messages, ProtoCoder.

com.google.cloud.dataflow.sdk.io - package com.google.cloud.dataflow.sdk.io

Defines transforms for reading and writing common storage formats, including AvroIO, BigQueryIO, and TextIO.

com.google.cloud.dataflow.sdk.io.bigtable - package com.google.cloud.dataflow.sdk.io.bigtable

Defines transforms for reading and writing from Google Cloud Bigtable.

com.google.cloud.dataflow.sdk.io.datastore - package com.google.cloud.dataflow.sdk.io.datastore

Provides an API for reading from and writing to Google Cloud Datastore over different versions of the Cloud Datastore Client libraries.

com.google.cloud.dataflow.sdk.io.range - package com.google.cloud.dataflow.sdk.io.range

Provides thread-safe helpers for implementing dynamic work rebalancing in position-based bounded sources.

com.google.cloud.dataflow.sdk.options - package com.google.cloud.dataflow.sdk.options

Defines PipelineOptions for configuring pipeline execution.

com.google.cloud.dataflow.sdk.runners - package com.google.cloud.dataflow.sdk.runners

Defines runners for executing Pipelines in different modes, including DirectPipelineRunner and DataflowPipelineRunner.

com.google.cloud.dataflow.sdk.runners.inprocess - package com.google.cloud.dataflow.sdk.runners.inprocess

Defines the InProcessPipelineRunner, which executes both Bounded and Unbounded Pipelines on the local machine.

com.google.cloud.dataflow.sdk.testing - package com.google.cloud.dataflow.sdk.testing

Defines utilities for unit testing Dataflow pipelines.

com.google.cloud.dataflow.sdk.transforms - package com.google.cloud.dataflow.sdk.transforms

Defines PTransforms for transforming data in a pipeline.

com.google.cloud.dataflow.sdk.transforms.display - package com.google.cloud.dataflow.sdk.transforms.display

Defines HasDisplayData for annotating components which provide display data used within UIs and diagnostic tools.

com.google.cloud.dataflow.sdk.transforms.join - package com.google.cloud.dataflow.sdk.transforms.join

Defines the CoGroupByKey transform for joining multiple PCollections.

com.google.cloud.dataflow.sdk.transforms.windowing - package com.google.cloud.dataflow.sdk.transforms.windowing

Defines the Window transform for dividing the elements in a PCollection into windows, and the Trigger for controlling when those elements are output.

com.google.cloud.dataflow.sdk.values - package com.google.cloud.dataflow.sdk.values

Defines PCollection and other classes for representing data in a Pipeline.

Combine - Class in com.google.cloud.dataflow.sdk.transforms

PTransforms for combining PCollection elements globally and per-key.

combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn

Combines the given output times, which must be from the same window, into an output time for a computed value.

combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults

Combines the given output times, which must be from the same window, into an output time for a computed value.

combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow

Combines the given output times, which must be from the same window, into an output time for a computed value.

Combine.AccumulatingCombineFn<InputT,AccumT extends Combine.AccumulatingCombineFn.Accumulator<InputT,AccumT,OutputT>,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

A CombineFn that uses a subclass of Combine.AccumulatingCombineFn.Accumulator as its accumulator type.

Combine.AccumulatingCombineFn.Accumulator<InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms

The type of mutable accumulator values used by this AccumulatingCombineFn.

Combine.BinaryCombineDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms

An abstract subclass of Combine.CombineFn for implementing combiners that are more easily and efficiently expressed as binary operations on doubles.

Combine.BinaryCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms

An abstract subclass of Combine.CombineFn for implementing combiners that are more easily expressed as binary operations.

Combine.BinaryCombineIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms

An abstract subclass of Combine.CombineFn for implementing combiners that are more easily and efficiently expressed as binary operations on ints

Combine.BinaryCombineLongFn - Class in com.google.cloud.dataflow.sdk.transforms

An abstract subclass of Combine.CombineFn for implementing combiners that are more easily and efficiently expressed as binary operations on longs.

Combine.CombineFn<InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

A CombineFn<InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT into a single output value of type OutputT.

Combine.Globally<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

Combine.Globally<InputT, OutputT> takes a PCollection<InputT> and returns a PCollection<OutputT> whose elements are the result of combining all the elements in each window of the input PCollection, using a specified CombineFn<InputT, AccumT, OutputT>.

Combine.GloballyAsSingletonView<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

Combine.GloballyAsSingletonView<InputT, OutputT> takes a PCollection<InputT> and returns a PCollectionView<OutputT> whose elements are the result of combining all the elements in each window of the input PCollection, using a specified CombineFn<InputT, AccumT, OutputT>.

Combine.GroupedValues<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

GroupedValues<K, InputT, OutputT> takes a PCollection<KV<K, Iterable<InputT>>>, such as the result of GroupByKey, applies a specified KeyedCombineFn<K, InputT, AccumT, OutputT> to each of the input KV<K, Iterable<InputT>> elements to produce a combined output KV<K, OutputT> element, and returns a PCollection<KV<K, OutputT>> containing all the combined output elements.

Combine.Holder<V> - Class in com.google.cloud.dataflow.sdk.transforms

Holds a single value value of type V which may or may not be present.

Combine.IterableCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms

Converts a SerializableFunction from Iterable<V>s to Vs into a simple Combine.CombineFn over Vs.

Combine.KeyedCombineFn<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

A KeyedCombineFn<K, InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT, associated with a key of type K, into a single output value of type OutputT.

Combine.PerKey<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

PerKey<K, InputT, OutputT> takes a PCollection<KV<K, InputT>>, groups it by key, applies a combining function to the InputT values associated with each key to produce a combined OutputT value, and returns a PCollection<KV<K, OutputT>> representing a map from each distinct key of the input PCollection to the corresponding combined value.

Combine.PerKeyWithHotKeyFanout<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

Like Combine.PerKey, but sharding the combining of hot keys.

Combine.SimpleCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms

Deprecated.

CombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

 

CombineFnBase - Class in com.google.cloud.dataflow.sdk.transforms

This class contains the shared interfaces and abstract classes for different types of combine functions.

CombineFnBase() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFnBase

 

CombineFnBase.GlobalCombineFn<InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms

A GloballyCombineFn<InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT into a single output value of type OutputT.

CombineFnBase.PerKeyCombineFn<K,InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms

A PerKeyCombineFn<K, InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT, associated with a key of type K, into a single output value of type OutputT.

CombineFns - Class in com.google.cloud.dataflow.sdk.transforms

Static utility methods that create combine function instances.

CombineFns() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns

 

CombineFns.CoCombineResult - Class in com.google.cloud.dataflow.sdk.transforms

A tuple of outputs produced by a composed combine functions.

CombineFns.ComposeCombineFnBuilder - Class in com.google.cloud.dataflow.sdk.transforms

A builder class to construct a composed CombineFnBase.GlobalCombineFn.

CombineFns.ComposedCombineFn<DataT> - Class in com.google.cloud.dataflow.sdk.transforms

A composed Combine.CombineFn that applies multiple CombineFns.

CombineFns.ComposedCombineFnWithContext<DataT> - Class in com.google.cloud.dataflow.sdk.transforms

A composed CombineWithContext.CombineFnWithContext that applies multiple CombineFnWithContexts.

CombineFns.ComposedKeyedCombineFn<DataT,K> - Class in com.google.cloud.dataflow.sdk.transforms

A composed Combine.KeyedCombineFn that applies multiple KeyedCombineFns.

CombineFns.ComposedKeyedCombineFnWithContext<DataT,K> - Class in com.google.cloud.dataflow.sdk.transforms

A composed CombineWithContext.KeyedCombineFnWithContext that applies multiple KeyedCombineFnWithContexts.

CombineFns.ComposeKeyedCombineFnBuilder - Class in com.google.cloud.dataflow.sdk.transforms

A builder class to construct a composed CombineFnBase.PerKeyCombineFn.

CombineFnWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext

 

combineOutputTimes(OutputTimeFn<?>, Iterable<? extends Instant>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFns

Applies the given OutputTimeFn to the given output times, obtaining the output time for a value computed.

CombineWithContext - Class in com.google.cloud.dataflow.sdk.transforms

This class contains combine functions that have access to PipelineOptions and side inputs through CombineWithContext.Context.

CombineWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext

 

CombineWithContext.CombineFnWithContext<InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

A combine function that has access to PipelineOptions and side inputs through CombineWithContext.Context.

CombineWithContext.Context - Class in com.google.cloud.dataflow.sdk.transforms

Information accessible to all methods in CombineFnWithContext and KeyedCombineFnWithContext.

CombineWithContext.KeyedCombineFnWithContext<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

A keyed combine function that has access to PipelineOptions and side inputs through CombineWithContext.Context.

CombineWithContext.RequiresContextInternal - Interface in com.google.cloud.dataflow.sdk.transforms

An internal interface for signaling that a GloballyCombineFn or a PerKeyCombineFn needs to access CombineWithContext.Context.

commit(Instant) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.InProcessPipelineRunner.UncommittedBundle

Commits this InProcessPipelineRunner.UncommittedBundle, returning an immutable InProcessPipelineRunner.CommittedBundle containing all of the elements that were added to it.

compact(AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.

compact(List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn

 

compact(K, AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn

Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.

compact(Object[]) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn

 

compact(Object[], CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext

 

compact(K, Object[]) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn

 

compact(K, Object[], CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext

 

compact(AccumT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext

Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.

compact(K, AccumT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext

Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.

compare(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.Largest

 

compare(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.Smallest

 

compare(KV<K, V>, KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.values.KV.OrderByKey

 

compare(KV<K, V>, KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.values.KV.OrderByValue

 

compareTo(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey

ByteKey implements Comparable<ByteKey> by comparing the arrays in lexicographic order.

compareTo(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow

 

compose() - Static method in class com.google.cloud.dataflow.sdk.transforms.CombineFns

Returns a CombineFns.ComposeCombineFnBuilder to construct a composed CombineFnBase.GlobalCombineFn.

ComposeCombineFnBuilder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeCombineFnBuilder

 

composeKeyed() - Static method in class com.google.cloud.dataflow.sdk.transforms.CombineFns

Returns a CombineFns.ComposeKeyedCombineFnBuilder to construct a composed CombineFnBase.PerKeyCombineFn.

ComposeKeyedCombineFnBuilder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder

 

CompressedReader(CompressedSource<T>, FileBasedSource.FileBasedReader<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader

Create a CompressedReader from a CompressedSource and delegate reader.

CompressedSource<T> - Class in com.google.cloud.dataflow.sdk.io

A Source that reads from compressed files.

CompressedSource.CompressedReader<T> - Class in com.google.cloud.dataflow.sdk.io

Reader for a CompressedSource.

CompressedSource.CompressionMode - Enum in com.google.cloud.dataflow.sdk.io

Default compression types supported by the CompressedSource.

CompressedSource.DecompressingChannelFactory - Interface in com.google.cloud.dataflow.sdk.io

Factory interface for creating channels that decompress the content of an underlying channel.

computeTargetTimestamp(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement

 

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder

 

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder

NullableCoder is consistent with equals if the nested Coder is.

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

contains(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow

Returns whether this window contains the given window.

containsInAnyOrder(T...) - Method in interface com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert

Asserts that the iterable in question contains the provided elements.

containsInAnyOrder(Iterable<T>) - Method in interface com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert

Asserts that the iterable in question contains the provided elements.

containsKey(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange

Returns true if the specified ByteKey is contained within this range.

Context(boolean) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.Context

 

Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.Context

 

Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn.Context

 

Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context

 

copyFrom(byte[]) - Static method in class com.google.cloud.dataflow.sdk.io.range.ByteKey

Creates a new ByteKey backed by a copy of the specified byte[].

copyToOutputFiles(List<String>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation

Copy temporary files to final output filenames using the file naming template.

Count - Class in com.google.cloud.dataflow.sdk.transforms

PTransorms to count the elements in a PCollection.

Count.PerElement<T> - Class in com.google.cloud.dataflow.sdk.transforms

Count.PerElement<T> takes a PCollection<T> and returns a PCollection<KV<T, Long>> representing a map from each distinct element of the input PCollection to the number of times that element occurs in the input.

CounterMark(long, Instant) - Constructor for class com.google.cloud.dataflow.sdk.io.CountingSource.CounterMark

Creates a checkpoint mark reflecting the last emitted value.

CountingInput - Class in com.google.cloud.dataflow.sdk.io

A PTransform that produces longs.

CountingInput() - Constructor for class com.google.cloud.dataflow.sdk.io.CountingInput

 

CountingInput.BoundedCountingInput - Class in com.google.cloud.dataflow.sdk.io

A PTransform that will produce a specified number of Longs starting from 0.

CountingInput.UnboundedCountingInput - Class in com.google.cloud.dataflow.sdk.io

A PTransform that will produce numbers starting from 0 up to Long.MAX_VALUE.

CountingSource - Class in com.google.cloud.dataflow.sdk.io

A source that produces longs.

CountingSource.CounterMark - Class in com.google.cloud.dataflow.sdk.io

The checkpoint for an unbounded CountingSource is simply the last value produced.

create(List<? extends Coder<?>>) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderFactory

Returns a Coder<?>, given argument coder to use for values of a particular type, given the Coders for each of the type's generic parameter types.

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions.StandardOutputFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.DataflowClientFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.PathValidatorFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.StagerFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions.JobNameFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory

 

create(PipelineOptions) - Method in interface com.google.cloud.dataflow.sdk.options.DefaultValueFactory

Creates a default value for a getter marked with Default.InstanceFactory.

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.CredentialDirFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.DefaultProjectFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.GcpUserCredentialsFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcsOptions.ExecutorServiceFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptions.AtomicLongFactory

 

create() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder

Creates and returns an object that implements PipelineOptions using the values configured on this builder during construction.

create() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory

Creates and returns an object that implements PipelineOptions.

create(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.Pipeline

Constructs a pipeline from the provided options.

create(DataflowPipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline

Creates and returns a new DataflowPipeline instance for tests.

create(Combine.GroupedValues<K, InputT, OutputT>, PCollection<KV<K, Iterable<InputT>>>, boolean, Random) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn

 

create() - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.ExecutorServiceFactory

Create a new ExecutorService.

create() - Static method in class com.google.cloud.dataflow.sdk.runners.inprocess.NanosOffsetClock